A complete rewrite of the Linux scripts

Re: A complete rewrite of the Linux scripts

Postby mani » 21.01.2018, 15:34

The newest wget version 1.19 has an option "--compression=none" to avoid the new feature of automatic decompression........

after editing 40-configure-downloaders.bash (wget_common_options) it worked for this run, but the option is naturally gone after the update ;-)

BTW: can you add a delete of the update files before the download, because it doesn't get downloaded from wget again, due to existence of this files in the tmp directory

BTW: i would be great to have an extra add-on option in the preferences to avoid the adding after every update

Thanks
Mani
mani
 
Posts: 11
Joined: 04.04.2017, 15:50

Re: A complete rewrite of the Linux scripts

Postby hbuhrmester » 21.01.2018, 15:43

I have uploaded the file again as "sh-new-1.0.tgz".

The server seems to handle this file slightly differently: The content-type is now "application/x-gtar", and there is no content-encoding set. Then wget should not be lead to decompress the content on the fly.

Code: Select all
hb1@debian:~/tmp$ wget --debug http://downloads.hartmut-buhrmester.de/sh-new-1.0.tgz
DEBUG output created by Wget 1.18 on linux-gnu.

Reading HSTS entries from /home/hb1/.wget-hsts
URI encoding = »UTF-8«
Converted file name 'sh-new-1.0.tgz' (UTF-8) -> 'sh-new-1.0.tgz' (UTF-8)
--2018-01-21 15:24:44--  http://downloads.hartmut-buhrmester.de/sh-new-1.0.tgz
Auflösen des Hostnamens »downloads.hartmut-buhrmester.de (downloads.hartmut-buhrmester.de)« … 80.237.132.245, 2a01:488:42:1000:50ed:84f5:3f:9682
Caching downloads.hartmut-buhrmester.de => 80.237.132.245 2a01:488:42:1000:50ed:84f5:3f:9682
Verbindungsaufbau zu downloads.hartmut-buhrmester.de (downloads.hartmut-buhrmester.de)|80.237.132.245|:80 … verbunden.
Created socket 3.
Releasing 0x00920240 (new refcount 1).

---request begin---
GET /sh-new-1.0.tgz HTTP/1.1
User-Agent: Wget/1.18 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: downloads.hartmut-buhrmester.de
Connection: Keep-Alive

---request end---
HTTP-Anforderung gesendet, auf Antwort wird gewartet …
---response begin---
HTTP/1.1 200 OK
Date: Sun, 21 Jan 2018 14:24:45 GMT
Content-Type: application/x-gtar
Content-Length: 242429
Connection: keep-alive
Server: Apache
Last-Modified: Sun, 21 Jan 2018 14:24:21 GMT
ETag: "3b2fd-5634a11d31684"
Accept-Ranges: bytes

---response end---
200 OK
Registered socket 3 for persistent reuse.
Länge: 242429 (237K) [application/x-gtar]
Wird in »»sh-new-1.0.tgz«« gespeichert.

sh-new-1.0.tgz            100%[=====================================>] 236,75K   631KB/s    in 0,4s   

2018-01-21 15:24:45 (631 KB/s) - »»sh-new-1.0.tgz«« gespeichert [242429/242429]


I've changed the file available-version.txt as well.

The file type, as identified by xdg-open, is still the same:

Code: Select all
hb1@debian:~/tmp$ xdg-mime query filetype sh-new-1.0.tar.gz
application/x-compressed-tar
hb1@debian:~/tmp$ xdg-mime query filetype sh-new-1.0.tgz
application/x-compressed-tar


Regards,
hbuhrmester
hbuhrmester
 
Posts: 271
Joined: 11.10.2013, 20:59

Re: A complete rewrite of the Linux scripts

Postby mani » 21.01.2018, 16:11

there is definitely a behavior change in wget

Differences:
wget -S http://downloads.hartmut-buhrmester.de/ ... 1.0.tar.gz
1.18:
Content-Encoding: x-gzip
2018-01-21 14:58:22 (789 KB/s) - ‘sh-new-1.0.tar.gz’ saved [242429/242429]
1.19:
Content-Encoding: gzip
2018-01-21 16:06:36 (777 KB/s) - »sh-new-1.0.tar.gz« gespeichert [583680]

Note: the name is not changed!!!!!
on 1.19 with option compression=none is the behavior similar as 1.18
wget --compression=none -S http://downloads.hartmut-buhrmester.de/ ... 1.0.tar.gz
Content-Encoding: x-gzip
2018-01-21 16:13:35 (780 KB/s) - »sh-new-1.0.tar.gz.2« gespeichert [242429/242429]
mani
 
Posts: 11
Joined: 04.04.2017, 15:50

Re: A complete rewrite of the Linux scripts

Postby Dalai » 21.01.2018, 16:16

mani wrote:It looks like the newest debian version has some strange changes.....

Looks like it. The NEWS file of wget-1.19.4.tar.gz (at https://ftp.gnu.org/gnu/wget/) contains this:
Code: Select all
* Changes in Wget 1.19.4
* A major bug that caused GZip'ed pages to never be decompressed has been fixed
* Support for Content-Encoding and Transfer-Encoding have been marked as
  experimental and disabled by default

* Changes in Wget 1.19.3
* Prevent erroneous decompression of .gz and .tgz files with broken servers

[...]

* Changes in Wget 1.19.2
[...]
* New option --compression for gzip Content-Encoding


Regards
Dalai
Dalai
 
Posts: 454
Joined: 12.07.2016, 21:00

Re: A complete rewrite of the Linux scripts

Postby hbuhrmester » 21.01.2018, 16:29

Note: the name is not changed!!!!!


This is not a redirect; it is just another upload with a different name...

You surely need to try the new URL http://downloads.hartmut-buhrmester.de/sh-new-1.0.tgz

Regards,
hbuhrmester
hbuhrmester
 
Posts: 271
Joined: 11.10.2013, 20:59

Re: A complete rewrite of the Linux scripts

Postby mani » 21.01.2018, 16:57

Dear Hartmut
I meant that the decompressed file has still the compressed file name

the tgz file works perfect with wget version 1.19.2, because wget (option -S) doesn't find a Content-Encoding
2018-01-21 16:56:09 (783 KB/s) - »sh-new-1.0.tgz« gespeichert [242429/242429]

Mani

P.S.: lets wait for the version 1.19.4 on Debian testing and than I will check again
mani
 
Posts: 11
Joined: 04.04.2017, 15:50

Re: A complete rewrite of the Linux scripts

Postby mani » 27.01.2018, 11:33

good morning
Today I installed wget version 1.19.3-2 and it doesn't deflate the tar.gz files automatic anymore and the update runs flawless.

best regards
Mani
mani
 
Posts: 11
Joined: 04.04.2017, 15:50

Re: A complete rewrite of the Linux scripts

Postby hbuhrmester » 27.01.2018, 14:09

Very well,

the self update of the Linux scripts should also work with wget 1.19.2 now, because I uploaded the file as "sh-new-1.0.tgz", and I also changed the files:

http://downloads.hartmut-buhrmester.de/ ... ersion.txt
http://downloads.hartmut-buhrmester.de/ ... ew-1.0.txt

to point to the new tgz archive, rather than the tar.gz file.

The self-update script for the Linux scripts downloads the file "available-version.txt" first. This file has four pieces of information, separated by spaces:

  1. the version number
  2. the URL of the archive
  3. the URL of the hashes file
  4. the name of the expanded directory

This information can be read into four variables with the bash built-in command "read", and these are then used for the self-update of the Linux scripts.

So these URLs are not hard-coded in the script, and they could be adjusted to work with wget 1.19.2 as well.

This is similar to the self-update of WSUS Offline Update: The file "StaticDownloadLink-recent.txt" has the URL for the most recent version of WSUS Offline Update.

Regards,
hbuhrmester
hbuhrmester
 
Posts: 271
Joined: 11.10.2013, 20:59

Previous

Return to Linux

Who is online

Users browsing this forum: No registered users and 6 guests