Download gets omitted after failing with result code 4

Download gets omitted after failing with result code 4

Postby n8marti » 12.10.2017, 09:07

I have found wsusoffline to be tremendously useful for doing windows updates in my location where the internet connection is very slow and erratic. However, I'm hampered by the fact that the linux script seems to skip re-downloading or continuing to download a partially downloaded file because it checks the server and finds the remote file to be the same version it attempted to download previously, even though the download didn't complete correctly. See the output from download.log below.

2017-10-12 08:01:39 - Info: Downloading/validating mpam-fex64.exe, try 1 ...
--2017-10-12 08:01:39-- http://download.microsoft.com/download/ ... -fex64.exe
Resolving download.microsoft.com (download.microsoft.com)... 222.163.198.100, 113.207.33.152, 2a02:26f0:c000:298::e59, ...
Connecting to download.microsoft.com (download.microsoft.com)|222.163.198.100|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Connection: close
Content-Disposition: attachment
Date: Wed, 11 Oct 2017 19:14:05 GMT
Powered-By-ChinaCache: HIT from CNC-SY-d-3Wm.1
Powered-By-ChinaCache: HIT from CNC-YG-1-3W3
Last-Modified: Wed, 11 Oct 2017 17:23:56 GMT
store-version: _2621441507742636
Expires: Thu, 12 Oct 2017 07:14:05 GMT
X-CID: 6
Content-Length: 109037328
X-CCC: CN
Age: 42456
Accept-Ranges: bytes
Server: Microsoft-IIS/8.5
Cache-Control: public,max-age=14400
CC_CACHE: TCP_HIT
Length: 109037328 (104M) [application/octet-stream]
Saving to: '../client/msse/x64-glb/mpam-fex64.exe'

0K ........ ........ ........ ........ ........ ........ 2% 34.5K 49m57s
3072K ........ ........ ........ ........ ........ ........ 5% 49.0K 41m19s
6144K ........ ........ ........ ........ ........ ........ 8% 36.8K 41m23s
9216K ........ ........ ........ ........ ........ ........ 11% 31.5K 42m31s
12288K ........ ........ ........ ........ ........ ........ 14% 35.5K 41m29s
15360K ........ ........ ........ ........ ........ ........ 17% 42.9K 39m6s
18432K ....... 17% 58.0K=8m19s

2017-10-12 08:11:02 (37.9 KB/s) - Read error at byte 19339756/109037328 (Success). Giving up.

2017-10-12 08:11:02 - Error: Download/validation of mpam-fex64.exe failed with result code 4
2017-10-12 08:11:02 - Info: Restarting download in 10 seconds ...
2017-10-12 08:11:12 - Info: Downloading/validating mpam-fex64.exe, try 2 ...
--2017-10-12 08:11:12-- http://download.microsoft.com/download/ ... -fex64.exe
Resolving download.microsoft.com (download.microsoft.com)... 92.123.89.7
Connecting to download.microsoft.com (download.microsoft.com)|92.123.89.7|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 304 Not Modified
Content-Type: application/octet-stream
Last-Modified: Wed, 11 Oct 2017 19:18:50 GMT
ETag: "6ebcfc4c542d31:0"
Date: Thu, 12 Oct 2017 07:11:15 GMT
Connection: close
File '../client/msse/x64-glb/mpam-fex64.exe' not modified on server. Omitting download.


I would expect that on "try 2" the download would continue where it failed rather than being omitted. Is there a tweak I can use in the script to avoid this problem?
Thanks, Nate.
n8marti
 

Re: Download gets omitted after failing with result code 4

Postby hbuhrmester » 13.10.2017, 20:03

The download of five particular files often results in damaged files, if wget is allowed to continue a partial download. These are the four virus definition files (mpam-fe.exe, mpam-fex64.exe, mpas-fe.exe, mpas-feX64.exe) and the WSUS catalog file wsusscn2.cab.

The problem with these files is, that there are up to three different versions of the same file in the Microsoft content delivery network. They all have the same download url, but different modification dates and file sizes.

You can actually see this in the protocol you posted. The modification dates of the two files are different:

Code: Select all
Last-Modified: Wed, 11 Oct 2017 17:23:56 GMT
...
Last-Modified: Wed, 11 Oct 2017 19:18:50 GMT


This is a typical result for the virus definition files: they are updated every two hours, but the content delivery network is not fast enough, to synchronize all servers immediately. Then you may get two different files on two tries.

If wget were allowed to simply "continue" a partial download, then it would mix parts of two different files into one download. The downloaded file will be damaged, because the digital file signature will not verify, and it cannot be installed.

Therefore, I created a special "failsafe download" especially for these files: wget only gets one try for each download, and it must not continue a partial download. Instead, the script always restarts the download from scratch for up to ten times. The download succeeds, if the file can be completely downloaded in one try.

wget and aria2 use the conditional header If-Modified-Since for timestamping. The modification date of the local file is sent along with this header. Then the server must decide, if the file on the server is newer. The server can respond with "200 OK" and then wget will download the file. Or the server can respond with "304 Not Modified" and then the download is omitted.

But apparently, there is a bug in the script: wget does not set the modification date for a partially downloaded file, and then wget sends a wrong reference date on the second try. If the reference date is the current date, then the server will always respond with "304 Not Modified". The script already removes partial files after 10 tries, but actually, it should remove such files after each try.

The script creates a backup copy of the virus definition files, which will be restored, if a download completely fails. This file should also be copied back after failed download attempts, to let wget use the correct reference date in subsequent tries.

With these changes, the function download_single_file_failsafe in the file 40-configure-downloaders.bash would look like:

Code: Select all
function download_single_file_failsafe ()
{
    local download_dir="$1"
    local download_link="$2"
    local filename="${download_link##*/}"
    local result_code=1
    local try_count=1
    local max_tries=10
    local wait_time=10
    mkdir -p "$download_dir"

    until (( result_code == 0 )) || (( try_count > max_tries )); do

        log_info_message "Downloading/validating ${filename}, try $try_count ..."
        if "${downloader}" ${common_options} ${failsafe_options} \
            "${logfile_prefix}${logfile}" \
            "${download_dir_prefix}${download_dir}" \
            "${download_link}"
        then
            result_code=0
            log_debug_message "Download/validation of ${filename} succeeded"
        else
            result_code="$?"
            log_error_message "Download/validation of ${filename} failed with result code $result_code"

            # The modification date of partial downloads is set to
            # the current date and time. This will prevent the proper
            # timestamping on subsequent tries. Therefore, partial
            # downloads should always be deleted at this point.
            if [[ -f "${download_dir}/${filename}" ]]; then
                log_info_message "Removing partial download ${filename} ..."
                rm "${download_dir}/${filename}"
            fi

            # If there is a backup copy of the local file, then it should
            # be copied back (but not yet moved back). Then wget and aria2
            # can use the correct modification date for timestamping.
            #
            # The creation and removal of backup files is handled by
            # other functions, to make the approach more modular.
            if [[ -f "${download_dir}/${filename}.bak" ]]; then
                log_info_message "Restoring ${filename} from backup ..."
                cp -a "${download_dir}/${filename}.bak" "${download_dir}/${filename}"
            fi

            if (( try_count < max_tries )); then
                log_info_message "Restarting download in $(( try_count * wait_time )) seconds ..."
                sleep "$(( try_count * wait_time ))"
            else
                log_info_message "Maximum number of tries reached. Giving up."
                runtime_errors="$(( runtime_errors + 1 ))"
            fi
        fi

        try_count=$(( try_count + 1 ))
    done

    return 0
}


Of course, everything worked fine, when I tried it. I was using Debian 8 Jessie for development. This was released a few years ago, and it used an older version of wget. That version did not use the conditional header If-Modified-Since. Instead, files were downloaded, if the file on the server was newer, or if the local and the remote files had different file sizes. The second condition was always true for partial downloads, and then the file would always be downloaded, regardless of the modification date.
hbuhrmester
 
Posts: 525
Joined: 11.10.2013, 20:59

Re: Download gets omitted after failing with result code 4

Postby n8marti » 14.10.2017, 21:06

Thanks for the help, hbuhrmester. I substituted your update to the function, and it seems to be working as intended. (And, no, I don't have to wait too long to see a download interrupted around here.)

2017-10-14 20:48:44 - Info: Downloading/validating mpam-fex64.exe, try 1 ...
--2017-10-14 20:48:44-- http://download.microsoft.com/download/ ... -fex64.exe
Resolving download.microsoft.com (download.microsoft.com)... 92.123.89.7, 2a02:26f0:c000:294::e59, 2a02:26f0:c000:298::e59
Connecting to download.microsoft.com (download.microsoft.com)|92.123.89.7|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Accept-Ranges: bytes
Server: Microsoft-IIS/8.5
Content-Disposition: attachment
Last-Modified: Sat, 14 Oct 2017 14:31:46 GMT
ETag: "4e83d129f944d31:0"
Content-Length: 110782736
Date: Sat, 14 Oct 2017 19:48:47 GMT
Connection: close
Length: 110782736 (106M) [application/octet-stream]
Saving to: '../client/msse/x64-glb/mpam-fex64.exe'

0K ........ ........ ........ ........ ........ .. 2% 11.4K=3m58s

2017-10-14 20:53:45 (11.4 KB/s) - Read error at byte 2782329/110782736 (Success). Giving up.

2017-10-14 20:53:45 - Error: Download/validation of mpam-fex64.exe failed with result code 4
2017-10-14 20:53:45 - Info: Removing partial download mpam-fex64.exe ...
2017-10-14 20:53:45 - Info: Restoring mpam-fex64.exe from backup ...
2017-10-14 20:53:45 - Info: Restarting download in 10 seconds ...
2017-10-14 20:53:55 - Info: Downloading/validating mpam-fex64.exe, try 2 ...
--2017-10-14 20:53:55-- http://download.microsoft.com/download/ ... -fex64.exe
Resolving download.microsoft.com (download.microsoft.com)... 123.159.202.140, 119.188.138.14, 2a02:26f0:c000:294::e59, ...
Connecting to download.microsoft.com (download.microsoft.com)|123.159.202.140|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Connection: close
Content-Disposition: attachment
Date: Sat, 14 Oct 2017 19:53:59 GMT
Powered-By-ChinaCache: MISS from CNC-LQ-h-3WD.2
Powered-By-ChinaCache: MISS from CNC-WZ-3-3W6
Content-Length: 110784272
X-CID: 6
Expires: Sun, 15 Oct 2017 07:53:59 GMT
Last-Modified: Sat, 14 Oct 2017 16:32:53 GMT
X-CCC: CN
Accept-Ranges: bytes
Server: Microsoft-IIS/8.5
Cache-Control: public,max-age=14400
CC_CACHE: TCP_REFRESH_MISS
Length: 110784272 (106M) [application/octet-stream]
Saving to: '../client/msse/x64-glb/mpam-fex64.exe'

0K ........ ........ .......
n8marti
 

Re: Download gets omitted after failing with result code 4

Postby hbuhrmester » 15.10.2017, 10:22

These restarts can be annoying, but the download usually succeeds after a few tries.

But you can see the point: The Microsoft content delivery network again offered two different files, with the file modification date off by two hours:

Code: Select all
Last-Modified: Sat, 14 Oct 2017 14:31:46 GMT
Content-Length: 110782736
...
Last-Modified: Sat, 14 Oct 2017 16:32:53 GMT
Content-Length: 110784272


Note, that the Internet has developed its own prime time, which is now in the early evening. Video and music streaming has a large impact, and everybody starts watching at the same time in the evening, just like the traditional TV prime time.


Larger downloads seem to work more reliably, if tried in the early morning.
hbuhrmester
 
Posts: 525
Joined: 11.10.2013, 20:59

Re: Download gets omitted after failing with result code 4

Postby n8marti » 15.10.2017, 21:37

Yeah, unfortunately where I am it can literally take all day to get one of Windows 10's gargantuan 1+GB updates. I have my own script which runs several iterations of wsusoffline's shell script to cover the computers I support here, and it has run all weekend and still gave up on some of the downloads. The best download speed I get is about 60KB/s. The worst is more like 5 or 10KB/s, which the average probably being around 30 or 40KB/s. That translates to at least 5.5 hours for a single 1GB download, as long as there's no break in the connection during that time.
n8marti
 


Return to Linux

Who is online

Users browsing this forum: No registered users and 6 guests