Error downloading Windows Defender definitions

[simone73]

Using WSUS 10.2.1
If i try to download updates for all windows 7, windows 8.1 and windows 10, and i check "include Windows defender definitions" i get this error:

Code: Select all

Warning: Integrity database ..\client\md\hashes-wddefs.txt not found.
Downloading/validating Windows Defender definition files...

11/14 23:59:25 [ERROR] CUID#6 - Download aborted. URI=http://download.microsoft.
com/download/DefinitionUpdates/mpas-fe.exe
Exception: [AbstractCommand.cc:399] errorCode=1 URI=http://download.microsoft.co
m/download/DefinitionUpdates/mpas-fe.exe
  -> [RequestGroup.cc:714] errorCode=1 Download aborted.
  -> [DefaultBtProgressInfoFile.cc:277] errorCode=1 total length mismatch. expec
ted: 40024336, actual: 39892760

11/14 23:59:25 [NOTICE] Download GID#fb77ed42a084cbaa not complete: ../client/wd
defs/x86-glb/mpas-fe.exe

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
fb77ed|ERR |       0B/s|../client/wddefs/x86-glb/mpas-fe.exe

Status Legend:
(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page
 for details.

ERROR: Download failure for w61 glb.

[aker]

This is a general problem with the definition files. MS' servers are not in sync, every server provides a different version of the definition file. So if the download crashes (due to an connection error), wsusou will not be able to download the complete file as it will connect to a different server with a different version of the file.
So just retry.

[hbuhrmester]

aria2 can be much better configured than wget, and I once suggested some options, which may help with the download of the virus definition files:

viewtopic.php?f=3&t=5019

But maybe these options should be revised again:

I still think, that the idea is correct, to let aria2 restart each download from scratch, rather than resuming the same download. But using 10 simultaneous connections will spoil that approach and make everything worse: If aria2 starts to download 3 different versions of the same file from 10 different servers, it can easily break things:

Code: Select all

2015-06-26 21:31:30.901678 ERROR - [AbstractCommand.cc:303]CUID#6 - Download aborted. URI=http://download.microsoft.com/download/DefinitionUpdates/mpas-feX64.exe
Exception: [AbstractCommand.cc:303] errorCode=8 URI=http://download.microsoft.com/download/DefinitionUpdates/mpas-feX64.exe
  -> [HttpResponse.cc:109] errorCode=8 Invalid range header. Request: 537049-0/35967248, Response: 537049-35967247/35992344
2015-06-26 21:31:30.902328 NOTICE - [RequestGroupMan.cc:392]Download GID#1 not complete: ../client/wddefs/x64-glb/mpas-feX64.exe

Actually, we should have two sets of options:

• One set of options optimized for speed. This could be used for large files, which never change.
  
• Another set of options for the slower, but more reliable download of problematic files. Multiple simultaneous connections should not be used in this case.

This could be sorted out like this:

Common options for timestamping, timeouts and logging:

Code: Select all

--allow-overwrite=true --auto-file-renaming=false --remote-time=true --conditional-get=true --max-tries=10 --retry-wait=10 --timeout=60 --log=..\download.log --log-level=notice

The logfile should always be used with aria2, because it writes a status line to the terminal window and more detailed information to the logfile. This means, aria2 can divert the information to both places. With wget, the complete output is either to the terminal window or to the logfile.

Optimized options for large files, which don't change:

Code: Select all

--max-connection-per-server=5

Failsafe options for problematic files:

Code: Select all

--always-resume=false --max-resume-failure-tries=0

Summary

Multiple connections may not work well for the virus definition downloads. So far, I suggest:

1. Find the file custom\SetAria2EnvVars.cmd, which is created by the script ActivateAria2Downloads.cmd
   
2. Delete the options -x10 -j10 -s10 -k1M -R

The meaning of these options is:

Code: Select all

-x, --max-connection-per-server=<NUM>
      The  maximum  number of connections to one server for each down‐
      load.  Default: 1

-j, --max-concurrent-downloads=<N>
      Set maximum  number  of  parallel  downloads  for  every  static
      (HTTP/FTP)  URI,  torrent and metalink. See also --split option.
      Default: 5

-s, --split=<N>
      Download a file using N connections.  If more than  N  URIs  are
      given,  first  N  URIs  are used and remaining URIs are used for
      backup.  If less than N URIs are given, those URIs are used more
      than  once  so that N connections total are made simultaneously.
      The number of connections to the  same  host  is  restricted  by
      --max-connection-per-server  option.   See also --min-split-size
      option.  Default: 5

-k, --min-split-size=<SIZE>
      aria2  does not split less than 2*SIZE byte range.  For example,
      let's consider downloading 20MiB file. If SIZE is 10M, aria2 can
      split file into 2 range [0-10MiB) and [10MiB-20MiB) and download
      it using 2 sources(if --split >= 2, of course).  If SIZE is 15M,
      since  2*15M  > 20MiB, aria2 does not split file and download it
      using 1 source.  You can append K or M (1K = 1024, 1M =  1024K).
      Possible Values: 1M -1024M Default: 20M

 -R, --remote-time[=true|false]
      Retrieve timestamp of the remote file from the  remote  HTTP/FTP
      server  and  if  it  is  available,  apply it to the local file.
      Default: false

The option -R is actually the same as --remote-time=true, but I prefer the long options, because they are more self-explaining.

[WSUSUpdateAdmin]

Hi!

Thanks, but sorry, I didn't get which additional modifications for ActivateAria2Downloads.cmd you suggest.

Regards
T. Wittrock

[hbuhrmester]

There are no additional options.

A small solution for ActivateAria2Downloads.cmd would be to simply delete the options -x10 -j10 -s10 -k1M -R

The first three options are for multiple downloads, and the short option -R is the same as the long option --remote-time=true

A larger solution would require changes to DownloadUpdates.cmd as well as ActivateAria2Downloads.cmd. Then the various options could be better sorted out, depending on the download:

• msse and wddefs
  common and failsafe options:
  No multiple, simultaneous downloads of the same file; restart rather than resume downloads on error.
  
• all other downloads
  common and optimized options:
  Multiple downloads may be useful for larger files and slow connections, or to speed up the processing of long download lists.

[boco]

That problem applies to wget, too. Unfortunately, you can't tell wget to *NOT* automatically resume (or only if the filesize and timestamp is the same). There's no such check, leading to a corrupted file.

I'm not downloading Signatures anymore, it's a lost battle. They are updated far too often. Maybe the installer could, if desired, download and install the latest signatures on the target system, in case that system is not offline.