Release date: 2021-05-11
Development branch: master
Supported version: WSUS Offline Update, Community Edition 12.5 (b66)
This is a first beta version of the Linux download scripts, version 2.4, to keep up with the changes in WSUS Offline Update, Community Edition 12.5 (b66).
This version changes the way, how superseded updates and dynamic updates for both Windows and Office are calculated: All dynamic updates are now extracted by the ProductId, separately for each Windows and Office version. Therefore, the download directory ../client/ofc is no longer used and may be deleted safely. The list of superseded updates is more complete, so that fewer updates for Windows Server 2012 and Windows 8.1 / Server 2012 R2 are downloaded.
Bugfixes
The Security-only Safety Guard did not work with GitLab
If security-only downloads are selected, then the full update rollups must be excluded from both download and installation. This requires a manual configuration, because incremental security-only updates and cumulative update rollups cannot be distinguished automatically.
The function seconly_safety_guard tries to make sure, that several configuration files in the directories client/static and client/exclude have been updated after the last official patch day; otherwise the download will be postponed.
Previously, the file modification date of these files would be compared to the current date. This worked well with Trac/svn. It does not work anymore with GitLab, because GitLab does not set the Last-Modified header for files, which are extracted from the version control system. Therefore, newly downloaded files will always have the file modification date set to the current date and time.
The Security-only Safety Guard now searches literally for the expected month name in the configuration files.
New features
Added two scripts for development
The new script reset-wsusoffline.bash deletes all automatically created files: timestamps, hashdeep files, cached files, superseded updates and the ETag database for GitLab. It keeps the manually created file preferences.bash, to preserve custom settings.
The file syntax-check.bash does a quick syntax check of all scripts, using both bash and shellcheck.
Both scripts are meant for development.
Changed features
New method for the calculation of dynamic updates
Dynamic updates for both Windows and Office are extracted by the ProductId, separately for each product. Dynamic Office updates are now downloaded to the directories ../client/o2k13 and ../client/o2k16. The directory ../client/ofc is no longer used and may be deleted safely.
New method for the calculation of superseded updates
The calculation of superseded updates was changed in two ways:
aker introduced a patch to recognize updates, which are only superseded in certain contexts. This may reduce the number of "exceptions", which are maintained in the files ExcludeList-superseded-exclude.txt and ExcludeList-superseded-exclude-seconly.txt.
The list of superseded updates is also more complete, because the restriction to two file paths was removed from the file extract-update-cab-exe-ids-and-locations.xsl. This means, that more superseded updates will be recognized and fewer updates for Windows Server 2012 and Windows 8.1 / Server 2012 R2 will be downloaded. This is basically the Windows 10 patch, that I suggested about a year ago:
- Windows 10 64 bit download folder
https://forums.wsusoffline.net/viewtopic.php?p=30568#p30568
Static download files for .NET Frameworks are simplified
The files:
- Code: Select all
StaticDownloadLinks-dotnet-x86-deu.txt
StaticDownloadLinks-dotnet-x64-deu.txt
are combined to:
- Code: Select all
StaticDownloadLinks-dotnet-deu.txt
Hashdeep files are calculated with the bare mode option
The hashdeep relative path mode option -l was replaced with the bare mode option -b. This removes all path information from the filenames of the hashed files. The calculation of the integrity database can be much simplified:
- The directory changes with pushd/popd are not necessary anymore.
- Since the bare mode removes all directory information, there is no distinction between a Windows path and Linux path.
- The counting of the input files was removed. This was mostly done for the dotnet directory, because single *.exe files were used as input. If these were missing, then the input would be empty, and hashdeep might try to read from standard input instead. The script could hang at this point. After the removal of the dotnet subdirectories, all directories are scanned recursively, and this prevents the possible bug.
Hashdeep files created with the relative path mode must be deleted once, because they would cause all updates to be reported as "moved".
The file hashes-wddefs.txt was replaced with two separate files for the subdirectories x86-glb and x64-glb.
Improved support for service packs
The option -includesp is supported by the scripts copy-to-target.bash, create-iso-image.bash and download-updates.bash. Service packs are read from the files:
- Code: Select all
../exclude/custom/ExcludeList-SPs.txt
../client/static/StaticUpdateIds-w63-upd1.txt
../client/static/StaticUpdateIds-w63-upd2.txt
This way, users can provide their own service packs with a custom ExcludeList-SPs.txt.
The support for the option -includesp also means, that some downloads may be missing, if this option is not used. However, the option was named -includesp rather than -excludesp for consistency with the other optional downloads.
Internal changes
The file ../client/exclude/ExcludeList.txt can be updated from GitLab.
The files StaticUpdateIds-w63-upd1.txt and StaticUpdateIds-w63-upd2.txt are used for the calculation of the ExcludeList-Linux-superseded-seconly.txt.
Removed features
Removed the "revised" method for calculating superseded updates, if security-only updates are selected
One of Microsoft's original goals for cumulative monthly update rollups was to include former updates:
Over time, Windows will also proactively add patches to the Monthly Rollup that have been released in the past. Our goal is eventually to include all of the patches we have shipped in the past since the last baseline, so that the Monthly Rollup becomes fully cumulative and you need only to install the latest single rollup to be up to date.
This refers to the cumulative monthly update rollups only, not to the incremental security-only updates. Therefore, sometimes updates will superseded by the monthly update rollups, but not by the security-only updates. These updates would be added to the file ExcludeList-superseded-exclude-seconly.txt. However, this is only done, if these updates are found to be missing during installation.
The revised method for calculating superseded updates tried to recognize this situation automatically:
- The file HideList-seconly.txt contains a list of all known update rollups. The kb numbers are traced back to the FileIds, and then to the parent bundle records.
- The RevisionIds of the parent bundle records are removed from the list of superseding updates: If update rollups are excluded from download, then they cannot possibly supersede other updates.
- The list of superseded updates is then calculated as before. The file ExcludeList-Linux-superseded-seconly-revised.txt will be similar to the file ExcludeList-Linux-superseded-seconly.txt, but with an additional correction for the update rollups from the file HideList-seconly.txt.
This method actually worked – in two cases it could predict missing updates, before they were reported as missing:
- Some update issues
https://forums.wsusoffline.net/viewtopic.php?f=4&t=7085
Here, two updates were reported as missing. The update, which could be recovered by the revised method, was "windows6.1-kb2631813-x64".
windows6.1-kb2631813 is still listed in the file ExcludeList-superseded-exclude.txt of the ESR-version:
https://gitlab.com/wsusoffline/wsusoffline/-/blob/esr-11.9/exclude/ExcludeList-superseded-exclude.txt - kb3003743
https://forums.wsusoffline.net/viewtopic.php?f=2&t=8697
But, as soon as all missing updates are added to the files ExcludeList-superseded-exclude.txt or ExcludeList-superseded-exclude-seconly.txt, this method would not make any difference anymore.
The revised method was introduced in the Linux download scripts, version 1.11 (2019-04-04) . It is removed now, because I don't know, how it fits into the new implementation of calculating superseded updates, or if it is still useful at all.
TODO
Download of Microsoft Edge (Chromium)
Downloads
The download links for this version are:
- http://downloads.hartmut-buhrmester.de/linux-scripts-2.4-b1.tgz
- http://downloads.hartmut-buhrmester.de/linux-scripts-2.4-b1_hashes.txt
- http://downloads.hartmut-buhrmester.de/linux-scripts-2.4-b1_virusreport.pdf
- http://downloads.hartmut-buhrmester.de/linux-scripts-2.4-b1_virusreport-fullpage.png
Link for the online check at VirusTotal. Note, that the URL includes the SHA-256 hash of the archive:
The hashes for the archive linux-scripts-2.4-b1.tgz are:
- Code: Select all
MD5 40f3529c17f4ed5ecb4138907d82e7dd
SHA-1 3b8aae4811bd6635b3e54690f0c79f8752d7f089
SHA-256 a0e3d0065f8f69501701d6829cf5a7abac71b0851ec5e87409b42c620b2d9779
Further Reading
The first three posts in the original topic A complete rewrite of the Linux scripts are still being updated, to point to the latest version:
