NEWS for Linux download scripts, version 1.20-ESR beta-1

NEWS for Linux download scripts, version 1.20-ESR beta-1

Postby hbuhrmester » 06.03.2021, 00:17

NEWS for the Linux download scripts, version 1.20-ESR (beta-1)

Release date: 2021-03-05
Development branch: esr-11.9
Supported version: WSUS Offline Update, Community Edition 11.9.8 (b39)

This is a first beta version of the Linux download scripts, version 1.20-ESR, to keep up with the changes in WSUS Offline Update, Community Edition 11.9.8 (b39).

This version changes the way, how superseded updates and dynamic updates for both Windows and Office are calculated: All dynamic updates are now extracted by the ProductId, separately for each Windows and Office version. Therefore, the download directories ../client/ofc, ../client/dotnet/x86-glb and ../client/dotnet/x64-glb are no longer used and may be deleted safely. The list of superseded updates is more complete, so that fewer updates for Windows Server 2012 and Windows 8.1 / Server 2012 R2 are downloaded.


New features

Added two scripts for development

The new script reset-wsusoffline.bash deletes all automatically created files: timestamps, hashdeep files, cached files, superseded updates and the ETag database for GitLab. It keeps the manually created file preferences.bash, to preserve custom settings.

The file syntax-check.bash does a quick syntax check of all scripts, using both bash and shellcheck.

Both scripts are meant for development.


Changed features

New method for the calculation of dynamic updates

Dynamic updates for both Windows and Office are extracted by the ProductId, separately for each product. Dynamic Office updates are now downloaded to the directories ../client/o2k13 and ../client/o2k16. The directory ../client/ofc is no longer used and may be deleted safely.

Dynamic updates for .NET Frameworks are downloaded the respective Windows directories. The subdirectories ../client/dotnet/x86-glb and ../client/dotnet/x64-glb are no longer used and should be deleted manually. Do not delete the parent directory ../client/dotnet, though, because it is still needed for statically defined downloads (.NET Framework installers and language packs).

New method for the calculation of superseded updates

The calculation of superseded updates was changed in two ways:

aker introduced a patch to recognize updates, which are only superseded in certain contexts. This may reduce the number of "exceptions", which are maintained in the files ExcludeList-superseded-exclude.txt and ExcludeList-superseded-exclude-seconly.txt.

The list of superseded updates is also more complete, because the restriction to two file paths was removed from the file extract-update-cab-exe-ids-and-locations.xsl. This means, that more superseded updates will be recognized and fewer updates for Windows Server 2012 and Windows 8.1 / Server 2012 R2 will be downloaded. This is basically the Windows 10 patch, that I suggested about a year ago:


Static download files for .NET Frameworks are simplified

The files:

Code: Select all
StaticDownloadLinks-dotnet-x86-deu.txt
StaticDownloadLinks-dotnet-x64-deu.txt


are combined to:

Code: Select all
StaticDownloadLinks-dotnet-deu.txt


Hashdeep files are calculated with the bare mode option

The hashdeep relative path mode option -l was replaced with the bare mode option -b. This removes all path information from the filenames of the hashed files. The calculation of the integrity database can be much simplified:

  • The directory changes with pushd/popd are not necessary anymore.
  • Since the bare mode removes all directory information, there is no distinction between a Windows path and Linux path anymore.
  • The counting of the input files was removed. This was mostly done for the dotnet directory, because single *.exe files were used as input. If these were missing, then the input would be empty, and hashdeep might try to read from standard input instead. The script could hang at this point. After the removal of the dotnet subdirectories, all directories are scanned recursively, and this prevents the possible bug.

Hashdeep files created with the relative path mode must be deleted once, because they would cause all updates to be reported as "moved".

The files hashes-msse.txt and hashes-wddefs.txt were replaced with separate files for the subdirectories x86-glb and x64-glb.

Improved support for service packs

The option -includesp is supported by the scripts copy-to-target.bash, create-iso-image.bash and download-updates.bash. Service packs are read from the files:

Code: Select all
../exclude/ExcludeList-SPs.txt
../exclude/custom/ExcludeList-SPs.txt
../client/static/StaticUpdateIds-w63-upd1.txt
../client/static/StaticUpdateIds-w63-upd2.txt


This way, users can provide their own service packs with a custom ExcludeList-SPs.txt.

The support for the option -includesp also means, that some downloads may be missing, if this option is not used. However, the option was named -includesp rather than -excludesp for consistency with the other optional downloads.


Removed features

Removed the "revised" method for calculating superseded updates, if security-only updates are selected

One of Microsoft's original goals for cumulative monthly update rollups was to include former updates:

Over time, Windows will also proactively add patches to the Monthly Rollup that have been released in the past. Our goal is eventually to include all of the patches we have shipped in the past since the last baseline, so that the Monthly Rollup becomes fully cumulative and you need only to install the latest single rollup to be up to date.

https://techcommunity.microsoft.com/t5/windows-blog-archive/further-simplifying-servicing-models-for-windows-7-and-windows-8/ba-p/166772


This refers to the cumulative monthly update rollups only, not to the incremental security-only updates. Therefore, sometimes updates will superseded by the monthly update rollups, but not by the security-only updates. These updates would be added to the file ExcludeList-superseded-exclude-seconly.txt. However, this is only done, if these updates are found to be missing during installation.

The revised method for calculating superseded updates tried to recognize this situation automatically:

  1. The file HideList-seconly.txt contains a list of all known update rollups. The kb numbers are traced back to the FileIds, and then to the parent bundle records.
  2. The RevisionIds of the parent bundle records are removed from the list of superseding updates: If update rollups are excluded from download, then they cannot possibly supersede other updates.
  3. The list of superseded updates is then calculated as before. The file ExcludeList-Linux-superseded-seconly-revised.txt will be similar to the file ExcludeList-Linux-superseded-seconly.txt, but with an additional correction for the update rollups from the file HideList-seconly.txt.

This method actually worked – in two cases it could predict missing updates, before they were reported as missing:


But, as soon as all missing updates are added to the files ExcludeList-superseded-exclude.txt or ExcludeList-superseded-exclude-seconly.txt, this method would not make any difference anymore.

The revised method was introduced in the Linux download scripts, version 1.11 (2019-04-04) . It is removed now, because I don't know, how it fits into the new implementation of calculating superseded updates, or if it is still useful at all.


Other changes

Simplified versioning of the Linux download scripts

The Linux download scripts for the development branch "esr-11.9" are now named:

  • Linux download scripts, version 1.x-ESR

The current and the next versions are:

  • Linux download scripts, version 1.20-ESR
  • Linux download scripts, version 1.21-ESR
  • Linux download scripts, version 1.22-ESR

There is no need to add a third version number or to stay at version 1.19.x-ESR any longer.

The download scripts for the "master" development branch are named:

  • Linux download scripts, version 2.x

The current and the next versions are:

  • Linux download scripts, version 2.4
  • Linux download scripts, version 2.5
  • Linux download scripts, version 2.6

So, version 1.x-ESR is the continuation of the extended support release with support for Windows Server 2008 and Windows 7 / Server 2008 R2.

Version 2.x is the current release.


Downloads

The download links for this version are:


Link for the online check at VirusTotal. Note, that the URL includes the SHA-256 hash of the archive:


The hashes for the archive linux-scripts-1.20-b1.tgz are:
Code: Select all
MD5      1d37c653a69eb4add523b2579c52d97a
SHA-1    5607daa47732a514904e7a7f9d3abc0ecd18027d
SHA-256  d7227a675cd5259f4d7b7fb0988c317219d3c4a3ec0f2ba2679586aad1310aec



Further Reading

The first three posts in the original topic A complete rewrite of the Linux scripts are still being updated, to point to the latest version:

hbuhrmester
 
Posts: 506
Joined: 11.10.2013, 20:59

Re: NEWS for Linux download scripts, version 1.20-ESR beta-1

Postby aker » 06.03.2021, 20:36

Great news.
If there is anything I should change in the Windows version, please tell me.
Wer Rechtschreibfehler findet, darf sie behalten oder an den Meistbietenden versteigern. / Everybody finding a misspelling is allowed to keep or sell it.
aker


WSUS Offline Update „Community Edition“ 12.5.1 / 11.9.9esr
https://gitlab.com/wsusoffline
aker
 
Posts: 3881
Joined: 02.03.2011, 15:32


Return to Linux

Who is online

Users browsing this forum: No registered users and 5 guests