A complete rewrite of the Linux scripts

A complete rewrite of the Linux scripts

Postby hbuhrmester » 14.12.2016, 13:53

Introducing a complete rewrite of the Linux download scripts

I like to introduce a complete rewrite of the Linux download scripts for the project WSUS Offline Update. These scripts offer many improvements over the legacy script DownloadUpdates.sh:

  • Separation of a frontend and backend script

    The script update-generator.bash is used to interactively select the update, language and download options. The script download-updates.bash fetches the selected updates without any user interaction. This separation makes the structure of both files more straightforward.

  • Highly modular approach

    Both scripts are further split into libraries, common tasks, setup tasks and download tasks. Each script does one task only in the most straightforward manner. This resembles the flow of control and makes the scripts easily expandable and more maintainable.

  • Unified language settings

    There is no distinction between default languages, custom languages and update languages.
    Users can specify one language on the command line, and then they will get downloads for the specified language only, and nothing more.

  • Verification of downloaded files

    SHA-1 hashes are embedded into the filename of all security updates, as a number of 40 hexadecimal digits. These are compared to the checksums, which are calculated by hashdeep.
    The verification of digital file signatures with Sysinternals Sigcheck running under wine was tried, but it doesn't really work without the necessary root certificates.

  • Compatibility

    The download script uses the same algorithms for calculating superseded and dynamic updates as the Windows script DownloadUpdates.cmd. The compliance with the Windows scripts can be tested with the scripts compare-integrity-database.bash and compare-update-tables.bash.

  • Desktop integration

    Obsolete updates are not deleted immediately, but moved into the trash. GNOME and most other GTK+ based desktop environments use GVFS to handle the trash. The package trash-cli can be used with other desktop environments or window managers. trash-cli should also work without any graphical environment.

  • Self updates of WSUS Offline Update

    Both the setup and the download script check for new versions of WSUS Offline Update. They also handle updates of the configuration files in the static and exclude directories.

  • Same day rules

    Same day rules are used to prevent the repeated evaluation of the same tasks in adjacent runs of the download script.

  • Documentation

    There is even a complete documentation.


Download and installation

An archive and the corresponding hashes file can be downloaded at:

http://downloads.hartmut-buhrmester.de/ ... a-2.tar.gz
http://downloads.hartmut-buhrmester.de/ ... beta-2.txt
http://downloads.hartmut-buhrmester.de/ ... beta-2.pdf

Verify the archive with:

Code: Select all
hashdeep -a -v -v -l -k hashes-sh-new-1.0-beta-2.txt sh-new-1.0-beta-2.tar.gz


Then unpack it into the wsusoffline directory. It should create a directory "sh-new-1.0-beta-2" along the existing directory "sh".


Edited on 2017-01-09 to point to the new version 1.0-beta-2
Last edited by hbuhrmester on 09.01.2017, 13:01, edited 1 time in total.
Reason: Aktualisierung der Download Links für Version 1.0-beta-2
hbuhrmester
 
Posts: 194
Joined: 11.10.2013, 20:59

Re: A complete rewrite of the Linux scripts

Postby Rush » 20.12.2016, 20:47

can anybody explane me how i used it under ubuntu linux in german please :) ?

thanks :)
Rush
 
Posts: 45
Joined: 09.08.2015, 19:05

Quick installation guide

Postby hbuhrmester » 22.12.2016, 15:38

Quick installation guide

Install the required and recommended packages...

on Debian:
Code: Select all
su -
aptitude install cabextract hashdeep wget xmlstarlet trash-cli


on Ubuntu:
Code: Select all
sudo apt-get install cabextract hashdeep wget xmlstarlet trash-cli


Download the archiv and the hashes file to the directory wsusoffline. This is the directory, where the Windows utility UpdateGenerator.exe resides.
Code: Select all
wget http://downloads.hartmut-buhrmester.de/sh-new-1.0-beta-2.tar.gz
wget http://downloads.hartmut-buhrmester.de/hashes-sh-new-1.0-beta-2.txt


Verify the integrity of the archive:
Code: Select all
hashdeep -a -v -v -l -k hashes-sh-new-1.0-beta-2.txt sh-new-1.0-beta-2.tar.gz


Unpack the archive in the directory wsusoffline:
Code: Select all
tar xvzf sh-new-1.0-beta-2.tar.gz


Change to the directory sh-new-1.0-beta-2 and run the script update-generator.bash, to interactively select the desired updates to fetch:
Code: Select all
cd sh-new-1.0-beta-2
./update-generator.bash


Note: the complete documentation is in the subdirectory documentation.


Edited on 2017-01-09 to point to the new version 1.0-beta-2
hbuhrmester
 
Posts: 194
Joined: 11.10.2013, 20:59

Kurzinstallationsanleitung

Postby hbuhrmester » 22.12.2016, 15:39

Kurzinstallationsanleitung

Installiere die benötigten und empfohlenen Pakete...

für Debian:
Code: Select all
su -
aptitude install cabextract hashdeep wget xmlstarlet trash-cli


für Ubuntu:
Code: Select all
sudo apt-get install cabextract hashdeep wget xmlstarlet trash-cli


Lade das Archiv und die Prüfsummendatei in das Verzeichnis wsusoffline herunter. Dies ist das Verzeichnis, in dem sich das Windows-Programm UpdateGenerator.exe befindet.
Code: Select all
wget http://downloads.hartmut-buhrmester.de/sh-new-1.0-beta-2.tar.gz
wget http://downloads.hartmut-buhrmester.de/hashes-sh-new-1.0-beta-2.txt


Überprüfe die Integrität des Archivs:
Code: Select all
hashdeep -a -v -v -l -k hashes-sh-new-1.0-beta-2.txt sh-new-1.0-beta-2.tar.gz


Entpacke das Archiv im Verzeichnis wsusoffline:
Code: Select all
tar xvzf sh-new-1.0-beta-2.tar.gz


Wechsle in das Verzeichnis sh-new-1.0-beta-2 und starte das Skript update-generator.bash, um die gewünschten Updates interaktiv auszuwählen:
Code: Select all
cd sh-new-1.0-beta-2
./update-generator.bash


Hinweis: Die vollständige Dokumentation befindet sich im Unterverzeichnis documentation.


Editiert am 09.01.2017 zur Anpassung an die neue Version 1.0-beta-2
hbuhrmester
 
Posts: 194
Joined: 11.10.2013, 20:59

New version 1.0-beta-2

Postby hbuhrmester » 09.01.2017, 13:08

New version 1.0-beta-2

A new version 1.0-beta-2 of the improved Linux download scripts has been released on 2017-01-09. This version brings two major improvements:

  1. The file client/autostart.ini will be rewritten to show an icon of the UpdateInstaller.exe and the built date of the medium. This file only works in Windows, and only, if it is in the root directory of a mounted ISO image, a real CD/DVD or a disk partition.

  2. A configuration variable $prefer_seconly is introduced to prefer security-only update rollups over the full quality update rollups for Windows 7 and Windows Server 2008 R2, Windows Server 2012, Windows 8.1 and Windows Server 2012 R2.

    This variable is supposed to be a permanent setting. It is defined and set to disabled in the script download-updates.bash. It should be edited in the file preferences.bash.

The downloads are available at:

http://downloads.hartmut-buhrmester.de/ ... a-2.tar.gz
http://downloads.hartmut-buhrmester.de/ ... beta-2.txt
http://downloads.hartmut-buhrmester.de/ ... beta-2.pdf

Note: The Introduction ( viewtopic.php?f=9&t=6180#p21327 ), Quick Installation Guide ( viewtopic.php?f=9&t=6180#p21449 ) and Kurzinstallationsanleitung ( viewtopic.php?f=9&t=6180#p21450 ) have been edited to point to the new download locations.
hbuhrmester
 
Posts: 194
Joined: 11.10.2013, 20:59

Re: A complete rewrite of the Linux scripts

Postby boco » 09.01.2017, 16:24

Maybe the Linux project should better be forked and run as a separate community project. There are more than a few differences between those OS (line endings, separators, case sensitivity etc.).
Microsoft update catalog: http://catalog.update.microsoft.com/v7/site/
Windows Install media creator: https://support.microsoft.com/en-us/help/15088/windows-create-installation-media
boco
 
Posts: 1679
Joined: 24.11.2009, 17:00
Location: Germany

Re: A complete rewrite of the Linux scripts

Postby crashmaster » 11.01.2017, 19:58

hbuhrmester,

First - Thank you for the excellent rewrite. We've been using it for a couple of weeks, it's fantastic and extremely well done with excellent documentation. It should completely replace the old shell scripts, or as another poster suggested, perhaps forked as there are several improvements and enhancements that we really like.

Second - We noticed that after the script finds a WSUS Offline upgrade available, it asks if you want to upgrade and it defaults to "N". If the user selects "Y", it upgrades wsus offline update and re-runs the comparison. This worked excellent, however is there any particular reason we could not adjust this to be "Y" or "Yes" instead? Or perhaps a new parameter with a boolean option so that we could set so that we update it automatically. We ask this because we are rarely at the console when we run your scripts - they are scheduled via cron. We'd rather it be auto upgraded if practical.

Third - Procedure to upgrade your sh-new / linux scripts themselves. I see there's a beta 2 version available. Perhaps I'm missing it in the preferences, or perhaps it's obvious - but is there an autoupdate mechanism/documentation for your scripts, or should we follow the install instructions and reinstall and just monitor this forum for changes?

Thank you!
crashmaster
 
Posts: 1
Joined: 11.01.2017, 18:32

Re: A complete rewrite of the Linux scripts

Postby WSUSUpdateAdmin » 23.01.2017, 14:22

Moin!

Das wird auf jeden Fall kommen.
Ich möchte nur "falcon" nicht vorgreifen bzw. in die Parade fahren, der sich ja dankenswerterweise bereit erklärt hat, die Pflege der Linux-Skripte zu übernehmen (vgl. viewtopic.php?f=9&t=5955).

Vielen Dank & viele Grüße,
Torsten
WSUSUpdateAdmin
Administrator
 
Posts: 1971
Joined: 07.07.2009, 14:38


Return to Linux

Who is online

Users browsing this forum: No registered users and 3 guests