Processing of dynamic Office updates (ofc) is notoriously slow. This is mostly because of two nested loops, which do numerous line-by-line comparisons of two input files:
DownloadUpdates.cmd, version 10.3.2, lines 1302 - 1309
- Code: Select all
for /F "usebackq tokens=1,2 delims=," %%i in ("%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt") do (
for /F "usebackq tokens=1,2 delims=," %%k in ("%TEMP%\OfficeUpdateAndFileIds.txt") do (
if /i "%%l"=="%%i" (
echo %%j>>"%TEMP%\DynamicDownloadLinks-%1-%2.txt"
echo %%k,%%j>>"%TEMP%\UpdateTableURL-%1-%2.csv"
)
)
)
Typical line counts for the files while calculating "ofc glb" are:
- Code: Select all
2529 lines - OfficeUpdateCabExeIdsAndLocations.txt
8934 lines - OfficeUpdateAndFileIds.txt
2529 lines - DynamicDownloadLinks-ofc-glb.txt
2595 lines - UpdateTableURL-ofc-glb.csv
If the first file is read only once, this makes:
- Code: Select all
read operations: 1 + 2529 = 2530
write operations: 2529 + 2595 = 5124
total file operations: 2530 + 5124 = 7654
If a virus scanner intercepts all operations, this will be surely slow. But all this can be replaced by just two calls of "join":
- Code: Select all
..\bin\join -t "," -o "1.2" "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt" "%TEMP%\OfficeUpdateAndFileIds.txt" > "%TEMP%\DynamicDownloadLinks-%1-%2-Unsorted.txt"
..\bin\join -t "," -o "2.2,1.2" "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt" "%TEMP%\OfficeUpdateAndFileIds.txt" > "%TEMP%\UpdateTableURL-%1-%2.csv"
"join" is a classical Unix command and also part of the GNU Core Utilities package. Ports are available in the GNUWin32 project:
https://en.wikipedia.org/wiki/Join_%28Unix%29
https://en.wikipedia.org/wiki/GNU_Core_Utilities
https://en.wikipedia.org/wiki/GnuWin32
So they can probably be included in WSUS Offline Update, just like wget and cabextract. A working example would be:
- Code: Select all
@echo off
setlocal enableextensions enabledelayedexpansion
if errorlevel 1 goto NoExtensions
set CSCRIPT_PATH=%SystemRoot%\System32\cscript.exe
rem This is an excerpt from the function DownloadCore for the calculation
rem of dynamic office updates. It goes from the label :DetermineOffice
rem to :ExcludeOffice. It assumes, that the file "%TEMP%\package.xml" has
rem already been created.
rem
rem The GNU utilities gsort and join from the project GNUWin32 must be placed
rem in ..\bin. "gsort.exe" can be used instead of "sort.exe". It is the same
rem file, only renamed to avoid ambiguities. Two libraries are also needed,
rem so there are four new files in ..\bin:
rem
rem gsort.exe, join.exe, libiconv2.dll, libintl3.dll
rem
rem For comparing and extracting fields with join, both input files must be
rem sorted. Both GNU utilities should honor the environment variable LC_ALL
rem for sorting and comparison.
set LC_ALL=C
set LANG_SHORT=
call :DetermineOffice ofc glb
set LANG_SHORT=en
call :DetermineOffice ofc enu
set LANG_SHORT=de
call :DetermineOffice ofc deu
goto EoF
:DetermineOffice
rem *** Determine dynamic update urls for %1 %2 ***
echo %TIME% - Determining dynamic update urls for %1 %2 (please be patient, this will take a while)...
%CSCRIPT_PATH% //Nologo //E:vbs XSLT.vbs "%TEMP%\package.xml" ..\xslt\ExtractUpdateCategoriesAndFileIds.xsl "%TEMP%\UpdateCategoriesAndFileIds.txt"
if errorlevel 1 goto DownloadError
%CSCRIPT_PATH% //Nologo //E:vbs XSLT.vbs "%TEMP%\package.xml" ..\xslt\ExtractUpdateCabExeIdsAndLocations.xsl "%TEMP%\UpdateCabExeIdsAndLocations-Unsorted.txt"
if errorlevel 1 goto DownloadError
rem sort file using GNU sort
..\bin\gsort --unique "%TEMP%\UpdateCabExeIdsAndLocations-Unsorted.txt" > "%TEMP%\UpdateCabExeIdsAndLocations.txt"
del "%TEMP%\UpdateCabExeIdsAndLocations-Unsorted.txt"
if exist "%TEMP%\OfficeUpdateAndFileIds.txt" del "%TEMP%\OfficeUpdateAndFileIds.txt"
if exist "%TEMP%\OfficeFileIds.txt" del "%TEMP%\OfficeFileIds.txt"
set UPDATE_ID=
set UPDATE_CATEGORY=
set UPDATE_LANGUAGES=
for /F "usebackq tokens=1,2 delims=;" %%i in ("%TEMP%\UpdateCategoriesAndFileIds.txt") do (
if "%%j"=="" (
if "!UPDATE_CATEGORY!"=="477b856e-65c4-4473-b621-a8b230bb70d9" (
for /F "tokens=1-3 delims=," %%k in ("%%i") do (
if "%%l" NEQ "" (
if /i "%2"=="glb" (
if "!UPDATE_LANGUAGES!_%%m"=="_" (
rem Swap the field order in OfficeUpdateAndFileIds.txt
echo %%l,!UPDATE_ID!>>"%TEMP%\OfficeUpdateAndFileIds-Unsorted.txt"
echo %%l>>"%TEMP%\OfficeFileIds-Unsorted.txt"
)
if "!UPDATE_LANGUAGES!_%%m"=="en_en" (
echo %%l,!UPDATE_ID!>>"%TEMP%\OfficeUpdateAndFileIds-Unsorted.txt"
echo %%l>>"%TEMP%\OfficeFileIds-Unsorted.txt"
)
) else (
if "%%m"=="%LANG_SHORT%" (
echo %%l,!UPDATE_ID!>>"%TEMP%\OfficeUpdateAndFileIds-Unsorted.txt"
echo %%l>>"%TEMP%\OfficeFileIds-Unsorted.txt"
)
)
)
)
)
) else (
for /F "tokens=1 delims=," %%k in ("%%i") do (
set UPDATE_ID=%%k
)
for /F "tokens=1* delims=," %%k in ("%%j") do (
set UPDATE_CATEGORY=%%k
set UPDATE_LANGUAGES=%%l
)
)
)
set UPDATE_ID=
set UPDATE_CATEGORY=
set UPDATE_LANGUAGES=
del "%TEMP%\UpdateCategoriesAndFileIds.txt"
rem Sort both output files using GNU sort
..\bin\gsort --unique "%TEMP%\OfficeUpdateAndFileIds-Unsorted.txt" > "%TEMP%\OfficeUpdateAndFileIds.txt"
..\bin\gsort --unique "%TEMP%\OfficeFileIds-Unsorted.txt" > "%TEMP%\OfficeFileIds.txt"
del "%TEMP%\OfficeUpdateAndFileIds-Unsorted.txt"
del "%TEMP%\OfficeFileIds-Unsorted.txt"
rem Field order
rem File 1: OfficeFileIds.txt
rem - Field 1: FileId
rem File 2: UpdateCabExeIdsAndLocations.txt
rem - Field 1: FileId
rem - Field 2: Location (URL)
rem Write FileIds and Locations. Since both input files are sorted by the
rem first field, the output file will also be sorted.
..\bin\join -t "," "%TEMP%\OfficeFileIds.txt" "%TEMP%\UpdateCabExeIdsAndLocations.txt" > "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt"
del "%TEMP%\OfficeFileIds.txt"
del "%TEMP%\UpdateCabExeIdsAndLocations.txt"
rem Revised field order
rem File 1: OfficeUpdateCabExeIdsAndLocations.txt
rem - Field 1.1: FileId
rem - Field 1.2: Location (URL)
rem File 2: OfficeUpdateAndFileIds.txt
rem - Field 2.1: FileId
rem - Field 2.2: Bundle UpdateId
rem Write Locations only
..\bin\join -t "," -o "1.2" "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt" "%TEMP%\OfficeUpdateAndFileIds.txt" > "%TEMP%\DynamicDownloadLinks-%1-%2-Unsorted.txt"
..\bin\gsort --unique "%TEMP%\DynamicDownloadLinks-%1-%2-Unsorted.txt" > "%TEMP%\DynamicDownloadLinks-%1-%2.txt"
rem Write UpdateIds and Locations
..\bin\join -t "," -o "2.2,1.2" "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt" "%TEMP%\OfficeUpdateAndFileIds.txt" > "%TEMP%\UpdateTableURL-%1-%2.csv"
del "%TEMP%\OfficeUpdateAndFileIds.txt"
del "%TEMP%\OfficeUpdateCabExeIdsAndLocations.txt"
rem del "%TEMP%\DynamicDownloadLinks-%1-%2-Unsorted.txt"
if not exist ..\client\ofc\nul md ..\client\ofc
%CSCRIPT_PATH% //Nologo //E:vbs ExtractIdsAndFileNames.vbs "%TEMP%\UpdateTableURL-%1-%2.csv" ..\client\ofc\UpdateTable-%1-%2.csv
del "%TEMP%\UpdateTableURL-%1-%2.csv"
echo %TIME% - Done.
:ExcludeOffice
goto :eof
:NoExtensions
:DownloadError
:EoF
endlocal
This calculates the three files DynamicDownloadLinks-ofc-glb.txt, DynamicDownloadLinks-ofc-enu.txt, and DynamicDownloadLinks-ofc-deu.txt in a new record time.