Pass through "ofc" only once per UpdateGenerator execution

[banzeo]

Hello,

In my UpdateGenerator download options I chose to download updates for O2k7 O2k10 o2k13 & O2k16 (french)
It seems that the tool passes through "ofc" (glb and fra) for each version of office which takes a long time (>10min)
Would it be possible to pass through ofc only once when more than one office version is checked ?

KRgds

[hbuhrmester]

There is a bug: "ofc glb" is calculated twice, if Office 2016 is selected.

It is calculated once at this point:

DownloadUpdates.cmd 10.2.1+ (r710), line 1026

Code: Select all

for %%i in (o2k7 o2k10 o2k13 o2k16) do (
  if /i "%1"=="%%i" (
    call :DownloadCore ofc %2 %TARGET_ARCH% %SKIP_PARAM%
    if errorlevel 1 goto Error
  )
)

The difference between o2k7, o2k10, o2k13 and o2k16 is:

• In the UpdateGenerator.exe, o2k7, o2k10, o2k13 are called with a real language: enu, deu, fra or some other language. Only these real languages should be processed in this loop.
• o2k16 is always called with "glb".

So, for example, selecting o2k7 fra and o2k16 glb in UpdateGenerator.exe would be translated to:

Code: Select all

call :DownloadCore ofc fra for o2k7
call :DownloadCore ofc glb for o2k16

"ofc glb" is processed a second time for all Office versions at this point:

DownloadUpdates.cmd 10.2.1+ (r710), line 1046

Code: Select all

for %%i in (w60 w60-x64 w61 w61-x64 w62 w62-x64 w63 w63-x64 w100 w100-x64 ofc) do (
  if /i "%1"=="%%i" (
    call :DownloadCore %1 %2 %TARGET_ARCH% %SKIP_PARAM%
    if errorlevel 1 goto Error
  )
)

Deleting o2k16 from the first loop will probably fix it:

Code: Select all

for %%i in (o2k7 o2k10 o2k13) do (
  if /i "%1"=="%%i" (
    call :DownloadCore ofc %2 %TARGET_ARCH% %SKIP_PARAM%
    if errorlevel 1 goto Error
  )
)

Then "ofc glb" should be processed only once, even if several Office versions are selected.

"ofc fra" will still get processed three times, but this will be much faster.

An option to prevent any repeated processing would be the introduction of "same day" rules for the main updates, similar to the optional downloads dotnet, cpp, and wle.

[WSUSUpdateAdmin]

Hi!

The reason is slightly different, but yes, it's a bug which I fixed now.

Thanks & regards,
Torsten Wittrock

[hbuhrmester]

Implementing a same day rule for main updates

Same day rules can be used to prevent a repeated processing of the same tasks, if the script DownloadUpdates.cmd is run multiple times. The main update will always be different, but some tasks will be the same for each run:

• The optional downloads Visual C++ Runtime Libraries and Windows Live Essentials are the same for all supported Windows versions. Other optional downloads like .NET Frameworks only differ in the architecture.
• All Windows downloads include processing the task (or download target) "win glb".
• All Office downloads include the task "ofc glb".
• Office 2007, 2010 and 2013 also share the common task "ofc <language>", where language can be deu, enu, fra or another supported language.

There are already same day rules for all optional downloads. The existing rules for C++ Runtime Libraries and Windows Essentials 2012 look like:

Code: Select all

rem *** Download C++ Runtime Libraries' installation files ***

echo %DATE% %TIME% - Info: Verified integrity of C++ Runtime Libraries' installation files>>%DOWNLOAD_LOGFILE%
for %%i in (..\client\md\hashes-cpp.txt) do echo _%%~ti | %SystemRoot%\System32\find.exe "_%DATE:~-10%" >nul 2>&1
if not errorlevel 1 (
  echo Skipping download/validation of C++ Runtime Libraries' installation files due to 'same day' rule.
  echo %DATE% %TIME% - Info: Skipped download/validation of C++ Runtime Libraries' installation files due to 'same day' rule>>%DOWNLOAD_LOGFILE%
  goto SkipCPP
)
del ..\client\md\hashes-cpp.txt


rem *** Download Windows Essentials 2012 - not required for w60 ***

echo %DATE% %TIME% - Info: Verified integrity of Windows Essentials 2012 installation files>>%DOWNLOAD_LOGFILE%
for %%i in (..\client\md\hashes-wle.txt) do echo _%%~ti | %SystemRoot%\System32\find.exe "_%DATE:~-10%" >nul 2>&1
if not errorlevel 1 (
  echo Skipping download/validation of Windows Essentials 2012 installation files due to 'same day' rule.
  echo %DATE% %TIME% - Info: Skipped download/validation of Windows Essentials 2012 installation files due to 'same day' rule>>%DOWNLOAD_LOGFILE%
  goto SkipWLE
)
del ..\client\md\hashes-wle.txt

This can be used as a template to create a same day rule for the main updates — Windows and Office updates, and dynamic updates for the .NET Frameworks. There are only a few changes:

1. The name of the hashes file is changed from hashes-wle.txt to hashes-%1-%2.txt
2. The goto statement is changed from "goto SkipWLE" to "goto :eof". Since the label DownloadCore is treated as a function, this is the same as a return in other languages. It does not quit the script, but only leaves the function DownloadCore.
3. The messages are changed appropriately.

Then the following code block:

Lines 1061 - 1075 in DownloadUpdates.cmd version 10.3.1

Code: Select all

if exist ..\client\md\hashes-%1-%2.txt (
  echo Verifying integrity of existing updates for %1 %2...
  pushd ..\client\md
  ..\bin\%HASHDEEP_EXE% -a -l -vv -k hashes-%1-%2.txt -r ..\%1\%2
  if errorlevel 1 (
    popd
    goto IntegrityError
  )
  del hashes-%1-%2.txt
  popd
  echo %DATE% %TIME% - Info: Verified integrity of existing updates for %1 %2>>%DOWNLOAD_LOGFILE%
) else (
  echo Warning: Integrity database ..\client\md\hashes-%1-%2.txt not found.
  echo %DATE% %TIME% - Warning: Integrity database ..\client\md\hashes-%1-%2.txt not found>>%DOWNLOAD_LOGFILE%
)

can be changed to:

Code: Select all

if exist ..\client\md\hashes-%1-%2.txt (
  echo Verifying integrity of existing updates for %1 %2...
  pushd ..\client\md
  ..\bin\%HASHDEEP_EXE% -a -l -vv -k hashes-%1-%2.txt -r ..\%1\%2
  if errorlevel 1 (
    popd
    goto IntegrityError
  )
  rem *** Copy the following command and paste it below the same day rule
  rem del hashes-%1-%2.txt
  rem ***
  popd
  echo %DATE% %TIME% - Info: Verified integrity of existing updates for %1 %2>>%DOWNLOAD_LOGFILE%
  rem *** Insert the same day rule here
  for %%i in (..\client\md\hashes-%1-%2.txt) do echo _%%~ti | %SystemRoot%\System32\find.exe "_%DATE:~-10%" >nul 2>&1
  if not errorlevel 1 (
    echo Skipping download/validation of %1 %2 due to 'same day' rule.
    echo %DATE% %TIME% - Info: Skipped download/validation of %1 %2 due to 'same day' rule>>%DOWNLOAD_LOGFILE%
    goto :eof
  )
  rem ***
  rem *** Insert the command copied from above, prepending the correct path
  del ..\client\md\hashes-%1-%2.txt
  rem ***
) else (
  echo Warning: Integrity database ..\client\md\hashes-%1-%2.txt not found.
  echo %DATE% %TIME% - Warning: Integrity database ..\client\md\hashes-%1-%2.txt not found>>%DOWNLOAD_LOGFILE%
)

In my tests with Office 2007, 2010, 2013 enu and deu, and Office 2016 glb, it seems to work well.

[aker]

@hbuhrmester
This might cause problems, if wsusou downloads a newer wsuscn2.cab but skips "ofc". If another Office update has been added, wsusou will list it as missing, as it wasn't downloaded. A rule, not to download dynamic updates for existing platforms with 100% matching hashes, if wsusscn2.cab hasn't changed wouldn't cause this issue.

[Denniss]

A newer update database will always cause issues if download sections are excluded by same day rules. Unless this "same day" may be tied to the database checksum.
One isue with OFC is the long time required to determine the dynamic updates

[aker]

Wouldn't it be useful, to update ofc only, if wsusscn2.cab changes?

[hbuhrmester]

Scheduling of dynamic updates could be done by changing the file modification date of the hashes files:

Trigger a rebuild of dynamic updates

If the files wsusscn2.cab and ExcludeList-superseded-exclude.txt are updated, set the modification date of the hashes files to one day in the past. This would be appended to the calculation of superseded updates.

That much would work with the same day rule, as suggested in my previous post.


Postpone dynamic updates

If ofc updates have been recalculated, set the modification date of the hashes files to one week in the future.

For postponing updates, the same day rule would need to be modified: It should not simply check for the "same day", but rather if the hashes files are newer or older than the current date. If the files are older, then dynamic updates are recalculated.

[WSUSUpdateAdmin]

Hi.

Thank you very much for your detailed suggestions, hbuhrmester!

I'll review the code and implement sort of a "same day rule" for "ofc glb" for the next version, which will also eliminate the whole bunch of IE versions (see http://www.welt.de/wirtschaft/webwelt/a ... is-10.html).

For now, it's an improvement to have "ofc glb" calculated only once per download run.

Thanks & best regards,
Torsten Wittrock

[WSUSUpdateAdmin]

Hi!

I implemented the following now (line 1070+):

Code: Select all

  echo %DATE% %TIME% - Info: Verified integrity of existing updates for %1 %2>>%DOWNLOAD_LOGFILE%
  for %%i in (..\client\md\hashes-%1-%2.txt) do echo _%%~ti | %SystemRoot%\System32\find.exe "_%DATE:~-10%" >nul 2>&1
  if not errorlevel 1 (
    if exist ..\exclude\ExcludeList-superseded.txt (
      for %%i in (..\exclude\ExcludeList-superseded.txt) do echo _%%~ti | %SystemRoot%\System32\find.exe "_%DATE:~-10%" >nul 2>&1
      if errorlevel 1 (
        echo Skipping download/validation of %1 %2 due to 'same day' rule.
        echo %DATE% %TIME% - Info: Skipped download/validation of %1 %2 due to 'same day' rule>>%DOWNLOAD_LOGFILE%
        goto :eof
      )
    )
  )
  del ..\client\md\hashes-%1-%2.txt


That means, 'same day' rule will fire if hashes-%1-%2.txt has timestamp from today (thanks, Hartmut! ) and ExcludeList-superseded.txt has timestamp not from today, i.e. older.
This will also consider changed wsusscn2.cab files, because ExcludeList-superseded.txt will always be recalculated then.

Do you agree?

Cheers,
Torsten