File integrity verification failure

File integrity verification failure

Postby lioninstreet » 13.10.2018, 21:43

Running 9.2.4 on xp

Yesterday, to fix sig check errors and missing root certificates I downloaded a few replacement files (sigcheck 2.3, rootsupd.exe, rvkroots.exe) and added to lines to .\client\md\hashes-win.glb.txt. Downloads completed no warnings or errors so it seemed like all was OK.

Today I tried running downloads to see if anything else would be found and the run ended with a file integrity verification error but without any other errors or warnings. Log is below:

Code: Select all
Sat 10/13/2018 14:04:00.50 - Info: Starting WSUS Offline Update download (v. 9.2.4) for wxp enu
Sat 10/13/2018 14:04:00.50 - Info: Option /includedotnet detected
Sat 10/13/2018 14:04:00.51 - Info: Option /verify detected
Sat 10/13/2018 14:04:00.53 - Info: Option /exitonerror detected
Sat 10/13/2018 14:04:00.59 - Info: Set time zone to LOC4:00
Sat 10/13/2018 14:04:01.09 - Info: Updated static download definitions
Sat 10/13/2018 14:04:01.37 - Info: Downloaded/validated mkisofs tool
Sat 10/13/2018 14:04:01.57 - Info: Found sigcheck.exe version 2.30.0.0 (common options: /accepteula -q -c)
Sat 10/13/2018 14:04:01.57 - Info: Verified integrity of Windows Update Agent installation and catalog files
Sat 10/13/2018 14:04:10.26 - Info: Downloaded/validated most recent Windows Update Agent installation and catalog files
Sat 10/13/2018 14:04:10.26 - Info: Verified digital file signatures of Windows Update Agent installation and catalog files
Sat 10/13/2018 14:04:10.26 - Info: Created integrity database for Windows Update Agent installation and catalog files
Sat 10/13/2018 14:04:20.90 - Info: Verified integrity of .NET Frameworks' installation files
Sat 10/13/2018 14:04:28.50 - Info: Downloaded/validated installation files for .NET Frameworks 3.5 SP1 and 4.x
Sat 10/13/2018 14:04:28.51 - Info: Verified integrity of existing updates for dotnet x86-glb
Sat 10/13/2018 14:04:40.50 - Info: Determined static update urls for dotnet x86-glb
Sat 10/13/2018 14:05:34.57 - Info: Found valid list of superseded updates
Sat 10/13/2018 14:05:51.64 - Info: Determined dynamic update urls for dotnet x86-glb
Sat 10/13/2018 14:05:51.81 - Info: Downloaded/validated 1 statically defined updates for dotnet x86-glb
Sat 10/13/2018 14:06:04.89 - Info: Downloaded/validated 117 dynamically determined updates for dotnet x86-glb
Sat 10/13/2018 14:06:07.42 - Info: Cleaned up client directory for dotnet x86-glb
Sat 10/13/2018 14:06:07.42 - Info: Removed NTFS alternate data streams for dotnet x86-glb
Sat 10/13/2018 14:06:13.03 - Info: Verified digital file signatures for dotnet x86-glb
Sat 10/13/2018 14:06:22.57 - Info: Created integrity database for dotnet x86-glb
Sat 10/13/2018 14:06:22.75 - Info: Cleaned up client directory for .NET Frameworks 3.5 SP1 and 4.x
Sat 10/13/2018 14:06:22.75 - Info: Verified digital file signatures for .NET Frameworks' installation files
Sat 10/13/2018 14:06:22.75 - Info: Created integrity database for .NET Frameworks' installation files
Sat 10/13/2018 14:06:29.29 - Info: Verified integrity of C++ Runtime Libraries' installation files
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2005_x64.exe to vcredist_x64.EXE
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x64.EXE to vcredist2005_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2008_x64.exe to vcredist_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x64.exe to vcredist2008_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2010_x64.exe to vcredist_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x64.exe to vcredist2010_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2012_x64.exe to vcredist_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x64.exe to vcredist2012_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2013_x64.exe to vcredist_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x64.exe to vcredist2013_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2017_x64.exe to VC_redist.x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\VC_redist.x64.exe to vcredist2017_x64.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2005_x86.exe to vcredist_x86.EXE
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x86.EXE to vcredist2005_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2008_x86.exe to vcredist_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x86.exe to vcredist2008_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2010_x86.exe to vcredist_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x86.exe to vcredist2010_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2012_x86.exe to vcredist_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x86.exe to vcredist2012_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2013_x86.exe to vcredist_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist_x86.exe to vcredist2013_x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\vcredist2017_x86.exe to VC_redist.x86.exe
Sat 10/13/2018 14:06:30.26 - Info: Renamed file ..\client\cpp\VC_redist.x86.exe to vcredist2017_x86.exe
Sat 10/13/2018 14:06:31.92 - Info: Downloaded/validated installation files for C++ Runtime Libraries
Sat 10/13/2018 14:06:32.31 - Info: Cleaned up client directory for C++ Runtime Libraries
Sat 10/13/2018 14:06:32.31 - Info: Verified digital file signatures for C++ Runtime Libraries' installation files
Sat 10/13/2018 14:06:32.31 - Info: Created integrity database for C++ Runtime Libraries' installation files
Sat 10/13/2018 14:06:35.87 - Error: File integrity verification failure


On a few other posts it looks like the fix is to go to to .\client\md and delete all the .txt files inside. In this case it seems odd because part of the fix for the missing root certificates was to add content to the .\client\md\hashes-win.glb.txt file.

Would this fix apply here as well?
lioninstreet
 
Posts: 37
Joined: 21.06.2018, 01:06

Re: File integrity verification failure

Postby Dalai » 13.10.2018, 23:55

File verification failures appear as soon as the contents of the (hashed) directories are changed, be it files changed, added or removed. Disable /exitonerror option to see the actual error. If that doesn't help, you can check where the error is yourself by executing HashDeep commands like this:
Code: Select all
wsusoffline\client\bin\hashdeep.exe -a -l -vv -k wsusoffline\client\md\hashes-dotnet.txt wsusoffline\client\dotnet\*.exe
Repeat that for all other hash files in wsusoffline\client\md, changing the paths, hash files and file specification accordingly.

But, at first I'd give it a try by renaming all files in wsusoffline\client\md. If it doesn't help, you can revert that change.

Regards
Dalai
Dalai
 
Posts: 633
Joined: 12.07.2016, 22:00

Re: File integrity verification failure

Postby lioninstreet » 14.10.2018, 01:11

Dalai wrote:File verification failures appear as soon as the contents of the (hashed) directories are changed, be it files changed, added or removed. Disable /exitonerror option to see the actual error.


Tried to find /exitonerror to disable in UpdateGenerator configuration settings. I didn't see a command line with this name. Where should I be looking for it? was going to change the names of all the files except the one I edited yesterday and see if that's what is throwing the error.
lioninstreet
 
Posts: 37
Joined: 21.06.2018, 01:06

Re: File integrity verification failure

Postby Dalai » 14.10.2018, 16:17

lioninstreet wrote:Tried to find /exitonerror to disable in UpdateGenerator configuration settings. I didn't see a command line with this name. Where should I be looking for it?

Sorry, I thought there is some checkbox in UpdateGenerator that toggles that switch. Well, don't use UpdateGenerator too much, so what do I know :oops:. After looking at the code, it's clear that the switch is set if "Only create collection script" is unchecked. So, to disable the switch, check this option, open a command line prompt (CMD) from the wsusoffline\cmd\custom directory and execute RunAll.cmd from there. It sounds more complicated than it is (hopefully ;)).

Regards
Dalai
Dalai
 
Posts: 633
Joined: 12.07.2016, 22:00

Re: File integrity verification failure

Postby lioninstreet » 16.10.2018, 05:08

Dalai wrote:Sorry, I thought there is some checkbox in UpdateGenerator that toggles that switch. Well, don't use UpdateGenerator too much, so what do I know :oops:. After looking at the code, it's clear that the switch is set if "Only create collection script" is unchecked. So, to disable the switch, check this option, open a command line prompt (CMD) from the wsusoffline\cmd\custom directory and execute RunAll.cmd from there. It sounds more complicated than it is (hopefully ;)).


Stupid question from my side. If I uncheck create collection script to permit the download error reporting, how will I be able to see the report showing what the error is other than trying to catch want the system says on the command line? Will the download history still be logged in the .txt file even though I unchecked showing the report?
lioninstreet
 
Posts: 37
Joined: 21.06.2018, 01:06

Re: File integrity verification failure

Postby Dalai » 16.10.2018, 14:59

WSUS Offline always logs the download operations to wsusoffline\log\download.log, regardless of the options selected in UpdateGenerator or if UpdateGenerator is used at all (i.e. called by a script instead).

Hope that answers your question.

Regards
Dalai
Dalai
 
Posts: 633
Joined: 12.07.2016, 22:00

Re: File integrity verification failure

Postby lioninstreet » 21.10.2018, 08:08

Kind of odd but the file integrity verification error has mysteriously disappeared.

I followed your suggestion and simply unchecked verify downloaded files and reran the download. No warnings or errors showed in the download log. I ran the download again, this time with verification checked and the log threw warning stating that a handful of *\client\hashes\*.txt files were not found again. No worries as that was solved with a third run.

Not sure why but on that second run, existing update NDP20SP2-KB2901111-x86.exe was deleted due to mismatching SHA-1 message digest.
lioninstreet
 
Posts: 37
Joined: 21.06.2018, 01:06

Re: File integrity verification failure

Postby hbuhrmester » 21.10.2018, 18:28

This really seems to be a complicated mess.

First of all, it seems that you downloaded 10 updates twice, with different file names:

In WSUS Offline Update 9.2.3, ten updates for .NET Frameworks were missing. Denniss suggested to add download links for the missing updates to the file StaticDownloadLinks-dotnet-x86-glb.txt (or a custom file with the same filename).
http://forums.wsusoffline.net/viewtopic ... 175#p16838

My suggestion was to only add the kb numbers to the file ExcludeList-superseded-exclude.txt. This also works, because the needed downloads really are in the WSUS catalog file, but they have been replaced (superseded) with newer downloads, which only work with the embedded Windows XP POSReady. The file ExcludeList-superseded-exclude.txt is meant for exactly this case: It enables superseded updates for download again.
http://forums.wsusoffline.net/viewtopic ... 650#p25118

This suggestion was accepted for WSUS Offline Update 9.2.4:
http://trac.wsusoffline.net/trac.fcgi/b ... xclude.txt

If you still keep the static download links added to version 9.2.3, then you will get two copies of the same files:

Code: Select all
NDP20SP2-KB2901111-x86.exe
ndp20sp2-kb2901111-x86_610083bd9139a2bc3fbdcd026e2d0cce11cafbc0.exe


The files really are identical, as they have the same SHA1 hashes:

Code: Select all
~/Downloads$ sha1sum NDP20SP2-KB2901111-x86.exe ndp20sp2-kb2901111-x86_610083bd9139a2bc3fbdcd026e2d0cce11cafbc0.exe
610083bd9139a2bc3fbdcd026e2d0cce11cafbc0  NDP20SP2-KB2901111-x86.exe
610083bd9139a2bc3fbdcd026e2d0cce11cafbc0  ndp20sp2-kb2901111-x86_610083bd9139a2bc3fbdcd026e2d0cce11cafbc0.exe


The SHA1 hash is plainly embedded into the filename of the second file. This is typical for all files, which are extracted from the WSUS catalog file. This allows an easy test:

After calculating the hashdeep files, column 3 (SHA1 hashes) and column 5 (pathnames) are extracted. The calculated and the embedded hashes are compared. If they don't match, then the file is deleted. This is when you get the error message "mismatching SHA-1 message digest".

Of course, the download script should delete the file ndp20sp2-kb2901111-x86_610083bd9139a2bc3fbdcd026e2d0cce11cafbc0.exe, because this is the file, which was tested. I guess, that it actually did, and only the error message pointed to the wrong file.

The file client/md/hashes-dotnet-x86-glb.txt must be rewritten without the failed hashes. At this point the download script may accidentally delete two lines, for both files. Then this file is still not correct, as it doesn't match the folder contents anymore, and the next "validation of existing files" may fail again.

So, you could do:

  • Double check, if Sigcheck 2.3 is actually working. SigCheck should catch damaged downloads first. Comparing the SHA1 hashes is only a second step. Do you use a really old CPU like Athlon XP or Pentium III?. Then you actually need SigCheck 2.1, because SigCheck 2.2 and higher need a CPU with SSE2 instruction set.

  • Remove the 10 static download links from the file StaticDownloadLinks-dotnet-x86-glb.txt, as they are not needed anymore, and also the 10 downloaded files.

  • Remove the file client/md/hashes-dotnet-x86-glb.txt, because it will not be in sync with the folder contents.

WSUS Offline Update 9.2.4 really solves most problems with missing updates, with the exception of the files rootsupd.exe / rvkroots.exe.
hbuhrmester
 
Posts: 324
Joined: 11.10.2013, 21:59


Return to Download

Who is online

Users browsing this forum: hbuhrmester and 13 guests