DownloadUpdates.sh - Timestamping and -O
Posted:
07.12.2011, 04:42
by nebyah
- Code: Select all
wget -nv -N --timeout=120 http://mse.dlservice.microsoft.com/download/0/2/C/02C8AB73-0774-4975-826F-9E8A0FD7F65D/dede/x86/mseinstall.exe -O ../client/msse/x86-glb/mseinstall-x86-deu.exe
WARNING: timestamping does nothing in combination with -O. See the manual
When -O is used, wget ignores -N (timestamping). This causes some files (vcredist*, mseinstall*, possibly others) to be downloaded several times, each time wsusoffline is run.
Might I suggest something like:
- Code: Select all
mkdir -p ../cache/mseinstall-x68-deu.exe \
&& wget -nv -N --timeout=120 http://mse.dlservice.microsoft.com/download/0/2/C/02C8AB73-0774-4975-826F-9E8A0FD7F65D/dede/x86/mseinstall.exe -P ../cache/mseinstall-x86-deu.exe/ \
&& cp -a ../cache/mseinstall-x86-deu.exe/mseinstall.exe ../client/msse/x86-glb/mseinstall-x86-deu.exe
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
12.12.2011, 12:26
by WSUSUpdateAdmin
Hi!
Yes, thanks, you're right, but the modification you suggest can't be done in that simple way, because all downloads are handled by the subroutine doWget(), so one has to modify this sub depending on required renaming.
Regards
Torsten Wittrock
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
13.12.2011, 02:53
by nebyah
I can see how that might be a problem.
(untested code)
- Code: Select all
foo()
{
oldname=$1
newname=$2
cachedir=`echo $oldname | tr -d "/:"`
cachefile=`basename $oldname`
mkdir -p ../cache/$cachedir \
&& doWget $oldname -P ../cache/$cachedir/ \
&& cp -a ../cache/$cachedir/$cachefile $newname
}
Then anywhere where you're currently using doWget $oldname -O /path/to/$newname
- Code: Select all
foo $oldname ../path/to/$newname
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
16.12.2011, 13:31
by WSUSUpdateAdmin
Hi nebyah,
I'ld like to have it the way the Windows part (DownloadUpdates.cmd) does it:
1. If exist <path>/<target name> rename <path>/<target name> to <path>/<URL file name>
2. doWget <URL> -P <path>
3. Rename <path>/<URL file name> to <path>/<target name>
This avoids both
- the warnings about wget -N -O parameter combination and
- the re-downloading of already existing files.
Do you have a Linux solution for that?
Thanks & regards
Torsten Wittrock
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
16.12.2011, 14:53
by harry
How about something like this:
- Code: Select all
...
if [ "$msse" == "1" ]; then
echo "Downloading MSSE files..."
if echo $sys | grep x64 > /dev/null 2>&1; then
mssestring=`cat ../temp/StaticUrls-msse-x64-glb.txt | grep ,`
arr=$(echo $mssestring | tr " " "\n")
for x in $arr
do
oldname=`echo $x | awk -F"," '{print $1}'`
newname=`echo $x | awk -F"," '{print $2}'`
tmpname=`basename $oldname`
mkdir -p ../client/msse/x64-glb
if [ -f "../client/msse/x64-glb/$newname" ]; then
mv -f "../client/msse/x64-glb/$newname" "../client/msse/x64-glb/$tmpname"
fi
doWget $oldname -P ../client/msse/x64-glb
if [ -n $newname ]; then
mv -f "../client/msse/x64-glb/$tmpname" "../client/msse/x64-glb/$newname"
fi
done
else
mssestring=`cat ../temp/StaticUrls-msse-x86-glb.txt | grep ,`
arr=$(echo $mssestring | tr " " "\n")
for x in $arr
do
oldname=`echo $x | awk -F"," '{print $1}'`
newname=`echo $x | awk -F"," '{print $2}'`
tmpname=`basename $oldname`
mkdir -p ../client/msse/x86-glb
if [ -f "../client/msse/x86-glb/$newname" ]; then
mv -f "../client/msse/x86-glb/$newname" "../client/msse/x86-glb/$tmpname"
fi
doWget $oldname -P ../client/msse/x86-glb
if [ -n $newname ]; then
mv -f "../client/msse/x86-glb/$tmpname" "../client/msse/x86-glb/$newname"
fi
done
fi
fi
...
or as a fucntion:
- Code: Select all
...
doWget_with_rename()
{
local old_new=$1
local loadtopath=$2
local oldname=`echo $old_new | awk -F"," '{print $1}'`
local newname=`echo $old_new | awk -F"," '{print $2}'`
local tmpname=`basename $oldname`
mkdir -p $loadtopath
if [ -f "$loadtopath/$newname" ]; then
mv -f "$loadtopath/$newname" "$loadtopath/$tmpname"
fi
doWget $oldname -P $loadtopath
local errWget=$?
if [ -n $newname ]; then
mv -f "$loadtopath/$tmpname" "$loadtopath/$newname"
fi
return $errWget
}
...
if [ "$msse" == "1" ]; then
echo "Downloading MSSE files..."
if echo $sys | grep x64 > /dev/null 2>&1; then
mssestring=`cat ../temp/StaticUrls-msse-x64-glb.txt | grep ,`
arr=$(echo $mssestring | tr " " "\n")
for x in $arr
do
doWget_with_rename $x ../client/msse/x64-glb
done
else
mssestring=`cat ../temp/StaticUrls-msse-x86-glb.txt | grep ,`
arr=$(echo $mssestring | tr " " "\n")
for x in $arr
do
doWget_with_rename $x ../client/msse/x86-glb
done
fi
fi
...
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
19.12.2011, 07:02
by nebyah
I ought to be less lazy, and test this stuff, but...
- Code: Select all
foo()
{
oldname=$1 #This would be the full download URL.
newname=$2 #This would be the full path we are saving to.
cachefile=`basename $oldname` #Filename part of URL.
#delete temp file if it exists.
#We had better do this, in case $newname does not exist and ../temp/$cachefile does exist, but represents a different $oldname.
#This situation should not occur, but could cause problems if it does occur. Therefore, to be safe, we delete first.
rm -f "../temp/$cachefile"
# copy existing. mv would be quicker, but this is safer - if dWget fails, current version will still exist.
# -a copies attributes such as modified time.
cp -a "$newname" "../temp/$cachefile"
doWget $oldname -P ../temp/ && mv ../temp/$cachefile $newname
rm -f "../temp/$cachefile" #Cleanup if doWget fails.
}
# later...
foo $oldname ../client/cpp/$newname
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
19.12.2011, 15:48
by Spider-Mann
In der "DownloadUpdates.cmd" wird das Problem folgendermaßen gelöst:
- Code: Select all
565 echo Downloading/validating installation files for C++ Runtime Libraries...
566 for %%i in (x64 x86) do (
567 for /F "tokens=1,2 delims=," %%j in (..\static\StaticDownloadLinks-cpp-%%i-glb.txt) do (
568 if "%%k" NEQ "" (
569 if exist ..\client\cpp\%%k (
570 echo Renaming file ..\client\cpp\%%k to %%~nxj...
571 ren ..\client\cpp\%%k %%~nxj
572 echo %DATE% %TIME% - Info: Renamed file ..\client\cpp\%%k to %%~nxj >>%DOWNLOAD_LOGFILE%
573 )
574 )
575 %WGET_PATH% -N -P ..\client\cpp %%j
576 if errorlevel 1 (
577 if exist ..\client\cpp\%%~nxj del ..\client\cpp\%%~nxj
578 echo Warning: Download of %%j failed.
579 echo %DATE% %TIME% - Warning: Download of %%j failed >>%DOWNLOAD_LOGFILE%
580 )
581 if "%%k" NEQ "" (
582 if exist ..\client\cpp\%%~nxj (
583 echo Renaming file ..\client\cpp\%%~nxj to %%k...
584 ren ..\client\cpp\%%~nxj %%k
585 echo %DATE% %TIME% - Info: Renamed file ..\client\cpp\%%~nxj to %%k >>%DOWNLOAD_LOGFILE%
586 )
587 )
588 )
589 )
und
- Code: Select all
669 echo Downloading/validating Microsoft Security Essentials installation files...
670 copy /Y ..\static\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt "%TEMP%\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt" >nul
671 if exist ..\static\custom\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt (
672 for /F %%i in (..\static\custom\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt) do echo %%i>>"%TEMP%\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt"
673 )
674 for /F "usebackq tokens=1,2 delims=," %%i in ("%TEMP%\StaticDownloadLinks-msse-%TARGET_ARCH%-glb.txt") do (
675 if "%%j" NEQ "" (
676 if exist ..\client\msse\%TARGET_ARCH%-glb\%%j (
677 echo Renaming file ..\client\msse\%TARGET_ARCH%-glb\%%j to %%~nxi...
678 ren ..\client\msse\%TARGET_ARCH%-glb\%%j %%~nxi
679 echo %DATE% %TIME% - Info: Renamed file ..\client\msse\%TARGET_ARCH%-glb\%%j to %%~nxi >>%DOWNLOAD_LOGFILE%
680 )
681 )
682 %WGET_PATH% -N -P ..\client\msse\%TARGET_ARCH%-glb %%i
683 if errorlevel 1 (
684 if exist ..\client\msse\%TARGET_ARCH%-glb\%%~nxi del ..\client\msse\%TARGET_ARCH%-glb\%%~nxi
685 echo Warning: Download of %%i failed.
686 echo %DATE% %TIME% - Warning: Download of %%i failed >>%DOWNLOAD_LOGFILE%
687 )
688 if "%%j" NEQ "" (
689 if exist ..\client\msse\%TARGET_ARCH%-glb\%%~nxi (
690 echo Renaming file ..\client\msse\%TARGET_ARCH%-glb\%%~nxi to %%j...
691 ren ..\client\msse\%TARGET_ARCH%-glb\%%~nxi %%j
692 echo %DATE% %TIME% - Info: Renamed file ..\client\msse\%TARGET_ARCH%-glb\%%~nxi to %%j >>%DOWNLOAD_LOGFILE%
693 )
694 )
695 )
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
19.12.2011, 21:56
by nebyah
You could skip the tests and send errors (eg file not found) to /dev/null:
- Code: Select all
mv -f "../client/cpp/$newname" "../client/cpp/$tmpname" 2>/dev/null
doWget $oldname -P ../client/cpp
mv -f "../client/cpp/$tmpname" "../client/cpp/$newname" 2>/dev/null
But is this code safe?
If wget fails half way through the download, will you be replacing an older but functional version of $newname with a corrupted version? Or does wget only replace the file after download is completed successfully?
Re: DownloadUpdates.sh - Timestamping and -O
Posted:
20.12.2011, 07:34
by Spider-Mann
In DownloadUpdates.sh Revision 326 steht noch folgendes:
- Code: Select all
Line 1234 -> doWget $oldname -O ../client/cpp/$newname
Line 1243 -> doWget $oldname -O ../client/cpp/$newname
Line 1263 -> doWget $oldname -O ../client/msse/x64-glb/$newname
Line 1272 -> doWget $oldname -O ../client/msse/x86-glb/$newname
und in den folgenden Zeilen stimmt auch etwas nicht:
- Code: Select all
Line 1167 -> doWget -i ../temp/StaticUrls-wddefs-x64-glb.txt -P ../client/wddefs/x64-glb
Line 1169 -> doWget -i ../temp/StaticUrls-wddefs-x86-glb.txt -P ../client/wddefs/x86-glb
Line 1286 -> doWget -i ../temp/StaticUrls-wddefs-x64-glb.txt -P ../client/wddefs
Line 1288 -> doWget -i ../temp/StaticUrls-wddefs-x86-glb.txt -P ../client/wddefs
entweder die ersten oder die zweiten beiden Zeilen sind fehlerhaft.