Sent a message to Tim Rühsen and got reply from Tim Rühsen (@rockdaboot) <[email protected]>
That said to sent email to this link. Have a program that uses wget2 under linux and it has no problems with using 13 threads to download 13 pages. Have found that in test with wine it has failures with threads set above 2, but they vary. Also, did some earlier test on windows 10 machine, and with native windows it wasn't getting errors till higher tests, but was doing whole process manually on windows.. Under Fedora 35 I ran this test script. for a in 1 2 3 4 5 6 7 8 9 10 11 12 13; do mkdir $a cd $a time wine wget2 --restrict-file-names=windows --max-threads=$a --secure-protocol=PFS -q --base="https://ssb-prod.ec.guamcc.edu/PROD/"; -i ../testlist.sav cd .. done cd 1 for b in 2 3 4 5 6 7 8 9 10 11 12 13; do for a in *; do echo $a; cmp $a ../$b/$a ; done; done>>../output 2>../output2 Conents of testlist.sav bwpkedir.P_NameDirectory?dir_indx=&next_row=001&prev_row=000&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=026&prev_row=025&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=051&prev_row=050&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=076&prev_row=075&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=101&prev_row=100&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=126&prev_row=125&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=151&prev_row=150&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=176&prev_row=175&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=201&prev_row=200&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=226&prev_row=225&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=251&prev_row=250&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=276&prev_row=275&searchtype=N&ButtonSelected= bwpkedir.P_NameDirectory?dir_indx=&next_row=301&prev_row=300&searchtype=N&ButtonSelected= The times don't vary much with 1 task taking about 7 seconds, and 2 being about 6. Above 2 the times tend to take longer, except for the ones with failures where those pages are not downloded completely. The output2 shows these errors. 1, 2, 4, and 5 seemed to download all files the same, but others had differences. Only ran once, so not sure if results would always be same. Running under linux files are always identicle regardless of multi-task. cmp: EOF on ../3/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=026&prev_row=025&searchtype=N&ButtonSelected= after byte 24464, in line 544 cmp: EOF on ../6/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=001&prev_row=000&searchtype=N&ButtonSelected= after byte 15816, in line 322 cmp: EOF on ../7/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=026&prev_row=025&searchtype=N&ButtonSelected= after byte 15816, in line 319 cmp: EOF on ../8/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=001&prev_row=000&searchtype=N&ButtonSelected= after byte 8080, in line 125 cmp: EOF on ../8/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=126&prev_row=125&searchtype=N&ButtonSelected= after byte 8080, in line 121 cmp: EOF on ../9/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=201&prev_row=200&searchtype=N&ButtonSelected= after byte 32656, in line 756 cmp: EOF on ../10/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=076&prev_row=075&searchtype=N&ButtonSelected= after byte 8080, in line 125 cmp: EOF on ../10/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=176&prev_row=175&searchtype=N&ButtonSelected= after byte 8080, in line 125 cmp: EOF on ../11/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=126&prev_row=125&searchtype=N&ButtonSelected= after byte 8080, in line 121 cmp: ../11/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=176&prev_row=175&searchtype=N&ButtonSelected=: No such file or directory cmp: EOF on ../12/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=126&prev_row=125&searchtype=N&ButtonSelected= after byte 8080, in line 121 cmp: EOF on ../12/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=276&prev_row=275&searchtype=N&ButtonSelected= after byte 8080, in line 118 cmp: EOF on ../13/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=026&prev_row=025&searchtype=N&ButtonSelected= after byte 15816, in line 319 cmp: EOF on ../13/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=126&prev_row=125&searchtype=N&ButtonSelected= after byte 32656, in line 740 cmp: EOF on ../13/bwpkedir.P_NameDirectory%3Fdir_indx=&next_row=201&prev_row=200&searchtype=N&ButtonSelected= after byte 16211, in line 335 Windows 10 machine is current disconnected, so haven't done tests on a native windows. Last time, the errors didn't show till higher numbers. Once got up to 10 threads with no errors, but then ran again at 10, and had errors. With the windows native it was a manual process since scripting doesn't work as nice. +------------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor (Retired) mailto:[email protected] mailto:[email protected] Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +------------------------------------------------------------+
