Hello, First of all, apologies, this doesn't fit the category of a bug, because I can't figure out whether it's possible or an ever-intended solution for wget. I reviewed the man page and did some searching, but didn't find a solution for the problem.
I was interested in leveraging wget to keep a remote list of files in sync. I construct a file list on the fly that I'm feeding to wget, and a location I'm downloading files to. wget -nc -a $logfile -i $filePath/filesynclist.txt -P $folderSyncPath The issue is that these are coming from a CDN and are signed URLs. So they come with a temporary token value appended onto the file name, for example: installer.pkg?token=088817451a9c490093t5ob22eyaorncifukx However, the next time those URLs are looked up, that token value may change. Thus, that file list will also change because the old token value is not present/valid any longer. Ultimately, wget needs the token value to access the file, verify its presence, see if there has been a change, and download it, but because the token value itself is dynamic, it results in the content being downloaded over and over again. Is it possible to tell wget to ignore the token for the purposes of the comparison in the existing file list (eg wget looks up the full url, but then removes the token string and confirms if that matches something locally downloaded), and only download if there is not a "base url string" that matches? This is quite a complex problem. Thank you for the help. Casey Jensen Associate Experience Engineering Macintosh Engineering [email protected] Click to engage our team <https://intake.cloud.capitalone.com/mac-engineering> ______________________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
