Huge POST requests with WGET & "multipart/form-data" ----------------------------------------------------
Hello I have been using WGET for many years and appreciate it. Unfortunately I recently ran into trouble due to changed behaviour of some servers. Some POST requests started to fail silently with error code 4. No error or rejection message is showed. Even "-d" does not bring up any useful hints. The thing simply aborts the communication after having tried to send the POST-data. Finally I figured out that the server apparently started rejecting POST-data if it is bigger than 8 KiO and expects "multipart/form-data" instead. RTFM: > In particular, they both expect content of the form > key1=value1&key2=value2, with percent-encoding for special characters; > the only difference is that one expects its content as a command-line > parameter and the other accepts its content from a file. In particular, > "--post-file" is not for transmitting files as form attachments: > those must appear as key=value data (with appropriate percent-coding) > just like everything else. Wget does not currently support > multipart/form-data for transmitting POST data; only > application/x-www-form-urlencoded. ... > Currently, "--body-file" is not for transmitting files as a whole. > Wget does not currently support multipart/form-data for transmitting > data; only application/x-www-form-urlencoded. In the future, this may > be changed so that wget sends the "--body-file" as a complete file > instead of sending its contents to the server. I am less smart than I would like to be after having read this, especially the statement "sends the "--body-file" as a complete file instead of sending its contents" is incomprehensible to me. How can I send huge POST requests via "multipart/form-data" now ? * Can this be workarounded with existing WGET versions (prepare data manually and use "--body-file" to send "raw" data) ? * Is there a chance that WGET will support "multipart/form-data" soon (apparently this has been on the wishlist for a long time) ? * Is there some other simple HTTPS program that can do it (I need HTTPS/TLS, cookies, and huge POST "multipart/form-data") ? I need to send something like this: POST /w/api.php HTTP/1.1 User-Agent: Bot Content-Type: multipart/form-data; boundary=---------------------------8ce61ec834cf268 Host: en.wikipedia.org Cookie: <removed> Content-Length: 9999 Accept-Encoding: gzip Connection: Keep-Alive -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="action" edit -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="title" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Title here -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="text" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here Huge text here -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="summary" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="token" 1f287ba00a908e9622045e7b18ffa352+\ -----------------------------8ce61ec834cf268 Content-Disposition: form-data; name="assert" user -----------------------------8ce61ec834cf268-- Any help would be appreciated. Thank you. Sincerely ... Taylor
