[Bug-wget] [GSoC Update] Week 9
= SUMMARY == My public fork of Wget2 project is available here [0]. I will continuously keep pushing my work so anyone interested can track me there. Feel free to participate in the discussions going on merge request with my mentors. Your feedback are highly appreciated. === INTRODUCTION === The purpose of this project is to use Libmicrohttpd as test suite for Wget2. I plan to do this by do some changes on function wget_test_start_server() also wget_test_stop_server() on src/libtest.c of Wget2. With this approach, I don't need to change existing test suite which call the internal server code through functions mentioned above. I've count there are 38 test file which use wget_test_start_server(). I must ensure all the test passed when I've done the transition. For installation prerequisite, I make that Libmicrohttpd support will needed when we need to run `make check` or run test suite. Then I need to modify configure.ac. I will give proper warning about this requirement. There is a section in README.md where I must explain to user to provide Libmicrohttpd to make all test running correctly. With Libmicrohttpd I can add new test using feature that not yet implemented in old server code, but ready on Libmicrohttpd, such as HTTP digest authentication and concurrent request checking. Mentors: Darshit Shah Ander Juaristi == UPDATES = Things which were done until this week: * Started working on wget_test_start_server() to use Libmicrohttpd as HTTP server. Workflow to resolve this: - Remove initial process for HTTP server socket - Create _http_server_start() function, wrapper for Libmicrohttpd. There is also function _answer_to_connection() which use to create proper HTTP response - Use select method (MHD_USE_SELECT_INTERNALLY) for threading model in Libmicrohttpd to get better compatibility - http_server_port seized automatically using Libmicrohttpd function by passing MHD_DAEMON_INFO_BIND_PORT or MHD_DAEMON_INFO_LISTEN_FD parameter to MHD_get_daemon_info() - Using iteration to parse urls data in _answer_to_connection(). This guarantee we can pass any variadic data to Libmicrohttpd and prevent segmentation fault - Fix _answer_to_connection() function to create proper HTTP response: - Handle arguments (query string) on URL - Handle redirection - Handle directory creation - Handle URL with IRI object - Handle URL with IDN hostname - Handle URL with query string which contains space - Handle If-Modified-Since header - Handle Byte Range request - Handle HTTP basic authentication - Handle HTTPS - Handle chunked tranfer encoding (though still need more discussion) * Fix compiler warning -Wunused-param caused by MHD API * Use persistent connection (Connection: keep-alive) instead multiple connection (Connection: close). This prevent hiding several problems of the connection between HTTP server and client = NEXT STEPS === Things which would be done in the coming week: * Fix CI testing on Gitlab MingW64 and Travis OSX - MingW64: Need a script for complete setup Libmicrohttpd with HTTPS support - OSX: Need more verbose debug since it fails to run HTTPS server * Add HTTP digest authentication test [0]: https://gitlab.com/gnuwget/wget2/tree/tmp-use-mhd Regards, Didik Setiawan
Re: [Bug-wget] Gzip Content-Encoding Patches
On Montag, 31. Juli 2017 23:17:42 CEST Tim Schlueter wrote: > Hi, > > Please see the attached patches which add automatic gzip decompression > for HTTP files with the Content-Encoding response header set correctly. > > It also adjusts a downloaded file's extension for br, compress, and > deflate Content-Encodings. > > Since the first patch set: > * doc/wget.texi has been updated to reflect the changes in the patches. > * Commit messages have been changed to be in the GNU change log style. > * The patches are attached to this email instead of being in the body. Very good work ! I pushed your commit to branch 'gzip' at https://gitlab.com/gnuwget/wget for further review and CI testing (just had time to glance at it and made a few gzipped downloads :-)) > I have not yet had a chance to look at what would be involved to add > automated tests for this patch set. IMO, the python test server should realize the offered encoding and compress the response body appropriately. This might be trivial to a python coder. Anyways, if you have resources to help out at wget2 (https://gitlab.com/ gnuwget/wget), that would be awesome. Will test your path(es) tomorrow and give you feedback. With Best Regards, Tim signature.asc Description: This is a digitally signed message part.
