Hi Damjan, Sounds great!
Can we have that as a Pull Request? Regards, Matthias Am 26.04.22 um 19:56 schrieb Damjan Jovanovic: > On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <j...@jagunet.com > <mailto:j...@jagunet.com>> wrote: > > I'm gonna look into the serf->(lib)curl option... Since we don't > use any of the fancy features of serf, I'm thinking that the easy > option might be best > > > > Hi > > I've ported our WebDAV content provider module from Serf to Curl. > > While it ended well, and several other bugs were found and fixed, it > definitely wasn't the "easy option" Jim ;). Starting with conservative > changes, it ended up needing total restructuring, and became more of a > rewrite. The crashes were frequent and hung connections many, and I > had to read up on the HTTP protocol, and read Curl and Serf's source > code, but eventually I prevailed, and a clean elegant stable Curl > WebDAV module emerged. > > The huge patch is attached for anyone wishing to review and test. > Unless there are major objections, I'll push it in a couple of days. > > STATUS > > It builds and works well on FreeBSD and Windows. > > Most of the code was reused, and all the operations and semantics > previously present with Serf, should have been preserved. > > Browsing WebDAV files and directories, loading files, overwriting them > ("Save"), creating them ("Save As"), renaming and deleting them, all > works. > > HTTP and HTTPS proxies work. Unlike Serf, Curl could also support > SOCKS proxies (with minimal further changes), but AOO lacks that > setting in the proxy UI and configuration. > > Authentication works, both to the destination server and to the proxy > server. I've successfully tested Basic and Digest authentication. Curl > supports every authentication method Serf does and more. > > HTTPS works, with a custom certificate verification function, using > our own certificate store from NSS and its API (like the Serf code > used). A bug was discovered (which is in the Serf implementation too) > where self-signed certificates were being unconditionally rejected; > apparently NSS wants to see that a copy of that certificate in its > certificate chain parameter too. Now they work, and the user gets > prompted to allow access. > > HTTPS and authentication can be used together on the same connection > and work well, both bringing up their UI dialogs as needed. > > A bug was fixed where when username and password were both present in > the URL (dav://user:pass@host/path), the code was trying to split them > at the "@" instead of ":". > > Unnecessary base64 encoding and decoding was removed, when importing > the TLS connection's certificates into our XSecurityEnvironment. They > now come in directly as ASN.1 DER, and still work. > > The code was greatly restructured and cleaned up, as Curl's API is > synchronous and blocking, with parameters set in advance instead of > through many callbacks, which has allowed using short clear methods, > and clean separation between the session and request classes. The > WebDAV content provider has shrunk from 35 to 21 C++ files, 43 to 29 > header files, and 19129 to 15991 lines of code. With WebDAV methods > centralized and refactored into only 10-20 lines of code each, instead > of scattered across 4 files, it is much more understandable and > maintainable now. > > Curl is vastly more portable than Serf. We should build easily now > even on OS/2. We can remain with existing build tools instead of > needing scons or cmake just to build Serf. > > 3 now unused dependencies were removed: apr, apr-util, and serf. Serf > isn't so bad. APR's pool idea is an ingenious way of doing resource > management in C. However Curl has excellent documentation, guides, > examples, and detailed explanations and even example code for each > setting, while Serf has no documentation. Serf might be worth it in a > project that already uses APR a lot, but we don't. > > Instead of the historical, crippled forms of logging like OSL_TRACE(), > which don't appear in release builds, I've made it use the newer > com.sun.star.logging UNO component (wrapped in > comphelper::EventLogger), which was inspired by java.util.logging, > with configurable verbosity levels, handlers (file and console) and > output formats (plain, csv), and importantly, which produces output in > release builds too. I've also made it so that on LogLevel::FINEST, > Curl's verbose mode is enabled and Curl's debug output is also logged > through us, with descriptions of what Curl is doing, and logs of all > HTTP traffic including headers and bodies, before encryption and after > decryption in the case of HTTPS, giving us tremendous detail that can > be used for troubleshooting problems. > > CURL CHANGED TO USE OPENSSL AND ZLIB > > Curl only supports the custom TLS certificate verification function we > use (the CURLOPT_SSL_CTX_FUNCTION option) when built with OpenSSL, > wolfSSL or mbedTLS providers. We currently use schannel on Windows > instead, which had to be changed. I also made it use zlib, which > generally helps, and WebDAV uses XML which is very verbose and > benefits from compression. On other OSes with system curl, it is now > checked for its SSL provider, and configure fails if it isn't OpenSSL. > > The new WebDAV module successfully builds and runs with both OpenSSL > 1.0.2 or 1.1.1. However 1 function was renamed between those versions, > so the OpenSSL version at runtime probably has to match the one used > at compile time (although building with 1.0.2 headers might allow > running with 1.1.1 - not tested). > > (After completing development and testing, it dawned on me there is a > completely different way to do the certification verification, which > should allow other SSL providers to be used and might be better in > various ways. See later.) > > We currently build zlib as a static library only, and on Windows its C > runtime library is linked statically (_MT), which can't be mixed with > curl's dynamic linking of the runtime library (_MD). Thus curl was > made to link it statically too. Most if not all of our modules link > the runtime library statically too (which begs the question, why do we > ship the msvcr* redistributables to users at all then?). > > ISSUES > > The file open dialog (Ctrl+O) can hang for several minutes when first > connecting to a server. This is not new - it happens with Serf as > well. This appears to be caused by autocompletion in the file dialog. > When typing in a URL like "davs://127.0.0.1 <http://127.0.0.1>", the > WebDAV content provider is first called with a partial "https://1";, > before the rest of the URL is entered. That "1" is (somehow) treated > as IP address "0.0.0.1", and a TCP connection to 0.0.0.1 is started. > Only after several minutes, when that connection times out and fails, > does the content provider get another request with the complete URL, > which succeeds. The distinctly unpleasant wait, is luckily only > present that the first time that server is used, as caching remembers > the URL even across AOO restarts, so the "1" is automatically expanded > to the full URL and the content provider never sees "https://1"; again. > > You can only enter credentials for the HTTP(S) proxy or the > destination, never both, as the credential manager caches credentials > per URL, not per host/realm, so while you are prompted for credentials > for both, and Curl is told about both, the destination credentials > overwrite the proxy credentials in the cache, so one Curl request > works but future Curl requests use the wrong credentials to the proxy. > This doesn't matter, as AOO doesn't support passwords for proxy > servers anyway, and Serf has the same issue. > > Damjan > > -- > > P.S. APACHE 2.4 SETUP FOR TESTING > > Put some files in /var/tmp/dav/files, then configure Apache HTTPD's > httpd.conf with something along these lines: > > Listen 127.0.0.1:8080 <http://127.0.0.1:8080> > LoadModule dav_module libexec/apache24/mod_dav.so > LoadModule dav_fs_module libexec/apache24/mod_dav_fs.so > > <VirtualHost 127.0.0.1:8080 <http://127.0.0.1:8080>> > DocumentRoot "/var/tmp/dav" > ErrorLog "/var/tmp/dav/errors.txt" > TransferLog "/var/tmp/dav/access.txt" > DavLockDB "/var/tmp/dav/DavLock" > > # If using TLS, generate these self-signed certificates too: > # SSLEngine on > # SSLCertificateFile "/var/tmp/dav/cert.pem" > # SSLCertificateKeyFile "/var/tmp/dav/key.pem" > > <Directory "/var/tmp/dav/files"> > AllowOverride none > Require valid-user > # Require all granted > > AuthType Basic > AuthName "WebDAV" > AuthBasicProvider file > AuthUserFile "/var/tmp/dav/passwords" > > Dav on > </Directory> > </VirtualHost> > > Enter your username and password with: > htpasswd -c /var/tmp/dav/passwords username > > Then in the file open dialog, enter "dav://127.0.0.1:8080/files/ > <http://127.0.0.1:8080/files/>" (or use scheme davs:// instead, if > using HTTPS). > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > For additional commands, e-mail: dev-h...@openoffice.apache.org
smime.p7s
Description: S/MIME Cryptographic Signature
