Am 27.04.22 um 17:44 schrieb Matthias Seidel:
Sounds great!
I also think that it's a great progress.
As none-developer I cannot really judge how good and valuable the
changes are. But all in all I'm happy to see this refactored to have
less problems.
Well done. :-)
Marcus
Am 26.04.22 um 19:56 schrieb Damjan Jovanovic:
On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <j...@jagunet.com
<mailto:j...@jagunet.com>> wrote:
I'm gonna look into the serf->(lib)curl option... Since we don't
use any of the fancy features of serf, I'm thinking that the easy
option might be best
Hi
I've ported our WebDAV content provider module from Serf to Curl.
While it ended well, and several other bugs were found and fixed, it
definitely wasn't the "easy option" Jim ;). Starting with conservative
changes, it ended up needing total restructuring, and became more of a
rewrite. The crashes were frequent and hung connections many, and I
had to read up on the HTTP protocol, and read Curl and Serf's source
code, but eventually I prevailed, and a clean elegant stable Curl
WebDAV module emerged.
The huge patch is attached for anyone wishing to review and test.
Unless there are major objections, I'll push it in a couple of days.
STATUS
It builds and works well on FreeBSD and Windows.
Most of the code was reused, and all the operations and semantics
previously present with Serf, should have been preserved.
Browsing WebDAV files and directories, loading files, overwriting them
("Save"), creating them ("Save As"), renaming and deleting them, all
works.
HTTP and HTTPS proxies work. Unlike Serf, Curl could also support
SOCKS proxies (with minimal further changes), but AOO lacks that
setting in the proxy UI and configuration.
Authentication works, both to the destination server and to the proxy
server. I've successfully tested Basic and Digest authentication. Curl
supports every authentication method Serf does and more.
HTTPS works, with a custom certificate verification function, using
our own certificate store from NSS and its API (like the Serf code
used). A bug was discovered (which is in the Serf implementation too)
where self-signed certificates were being unconditionally rejected;
apparently NSS wants to see that a copy of that certificate in its
certificate chain parameter too. Now they work, and the user gets
prompted to allow access.
HTTPS and authentication can be used together on the same connection
and work well, both bringing up their UI dialogs as needed.
A bug was fixed where when username and password were both present in
the URL (dav://user:pass@host/path), the code was trying to split them
at the "@" instead of ":".
Unnecessary base64 encoding and decoding was removed, when importing
the TLS connection's certificates into our XSecurityEnvironment. They
now come in directly as ASN.1 DER, and still work.
The code was greatly restructured and cleaned up, as Curl's API is
synchronous and blocking, with parameters set in advance instead of
through many callbacks, which has allowed using short clear methods,
and clean separation between the session and request classes. The
WebDAV content provider has shrunk from 35 to 21 C++ files, 43 to 29
header files, and 19129 to 15991 lines of code. With WebDAV methods
centralized and refactored into only 10-20 lines of code each, instead
of scattered across 4 files, it is much more understandable and
maintainable now.
Curl is vastly more portable than Serf. We should build easily now
even on OS/2. We can remain with existing build tools instead of
needing scons or cmake just to build Serf.
3 now unused dependencies were removed: apr, apr-util, and serf. Serf
isn't so bad. APR's pool idea is an ingenious way of doing resource
management in C. However Curl has excellent documentation, guides,
examples, and detailed explanations and even example code for each
setting, while Serf has no documentation. Serf might be worth it in a
project that already uses APR a lot, but we don't.
Instead of the historical, crippled forms of logging like OSL_TRACE(),
which don't appear in release builds, I've made it use the newer
com.sun.star.logging UNO component (wrapped in
comphelper::EventLogger), which was inspired by java.util.logging,
with configurable verbosity levels, handlers (file and console) and
output formats (plain, csv), and importantly, which produces output in
release builds too. I've also made it so that on LogLevel::FINEST,
Curl's verbose mode is enabled and Curl's debug output is also logged
through us, with descriptions of what Curl is doing, and logs of all
HTTP traffic including headers and bodies, before encryption and after
decryption in the case of HTTPS, giving us tremendous detail that can
be used for troubleshooting problems.
CURL CHANGED TO USE OPENSSL AND ZLIB
Curl only supports the custom TLS certificate verification function we
use (the CURLOPT_SSL_CTX_FUNCTION option) when built with OpenSSL,
wolfSSL or mbedTLS providers. We currently use schannel on Windows
instead, which had to be changed. I also made it use zlib, which
generally helps, and WebDAV uses XML which is very verbose and
benefits from compression. On other OSes with system curl, it is now
checked for its SSL provider, and configure fails if it isn't OpenSSL.
The new WebDAV module successfully builds and runs with both OpenSSL
1.0.2 or 1.1.1. However 1 function was renamed between those versions,
so the OpenSSL version at runtime probably has to match the one used
at compile time (although building with 1.0.2 headers might allow
running with 1.1.1 - not tested).
(After completing development and testing, it dawned on me there is a
completely different way to do the certification verification, which
should allow other SSL providers to be used and might be better in
various ways. See later.)
We currently build zlib as a static library only, and on Windows its C
runtime library is linked statically (_MT), which can't be mixed with
curl's dynamic linking of the runtime library (_MD). Thus curl was
made to link it statically too. Most if not all of our modules link
the runtime library statically too (which begs the question, why do we
ship the msvcr* redistributables to users at all then?).
ISSUES
The file open dialog (Ctrl+O) can hang for several minutes when first
connecting to a server. This is not new - it happens with Serf as
well. This appears to be caused by autocompletion in the file dialog.
When typing in a URL like "davs://127.0.0.1 <http://127.0.0.1>", the
WebDAV content provider is first called with a partial "https://1
<https://1>", before the rest of the URL is entered. That "1" is
(somehow) treated as IP address "0.0.0.1", and a TCP connection to
0.0.0.1 is started. Only after several minutes, when that connection
times out and fails, does the content provider get another request
with the complete URL, which succeeds. The distinctly unpleasant wait,
is luckily only present that the first time that server is used, as
caching remembers the URL even across AOO restarts, so the "1" is
automatically expanded to the full URL and the content provider never
sees "https://1 <https://1>" again.
You can only enter credentials for the HTTP(S) proxy or the
destination, never both, as the credential manager caches credentials
per URL, not per host/realm, so while you are prompted for credentials
for both, and Curl is told about both, the destination credentials
overwrite the proxy credentials in the cache, so one Curl request
works but future Curl requests use the wrong credentials to the proxy.
This doesn't matter, as AOO doesn't support passwords for proxy
servers anyway, and Serf has the same issue.
Damjan
--
P.S. APACHE 2.4 SETUP FOR TESTING
Put some files in /var/tmp/dav/files, then configure Apache HTTPD's
httpd.conf with something along these lines:
Listen 127.0.0.1:8080 <http://127.0.0.1:8080>
LoadModule dav_module libexec/apache24/mod_dav.so
LoadModule dav_fs_module libexec/apache24/mod_dav_fs.so
<VirtualHost 127.0.0.1:8080 <http://127.0.0.1:8080>>
DocumentRoot "/var/tmp/dav"
ErrorLog "/var/tmp/dav/errors.txt"
TransferLog "/var/tmp/dav/access.txt"
DavLockDB "/var/tmp/dav/DavLock"
# If using TLS, generate these self-signed certificates too:
# SSLEngine on
# SSLCertificateFile "/var/tmp/dav/cert.pem"
# SSLCertificateKeyFile "/var/tmp/dav/key.pem"
<Directory "/var/tmp/dav/files">
AllowOverride none
Require valid-user
# Require all granted
AuthType Basic
AuthName "WebDAV"
AuthBasicProvider file
AuthUserFile "/var/tmp/dav/passwords"
Dav on
</Directory>
</VirtualHost>
Enter your username and password with:
htpasswd -c /var/tmp/dav/passwords username
Then in the file open dialog, enter "dav://127.0.0.1:8080/files/
<http://127.0.0.1:8080/files/>" (or use scheme davs:// instead, if
using HTTPS).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org
