Bug#946359: pg-snakeoil: Selftest apears to be broken
Re: Sebastian Andrzej Siewior 2019-12-19 <20191219193659.t6qzowamewru7yfk@flow> > > One bit that could have been relevant is that I'm running on schroot > > with tmpfs on an overlay fs. > > But that part is transparent. There's one thing that doesn't work on overlayfs, renaming (underlay?) directories. We appear not to have problems with that in Debian, but yum operations fail horribly in a centos chroot. But that's not the problem here. Christoph
Processed: Re: Bug#946359: Info received (Bug#946359: pg-snakeoil: Selftest apears to be broken)
Processing commands for cont...@bugs.debian.org:
> forwarded 946359 https://bugzilla.clamav.net/show_bug.cgi?id=12440
Bug #946359 {Done: Christoph Berg } [pg-snakeoil] pg-snakeoil:
Selftest apears to be broken
Set Bug forwarded-to-address to
'https://bugzilla.clamav.net/show_bug.cgi?id=12440'.
>
End of message, stopping processing here.
Please contact me if you need assistance.
--
946359: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946359
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Bug#946359: pg-snakeoil: Selftest apears to be broken
On 2019-12-19 10:04:48 [+0100], Christoph Berg wrote: > Re: Sebastian Andrzej Siewior 2019-12-18 > <20191218225837.qttuxpwrbo5ukpr3@flow> > > > $ sudo -u clamav freshclam --verbose > > > > what happens if you strip the sudo part? One of the first thing is to > > change to the clamav user (well so is my memory and the /var/…/clamav is > > owned by clamav so…)? However after I install sudo in my chroot and try > > this it still works :/ > > Now it just works, both with "sudo freshclam --verbose" and "sudo -u > clamav freshclam --verbose": I meant without sudo but here you go. > Thu Dec 19 10:00:36 2019 -> *updatedb: Running g_cb_download_complete > callback... and now out of the sudden it is no longer outdated. > > > Time: 2.4s, ETA; 0.0s [===>] > > > 52.81MiB/52.81MiB > > > * Connection #0 to host database.clamav.net left intact > > > Wed Dec 18 11:56:13 2019 -> ^Mirror https://database.clamav.net is not > > > synchronized. > > > > So I don't have this. And for that to happen you need an out-dated > > database. And somehow you have that and the ci host. Reproducible. > > Maybe there was one bad server in the mirror list... right. And the same server is used ci.debian.net. For days. The database server sits behind cloudflare's CDN [0]. Which means something would bad with the CDN. But then it appears to work with the old freshclam while new one throws the problem. So it might be a problem somewhere else. [0] https://blog.clamav.net/2018/09/want-to-improve-your-clamav-experience.html > > If the `sudo' part makes no difference, can you stash me your chroot or > > the other way around? There must be something that is different. > > One bit that could have been relevant is that I'm running on schroot > with tmpfs on an overlay fs. But that part is transparent. I would expect a transparent proxy, dns-preload library or an odd package which somehow influeces curl/freshclam. > Christoph Sebastian
Bug#946359: pg-snakeoil: Selftest apears to be broken
Re: Sebastian Andrzej Siewior 2019-12-18 <20191218225837.qttuxpwrbo5ukpr3@flow> > > $ sudo -u clamav freshclam --verbose > > what happens if you strip the sudo part? One of the first thing is to > change to the clamav user (well so is my memory and the /var/…/clamav is > owned by clamav so…)? However after I install sudo in my chroot and try > this it still works :/ Now it just works, both with "sudo freshclam --verbose" and "sudo -u clamav freshclam --verbose": $ sudo freshclam --verbose Thu Dec 19 10:00:32 2019 -> ClamAV update process started at Thu Dec 19 10:00:32 2019 Thu Dec 19 10:00:32 2019 -> *Current working dir is /var/lib/clamav/ Thu Dec 19 10:00:32 2019 -> *Querying current.cvd.clamav.net Thu Dec 19 10:00:32 2019 -> *TTL: 539 Thu Dec 19 10:00:32 2019 -> *fc_dns_query_update_info: Software version from DNS: 0.102.1 Thu Dec 19 10:00:32 2019 -> *Current working dir is /var/lib/clamav/ Thu Dec 19 10:00:32 2019 -> *check_for_new_database_version: No local copy of "daily" database. Thu Dec 19 10:00:32 2019 -> *query_remote_database_version: daily.cvd version from DNS: 25667 Thu Dec 19 10:00:32 2019 -> daily database available for download (remote version: 25667) Thu Dec 19 10:00:32 2019 -> *Retrieving https://database.clamav.net/daily.cvd Thu Dec 19 10:00:32 2019 -> *downloadFile: Download source: https://database.clamav.net/daily.cvd Thu Dec 19 10:00:32 2019 -> *downloadFile: Download destination: /var/lib/clamav/tmp/clamav-a0eebaf13c63bb204c5e5a77e26f717c.tmp * Trying 2606:4700::6810:db54:443... * TCP_NODELAY set * Connected to database.clamav.net (2606:4700::6810:db54) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use h2 * Server certificate: * subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl392509.cloudflaressl.com * start date: Aug 24 00:00:00 2019 GMT * expire date: Mar 1 23:59:59 2020 GMT * subjectAltName: host "database.clamav.net" matched cert's "*.clamav.net" * issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO ECC Domain Validation Secure Server CA 2 * SSL certificate verify ok. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x55c822a6d790) > GET /daily.cvd HTTP/2 Host: database.clamav.net user-agent: ClamAV/0.102.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) accept: */* connection: close * old SSL session ID is stale, removing * Connection state changed (MAX_CONCURRENT_STREAMS == 256)! < HTTP/2 200 < date: Thu, 19 Dec 2019 09:00:33 GMT < content-type: application/octet-stream < content-length: 55429776 < set-cookie: __cfduid=d808352f8029efc872822e310079600b81576746033; expires=Sat, 18-Jan-20 09:00:33 GMT; path=/; domain=.clamav.net; HttpOnly; SameSite=Lax < last-modified: Wed, 18 Dec 2019 09:53:00 GMT < etag: "5df9f6fc-34dca90" < expires: Thu, 19 Dec 2019 13:00:33 GMT < cache-control: public, max-age=14400 < cf-cache-status: HIT < age: 3383 < accept-ranges: bytes < strict-transport-security: max-age=15552000 < x-content-type-options: nosniff < expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct; < server: cloudflare < cf-ray: 54782fd3fa8ed45b-HAM < Time: 3.5s, ETA; 0.0s [===>] 52.86MiB/52.86MiB * Connection #0 to host database.clamav.net left intact Thu Dec 19 10:00:36 2019 -> *updatedb: Running g_cb_download_complete callback... Thu Dec 19 10:00:36 2019 -> *download_complete_callback: Download complete for database : /var/lib/clamav/tmp/clamav-a0eebaf13c63bb204c5e5a77e26f717c.tmp-daily.cvd Thu Dec 19 10:00:36 2019 -> *download_complete_callback: fc_context->bTestDatabases : 1 Thu Dec 19 10:00:36 2019 -> *download_complete_callback: fc_context->bBytecodeEnabled : 1 Thu Dec 19 10:00:36 2019 -> Testing database: '/var/lib/clamav/tmp/clamav-a0eebaf13c63bb204c5e5a77e26f717c.tmp-daily.cvd' ... Thu Dec 19 10:00:36 2019 -> *Loading signatures from /var/lib/clamav/tmp/clamav-a0eebaf13c63bb204c5e5a77e26f717c.tmp-daily.cvd Thu Dec 19 10:00:40 2019 -> *Properly loaded 2061162 signatures from /var/lib/clamav/tmp/clamav-a0eebaf13c63bb204c5e5a77e26f717c.tmp-daily.cvd Thu Dec 19 10:00:41 2019 -> Database test passed. Thu Dec 19 10:00:41 2019 -> daily.cvd updated (version: 25667, sigs: 2061162, f-level: 63, builder: raynman) Thu Dec 19 10:00:41 2019 -> *fc_update_database: daily.cvd updated. Thu Dec 19 10:00:41 2019 -> *Current working dir is /var/lib/clamav/ Thu Dec 19 10:00:41 2019 -> *check_for_new_database_version: No local copy of "main" database. Thu Dec 19 10:00:41 2019 -> *query_remote_database_version: main.cvd version from DNS: 59 Thu Dec 19 10:00:41 2019 -> main database
Bug#946359: pg-snakeoil: Selftest apears to be broken
On 2019-12-18 11:59:50 [+0100], Christoph Berg wrote: > Nothing special, and the test started failing on ci.debian.net as well > as in my local sid chroot. Yes and this is absolute mystery to me. My up-to-date sid chroot I use in schroot for sbuild does not show this problem. If I install freshclam only without recommends I can't download anything because ca-certificate is missing. With this packages installed it works then… > > Could you start `freshclam' by hand with --verbose (not sure if --debug > > works) and provide more output? It appears that the version downloaded > > is one less than available and that is where things go south. > > In the sid chroot, with an empty /var/lib/clamav/: > > $ sudo -u clamav freshclam --verbose what happens if you strip the sudo part? One of the first thing is to change to the clamav user (well so is my memory and the /var/…/clamav is owned by clamav so…)? However after I install sudo in my chroot and try this it still works :/ > Time: 2.4s, ETA; 0.0s [===>] > 52.81MiB/52.81MiB > * Connection #0 to host database.clamav.net left intact > Wed Dec 18 11:56:13 2019 -> ^Mirror https://database.clamav.net is not > synchronized. So I don't have this. And for that to happen you need an out-dated database. And somehow you have that and the ci host. Reproducible. If the `sudo' part makes no difference, can you stash me your chroot or the other way around? There must be something that is different. > Christoph Sebastian
Bug#946359: pg-snakeoil: Selftest apears to be broken
Re: Sebastian Andrzej Siewior 2019-12-11 <20191211141451.tn2u64ssgarpgz25@flow> > > The test fails in my sid chroot as well because freshclam can't > > download the database, /var/lib/clamav/ is empty except for a "tmp" > > directory. > > Do you have a special inet setup? Kind of web proxy or something like > that. Nothing special, and the test started failing on ci.debian.net as well as in my local sid chroot. > Could you start `freshclam' by hand with --verbose (not sure if --debug > works) and provide more output? It appears that the version downloaded > is one less than available and that is where things go south. In the sid chroot, with an empty /var/lib/clamav/: $ sudo -u clamav freshclam --verbose Wed Dec 18 11:56:09 2019 -> ClamAV update process started at Wed Dec 18 11:56:09 2019 Wed Dec 18 11:56:09 2019 -> *Current working dir is /var/lib/clamav/ Wed Dec 18 11:56:09 2019 -> *Querying current.cvd.clamav.net Wed Dec 18 11:56:09 2019 -> *TTL: 503 Wed Dec 18 11:56:09 2019 -> *fc_dns_query_update_info: Software version from DNS: 0.102.1 Wed Dec 18 11:56:09 2019 -> *Current working dir is /var/lib/clamav/ Wed Dec 18 11:56:09 2019 -> *check_for_new_database_version: No local copy of "daily" database. Wed Dec 18 11:56:09 2019 -> *query_remote_database_version: daily.cvd version from DNS: 25667 Wed Dec 18 11:56:09 2019 -> daily database available for download (remote version: 25667) Wed Dec 18 11:56:09 2019 -> *Retrieving https://database.clamav.net/daily.cvd Wed Dec 18 11:56:09 2019 -> *downloadFile: Download source: https://database.clamav.net/daily.cvd Wed Dec 18 11:56:09 2019 -> *downloadFile: Download destination: /var/lib/clamav/tmp/clamav-88ed61b7591f35acdee87b5b900326e2.tmp * Trying 2606:4700::6810:db54:443... * TCP_NODELAY set * Connected to database.clamav.net (2606:4700::6810:db54) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use h2 * Server certificate: * subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl392509.cloudflaressl.com * start date: Aug 24 00:00:00 2019 GMT * expire date: Mar 1 23:59:59 2020 GMT * subjectAltName: host "database.clamav.net" matched cert's "*.clamav.net" * issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO ECC Domain Validation Secure Server CA 2 * SSL certificate verify ok. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x561ac8280980) > GET /daily.cvd HTTP/2 Host: database.clamav.net user-agent: ClamAV/0.102.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) accept: */* connection: close * old SSL session ID is stale, removing * Connection state changed (MAX_CONCURRENT_STREAMS == 256)! < HTTP/2 200 < date: Wed, 18 Dec 2019 10:56:10 GMT < content-type: application/octet-stream < content-length: 55374037 < set-cookie: __cfduid=da44da730a0bfc7a34a8990f54f1610a4157570; expires=Fri, 17-Jan-20 10:56:10 GMT; path=/; domain=.clamav.net; HttpOnly; SameSite=Lax < last-modified: Tue, 17 Dec 2019 09:54:00 GMT < etag: "5df8a5b8-34cf0d5" < expires: Wed, 18 Dec 2019 14:56:10 GMT < cache-control: public, max-age=14400 < cf-cache-status: HIT < age: 10753 < accept-ranges: bytes < strict-transport-security: max-age=15552000 < x-content-type-options: nosniff < expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct; < server: cloudflare < cf-ray: 54709bcece3a40ce-HAM < Time: 2.4s, ETA; 0.0s [===>] 52.81MiB/52.81MiB * Connection #0 to host database.clamav.net left intact Wed Dec 18 11:56:13 2019 -> ^Mirror https://database.clamav.net is not synchronized. Wed Dec 18 11:56:13 2019 -> !Unexpected error when attempting to update database: daily Wed Dec 18 11:56:13 2019 -> ^fc_update_databases: fc_update_database failed: Up-to-date (1) Wed Dec 18 11:56:13 2019 -> !Database update process failed: Up-to-date (1) Wed Dec 18 11:56:13 2019 -> !Update failed. > > Using a smaller database instead of downloading the whole thing for > > each test run makes sense. We implemented that now, the pg_snakeoil 1.3 testsuite will now look for the "The Quick Brown Fox" virus: https://github.com/credativ/pg_snakeoil/tree/master/testfiles Thanks for the tip! Christoph
