Re: [clamav-users] New Main & Daily CVD's are incoming

[Joel Esler (jesler) via clamav-users]

Technically what we do is publish a zero byte cdiff.  This makes freshclam 
force update and grab the whole cvd.  Then, from that point on, the new daily 
cvd will be much smaller, and updates should apply faster. Ultimately saving on 
bandwidth as the daily.cvd will be much smaller.  Again, like I just said in my 
other email, if you’re not using 0.103.3, you should start that upgrade engine.

— 
Sent from my  iPad

> On Jul 13, 2021, at 19:27, Mark Allan  wrote:
> 
>  According to the man page (and freshclam.conf) "ScriptedUpdates" is what 
> ClamAV calls the mechanism for performing daily incremental updates via cdiff 
> files rather than downloading the whole cvd.
> 
> Are you providing cdiff files for both main.cvd and daily.cvd or just the cvd 
> files?
> 
> Regards
> Mark
> 
>> On 13 Jul 2021, at 3:55 pm, Joel Esler (jesler)  wrote:
>> 
>> I am not sure what you mean by “scripted updates”?  If you are using 
>> FreshClam or cvdupdate, your downloads should happen fine.
>> 
 On Jul 13, 2021, at 10:29 AM, Mark Allan via clamav-users 
  wrote:
 
 Hi Joel,
 
 Will you be posting scripted updates for main.cvd and daily.cvd or just 
 the new cvd files in their entirety? I seem to remember processing the 
 cdiff files caused a lot of problems for people the last time main.cvd was 
 updated.
 
 Mark
 
 On 13 Jul 2021, at 3:05 pm, Joel Esler (jesler) via clamav-users 
  wrote:
 
 Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
 main.cvd and daily.cvd, as we do periodically to move more of the long 
 term signatures into the main.cvd and make the daily.cvd smaller again.  
 
 This will have an impact on your downloads of these files (as every ClamAV 
 instance will have to re-download both files), so you may see a spike in 
 your bandwidth usage.
 
 We will monitor the situation on the mirror side and make any adjustments 
 necessary, but we anticipate no issues.
 
 https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html
 
> 


smime.p7s
Description: S/MIME cryptographic signature

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] New Main & Daily CVD's are incoming

[Joel Esler (jesler) via clamav-users]

> On Jul 13, 2021, at 18:08, Paul Kosinski via clamav-users 
>  wrote:
> 
> On Tue, 13 Jul 2021 14:05:53 +
> "Joel Esler \(jesler\) via clamav-users"  
> wrote:
> 
>> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
>> main.cvd and daily.cvd, as we do periodically to move more of the long term 
>> signatures into the main.cvd and make the daily.cvd smaller again.  
>> 
>> This will have an impact on your downloads of these files (as every ClamAV 
>> instance will have to re-download both files), so you may see a spike in 
>> your bandwidth usage.
>> 
>> We will monitor the situation on the mirror side and make any adjustments 
>> necessary, but we anticipate no issues.
>> 
>> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
>> 
> 
> I wondered when (and if) you would be able to distribute a new main.cvd, 
> given your concerns about Cloudflare bandwidth usage. I assume this means 
> that there is (almost) no one still downloading ClamAV updates every second 
> or so.

Oh there are, but at this point most of them have been blocked outright, and 
then they file a ticket and apologize for doing it, or they are rate limited. 

We are also working with Cloudflare to enact some more specific rate limits 
(we’ve been working with them on the development of the feature) that will 
alleviate a lot of the problems we are having with any newer versions, and then 
slowly we are EOL’ing older versions of ClamAV.  The more people upgrade to 
103.2 or 103.3 (newest) the better the ecosystem will be.  Slowly over the next 
year or so, the ecosystem will normalize and our bandwidth usage will be 
extremely efficient.  

> I also presume that my IP address won't set off any alarms tomorrow by 
> downloading 3 complete copies of main.cvd and daily.cvd (since I gave up 
> trying to run my own mirror many months ago).
> 
I will have to make the rate limits bigger tomorrow for the main and daily




smime.p7s
Description: S/MIME cryptographic signature

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] running freshclam and 3rd party/clamav-unofficial-sigs.sh owner name changes occasionally

[G.W. Haywood via clamav-users]

Hi there,

On Tue, 13 Jul 2021, Robert Kudyba wrote:


... daily.cld was updated, presumably by freshclam.  That's good, as
nothing seems to have broken.  Can you confirm that happened from the
freshclam log?


here are the logs from 10:01 AM Jul 13:
Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, 
sigs: 3995778, f-level: 63, builder: raynman)
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, 
sigs: 3995778, f-level: 63, builder: raynman)
...
ps -auwx|grep freshclam
clamav  3818  0.0  0.0  28952 12864 ?Ss   12:00   0:00 
/usr/bin/freshclam -d --foreground=true


The logs contain a lot of duplicated lines.  Maybe you have both a
line like

StandardOutput=syslog

in your freshclam.service and *also* a line like

LogSyslog yes

in your freshclam.conf (or whatever passes for freshclam.conf in these
screwy RedHat systems).  Well, you want one or the other but not both.
I'd suggest commenting out the "LogSyslog yes" line and restarting the
freshclam daemon.


Are you sure that the system time gets set correctly at boot?  We
need to know that we can rely on the timestamps in the logs. ...


...
Jul 13 12:00:50 ourserver.edu systemd[1]: Starting NTP client/server...
Jul 13 12:01:34 ourserver.edu chronyd[3232]: Selected source 50.205.57.38 
(2.fedora.pool.ntp.org)


Looks good.


Anyway, suddenly the owner/group IDs have changed and you have both a
daily.cld and a daily.cvd - which isn't good news, especially as one
of them is over three weeks old.  Where did it come from?



Right, that's the question.


Looks like it was either the update or the reboot.  An easy way to
find out would be to just reboot.


Assuming that we can believe the timestamps, then any problems that
arose from ownership by the clamupdate user/group had already happened
at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
which caused them.

Is this the first time that clamav-unofficial-sigs.sh ran?


No it's been running all the time.


I think we're confusing each other.  The clamav-unofficial-sigs.sh
script doesn't run like a daemon runs.  The script is started by
something like a cron entry; it updates the configured databases if
needed, then stops.  The unofficial update script only updates (or
should only update) the third-party signature database files, that is
everything except 'main', 'daily' and 'bytecode'.  I meant was it at
12:14 that the clamav-unofficial-sigs.sh script ran?  Presumably it's
logging its activities somewhere, do you have that log?  The log
location will be set in the configuration for the unofficial script.


So are freshclam and clamav-unofficial-sigs.sh not supposed to run
as separate processes?


They are completely separate, they know nothing about each other and it
would be bad if they both tried to update the same files.  The ClamaV
team provide freshclam as part of the ClamAV 'official' distribution.
The clamav-unofficial-sigs.sh is optional, and is provided separately.
There are completely separate configuration files for both utilities.

I'm beginning to wonder if this is all down to installation of the
unofficial update script with a configuration which assumes that the
user:group should be the same user:group as the clamd daemon and not
the same user:group as the freshclam daemon.  The clamd daemon only
needs to read the dababase files but freshclam needs to write them.
I can sort of imagine somebody at RedHat thinking they'd make the
update process and the scanning processes use different UIDs & GIDs
for extra security, but it's not a lot of extra security, and it's
just asking for this kind of trouble when somebody installs another
utility which updates files in the same directory.


What's in the freshclam log about these times?


Nothing as the upgrade/reboot was still happening. The next freshclam is:
Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started at Tue 
Jul 13 14:00:58 2021
Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started at Tue 
Jul 13 14:00:58 2021
Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Can't create temporary 
directory /var/lib/clamav/tmp.21024dac47
Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory must be 
writable for UID 985 or GID 981
Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Update failed.
Jul 13 14:00:58 ourserver freshclam[3818]: Can't create temporary directory 
/var/lib/clamav/tmp.21024dac47
Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory must be 
writable for UID 985 or GID 981
Jul 13 14:00:58 ourserver freshclam[3818]: Update failed.
Jul 13 14:00:58 ourserver freshclam[3818]:


The "Hint:" part tells you what the problem is.  Something changed the
permissions on the directory 

Re: [clamav-users] New Main & Daily CVD's are incoming

[Mark Allan via clamav-users]

According to the man page (and freshclam.conf) "ScriptedUpdates" is what ClamAV 
calls the mechanism for performing daily incremental updates via cdiff files 
rather than downloading the whole cvd.

Are you providing cdiff files for both main.cvd and daily.cvd or just the cvd 
files?

Regards
Mark

> On 13 Jul 2021, at 3:55 pm, Joel Esler (jesler)  wrote:
> 
> I am not sure what you mean by “scripted updates”?  If you are using 
> FreshClam or cvdupdate, your downloads should happen fine.
> 
>> On Jul 13, 2021, at 10:29 AM, Mark Allan via clamav-users 
>> mailto:clamav-users@lists.clamav.net>> wrote:
>> 
>> Hi Joel,
>> 
>> Will you be posting scripted updates for main.cvd and daily.cvd or just the 
>> new cvd files in their entirety? I seem to remember processing the cdiff 
>> files caused a lot of problems for people the last time main.cvd was updated.
>> 
>> Mark
>> 
>>> On 13 Jul 2021, at 3:05 pm, Joel Esler (jesler) via clamav-users 
>>> mailto:clamav-users@lists.clamav.net>> 
>>> wrote:
>>> 
>>> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
>>> main.cvd and daily.cvd, as we do periodically to move more of the long term 
>>> signatures into the main.cvd and make the daily.cvd smaller again.  
>>> 
>>> This will have an impact on your downloads of these files (as every ClamAV 
>>> instance will have to re-download both files), so you may see a spike in 
>>> your bandwidth usage.
>>> 
>>> We will monitor the situation on the mirror side and make any adjustments 
>>> necessary, but we anticipate no issues.
>>> 
>>> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
>>> 
>>> 


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] New Main & Daily CVD's are incoming

[Paul Kosinski via clamav-users]

On Tue, 13 Jul 2021 14:05:53 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
> main.cvd and daily.cvd, as we do periodically to move more of the long term 
> signatures into the main.cvd and make the daily.cvd smaller again.  
> 
> This will have an impact on your downloads of these files (as every ClamAV 
> instance will have to re-download both files), so you may see a spike in your 
> bandwidth usage.
> 
> We will monitor the situation on the mirror side and make any adjustments 
> necessary, but we anticipate no issues.
> 
> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
> 




I wondered when (and if) you would be able to distribute a new main.cvd, given 
your concerns about Cloudflare bandwidth usage. I assume this means that there 
is (almost) no one still downloading ClamAV updates every second or so.

I also presume that my IP address won't set off any alarms tomorrow by 
downloading 3 complete copies of main.cvd and daily.cvd (since I gave up trying 
to run my own mirror many months ago).

P.S. When I was wondering about when a new 'main' might be released, it 
occurred to me that perhaps the CDIFF mechanism could be used to gradually 
remove permanent signatures from 'daily' and add the same ones to 'main', 
thereby significantly reducing the peak bandwidth load. (Of course this still 
adds to the long term bandwidth consumption -- unless the CDIFF mechanism has a 
"move signature" option added.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] running freshclam and 3rd party/clamav-unofficial-sigs.sh owner name changes occasionally

[Robert Kudyba]

> -rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd
>
> and a bunch of others which we're not concerned with.  Firstly, you
> really don't want both a bytecode.cld *and* a bytecode.cvd, so you
> should probably just delete the older one.


Done.


> Here's what happens just after 10AM on the 13th:
>
> Tue Jul 13 10:01:01 AM EDT 2021
> -rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd
> Tue Jul 13 10:02:01 AM EDT 2021
> -rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd
>
> So daily.cld was updated, presumably by freshclam.  That's good, as
> nothing seems to have broken.  Can you confirm that happened from the
> freshclam log?


here are the logs from 10:01 AM Jul 13:
Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
26230, sigs: 3995778, f-level: 63, builder: raynman)
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
26230, sigs: 3995778, f-level: 63, builder: raynman)
Jul 13 10:01:02 storm freshclam[3930506]: main.cvd database is up-to-date
(version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Jul 13 10:01:02 storm freshclam[3930506]: main.cvd database is up-to-date
(version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Jul 13 10:01:02 storm freshclam[3930506]: bytecode.cvd database is
up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Jul 13 10:01:02 storm freshclam[3930506]: bytecode.cvd database is
up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.hdb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.hdb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.ign2 is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.ign2 is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: javascript.ndb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: javascript.ndb is up-to-date
(version: custom database)
Jul 13 10:01:10 storm freshclam[3930506]: Testing database:
'/var/lib/clamav/tmp.f9e1fecbc3/clamav-7b04ccc60e7adc16d356b3b689db8e0f.tmp-spam_marketing.ndb'
...
Jul 13 10:01:10  ourserver   freshclam[3930506]: Testing database:
'/var/lib/clamav/tmp.f9e1fecbc3/clamav-7b04ccc60e7adc16d356b3b689db8e0f.tmp-spam_marketing.ndb'
...
Jul 13 10:01:10 ourserver freshclam[3930506]: Database test passed.
Jul 13 10:01:10  ourserver   freshclam[3930506]: Database test passed.
Jul 13 10:01:10  ourserver   freshclam[3930506]: spam_marketing.ndb updated
(version: custom database, sigs: 31016)
Jul 13 10:01:10  ourserver   freshclam[3930506]: spam_marketing.ndb updated
(version: custom database, sigs: 31016)
Jul 13 10:01:10  ourserver   freshclam[3930506]: securiteinfohtml.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10  ourserver   freshclam[3930506]: securiteinfohtml.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10  ourserver   freshclam[3930506]: securiteinfoascii.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10  ourserver   freshclam[3930506]: securiteinfoascii.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfoandroid.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfoandroid.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfoold.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfoold.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfopdf.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: securiteinfopdf.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: safebrowsing.gdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]: safebrowsing.gdb is
up-to-date (version: custom database)
Jul 13 10:01:11  ourserver   freshclam[3930506]:
--


> Is freshclam running from cron or as a daemon?
>

Daemon
ps -auwx|grep freshclam
clamav  3818  0.0  0.0  28952 12864 ?Ss   12:00   0:00

Re: [clamav-users] running freshclam and 3rd party/clamav-unofficial-sigs.sh owner name changes occasionally

[G.W. Haywood via clamav-users]

Hello again,

On Tue, 13 Jul 2021, Robert Kudyba wrote:


After an upgrade of Fedora and subsequent reboot the permission problem
returned. Same the files:
-rw-r--r-- 1 clamupdate clamupdate293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25  2019 main.cvd

as well as the directory:
ls -dl /var/lib/clamav
drwxr-xr-x 4 clamupdate clamupdate 8192 Jul 13 11:39 /var/lib/clamav

Also in the clamav-unofficial-sigs.log file
Jul 13 12:14:01 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav

Permission log file is available at
https://storm.cis.fordham.edu/~rkudyba/clam_perms.log


Now we're gettting somewhere. :)

The log starts with

Mon Jul 12 09:59:01 AM EDT 2021

and the first timestamp for daily.cld is

-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld

It is perhaps a little unfortunate that the log starts at the exact
time of the last modification of daily.cld - we might need to come
back to that but I hope not.  Also there are three timestamps where
I'd expect only one so I suspect something is a little bit squiffy in
the crontab, but that probably doesn't matter.
In the database directory at 09:59 you have the four files

-rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd

and a bunch of others which we're not concerned with.  Firstly, you
really don't want both a bytecode.cld *and* a bytecode.cvd, so you
should probably just delete the older one.  To cut down on the amount
of text I used this shell command to view the log:

$ grep '\(bytecode\|main\.\|daily\|clamupdate\|\(Mon\|Tue\) Jul 1\)' 
clam_perms.log  | less

Then I just searched for interesting things (I've had a lot of
practice at trawling through logs...)

Here's what happens just after 10AM on the 13th:

Tue Jul 13 10:01:01 AM EDT 2021
-rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd
Tue Jul 13 10:02:01 AM EDT 2021
-rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd

So daily.cld was updated, presumably by freshclam.  That's good, as
nothing seems to have broken.  Can you confirm that happened from the
freshclam log?  Is freshclam running from cron or as a daemon?

--

The next thing that I see of interest is

Tue Jul 13 11:10:02 AM EDT 2021
-rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25  2019 main.cvd
Tue Jul 13 12:02:01 PM EDT 2021
-rw-r--r-- 1 clamav clamav   1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamupdate clamupdate293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25  2019 main.cvd

There's a fifty minute gap in the log.  Why is that?  Presumably this
is about the time you updated and rebooted the system.  Are you sure
that the system time gets set correctly at boot?  We need to know that
we can rely on the timestamps in the logs.  All the logs.

Anyway, suddenly the owner/group IDs have changed and you have both a
daily.cld and a daily.cvd - which isn't good news, especially as one
of them is over three weeks old.  Where did it come from?


From the cron log file:
Jul 13 12:14:01 ourserver CROND[22349]: (clamav) CMD ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)
Jul 13 12:14:03  ourserver CROND[22318]: (clamav) CMDEND ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)


Assuming that we can believe the timestamps, then any problems that
arose from ownership by the clamupdate user/group had already happened
at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
which caused them.

Is this the first time that clamav-unofficial-sigs.sh ran?

What's in the freshclam log about these times?

--

73,
Ged.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

Re: [clamav-users] Qnap TS-259Pro+

[Andrew C Aitchison via clamav-users]

The TS-259Pro+ appears go have 1GB RAM, which is not really enough
to run clamav, so compiling from source is unlikely to be helpful.

On Tue, 13 Jul 2021, Eero Volotinen wrote:


You probably need to buy newer version of qnap nas
or compile clamav from sources.

Eero

On Tue 13. Jul 2021 at 19.41, Raymond Ng via clamav-users <
clamav-users@lists.clamav.net> wrote:

> My Qnap NAS suddenly stop updating Virus signature since March.
> It had a manual update but I can’t find where to download the latest
> signature file at Clamav home page.
> Kindly help to direct where I could download the latest signature so I
> could manual update the signature.
> Ive check on the Qnap Community site that there is a new version of
> Clamav
> but it’s not compatible with my model.
> 
> Regards

> *Raymond Ng*
> Sent from my iPhone
> 
> ___
> 
> clamav-users mailing list

> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> 
> Help us build a comprehensive ClamAV guide:

> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml
> 



--
Andrew C. Aitchison Kendal, UK
and...@aitchison.me.uk

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Qnap TS-259Pro+

[G.W. Haywood via clamav-users]

Hi there,

On Wed, 14 Jul 2021, Raymond Ng via clamav-users wrote:


My Qnap NAS suddenly stop updating Virus signature since March.
It had a manual update but I can’t find where to download the latest signature 
file at Clamav home page.
Kindly help to direct where I could download the latest signature so I could 
manual update the signature.
Ive check on the Qnap Community site that there is a new version of Clamav but 
it’s not compatible with my model.


Please search the archives of this mailing list for discussions of the
QNAP devices.  There are certainly some posts around early March 2021,
but do use a search tool to find more.

--

73,
Ged.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] running freshclam and 3rd party/clamav-unofficial-sigs.sh owner name changes occasionally

[Robert Kudyba]

After an upgrade of Fedora and subsequent reboot the permission problem
returned. Same the files:
-rw-r--r-- 1 clamupdate clamupdate293670 Apr  8 06:32 bytecode.cvd
-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25  2019 main.cvd

as well as the directory:
ls -dl /var/lib/clamav
drwxr-xr-x 4 clamupdate clamupdate 8192 Jul 13 11:39 /var/lib/clamav

Also in the clamav-unofficial-sigs.log file
Jul 13 12:14:01 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav

Permission log file is available at
https://storm.cis.fordham.edu/~rkudyba/clam_perms.log

>From the cron log file:
Jul 13 12:14:01 ourserver CROND[22349]: (clamav) CMD ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)
Jul 13 12:14:03  ourserver CROND[22318]: (clamav) CMDEND ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)

On Mon, Jul 12, 2021 at 12:31 PM Robert Kudyba  wrote:

>
>>
>> > grep clam /etc/passwd
>> > clamilt:x:989:985:Clamav Milter
>> User:/var/run/clamav-milter:/sbin/nologin
>> > clamav:x:985:981::/var/run/clamav:/sbin/nologin
>> > clamupdate:x:983:979:Clamav database update
>> user:/var/lib/clamav:/sbin/nologin
>> > clamscan:x:982:978:Clamav scanner user:/:/sbin/nologin
>>
>> Interesting.  The 'clamav' user seems not to have been created by the
>> same setup process which created the other three, since it didn't get
>> a text description.  There's a suspicious gap in the numeric IDs from
>> 985:981 to 989:985 like the milter IDs were added later.  Make sense?
>>
>> What does
>>
>> grep clam /etc/group
>>
>> give you?
>>
> grep clam /etc/group
> clamilt:x:985:clamav,clamscan
> clamav:x:981:clamscan,clamilt,clamupdate
> clamupdate:x:979:clamav
> clamscan:x:978:clamilt,clamav
> virusgroup:x:949:clamupdate,clamscan,clamilt
>
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] clamscan: permission denied on many files being used by another process

[G.W. Haywood via clamav-users]

Hi there,

On Tue, 13 Jul 2021, Michael Wang wrote:


My question is how I can let clamscan to read a file, as I have
shown that even I cannot "more" a file used by another process as
administrator.


As I explained in my earler reply to you:


It's up to you to arrange for the scanner to have permission to do
what you want it to do.


Obviously ClamAV can't protect you from a malicious program in a file
if you run the program in the file before you scan it with ClamAV.  It
is very likely that the first thing that a malicious program will do
will be to seek out anti-virus software and either disable it or make
it appear to give a clean bill of health to the malicious program.

If you cannot scan a file with ClamAV because another process is using
it then it is already too late to scan it.  You have simply failed to
use ClamAV in the way in which it is designed.

Your operating system has its own ideas about security.  The little I
know about Windows makes me wonder if the main idea isn't to bamboozle
the average user by making things incredibly complicated, but whatever
it does either you have to work with it or you have to work around it.
I would recommend working with it, because working around it will lead
to many problems.  You have to learn about the systems and tools that
you're using to be able to get the best out of them; you need to learn
(1) what ClamAV is designed to do, and also (because you seem to have
some ideas about that which aren't what the rest of us have) what it
is *not* designed to do; and (2) how to arrange for ClamAV to be able
to do what it is intended to do on your system.  If it won't do what
you want it to do because it was never designed to do that in the
first place, then there's really no point grumbling about it.


If clamscan cannot scan a file used by another process, then I question the
usefulness of the software because a hacker can just install a virus file
and use it, clamscan will not be able to detect it.


You are right to question the usefulness of any security tool, but not
for the reasons which you give, which make no sense.  If a hacker can
install a "virus file" then you have already lost the battle, because
he can presumably also compromise ClamAV itself.  And if ClamAV were
to attempt to defeat the security features of the operating system, I
should consider it to be a security threat.  Having gained a toe-hold
your hacker could use it to fully compromise the system.

To get back to the basics:

ClamAV is a suite of tools for you to use.

ClamAV looks for threats in files and data streams.

The ClamAV team provides ClamAV, and what we call a signature database
which is just a bunch of files which contain descriptions of threats.
The descriptions are given in a variety of forms, with which you need
to become familiar.

There are third parties who add many more signatures to the database.
You don't need to worry about them yet but bear it in mind for later.

You can add your own signatures, and also Yara rules.  Later.

You provide the files and the data streams for ClamAV to scan.

It's as simple as that.

--

73,
Ged.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Qnap TS-259Pro+

[Eero Volotinen]

You probably need to buy newer version of qnap nas
or compile clamav from sources.

Eero

On Tue 13. Jul 2021 at 19.41, Raymond Ng via clamav-users <
clamav-users@lists.clamav.net> wrote:

> My Qnap NAS suddenly stop updating Virus signature since March.
> It had a manual update but I can’t find where to download the latest
> signature file at Clamav home page.
> Kindly help to direct where I could download the latest signature so I
> could manual update the signature.
> Ive check on the Qnap Community site that there is a new version of Clamav
> but it’s not compatible with my model.
>
> Regards
> *Raymond Ng*
> Sent from my iPhone
>
> ___
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] clamscan: permission denied on many files being used by another process

[Kris Deugau]

Michael Wang wrote:
I understand "more" is not clamscan, I was just showing that the file in 
question cannot be opened with clamscan nor with "more" as 
administrator. I also understand if clamscan cannot read a file, it 
cannot scan it. My question is how I can let clamscan to read a file, as 
I have shown that even I cannot "more" a file used by another process as 
administrator.


Welcome to Windows.  If a file is open by some process, it fundamentally 
cannot be opened by any other process (possibly depending on the first 
process' open mode), *by definition*.  This is a very low-level 
restriction imposed by the Windows filesystem API.


Conventional antivirus scanners get around this by a) hooking into 
Windows' filesystem API (~~"scan-on-access", which IIRC Clam doesn't 
support - at least not well - on Windows) or b) scanning the memory 
space of the offending process (ClamAV doesn't scan memory chunks).


-kgd

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Qnap TS-259Pro+

[Raymond Ng via clamav-users]

My Qnap NAS suddenly stop updating Virus signature since March. 
It had a manual update but I can’t find where to download the latest signature 
file at Clamav home page. 
Kindly help to direct where I could download the latest signature so I could 
manual update the signature. 
Ive check on the Qnap Community site that there is a new version of Clamav but 
it’s not compatible with my model. 

Regards 
Raymond Ng
Sent from my iPhone
___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] clamscan: permission denied on many files being used by another process

[Michael Wang]

The version I am running is clamav-0.103.3-win-x64-portable.zip

from https://www.clamav.net/downloads#otherversions . The advantage of
using the portable version is that you do not need to install, but just to
use the software from the network path.

I understand "more" is not clamscan, I was just showing that the file in
question cannot be opened with clamscan nor with "more" as administrator. I
also understand if clamscan cannot read a file, it cannot scan it. My
question is how I can let clamscan to read a file, as I have shown that
even I cannot "more" a file used by another process as administrator.

If clamscan cannot scan a file used by another process, then I question the
usefulness of the software because a hacker can just install a virus file
and use it, clamscan will not be able to detect it.

On Mon, Jul 12, 2021 at 11:45 AM G.W. Haywood via clamav-users <
clamav-users@lists.clamav.net> wrote:

> Hi there,
>
> On Mon, 12 Jul 2021, Michael Wang via clamav-users wrote:
>
> > I run ClamAV on windows using the latest portable installation with all
> > default configuration.
>
> What version of ClamAV, and where did it come from?
>
> > I run the task scheduler under the SYSTEM user with the highest
> > credentials checked, but I still have lots of permission denied
> > messages.
>
> That's to be expected if the scanning process can't read the data.
>
> > I logged in locally and checked one of the files under a powershell
> window
> > as *ADMINISTRATOR*, and I got:
> >
> > *PS C:\Users\j.doe\AppData\local\Microsoft\Windows\WebCache> more
> .\V01.log*
> > *Get-Content : The process cannot access the file
> > 'C:\Users\j.doe\AppData\local\Microsoft\Windows\WebCache\V01.log' because
> > it is being used by another process.*
>
> The 'more' command is a pager, not a scanner.  In what you've posted I
> see no evidence of a ClamAV process doing (or failing to do) anything.
>
> > So do I have to live with it? If there is a virus file and this file is
> > being currently used, clamscan cannot detect it?
>
> Not necessarily.  If the scanner does not have permission to read
> something which you want it to scan, then obviously it cannot scan it.
> This applies just as much to devices and data streams via sockets as
> is does to files.  It's up to you to arrange for the scanner to have
> permission to do what you want it to do.  And in my view it's usually
> pointless to scan a log file with a virus scanner - if indeed that is
> what you're doing - and this applies especially to the log which is
> recording the progress of the scan.
>
> --
>
> 73,
> Ged.
>
> ___
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] New Main & Daily CVD's are incoming

[Joel Esler (jesler) via clamav-users]

I am not sure what you mean by “scripted updates”?  If you are using FreshClam 
or cvdupdate, your downloads should happen fine.

> On Jul 13, 2021, at 10:29 AM, Mark Allan via clamav-users 
>  wrote:
> 
> Hi Joel,
> 
> Will you be posting scripted updates for main.cvd and daily.cvd or just the 
> new cvd files in their entirety? I seem to remember processing the cdiff 
> files caused a lot of problems for people the last time main.cvd was updated.
> 
> Mark
> 
>> On 13 Jul 2021, at 3:05 pm, Joel Esler (jesler) via clamav-users 
>> mailto:clamav-users@lists.clamav.net>> wrote:
>> 
>> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
>> main.cvd and daily.cvd, as we do periodically to move more of the long term 
>> signatures into the main.cvd and make the daily.cvd smaller again.  
>> 
>> This will have an impact on your downloads of these files (as every ClamAV 
>> instance will have to re-download both files), so you may see a spike in 
>> your bandwidth usage.
>> 
>> We will monitor the situation on the mirror side and make any adjustments 
>> necessary, but we anticipate no issues.
>> 
>> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
>> 
>> 
>> -- 
>> Joel Esler
>> Manager, Communities Division
>> Cisco Talos Intelligence Group
>> https://www.talosintelligence.com  | 
>> https://www.snort.org  | https://www.clamav.net 
>>  
>> 
>> ___
>> 
>> clamav-users mailing list
>> clamav-users@lists.clamav.net 
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>> 
>> 
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>> 
>> http://www.clamav.net/contact.html#ml
> 
> 
> ___
> 
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml



smime.p7s
Description: S/MIME cryptographic signature

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] New Main & Daily CVD's are incoming

[Mark Allan via clamav-users]

Hi Joel,

Will you be posting scripted updates for main.cvd and daily.cvd or just the new 
cvd files in their entirety? I seem to remember processing the cdiff files 
caused a lot of problems for people the last time main.cvd was updated.

Mark

> On 13 Jul 2021, at 3:05 pm, Joel Esler (jesler) via clamav-users 
>  wrote:
> 
> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
> main.cvd and daily.cvd, as we do periodically to move more of the long term 
> signatures into the main.cvd and make the daily.cvd smaller again.  
> 
> This will have an impact on your downloads of these files (as every ClamAV 
> instance will have to re-download both files), so you may see a spike in your 
> bandwidth usage.
> 
> We will monitor the situation on the mirror side and make any adjustments 
> necessary, but we anticipate no issues.
> 
> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
> 
> 
> -- 
> Joel Esler
> Manager, Communities Division
> Cisco Talos Intelligence Group
> https://www.talosintelligence.com  | 
> https://www.snort.org  | https://www.clamav.net 
>  
> 
> ___
> 
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] New Main & Daily CVD's are incoming

[Joel Esler (jesler) via clamav-users]

Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
main.cvd and daily.cvd, as we do periodically to move more of the long term 
signatures into the main.cvd and make the daily.cvd smaller again.  

This will have an impact on your downloads of these files (as every ClamAV 
instance will have to re-download both files), so you may see a spike in your 
bandwidth usage.

We will monitor the situation on the mirror side and make any adjustments 
necessary, but we anticipate no issues.

https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 


-- 
Joel Esler
Manager, Communities Division
Cisco Talos Intelligence Group
https://www.talosintelligence.com | https://www.snort.org | 
https://www.clamav.net 

smime.p7s
Description: S/MIME cryptographic signature

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml