[clamav-users] Freshclam / mirror updates

[Joel Esler (jesler) via clamav-users]

Yesterday we made some updates to our CDN that distributes our CVD / CLD / 
CDIFF files.  These changes should result in a faster and more reliable 
download of these files.

Please let me know if you see any issues positive or negative!

Thanks!

--
Joel Esler
Manager, Communities Division
Cisco Talos Intelligence Group
http://www.talosintelligence.com

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Issue with clamav logical signature generation

[Tilman Schmidt]

Am 25.02.19 um 19:44 schrieb G.W. Haywood via clamav-users:
> Just as decimal strings are strings composed of decimal digits and can
> be any length, hexadecimal strings are strings composed of hexadecimal
> digits - and can also be any length.  They usually present as an even
> number of digits only because they generally represent the even numbers
> of four-bit binary numbers found in machine registers: 8-bit bytes and
> 16-bit words from decades ago, 32-bit, 64-bit and even 128-bit words
> (e.g. for IPv6 addresses) in more recent times.  I'm sure I did once
> use 12-bit word lengths for some reason, but I can't now remember what
> the hardware was.

Can't remember using 12 bit words, but one of the first machines I used,
a CDC 6600, had 60 bit data words and 18 bit addresses.
Text was normally uppercase-only, stored in 6-bit bytes ten characters
per word. But if you wanted to use lower case you could use a different
encoding which would occupy two of the 6-bit bytes per character, so in
a way you had 12-bit bytes.

Of course octal, not hex, was used to represent binary values back then.

Oh, and punchcards had 12 rows so a binary image of a punchcard would
actually consist of 80 12-bit words. (Or bytes.)

Tilman

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Issue with clamav logical signature generation

[SCOTT PACKARD via clamav-users]

> Can't remember using 12 bit words, but one of the first machines I used,
> a CDC 6600, had 60 bit data words and 18 bit addresses.
> Text was normally uppercase-only, stored in 6-bit bytes ten characters
> per word. But if you wanted to use lower case you could use a different
> encoding which would occupy two of the 6-bit bytes per character, so in
> a way you had 12-bit bytes

An instructor called it Consistency Doesn't Count (CDC).
That was the big machine on campus back in the day.



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Incompatible clamav.h changes

[Alessandro Vesely via clamav-users]

Hi Micah,

Thank you so much for your prompt reply.

On Wed 27/Feb/2019 20:48:44 +0100 Micah Snyder \(micasnyd\) via clamav-users 
wrote:
> 
> You're correct, there are non-backwards compatible changes in clamav.h in 
> version 0.101.  The libclamav major version number as also increased to 
> highlight the incompatibility.  
> 
> The changes are mentioned somewhat briefly in the release notes:
> https://blog.clamav.net/2018/12/clamav-01010-has-been-released.html
> https://github.com/Cisco-Talos/clamav-devel/blob/clamav-0.101.1/NEWS.md#some-of-the-more-obvious-changes
>  


That doc is rather brief about #define's.  I attach my attempt at matching 
them.  Besides having different values and going to different variables, 
CL_SCAN_* have new names.


> CL_SCAN_STDOPT previously enabled all parsers, plus heuristic alerts (which 
> were sometimes/inconsistently referred to as algorithmic detection): 
> https://github.com/Cisco-Talos/clamav-devel/blob/clamav-0.100.2/libclamav/clamav.h#L181
> 
> In 0.101, CL_SCAN_STDOPT has gone away.  Instead, you can get the same 
> functionality by setting the following, as shown in the example 
> (https://github.com/Cisco-Talos/clamav-devel/blob/dev/0.101/examples/ex1.c#L93):


I see.  I think I'll code #ifdef CL_SCAN_STDOPT for compile time switching, 
trying to be compatible with v100.


> options.parse |= ~0; /* enable all parsers */
> options.general |= CL_SCAN_GENERAL_HEURISTICS; /* enable heuristic alert 
> options */

The docs suggest AlgorithmicDetection becomes HeuristicAlerts, I'm not clear if 
that implies CL_SCAN_ALGORITHMIC becomes CL_SCAN_GENERAL_HEURISTICS (as claimed 
in my attachment).


Best
Ale
-- 




CL_ENGINE_* and CL_DB_* defines are untouched.

CL_SCAN_* migrated as follows(?):

-#define CL_SCAN_RAW0x0

-#define CL_SCAN_STDOPT (CL_SCAN_ARCHIVE | CL_SCAN_MAIL | CL_SCAN_OLE2 
| CL_SCAN_PDF | CL_SCAN_HTML | CL_SCAN_PE | CL_SCAN_ALGORITHMIC | CL_SCAN_ELF | 
CL_SCAN_SWF | CL_SCAN_XMLDOCS | CL_SCAN_HWP3)

-#define CL_SCAN_ARCHIVE0x1
+#define CL_SCAN_PARSE_ARCHIVE  0x1

-#define CL_SCAN_MAIL   0x2
+#define CL_SCAN_PARSE_MAIL 0x40

-#define CL_SCAN_OLE2   0x4
+#define CL_SCAN_PARSE_OLE2 0x80

-#define CL_SCAN_BLOCKENCRYPTED  0x8
+#define CL_SCAN_HEURISTIC_ENCRYPTED_ARCHIVE 0x40  /* alert if archive 
is encrypted (rar, zip, etc) */
+#define CL_SCAN_HEURISTIC_ENCRYPTED_DOC 0x80  /* alert if a 
document is encrypted (pdf, docx, etc) */

-#define CL_SCAN_HTML   0x10
+#define CL_SCAN_PARSE_HTML 0x100

-#define CL_SCAN_PE 0x20
+#define CL_SCAN_PARSE_PE   0x200

-#define CL_SCAN_BLOCKBROKEN 0x40
+#define CL_SCAN_HEURISTIC_BROKEN0x2   /* alert on broken 
PE and broken ELF files */

-#define CL_SCAN_MAILURL 0x80 /* ignored */

-#define CL_SCAN_BLOCKMAX0x100
+#define CL_SCAN_HEURISTIC_EXCEEDS_MAX   0x4   /* alert when files 
exceed scan limits (filesize, max scansize, or max recursion depth) */

-#define CL_SCAN_ALGORITHMIC 0x200
+#define CL_SCAN_GENERAL_HEURISTICS  0x4 /* option to enable 
heuristic alerts */

-#define CL_SCAN_PHISHING_BLOCKSSL   0x800 /* ssl mismatches, 
not ssl by itself*/
+#define CL_SCAN_HEURISTIC_PHISHING_SSL_MISMATCH 0x8   /* alert on SSL 
mismatches */

-#define CL_SCAN_PHISHING_BLOCKCLOAK 0x1000
+#define CL_SCAN_HEURISTIC_PHISHING_CLOAK0x10  /* alert on cloaked 
URLs in emails */

-#define CL_SCAN_ELF 0x2000
+#define CL_SCAN_PARSE_ELF   0x2

-#define CL_SCAN_PDF 0x4000
+#define CL_SCAN_PARSE_PDF   0x4

-#define CL_SCAN_STRUCTURED  0x8000
+#define CL_SCAN_HEURISTIC_STRUCTURED0x200 /* data loss 
prevention options, i.e. alert when detecting personal information */

-#define CL_SCAN_STRUCTURED_SSN_NORMAL   0x1
+#define CL_SCAN_HEURISTIC_STRUCTURED_SSN_NORMAL 0x400 /* alert when 
detecting social security numbers */

-#define CL_SCAN_STRUCTURED_SSN_STRIPPED 0x2
+#define CL_SCAN_HEURISTIC_STRUCTURED_SSN_STRIPPED   0x800 /* alert when 
detecting stripped social security numbers */

-#define CL_SCAN_PARTIAL_MESSAGE 0x4
+#define CL_SCAN_MAIL_PARTIAL_MESSAGE0x1

-#define CL_SCAN_HEURISTIC_PRECEDENCE0x8
+#define CL_SCAN_GENERAL_HEURISTIC_PRECEDENCE0x8 /* allow heuristic 
match to take precedence. */

-#define CL_SCAN_BLOCKMACROS 0x10
+#define CL_SCAN_HEURISTIC_MACROS0x20  /* alert on OLE2 
files containing macros */

-#define CL_SCAN_ALLMATCHES   

Re: [clamav-users] Incompatible clamav.h changes

[Paco Parra via clamav-users]

Please remove from your list.

El 28/02/2019 a las 12:55 p. m., Alessandro Vesely via clamav-users escribió:

Hi Micah,

Thank you so much for your prompt reply.

On Wed 27/Feb/2019 20:48:44 +0100 Micah Snyder \(micasnyd\) via clamav-users 
wrote:



You're correct, there are non-backwards compatible changes in clamav.h in 
version 0.101.  The libclamav major version number as also increased to 
highlight the incompatibility.

The changes are mentioned somewhat briefly in the release notes:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.clamav.net%2F2018%2F12%2Fclamav-01010-has-been-released.html&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770130980116&sdata=1XgAZinVLS3Ab8kyjtOlckWXaie0znypFQyiJAsZs%2BI%3D&reserved=0
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCisco-Talos%2Fclamav-devel%2Fblob%2Fclamav-0.101.1%2FNEWS.md%23some-of-the-more-obvious-changes&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770130980116&sdata=RA1T%2BVN4LSifcg3VZ%2B34MD4%2B60mczwTKZO6giQ4KHVc%3D&reserved=0




That doc is rather brief about #define's.  I attach my attempt at matching 
them.  Besides having different values and going to different variables, 
CL_SCAN_* have new names.




CL_SCAN_STDOPT previously enabled all parsers, plus heuristic alerts (which 
were sometimes/inconsistently referred to as algorithmic detection):
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCisco-Talos%2Fclamav-devel%2Fblob%2Fclamav-0.100.2%2Flibclamav%2Fclamav.h%23L181&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770130980116&sdata=BSOZzzz%2B%2FqPE6n3msSufGaRkyrpw5nO%2B1BGPOpsgQKs%3D&reserved=0

In 0.101, CL_SCAN_STDOPT has gone away.  Instead, you can get the same 
functionality by setting the following, as shown in the example 
(https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCisco-Talos%2Fclamav-devel%2Fblob%2Fdev%2F0.101%2Fexamples%2Fex1.c%23L93&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770130980116&sdata=B4%2B%2FT57guKkuBQRiWyfb7pOdzaeDPHkb9w%2FWj6fmC4M%3D&reserved=0):




I see.  I think I'll code #ifdef CL_SCAN_STDOPT for compile time switching, 
trying to be compatible with v100.




options.parse |= ~0; /* enable all parsers */
options.general |= CL_SCAN_GENERAL_HEURISTICS; /* enable heuristic alert 
options */



The docs suggest AlgorithmicDetection becomes HeuristicAlerts, I'm not clear if 
that implies CL_SCAN_ALGORITHMIC becomes CL_SCAN_GENERAL_HEURISTICS (as claimed 
in my attachment).


Best
Ale





___

clamav-users mailing list
clamav-users@lists.clamav.net
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.clamav.net%2Fmailman%2Flistinfo%2Fclamav-users&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770131000125&sdata=TLJpCqa57kSWru%2FnHHKbc89QftQqU%2BkD8Fw2jD%2FnMdM%3D&reserved=0


Help us build a comprehensive ClamAV guide:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvrtadmin%2Fclamav-faq&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770131000125&sdata=Kkw%2BnEdZc3kfYPxUlfUFTQqrhKIEHXH5r0KzXoJKj5U%3D&reserved=0

https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.clamav.net%2Fcontact.html%23ml&data=02%7C01%7C%7Cb49c566de1bd4af3c7af08d69dae7f35%7C84df9e7fe9f640afb435%7C1%7C0%7C636869770131000125&sdata=%2BcG%2BNAe2XAYoE9KnPWgLU60T3VwcDGogenV9cjq%2FGIQ%3D&reserved=0


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Incompatible clamav.h changes

[Micah Snyder (micasnyd) via clamav-users]

Alessandro,

Your attachment is correct. I will also note that the following macros enable 
the same feature.  The name "collect metadata" seemed to be more intelligible 
than simply "properties":

-#define CL_SCAN_FILE_PROPERTIES 0x1000
+#define CL_SCAN_GENERAL_COLLECT_METADATA0x2 /* collect metadata 
(--gen-json) */

To be clear about how the new scan options defines work... Instead of one 
bit-flag variable, we now have 5 bitflags variables to select options using the 
cl_scan_options structure. They are categorized by feature, like this:

struct cl_scan_options {
uint32_t general;   // <-- CL_SCAN_GENERAL_...
uint32_t parse;   // <-- CL_SCAN_PARSE_...
uint32_t heuristic;   // <-- CL_SCAN_HEURISTIC_...
uint32_t mail;   // <-- CL_SCAN_MAIL_...
uint32_t dev;   // <-- CL_SCAN_DEV_...
};

Come to think of it, I'm not sure why we don't include a macro in clamav.h to 
define the libclamav version number.  I will have to discuss it with the team.  
To programmatically query a string of the version number, you can use 
`cl_retver()`. 

Let me know if you have any other questions, and my apologies for the confusion.

Regards,
Micah

On 2/28/19, 1:56 PM, "Alessandro Vesely"  wrote:

Hi Micah,

Thank you so much for your prompt reply.

On Wed 27/Feb/2019 20:48:44 +0100 Micah Snyder \(micasnyd\) via 
clamav-users wrote:
> 
> You're correct, there are non-backwards compatible changes in clamav.h in 
version 0.101.  The libclamav major version number as also increased to 
highlight the incompatibility.  
> 
> The changes are mentioned somewhat briefly in the release notes:
> https://blog.clamav.net/2018/12/clamav-01010-has-been-released.html
> 
https://github.com/Cisco-Talos/clamav-devel/blob/clamav-0.101.1/NEWS.md#some-of-the-more-obvious-changes
 


That doc is rather brief about #define's.  I attach my attempt at matching 
them.  Besides having different values and going to different variables, 
CL_SCAN_* have new names.


> CL_SCAN_STDOPT previously enabled all parsers, plus heuristic alerts 
(which were sometimes/inconsistently referred to as algorithmic detection): 
> 
https://github.com/Cisco-Talos/clamav-devel/blob/clamav-0.100.2/libclamav/clamav.h#L181
> 
> In 0.101, CL_SCAN_STDOPT has gone away.  Instead, you can get the same 
functionality by setting the following, as shown in the example 
(https://github.com/Cisco-Talos/clamav-devel/blob/dev/0.101/examples/ex1.c#L93):


I see.  I think I'll code #ifdef CL_SCAN_STDOPT for compile time switching, 
trying to be compatible with v100.


> options.parse |= ~0; /* enable all parsers */
> options.general |= CL_SCAN_GENERAL_HEURISTICS; /* enable heuristic 
alert options */

The docs suggest AlgorithmicDetection becomes HeuristicAlerts, I'm not 
clear if that implies CL_SCAN_ALGORITHMIC becomes CL_SCAN_GENERAL_HEURISTICS 
(as claimed in my attachment).


Best
Ale
-- 







___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml