[Clamav-devel] Override the CA bundle file when using freshclam.
I was having trouble using the latest freshclam version on CentOS 6,
because the system wide CA bundle file used by libcurl is unable to
validate the ClamAV HTTPS certificates. So I fixed the problem with a
small patch that allows the user to override the CA bundle file
freshclam uses. You can set the path via the "CAFile" directive in the
freshclam.conf file, or dictate the path using freshclam command line,
as --ca=FILE" ... the patch is a little rough, I didn't test/document
the conf file option, and I haven't written unit tests for it, but since
it's a pretty important feature I'm submitting the patch to the list ... L~
diff --git a/freshclam/freshclam.c b/freshclam/freshclam.c
index b638ad7..1a15257 100644
--- a/freshclam/freshclam.c
+++ b/freshclam/freshclam.c
@@ -154,6 +154,7 @@ static void help(void)
printf("\n");
printf("--config-file=FILE Read configuration from FILE.\n");
printf("--log=FILE -l FILE Log into FILE\n");
+printf("--ca=FILEOverride the certificate bundle FILE location\n");
printf("--daemon -d Run in daemon mode\n");
printf("--pid=FILE -p FILE Save daemon's pid in FILE\n");
#ifndef _WIN32
@@ -856,6 +857,7 @@ static fc_error_t initialize(struct optstruct *opts)
if ((optget(opts, "LocalIPAddress"))->enabled)
fcConfig.localIP = (optget(opts, "LocalIPAddress"))->strarg;
+fcConfig.caFile = optget(opts, "CAFile")->strarg;
fcConfig.databaseDirectory = optget(opts, "DatabaseDirectory")->strarg;
/* Select a path for the temp directory: databaseDirectory/tmp */
diff --git a/libfreshclam/libfreshclam.c b/libfreshclam/libfreshclam.c
index c32646c..23e70be 100644
--- a/libfreshclam/libfreshclam.c
+++ b/libfreshclam/libfreshclam.c
@@ -206,6 +206,9 @@ fc_error_t fc_initialize(fc_config *fcConfig)
if (NULL != fcConfig->proxyPassword) {
g_proxyPassword = cli_strdup(fcConfig->proxyPassword);
}
+if (NULL != fcConfig->caFile) {
+g_caFile = cli_strdup(fcConfig->caFile);
+}
#ifdef _WIN32
if ((fcConfig->databaseDirectory[strlen(fcConfig->databaseDirectory) - 1] != '/') &&
@@ -270,6 +273,10 @@ void fc_cleanup(void)
free(g_userAgent);
g_userAgent = NULL;
}
+if (NULL != g_caFile) {
+free(g_caFile);
+g_caFile = NULL;
+}
if (NULL != g_proxyServer) {
free(g_proxyServer);
g_proxyServer = NULL;
diff --git a/libfreshclam/libfreshclam.h b/libfreshclam/libfreshclam.h
index a83a338..a32f29b 100644
--- a/libfreshclam/libfreshclam.h
+++ b/libfreshclam/libfreshclam.h
@@ -50,6 +50,7 @@ typedef struct fc_config_ {
uint32_t connectTimeout; /**< CURLOPT_CONNECTTIMEOUT, Timeout for the. connection phase (seconds). */
uint32_t requestTimeout; /**< CURLOPT_TIMEOUT, Timeout for libcurl transfer operation (seconds). */
uint32_t bCompressLocalDatabase; /**< If set, will apply gz compression to CLD databases. */
+const char *caFile; /**< (optional) CA file to use for certificate verification, if desired. */
const char *logFile; /**< (optional) Filepath to use for log output, if desired. */
const char *logFacility; /**< (optional) System logging facility (I.e. "syslog"), if desired. */
const char *localIP; /**< (optional) client IP for multihomed systems. */
diff --git a/libfreshclam/libfreshclam_internal.c b/libfreshclam/libfreshclam_internal.c
index 9907254..3d98364 100644
--- a/libfreshclam/libfreshclam_internal.c
+++ b/libfreshclam/libfreshclam_internal.c
@@ -108,6 +108,7 @@ uint16_t g_proxyPort = 0;
char *g_proxyUsername = NULL;
char *g_proxyPassword = NULL;
+char *g_caFile= NULL;
char *g_tempDirectory = NULL;
char *g_databaseDirectory = NULL;
@@ -422,6 +423,12 @@ static fc_error_t create_curl_handle(
}
}
+if (g_caFile) {
+if (CURLE_OK != curl_easy_setopt(curl, CURLOPT_CAINFO, g_caFile)) {
+logg("!create_curl_handle: Failed to set CURLOPT_CAINFO (%s)!\n", g_caFile);
+}
+}
+
#if defined(C_DARWIN) || defined(_WIN32)
if (CURLE_OK != curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslctx_function)) {
logg("*create_curl_handle: Failed to set SSL CTX function. Your libcurl may use an SSL backend that does not support CURLOPT_SSL_CTX_FUNCTION.\n");
diff --git a/libfreshclam/libfreshclam_internal.h b/libfreshclam/libfreshclam_internal.h
index 747e3ee..1d01c62 100644
--- a/libfreshclam/libfreshclam_internal.h
+++ b/libfreshclam/libfreshclam_internal.h
@@ -46,6 +46,7 @@ extern uint16_t g_proxyPort;
extern char *g_proxyUsername;
extern char *g_proxyPassword;
+extern char *g_caFile;
extern char *g_tempDirectory;
extern char *g_databaseDirectory;
diff --git a/shared/optparser.c b/shared/optparser.c
index bb597c7..2c0ed39 100644
---
Re: [Clamav-devel] New Version of ClamAV
On 3/20/2013 9:47 AM, Eugene Crosser wrote: On 03/20/2013 06:35 PM, Matt Olney wrote: We're currently scoping out the next version of ClamAV. We have a number of ideas in house, but I wanted to solicit some feedback from our users about what you might be interested in seeing. API to scan data that is already read (or, more practically, mmap()'ed) into memory, a la scan_buff(void *start, size_t length, ...options etc.) Thanks, Eugene I second that! Right now we have to write an email message to disk just so we can pass in a file descriptor. I'd also advocate better support/documentation/testing of a lite ClamAV build configuration. Namely geared towards systems with limited memory and/or smaller than normal thread stack sizes, or strict security environments, yet still capable of examining large file(s). This would include being designed to run (as effectively as possible) without the builtin runtime. Security audits have flagged ClamAV in the past because it ships with LLVM code and could (theoretically) execute arbitrary code from a compromised definitions file. ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] Problem building 0.97.2 on FreeBSD
On 8/2/2011 9:23 AM, Renato Botelho wrote:
A FreeBSD user reported me he got following while building
0.97.2 on a recent -current system:
PASS: check1_clamscan.sh
FAIL: check2_clamd.sh
PASS: check3_clamd.sh
I encountered the same error under RHEL 6 = ERROR: Can't send to clamd:
Broken pipe. Increasing the resource limits fixed the issue for me.
Attached is the patch I used to workaround the problem with the 0.97.1
and 0.97.2 releases. See this bug report for information:
https://wwws.clamav.net/bugzilla/show_bug.cgi?id=1990
diff -U 3 -H -b -w -B -E -i -p -r -N -- clamav/unit_tests/check_common.sh
clamav.lavabit/unit_tests/check_common.sh
--- clamav/unit_tests/check_common.sh 2011-05-13 06:25:31.0 -0500
+++ clamav.lavabit/unit_tests/check_common.sh 2011-06-09 13:42:00.570566000
-0500
@@ -47,8 +47,6 @@ die()
# Setup test directory to avoid temporary and output file clashes
test_start() {
-ulimit -t 120 || true; ulimit -d 1024000 || true;
-ulimit -v 1024000 || true;
(cd test-$1 2/dev/null killclamd || true)
rm -rf test-$1
mkdir test-$1
___
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] [QUESTION] How does clamAV updates the signature database on-the-fly?
On 7/28/2010 6:18 PM, thyago wrote:
I'm researching ways of updating a signature database on-the-fly, so the way
clamAV does it, can really help me out...
I mean, what structures are there? how is it implemented?
Is there a data structure used to store the signatures on memory? If so, how
exactly is it updated?
what type of data structure? dynamic or static?
I need to know if you guys use a pointer to the structure, and then just set
it to point to the new updated structure,
and if for example, there's a condition, that limits when this pointer can
be changed...like a thread needing to finish first
I tried to look for the implementation on the code itself...but it's so
big...i don't know in which file to look =/
Thank you very much, for the help
Thyago
Attached is my implementation. As you can see I use a RW lock to
minimize contention.
/**
* @file /magma/providers/external/clamav.c
*
* @brief Interface for the ClamAV library.
*
* $Author: Ladar Levison $
* $Date: 2010/08/13 10:32:38 $
* $Revision: ecaee526d4ba88a141c5b889dd023b13c05c2654 $
*
*/
#include magma.h
/**
* The virus engine spool directory.
*/
char *virus_spool = NULL;
/**
* The mask used to generate temporary file names.
*/
char *virus_spool_mask = NULL;
/**
* The status of the signatures directory.
*/
struct cl_stat virus_stat;
/**
* The number of signatures loaded by the virus engine.
*/
unsigned int virus_sigs = 0;
/**
* The virus engine context pointer.
*/
struct cl_engine *virus_engine = NULL;
/**
* The virus engine read/write lock.
*/
pthread_rwlock_t virus_lock = PTHREAD_RWLOCK_INITIALIZER;
/**
* Obtains a virus engine read lock and records the number of virus signatures
loaded by the active ClamAV engine context.
*
* @return Returns the number of virus signatures loaded by the active ClamAV
engine context.
*/
uint64_t virus_sigs_loaded(void) {
uint64_t loaded = 0;
pthread_rwlock_rdlock(virus_lock);
loaded = virus_sigs;
pthread_rwlock_unlock(virus_lock);
return loaded;
}
/**
* Counts the number of official signatures available inside the ClamAV
database folder.
*
* @return Returns the number of official signatures available inside the
ClamAV database folder.
*/
uint64_t virus_sigs_total(void) {
int state;
unsigned int total = 0;
if ((state = cl_countsigs_d(magma.iface.virus.signatures,
CL_COUNTSIGS_OFFICIAL, total)) != CL_SUCCESS) {
log_error(ClamAV was unable to count the number of available
signatures. {cl_countsigs = %i = %s}, state, cl_strerror_d(state));
return 0;
}
return total;
}
/**
* Frees a ClamAV engine context and sets the pointer to NULL.
*
* @param target A doubly referenced pointer to a ClamAV engine context.
*/
void virus_engine_destroy(struct cl_engine **target) {
log_check(!target || !*target);
cl_engine_free_d(*target);
*target = NULL;
return;
}
/**
* Generates a new ClamAV engine context.
*
* @param signatures An optional pointer which will be used to record the
number of signatures loaded.
* @return Returns a pointer to the newly created context or NULL if an error
occurs.
*/
struct cl_engine * virus_engine_create(uint64_t *signatures) {
int state;
unsigned int loaded = 0;
struct cl_engine *target = NULL;
// Reset the signatures pointer if one was passed in.
if (*signatures) {
*signatures = 0;
}
// Allocate ClamAV engine context.
if ((target = cl_engine_new_d()) == NULL) {
log_error(ClamAV returned an error while allocating the engine
context. {cl_engine = NULL});
return NULL;
}
// Load the current signature database.
if ((state = cl_load_d(magma.iface.virus.signatures, target, loaded,
CL_DB_STDOPT)) != CL_SUCCESS) {
log_error(ClamAV returned an error while loading the database.
{cl_load = %i = %s}, state, cl_strerror_d(state));
cl_engine_free_d(target);
return NULL;
}
// Compile the internal lookup structures.
if ((state = cl_engine_compile_d(target)) != CL_SUCCESS) {
log_error(ClamAV database compilation error.
{cl_engine_compile = %i = %s}, state, cl_strerror_d(state));
cl_engine_free_d(target);
return NULL;
}
// Max scan size. 2048 MB.
// Sets the maximum amount of data to be scanned for each input file.
if ((state = cl_engine_set_num_d(target, CL_ENGINE_MAX_SCANSIZE, 2048ll
* 1048576ll)) != CL_SUCCESS) {
log_error(ClamAV configuration error. {cl_engine_set_num = %i
= %s}, state, cl_strerror_d(state));
cl_engine_free_d(target);
return NULL;
}
// Max file size. 512 MB.
// Files larger than this limit won't be scanned
Re: [Clamav-devel] [QUESTION] How does clamAV updates the signature database on-the-fly?
On 8/14/2010 3:19 AM, Török Edwin wrote:
// Scan the message. The OLE code has a bug in it that causes segfaults.
What bug ??
That comment was related to a bug I found in Feb/2008 and v0.92.1, but
has long since been patched. See this email thread for details:
http://marc.info/?l=clamav-develm=120442553919615
I had an internal patch floating around for awhile that fixed the issue
inside ole2_walk_property_tree() by incrementing rec_level. Somewhere
along the line the issue was fixed, but I never removed the comment. The
relevant lines in v0.96.2 increment rec_level just like my patch did. I
never submitted the patch because back in 2008 because you indicated
that wasn't the best solution.
// We ignore email that ClamAV thinks is a phishing
based on scanner's internal heuristic checks. else if
(starts_ci_bl_bl(Phishing, 8, virname, ns_get_length(virname)) ||
starts_ci_bl_bl(Joke, 4, virname, ns_get_length(virname)))
{ pthread_rwlock_unlock(virus_lock);
stats_increment_by_name(provider.virus.scan.total);
stats_increment_by_name(provider.virus.scan.clean); close(fd);
return 0; }
This is incorrect, if you want to match the heuristic Phishing
detection use Heuristics.Phishing.
There are signatures which contain *Phishing*, and *Joke*. ClamAV stops
on first match.
So if you get a zip that contains something ClamAV detects as
Phishing/Joke as first element in zip followed a real malware, then it
will only report the first match (Phishing/Joke). Your code will mark
it as clean, when in fact it could be infected.
(Note that this is not the case for Heuristics.Phishing where ClamAV
keeps on scanning and only reports the heuristics if it didn't find
anything else).
The proper way to deal with this is to not load the Phishing signatures
at all, there is an option you can pass to cl_load() for that.
For *Joke* there is no flag that you can pass though.
Is it possible to determine when ClamAV detects more than one virus and
iterate through the resulting names? I revisited the ex1.c file, and the
clamscan/manager.c file and they seem to suffer from the same issue. In
the case of clamscan, it only outputs the first virus name, which like
you pointed out could be innocuous compared to what else lies farther
along in the file.
If we are limited to only a single result, wouldn't it make more sense
to have a precendence order in place? Presumably malware would rate
ahead of phishing or jokes.
--
Ladar Levison
Lavabit LLC
http://lavabit.com
___
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] [QUESTION] How does clamAV updates the signature database on-the-fly?
On 8/14/2010 5:30 AM, Török Edwin wrote: Heuristics.Phishing.* will not stop the scan, and report only if nothing else is found. Other engine detections could be changed to behave the same way. Signature based detections however always stop on first match, and that is not configurable. If you want to ignore certain signature categories, it is best to not load them in the first place. To do that you can unpack the DBs, and remove the sigs you don't want. What I'm trying to do is let the user decide whether to enable a specific category, so removing the signatures from the database isn't an option for me. For awhile now, our users have been able to use the preferences portal on our website to enable/disable malware checks and/or phishing checks. The malware category is usually a reliable reason to send a message to the bit bucket. While the latter Phishing/Heuristic/Joke categories are more likely to generate a false positive. Perhaps its just me, but I would consider the ability to reliably determine what ClamAV found important. -- Ladar Levison Lavabit LLC http://lavabit.com ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] The upcoming 15 April kill-switch
On 4/7/2010 4:05 PM, David F. Skoll wrote: o The server could look at the Freshclam user-agent version number and not serve up the new database if it's too old. Wouldn't this provide a solution? I checked, and freshclam does provide the version number in the User-Agent string. I used the trusty packet sniffer to check first. And while I'm not sure what the end result/error would be, couldn't the servers be easily configured to redirect requests from old clients? Or better yet, somehow return the last valid/working image for 0.94.2 clients? I understand that F/OSS projects like ClamAV are at the mercy of what the developers would like to work on, and that generally means pushing the code forward rather than maintaining past releases; but common cutesy says they also shouldn't knowingly make technical decisions that will result their code exploding into a burst of flames at some arbitrary point in the future. As someone who regularly gets parachuted into burning buildings with a keyboard and a water bucket; I instinctively frown upon the people who go about setting fires on purpose. There is a big difference between a product that is no longer updated/maintained past a certain date but continues to provide the same humble functionality it had when it was abandoned, and one that refuses past a certain date. Personally speaking, I have a few servers in production right now that use 0.94.2 because moving past that point will require updating code to use the new library interface. I had assumed on those servers that ClamAV would simply continue working past the 15th, but would just be stuck using whatever ends up being the last compatible signatures database. This thread made me realize that if I don't disable freshclam on the 14th, I might be needing the water bucket on the 15th. For those sysadmins who don't notice this thread; I hope all that ends up happening is inbound mail gets delivered without being scanned. And that they are are able to revert their database to a working version or update their installs before the bad guys realize which systems are no longer scanning for viruses. Of course the unlucky ones will only hear the bells that ring when the bits stop flowing. Anyone feel like volunteering to create a wrapped version of ClamAV that is binary compatible with pre 0.95 installs? ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] [Clamav] Cannot dlopen: file not found - unrar support unavailable
Eugene Crosser wrote: I suspect that this may break on AMD64. It's more picky about PIC/non-PIC code (no pun intended), attempt to link objects from .a into .so often fails. I haven't tested this method on any AMD64 servers, but it wouldn't surprise if you were right. When I tried using this method on a Cent OS 5 system, I had to link 1-2 or other libraries before dlopen() would accept it. Any thoughts on a better way of accomplishing the same task? I like to combine all of the f/oss libraries my daemon uses into a single library (clamav, openssl, memcached, etc). It makes deployments easier (executable, f/oss library and config file). ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
[Clamav-devel] API concerns
There currently is no function call available via the public libclamav API to return the total number of signatures in the database directory. To find this number a developer would need to manually pass the file names to the CVD functions. Since there are multiple files stored in the directory, and filenames change, I don't feel that is a stable interface model. Why not store the number as part of the cl_stat structure, and return it with a simple function call: unsigned int cl_statsigtotal(const struct cl_stat *dbstat); I like to write this total to the log files so I periodically check to make sure the database is up to date, and has been refreshed inside the daemon. Checking the number against the one on the clamav.net home page is easiest, since I don't have freshclam installed on every server. I used to use the number of signatures returned by the cl_load function, but that function only returns the number of signatures loaded, not the number of signatures available. (When I first implemented my interface, there was no distinction.) ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
