Hi there, On Wed, 7 Apr 2021, Saurav Sarkar via clamav-users wrote:
We are using a HTTP enabled malware scanning service based on Clam AV.
Perhaps you will get better answers if you address your questions to the supplier of this service.
We have files like CAD files which can go in GBs and want to send to this malware scanning service.
Does the service which you are using permit that?
Is there a possibility to send the file in chunks and get it scanned in the server side in chunks.
Again you should ask your service because we on the mailing list know nothing about it. I imagine that it might be possible, but I would also guess that it would be pointless for your stated purpose.
I observed that there is a INSTREAM command in clamd for this purpose
The clamd 'man' page doesn't exactly say that. And assuming that this is related to your use of the service, do you know that your service actually uses clamd? The INSTREAM command is available so that you can send a stream of data to the scanner instead of telling it to scan some file. If you will read the clamd.conf 'man' page you will see that the stream of data must not exceed the value of the configured 'StreamMaxLength'. The default for that option is 25 Megabytes, a lot less than the GBs that you're talking about. If the maximum is exceeded by the length of the data stream sent after the INSTREAM command, clamd will return an error and the scan will fail. If I were running a Web service of the sort you've dfescribed I'd be very cautious about increasing the default StreamMaxLength because of the potential for abuse.
and also there is a 4GB size limit.
A number of limits depend on the configuration, and can be much less than that.
Read somewhere
Where?
the full file size is mapped to memory.
I do not know what that means. The scanner will use whatever memory is available to it. It needs around 1Gbyte for the current 'official' databases, and it can use considerably more than that if you add some of the various third-party datasases. But this memory is used to store signatures (or rather the compiled versions of them, which is what takes the time to start clamd or clamscan), not to store the data being scanned. It is not easy to predict how much memory will be used to scan a particular data stream.
Is it the case for INSTREAM command also ?
See my previous answer.
If it is the case then even if chunking is supported then the server side must have at least 4GB of memory.
Somewhere along your chain of logic you seem to have left me behind, but I would recommend at least 4GB of memory for anything which will be running the ClamAV scanner unless the user knows what he's doing. -- 73, Ged. _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
