there are no spaces in the filename - they all are replaced by '_'.
check if the assp process has full permission to the 'assp/tmp' folder !
also check ulimit for the assp user (max opened file handles) !
I never had any problems with any filename on windows - even with the
older Plugin versions.
I don't recommend to OCR images anyway with the plugin. ImageMagick
converts any image in to an uncompressed image for tesseract. It could be
possible that a 1MB jpeg grows to 30MB or more.
Even with a very fast CPU, it will take simply too long to convert and to
parse the image - which could lead in to DB and in to SMTP timeouts,
because the system calls for the convert and OCR are out of control of
assp and uninteruptable.
Notice, that a vanilla tesseract installation is unusable. You have to
extensive train tesseract - which has taken me some weeks , before I've
switched off the OCR! :)
The Bayesian and HMM engine (rebuildspamdb) calculates a hash value over
each found attachment / include. And the ASSP_AFC.pm Plugin is doing the
same - and will block if the hash (attachment/include) is known to be part
of SPAM.
Thomas
Von: aquilinux <[email protected]>
An: ASSP development mailing list <[email protected]>,
Datum: 16.10.2013 13:44
Betreff: Re: [Assp-test] Antwort: Re: Antwort: ASSP_OCR errors
thanks for your efforts, Thomas.
unfortunately the issue is still present, even if filename has been
epurated by spaces:
Oct-16-13 13:29:16 m1-22956-36876 [Worker_1] [TLS-out] [Plugin]
172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: (att) *file
png_with_spaces.png* found in mime part 2
Oct-16-13 13:29:16 m1-22956-36876 [Worker_1] [TLS-out] [Plugin]
172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
(attatched) file *png_with_spaces.png*
Oct-16-13 13:29:16 m1-22956-36876 [Worker_1] [TLS-out] [Plugin]
172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR::get_ocr(),
imagemagick convert problem? */usr/local/bin/convert
/opt/assp/tmp/ocr_pl13819229566793png_with_spaces.png -compress none
-colorspace rgb -contrast
/opt/assp/tmp/.tesseract_x_temp_1381922956455.tif, 256*
Oct-16-13 13:29:16 m1-22956-36876 [Worker_1] [TLS-out] [Plugin]
172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: Plugin(2.14)
successful called for runlevel 'complete mail'!
same thing for pdf, as i reported yesterday.
this is not a "big" problem for me since OCR errors are not so common and
mails are processed no matter if the plugin exits with errors or not.
but i'd like to have all errors fixed before going into production next
month.
thanks again for your efforts, ASSP really kicks the ass :-)
cheers!
On Wed, Oct 16, 2013 at 1:15 PM, Thomas Eckardt
<[email protected]>wrote:
> ASSP_OCR.pm version 2.14 removes the spaces from the file name.
>
> Thomas
>
>
>
>
>
> Von: aquilinux <[email protected]>
> An: ASSP development mailing list <[email protected]>,
> Datum: 15.10.2013 10:26
> Betreff: Re: [Assp-test] Antwort: Re: Antwort: ASSP_OCR errors
>
>
>
> all these commands work from command line and exit 0:
>
> /usr/local/bin/convert */tmp/png\ with\ spaces.png* -compress none
> -colorspace rgb -contrast "/opt/assp/tmp/test.tiff"
> /usr/local/bin/convert *'/tmp/png with spaces.png'* -compress none
> -colorspace rgb -contrast "/opt/assp/tmp/test.tiff"
> /usr/local/bin/convert *"/tmp/png with spaces.png"* -compress none
> -colorspace rgb -contrast "/opt/assp/tmp/test.tiff"
>
> moreover, the issue is not limited to DoImage:
>
> Oct-15-13 10:18:43 m1-25123-06842 [Worker_1] [TLS-out] [Plugin]
> 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: (att) file* pdf
with
> spaces.pdf* found in mime part 2
> Oct-15-13 10:18:43 m1-25123-06842 [Worker_1] [TLS-out] [Plugin]
> 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
> (attached)
> file *pdf with spaces.pdf*
> Oct-15-13 10:18:43 m1-25123-06842 [Worker_1] [TLS-out] [Plugin]
> 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: *error (Can't call
> method "_text_from_pdf" on an undefined value at
> /opt/assp/Plugins/ASSP_OCR.pm line 329*.
> )
> Oct-15-13 10:18:43 m1-25123-06842 [Worker_1] [TLS-out] [Plugin]
> 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: Plugin(2.13)
> successful called for runlevel 'complete mail'!
>
> so, maybe there's a problem writing *$tmpfile* instead of reading it,
but
> i'm only guessing.
>
> thanks.
>
>
> On Tue, Oct 15, 2013 at 6:56 AM, Thomas Eckardt
> <[email protected]>wrote:
>
> > >/usr/local/bin/convert
> > '/opt/assp/tmp/ocr_pl1381736958459png with spaces.png' -compress none
> > -colorspace rgb -contrast
> > /opt/assp/tmp/.tesseract_x_temp_1381736958354.tif, 256*
> >
> > is imagemagick not able to deal with file names containing whitespaces
> on
> > linux ?
> > or is there anything else wrong in this command?
> >
> > Thomas
> >
> >
> >
> >
> >
> > Von: aquilinux <[email protected]>
> > An: ASSP development mailing list
<[email protected]>,
> > Datum: 14.10.2013 10:10
> > Betreff: Re: [Assp-test] Antwort: ASSP_OCR errors
> >
> >
> >
> > Hi Thomas,
> > file name is now sanitized:
> >
> > *Oct-14-13 09:41:36 m1-36496-25693 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: (att) file
text1.sid
> > found in mime part 1*
> > *Oct-14-13 09:41:36 m1-36496-25693 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: (att) file
> xstress.jpg
> > found in mime part 2*
> > *Oct-14-13 09:41:36 m1-36496-25693 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
> (attatched)
> > file xstress.jpg*
> > *Oct-14-13 09:41:36 m1-36496-25693 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: OCR(2.13)
> > (TextFile(text1.sid), Image(xstress.jpg)) data extracted*
> > *Oct-14-13 09:41:36 m1-36496-25693 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: Plugin(2.13)
> > successful
> > called for runlevel 'complete mail'!*
> >
> > however, the plugin still has some problems dealing with spaces in
> > attachment filename:
> >
> > *Oct-14-13 09:49:18 m1-36958-26749 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
> (attatched)
> > file
> > png with spaces.png*
> > *Oct-14-13 09:49:18 m1-36958-26749 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR::get_ocr(),
> imagemagick
> > convert problem? /usr/local/bin/convert
> > '/opt/assp/tmp/ocr_pl1381736958459png with spaces.png' -compress none
> > -colorspace rgb -contrast
> > /opt/assp/tmp/.tesseract_x_temp_1381736958354.tif, 256*
> > *Oct-14-13 09:49:18 m1-36958-26749 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: OCR(2.13)
> > (TextFile(text1.sid)) data extracted*
> > *Oct-14-13 09:49:18 m1-36958-26749 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: Plugin(2.13)
> > successful
> > called for runlevel 'complete mail'!*
> > *
> > *
> > *Oct-14-13 09:54:35 m1-37275-12668 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: (att) file
> > png_without_spaces.png found in mime part 2*
> > *Oct-14-13 09:54:35 m1-37275-12668 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
> > (attatched) file png_without_spaces.png*
> > *Oct-14-13 09:54:36 m1-37275-12668 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: OCR(2.13)
> > (TextFile(text1.sid), Image(png_without_spaces.png)) data extracted*
> > *Oct-14-13 09:54:36 m1-37275-12668 [Worker_1] [TLS-out] [Plugin]
> > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR:
Plugin(2.13)
> > successful called for runlevel 'complete mail'!*
> >
> > one more thing.
> > i noticed that after OCR plugin is called, sometimes mysql database
> "goes
> > away".
> > sometimes "right after" the mail is processed and delivered, sometimes
> > during the following minute.
> >
> > *Oct-14-13 09:58:12 [Worker_10000] Warning: got database error
Database
> > select statement failed in whitelist(exists): MySQL server has gone
away
> -
> > MySQL server has gone away on table whitelist - try to reconnect*
> > *[... snip ...]*
> >
> > i only need to "bomb" ASSP with mails (3 is enough) with attachment to
> > reproduce the issue.
> > this is annoying, but it isn't really a problems yet, as long as it
> > doesn't
> > break my mysql multi-master replication.
> >
> > thanks for your work.
> > cheers!
> >
> >
> >
> > On Sat, Oct 12, 2013 at 7:25 AM, Thomas Eckardt
> > <[email protected]>wrote:
> >
> > > try ASSP_OCR version 2.13
> > >
> > > Thomas
> > >
> > >
> > >
> > >
> > >
> > > Von: aquilinux <[email protected]>
> > > An: ASSP development mailing list
> <[email protected]>,
> > > Datum: 11.10.2013 16:39
> > > Betreff: Re: [Assp-test] Antwort: ASSP_OCR errors
> > >
> > >
> > >
> > > hi Thomas, i tried ASSP_OCR.pm version 2.12 (even if i suspect there
> is
> > a
> > > version mismatch in file .pm because it says 2.11) but the issue
still
> > > occurs.
> > > it seems to happen randomly with jpg files (not all jpgs).
> > >
> > > doing some tests with xstress (a SMTP Stress Testing Tool) i noticed
> > that
> > > files i attach are prefixed with a / (slash) with obvious problem on
> > image
> > > file path:
> > >
> > > Oct-11-13 15:50:55 m1-99455-30185 [Worker_1] [TLS-out] [Plugin]
> > > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR: processing
> > > (attatched) file */xstress.jpg*
> > > Oct-11-13 15:50:55 m1-99455-30185 [Worker_1] [TLS-out] [Plugin]
> > > 172.20.17.13 <[email protected]> to: [email protected] ASSP_OCR::get_ocr(),
> > > imagemagick convertproblem? /usr/local/bin/convert *
> > > /opt/assp/tmp/ocr_pl13814994555344/xstress.jpg* -compress none
> > -colorspace
> > > rgb -contrast /opt/assp/tmp/.tesseract_x_temp_1381499455134.tif, 256
1
> > > Oct-11-13 15:50:55 m1-99455-30185 [Worker_1] [TLS-out] [Plugin]
> > > 172.20.17.13 <[email protected]> to:
> > > [email protected] *ASSP_OCR:
> > > OCR(2.11)* (TextFile(text1.sid)) data extracted
> > >
> > > i don't know if it's sender's mta fault (i'm using xstress for
testing
> > my
> > > ASSP servers), but maybe ASSP should sanitize file names before
> writing
> > > them in tmp directory for converting.
> > > by the way, these emails and attachments are correctly delivered
with
> > > correct attachment name (xstress.jpg).
> > >
> > > if you could fix this (if this can be a bug..), i would do some more
> > > exhaustive tests with all kind of attachments.
> > >
> > > cheers.
> > >
> > >
> > >
> > >
> > > On Mon, Oct 7, 2013 at 4:17 PM, Thomas Eckardt
> > > <[email protected]>wrote:
> > >
> > > > try ASSP_OCR.pm version 2.12 (just released)
> > > >
> > > > Thomas
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Von: aquilinux <[email protected]>
> > > > An: [email protected],
> > > > Datum: 07.10.2013 13:37
> > > > Betreff: [Assp-test] ASSP_OCR errors
> > > >
> > > >
> > > >
> > > > Hi all.
> > > > I'm running ASSP version 2.3.3(13276) with Perl v5.14.2 on a
Ubuntu
> > > x86_64
> > > > box.
> > > > I'm getting this error since ASSP deployment:
> > > >
> > > >
> > > > ASSP_OCR::get_ocr(), imagemagick convert problem?
> > > > "/usr/local/bin/convert"
> > > > "/opt/assp/tmp/ocr_pl13807957101765image001.jpg" -compress none
> > > > -colorspace
> > > > rgb -contrast "/opt/assp/tmp/.tesseract_x_temp_1380795710360.tif",
> -1
> > > >
> > > > all ASSP_OCR paths are correct, imagemagick is working fine
outside
> > > ASSP.
> > > >
> > > > here is module version:
> > > > Plugins::ASSP_OCR 2.09 / 2.09 enabled sourceforge
> > > >
> > > > did anyone encountered this issue yet?
> > > >
> > > > thanks.
> > > >
> > > > --
> > > > "Madness, like small fish, runs in hosts, in vast numbers of
> > instances."
> > > >
> > > > Nessuno mi pettina bene come il vento.
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------------------------------------
> > > > October Webinars: Code for Performance
> > > > Free Intel webinars can help you accelerate application
performance.
> > > > Explore tips for MPI, OpenMP, advanced profiling, and more. Get
the
> > most
> > > > from
> > > > the latest Intel processors and coprocessors. See abstracts and
> > register
> > > >
> > > >
> > >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
>
> >
> > >
> > > >
> > > > _______________________________________________
> > > > Assp-test mailing list
> > > > [email protected]
> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >
> > > >
> > > >
> > > >
> > > > DISCLAIMER:
> > > > *******************************************************
> > > > This email and any files transmitted with it may be confidential,
> > > legally
> > > > privileged and protected in law and are intended solely for the
use
> of
> > > the
> > > >
> > > > individual to whom it is addressed.
> > > > This email was multiple times scanned for viruses. There should be
> no
> > > > known virus in this email!
> > > > *******************************************************
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------------------------------------
> > > > October Webinars: Code for Performance
> > > > Free Intel webinars can help you accelerate application
performance.
> > > > Explore tips for MPI, OpenMP, advanced profiling, and more. Get
the
> > most
> > > > from
> > > > the latest Intel processors and coprocessors. See abstracts and
> > register
> > > >
> > > >
> > >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
>
> >
> > >
> > > > _______________________________________________
> > > > Assp-test mailing list
> > > > [email protected]
> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >
> > > >
> > >
> > >
> > > --
> > > "Madness, like small fish, runs in hosts, in vast numbers of
> instances."
> > >
> > > Nessuno mi pettina bene come il vento.
> > >
> > >
> >
> >
>
>
------------------------------------------------------------------------------
> > > October Webinars: Code for Performance
> > > Free Intel webinars can help you accelerate application performance.
> > > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
> most
> > > from
> > > the latest Intel processors and coprocessors. See abstracts and
> register
> > >
> > >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
>
> >
> > >
> > > _______________________________________________
> > > Assp-test mailing list
> > > [email protected]
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> > >
> > >
> > > DISCLAIMER:
> > > *******************************************************
> > > This email and any files transmitted with it may be confidential,
> > legally
> > > privileged and protected in law and are intended solely for the use
of
> > the
> > >
> > > individual to whom it is addressed.
> > > This email was multiple times scanned for viruses. There should be
no
> > > known virus in this email!
> > > *******************************************************
> > >
> > >
> > >
> > >
> > >
> >
> >
>
>
------------------------------------------------------------------------------
> > > October Webinars: Code for Performance
> > > Free Intel webinars can help you accelerate application performance.
> > > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
> most
> > > from
> > > the latest Intel processors and coprocessors. See abstracts and
> register
> > >
> > >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
>
> >
> > > _______________________________________________
> > > Assp-test mailing list
> > > [email protected]
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> >
> >
> > --
> > "Madness, like small fish, runs in hosts, in vast numbers of
instances."
> >
> > Nessuno mi pettina bene come il vento.
> >
> >
>
>
------------------------------------------------------------------------------
> > October Webinars: Code for Performance
> > Free Intel webinars can help you accelerate application performance.
> > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
most
> > from
> > the latest Intel processors and coprocessors. See abstracts and
register
> >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
>
> >
> > _______________________________________________
> > Assp-test mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
> >
> >
> > DISCLAIMER:
> > *******************************************************
> > This email and any files transmitted with it may be confidential,
> legally
> > privileged and protected in law and are intended solely for the use of
> the
> >
> > individual to whom it is addressed.
> > This email was multiple times scanned for viruses. There should be no
> > known virus in this email!
> > *******************************************************
> >
> >
> >
> >
> >
>
>
------------------------------------------------------------------------------
> > October Webinars: Code for Performance
> > Free Intel webinars can help you accelerate application performance.
> > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
most
> > from
> > the latest Intel processors and coprocessors. See abstracts and
register
> >
> >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>
> > _______________________________________________
> > Assp-test mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
>
>
> --
> "Madness, like small fish, runs in hosts, in vast numbers of instances."
>
> Nessuno mi pettina bene come il vento.
>
>
------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register
>
>
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>
> _______________________________________________
> Assp-test mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential,
legally
> privileged and protected in law and are intended solely for the use of
the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
>
>
>
------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register
>
>
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
> _______________________________________________
> Assp-test mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
--
"Madness, like small fish, runs in hosts, in vast numbers of instances."
Nessuno mi pettina bene come il vento.
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Assp-test mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/assp-test
DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Assp-test mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/assp-test
