Re: [dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
Dear list, Just following up on this for posterity. On Ubuntu at least the issue appears to have been fixed in a Ghostsript update later in December, 2018. In Ubuntu 16.04 the version of Ghostscript with the fix is 9.26~dfsg+0-0ubuntu0.16.04.3. PDFs that were problematic before now have thumbnails generated successfully via filter-media. See the following Launchpad discussion: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1806517 Regards, On Mon, Dec 3, 2018 at 5:25 PM Alan Orth wrote: > A few updates, though still no solution. > > It seems this only affects certain PDFs. I've tried to inspect some of the > affected PDFs to see if I can find a pattern, but haven't found anything > conclusive yet. The only small detail is that two problematic PDFs were > created with "Microsoft® Word 2016" and an unaffected PDF was created with > "Microsoft® Word for Office 365". I still believe it is related to > Ghostscript's pngalpha driver and or the FirstPage/LastPage options (DSpace > generates the thumbnail from page 0) because the problematic PDFs do not > work on my local development machine running Arch Linux with ghostscript > 9.26 either. I will keep an eye on ghostscript git and keep you posted. I'd > appreciate if others shared their experiences too. > > On a side note, while ImageMagick's convert and identify commands also > fail on these PDFs, GraphicsMagick's convert and identify somehow work > around the problem and work fine. GraphicsMagick—forked from ImageMagick in > 2003—is widely considered to be a better project (better code, better > tests, better docs). I wonder if we should switch to it... the im4java > project supports it. > > Regards, > > Regards, > > On Sun, Dec 2, 2018 at 12:21 PM Alan Orth wrote: > >> Dear list, >> >> Another recent update in Debian / Ubuntu has broken PDF thumbnail >> generation in DSpace 5 and 6, though this time it is the Ghostscript >> package rather than ImageMagick. You should see errors like this when you >> try to run filter-media: >> >> org.im4java.core.InfoException: org.im4java.core.CommandException: >> org.im4java.core.CommandException: identify: FailedToExecuteCommand `"gs" >> -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=5 >> -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=4 >> -dGraphicsAlphaBits=4 "-r72x72" -dFirstPage=1 -dLastPage=1 >> "-sOutputFile=/tmp/magick-12989PcFN0DnJOej7%d" >> "-f/tmp/magick-129895Bmp44lvUfxo" "-f/tmp/magick-12989C0QFG51fktLF"' (-1) @ >> error/delegate.c/ExternalDelegateCommand/461. >> >> See Ubuntu Security Notice 3831-1[0] as well as a StackOverflow >> discussion from this past weekend[1]. In this case it seems that there is a >> bug[2] in the upstream Ghostscript project related to the pngalpha device >> and there is currently no packaged version with a fix. One workaround for >> now, suggested in the StackOverflow discussion, is to use the png16m output >> device instead of pngalpha. I have confirmed that this works on standalone >> invocations of `gs`, but I am not sure what implications this has for >> speed, memory, or the quality of thumbnails. >> >> Regards, >> >> [0] https://usn.ubuntu.com/3831-1/ >> [1] >> https://stackoverflow.com/questions/53560755/ghostscript-9-26-update-breaks-imagick-readimage-for-multipage-pdf >> [2] https://bugs.ghostscript.com/show_bug.cgi?id=699815 >> >> On Fri, Oct 12, 2018 at 6:44 PM Alan Orth wrote: >> >>> Great idea, Tim! I just logged it in Jira as DS-4052. >>> >>> https://jira.duraspace.org/browse/DS-4052 >>> >>> Regards, >>> >>> On Thu, Oct 11, 2018 at 7:23 PM Tim Donohue >>> wrote: >>> Alan, Could you create a "documentation" ticket for this in our JIRA system (as that's sometimes an easier place to find "known issues")? This sounds like something we should get into the official documentation, and having a ticket to remind us (and as a place to add additional questions/comments) would be useful. Thanks, Tim On Thu, Oct 11, 2018 at 12:27 AM Alan Orth wrote: > Dear DSpace people, > > Recently I noticed that our PDFs were not getting thumbnails generated > for them. I ran the filter-media script manually and saw an error like > this: > > org.im4java.core.InfoException: org.im4java.core.CommandException: > org.im4java.core.CommandException: identify: not authorized > `/tmp/impdfthumb5039464037201498062.pdf' @ > error/constitute.c/ReadImage/412. > > According to an Ubuntu Security Notice from 2018-10 the Ubuntu > developers have decided to disable PDF processing by default in their > ImageMagick[0] package. This affects all currently supported Ubuntu > long-term support releases. > > If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, > you will need to change your system's ImageMagick policy.xml file to > re-enable PDF processing. On Ubuntu 16.04 the file is >
Re: [dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
A few updates, though still no solution. It seems this only affects certain PDFs. I've tried to inspect some of the affected PDFs to see if I can find a pattern, but haven't found anything conclusive yet. The only small detail is that two problematic PDFs were created with "Microsoft® Word 2016" and an unaffected PDF was created with "Microsoft® Word for Office 365". I still believe it is related to Ghostscript's pngalpha driver and or the FirstPage/LastPage options (DSpace generates the thumbnail from page 0) because the problematic PDFs do not work on my local development machine running Arch Linux with ghostscript 9.26 either. I will keep an eye on ghostscript git and keep you posted. I'd appreciate if others shared their experiences too. On a side note, while ImageMagick's convert and identify commands also fail on these PDFs, GraphicsMagick's convert and identify somehow work around the problem and work fine. GraphicsMagick—forked from ImageMagick in 2003—is widely considered to be a better project (better code, better tests, better docs). I wonder if we should switch to it... the im4java project supports it. Regards, Regards, On Sun, Dec 2, 2018 at 12:21 PM Alan Orth wrote: > Dear list, > > Another recent update in Debian / Ubuntu has broken PDF thumbnail > generation in DSpace 5 and 6, though this time it is the Ghostscript > package rather than ImageMagick. You should see errors like this when you > try to run filter-media: > > org.im4java.core.InfoException: org.im4java.core.CommandException: > org.im4java.core.CommandException: identify: FailedToExecuteCommand `"gs" > -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=5 > -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=4 > -dGraphicsAlphaBits=4 "-r72x72" -dFirstPage=1 -dLastPage=1 > "-sOutputFile=/tmp/magick-12989PcFN0DnJOej7%d" > "-f/tmp/magick-129895Bmp44lvUfxo" "-f/tmp/magick-12989C0QFG51fktLF"' (-1) @ > error/delegate.c/ExternalDelegateCommand/461. > > See Ubuntu Security Notice 3831-1[0] as well as a StackOverflow discussion > from this past weekend[1]. In this case it seems that there is a bug[2] in > the upstream Ghostscript project related to the pngalpha device and there > is currently no packaged version with a fix. One workaround for now, > suggested in the StackOverflow discussion, is to use the png16m output > device instead of pngalpha. I have confirmed that this works on standalone > invocations of `gs`, but I am not sure what implications this has for > speed, memory, or the quality of thumbnails. > > Regards, > > [0] https://usn.ubuntu.com/3831-1/ > [1] > https://stackoverflow.com/questions/53560755/ghostscript-9-26-update-breaks-imagick-readimage-for-multipage-pdf > [2] https://bugs.ghostscript.com/show_bug.cgi?id=699815 > > On Fri, Oct 12, 2018 at 6:44 PM Alan Orth wrote: > >> Great idea, Tim! I just logged it in Jira as DS-4052. >> >> https://jira.duraspace.org/browse/DS-4052 >> >> Regards, >> >> On Thu, Oct 11, 2018 at 7:23 PM Tim Donohue >> wrote: >> >>> Alan, >>> >>> Could you create a "documentation" ticket for this in our JIRA system >>> (as that's sometimes an easier place to find "known issues")? This sounds >>> like something we should get into the official documentation, and having a >>> ticket to remind us (and as a place to add additional questions/comments) >>> would be useful. >>> >>> Thanks, >>> >>> Tim >>> >>> On Thu, Oct 11, 2018 at 12:27 AM Alan Orth wrote: >>> Dear DSpace people, Recently I noticed that our PDFs were not getting thumbnails generated for them. I ran the filter-media script manually and saw an error like this: org.im4java.core.InfoException: org.im4java.core.CommandException: org.im4java.core.CommandException: identify: not authorized `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412. According to an Ubuntu Security Notice from 2018-10 the Ubuntu developers have decided to disable PDF processing by default in their ImageMagick[0] package. This affects all currently supported Ubuntu long-term support releases. If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, you will need to change your system's ImageMagick policy.xml file to re-enable PDF processing. On Ubuntu 16.04 the file is /etc/ImageMagick-6/policy.xml. Delete or comment out the following line: Regards, [0] https://usn.ubuntu.com/3785-1/ -- Alan Orth alan.o...@gmail.com https://picturingjordan.com https://englishbulgaria.net https://mjanja.ch "In heaven all the interesting people are missing." ―Friedrich Nietzsche -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical
Re: [dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
Dear list, Another recent update in Debian / Ubuntu has broken PDF thumbnail generation in DSpace 5 and 6, though this time it is the Ghostscript package rather than ImageMagick. You should see errors like this when you try to run filter-media: org.im4java.core.InfoException: org.im4java.core.CommandException: org.im4java.core.CommandException: identify: FailedToExecuteCommand `"gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=5 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=4 -dGraphicsAlphaBits=4 "-r72x72" -dFirstPage=1 -dLastPage=1 "-sOutputFile=/tmp/magick-12989PcFN0DnJOej7%d" "-f/tmp/magick-129895Bmp44lvUfxo" "-f/tmp/magick-12989C0QFG51fktLF"' (-1) @ error/delegate.c/ExternalDelegateCommand/461. See Ubuntu Security Notice 3831-1[0] as well as a StackOverflow discussion from this past weekend[1]. In this case it seems that there is a bug[2] in the upstream Ghostscript project related to the pngalpha device and there is currently no packaged version with a fix. One workaround for now, suggested in the StackOverflow discussion, is to use the png16m output device instead of pngalpha. I have confirmed that this works on standalone invocations of `gs`, but I am not sure what implications this has for speed, memory, or the quality of thumbnails. Regards, [0] https://usn.ubuntu.com/3831-1/ [1] https://stackoverflow.com/questions/53560755/ghostscript-9-26-update-breaks-imagick-readimage-for-multipage-pdf [2] https://bugs.ghostscript.com/show_bug.cgi?id=699815 On Fri, Oct 12, 2018 at 6:44 PM Alan Orth wrote: > Great idea, Tim! I just logged it in Jira as DS-4052. > > https://jira.duraspace.org/browse/DS-4052 > > Regards, > > On Thu, Oct 11, 2018 at 7:23 PM Tim Donohue > wrote: > >> Alan, >> >> Could you create a "documentation" ticket for this in our JIRA system (as >> that's sometimes an easier place to find "known issues")? This sounds like >> something we should get into the official documentation, and having a >> ticket to remind us (and as a place to add additional questions/comments) >> would be useful. >> >> Thanks, >> >> Tim >> >> On Thu, Oct 11, 2018 at 12:27 AM Alan Orth wrote: >> >>> Dear DSpace people, >>> >>> Recently I noticed that our PDFs were not getting thumbnails generated >>> for them. I ran the filter-media script manually and saw an error like this: >>> >>> org.im4java.core.InfoException: org.im4java.core.CommandException: >>> org.im4java.core.CommandException: identify: not authorized >>> `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412. >>> >>> According to an Ubuntu Security Notice from 2018-10 the Ubuntu >>> developers have decided to disable PDF processing by default in their >>> ImageMagick[0] package. This affects all currently supported Ubuntu >>> long-term support releases. >>> >>> If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, >>> you will need to change your system's ImageMagick policy.xml file to >>> re-enable PDF processing. On Ubuntu 16.04 the file is >>> /etc/ImageMagick-6/policy.xml. Delete or comment out the following line: >>> >>> >>> >>> Regards, >>> >>> [0] https://usn.ubuntu.com/3785-1/ >>> -- >>> Alan Orth >>> alan.o...@gmail.com >>> https://picturingjordan.com >>> https://englishbulgaria.net >>> https://mjanja.ch >>> "In heaven all the interesting people are missing." ―Friedrich Nietzsche >>> >>> -- >>> All messages to this mailing list should adhere to the DuraSpace Code of >>> Conduct: https://duraspace.org/about/policies/code-of-conduct/ >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech+unsubscr...@googlegroups.com. >>> To post to this group, send email to dspace-tech@googlegroups.com. >>> Visit this group at https://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> Tim Donohue >> Technical Lead for DSpace & DSpaceDirect >> DuraSpace.org | DSpace.org | DSpaceDirect.org >> > > > -- > Alan Orth > alan.o...@gmail.com > https://picturingjordan.com > https://englishbulgaria.net > https://mjanja.ch > "In heaven all the interesting people are missing." ―Friedrich Nietzsche > -- Alan Orth alan.o...@gmail.com https://picturingjordan.com https://englishbulgaria.net https://mjanja.ch "In heaven all the interesting people are missing." ―Friedrich Nietzsche -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at
Re: [dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
Great idea, Tim! I just logged it in Jira as DS-4052. https://jira.duraspace.org/browse/DS-4052 Regards, On Thu, Oct 11, 2018 at 7:23 PM Tim Donohue wrote: > Alan, > > Could you create a "documentation" ticket for this in our JIRA system (as > that's sometimes an easier place to find "known issues")? This sounds like > something we should get into the official documentation, and having a > ticket to remind us (and as a place to add additional questions/comments) > would be useful. > > Thanks, > > Tim > > On Thu, Oct 11, 2018 at 12:27 AM Alan Orth wrote: > >> Dear DSpace people, >> >> Recently I noticed that our PDFs were not getting thumbnails generated >> for them. I ran the filter-media script manually and saw an error like this: >> >> org.im4java.core.InfoException: org.im4java.core.CommandException: >> org.im4java.core.CommandException: identify: not authorized >> `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412. >> >> According to an Ubuntu Security Notice from 2018-10 the Ubuntu developers >> have decided to disable PDF processing by default in their ImageMagick[0] >> package. This affects all currently supported Ubuntu long-term support >> releases. >> >> If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, >> you will need to change your system's ImageMagick policy.xml file to >> re-enable PDF processing. On Ubuntu 16.04 the file is >> /etc/ImageMagick-6/policy.xml. Delete or comment out the following line: >> >> >> >> Regards, >> >> [0] https://usn.ubuntu.com/3785-1/ >> -- >> Alan Orth >> alan.o...@gmail.com >> https://picturingjordan.com >> https://englishbulgaria.net >> https://mjanja.ch >> "In heaven all the interesting people are missing." ―Friedrich Nietzsche >> >> -- >> All messages to this mailing list should adhere to the DuraSpace Code of >> Conduct: https://duraspace.org/about/policies/code-of-conduct/ >> --- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech+unsubscr...@googlegroups.com. >> To post to this group, send email to dspace-tech@googlegroups.com. >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> > -- > Tim Donohue > Technical Lead for DSpace & DSpaceDirect > DuraSpace.org | DSpace.org | DSpaceDirect.org > -- Alan Orth alan.o...@gmail.com https://picturingjordan.com https://englishbulgaria.net https://mjanja.ch "In heaven all the interesting people are missing." ―Friedrich Nietzsche -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
Alan, Could you create a "documentation" ticket for this in our JIRA system (as that's sometimes an easier place to find "known issues")? This sounds like something we should get into the official documentation, and having a ticket to remind us (and as a place to add additional questions/comments) would be useful. Thanks, Tim On Thu, Oct 11, 2018 at 12:27 AM Alan Orth wrote: > Dear DSpace people, > > Recently I noticed that our PDFs were not getting thumbnails generated for > them. I ran the filter-media script manually and saw an error like this: > > org.im4java.core.InfoException: org.im4java.core.CommandException: > org.im4java.core.CommandException: identify: not authorized > `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412. > > According to an Ubuntu Security Notice from 2018-10 the Ubuntu developers > have decided to disable PDF processing by default in their ImageMagick[0] > package. This affects all currently supported Ubuntu long-term support > releases. > > If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, you > will need to change your system's ImageMagick policy.xml file to re-enable > PDF processing. On Ubuntu 16.04 the file is /etc/ImageMagick-6/policy.xml. > Delete or comment out the following line: > > > > Regards, > > [0] https://usn.ubuntu.com/3785-1/ > -- > Alan Orth > alan.o...@gmail.com > https://picturingjordan.com > https://englishbulgaria.net > https://mjanja.ch > "In heaven all the interesting people are missing." ―Friedrich Nietzsche > > -- > All messages to this mailing list should adhere to the DuraSpace Code of > Conduct: https://duraspace.org/about/policies/code-of-conduct/ > --- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Tim Donohue Technical Lead for DSpace & DSpaceDirect DuraSpace.org | DSpace.org | DSpaceDirect.org -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] Public service announcement: Recent Ubuntu ImageMagick update breaks PDF thumbnails
Dear DSpace people, Recently I noticed that our PDFs were not getting thumbnails generated for them. I ran the filter-media script manually and saw an error like this: org.im4java.core.InfoException: org.im4java.core.CommandException: org.im4java.core.CommandException: identify: not authorized `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412. According to an Ubuntu Security Notice from 2018-10 the Ubuntu developers have decided to disable PDF processing by default in their ImageMagick[0] package. This affects all currently supported Ubuntu long-term support releases. If you are using DSpace 5.x or 6.x with ImageMagick thumbnail support, you will need to change your system's ImageMagick policy.xml file to re-enable PDF processing. On Ubuntu 16.04 the file is /etc/ImageMagick-6/policy.xml. Delete or comment out the following line: Regards, [0] https://usn.ubuntu.com/3785-1/ -- Alan Orth alan.o...@gmail.com https://picturingjordan.com https://englishbulgaria.net https://mjanja.ch "In heaven all the interesting people are missing." ―Friedrich Nietzsche -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
