RE: Fop Memory Leak
Eric- On the PNGRenderer code, the method stopRenderer() does not run a similar cleanup that TIFFRenderer (which also extends Java2DRenderer) calls. At the end of the TIFFRenderer method, clearViewportList(); is called. Can you try extending PNGRenderer with a custom renderer that overrides the stopRenderer() method with public void stopRenderer() throws IOException { super.stopRenderer(); clearViewportList(); } And see if that helps? I don't see it in trunk, and this is 100% guesswork on my part, and it could completely screw up PNGs created, but it might work. J -Josh From: Eric Douglas [mailto:edoug...@blockhouse.com] Sent: Thursday, May 26, 2011 10:03 AM To: fop-dev@xmlgraphics.apache.org Subject: Fop Memory Leak I could use some help tracking this down. I created a Print Preview program but it just runs out of memory after a dozen or two pages. I started with a program which generates XML data, transforms it into FO data, then transforms that. There are 2 transforms in it, one to create a PDF and one for Print Preview. When I tried to create a large FO and generate a PDF of 1000+ pages it crashed, out of memory. I changed it to break the output. It now stores an array of FO files for no more than 10 pages each, using XSL's intiial-page-number to store a starting point for each document fragment. To create a PDF I call the transformer to get the PDF from the Fop output stream in byte format for each FO. Then I load them into PDF objects using the pfdbox project, create a new PDF with it and copy in the pages to merge them. This works great for PDF. I was able to generate a report this way of over 1800 pages. Now I try it for print preview, generating 10 pages at a time, calling the render process again when a new size or a page from a different block is requested. The memory use never drops on these calls but in 2 places I see it jumping up, whenever it calls Transformer.transform with a PNGRenderer, and whenever I try to create a new image for a page even though I'm only generating 10 pages and on each render I replace the previous array of images. If I do something as simple as this the memory jumps up sometimes on the create. myBufferedImage = myImageArray.get(pageNum); myBufferedImage.flush(); myBufferedImage = new BufferedImage(sameWidthAsOldImage,sameHeightAsOldImage); It does this if I call PNGRenderer.getPageImage(pageNum) (from Java2DRenderer), so I tried creating the BufferedImage (or VolatileImage same issue) myself and using the Java2DRenderer.print method to draw the page on it. Memory use keeps increasing on the image create and on the transform. The transform is all done in one method, with only the FopFactory and TransformerFactory remaining (created static final to the class) after the method so a second call shouldn't use any more than the first. - The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify First Data immediately by replying to this message and deleting it from your computer.
RE: Fop Memory Leak
That would defeat the purpose. It appears stopRenderer gets called in the transform method. After the transform I need to get the Viewport values to generate page images. It appears the transform increases memory use temporarily, releasing it when done. It was just increasing my heap size on subsequent calls because I was using additional memory between calls so I was closer to the limit. The additional memory was in the class I was using to call the class doing the rendering. I was getting a copy of the image in an array used to display on a GUI window. I wasn't clearing that array so it was holding the old references. My memory problem now is just in the proprietary API I'm using (an extension of Oracle's webstart) where I do that rendering on a server then serialize it to a client to display. That render hits a memory limit I'm not sure why. From: Marquart, Joshua D [mailto:joshua.marqu...@firstdata.com] Sent: Friday, May 27, 2011 11:08 AM To: fop-dev@xmlgraphics.apache.org Subject: RE: Fop Memory Leak Eric- On the PNGRenderer code, the method stopRenderer() does not run a similar cleanup that TIFFRenderer (which also extends Java2DRenderer) calls. At the end of the TIFFRenderer method, clearViewportList(); is called. Can you try extending PNGRenderer with a custom renderer that overrides the stopRenderer() method with public void stopRenderer() throws IOException { super.stopRenderer(); clearViewportList(); } And see if that helps? I don't see it in trunk, and this is 100% guesswork on my part, and it could completely screw up PNGs created, but it might work. J -Josh From: Eric Douglas [mailto:edoug...@blockhouse.com] Sent: Thursday, May 26, 2011 10:03 AM To: fop-dev@xmlgraphics.apache.org Subject: Fop Memory Leak I could use some help tracking this down. I created a Print Preview program but it just runs out of memory after a dozen or two pages. I started with a program which generates XML data, transforms it into FO data, then transforms that. There are 2 transforms in it, one to create a PDF and one for Print Preview. When I tried to create a large FO and generate a PDF of 1000+ pages it crashed, out of memory. I changed it to break the output. It now stores an array of FO files for no more than 10 pages each, using XSL's intiial-page-number to store a starting point for each document fragment. To create a PDF I call the transformer to get the PDF from the Fop output stream in byte format for each FO. Then I load them into PDF objects using the pfdbox project, create a new PDF with it and copy in the pages to merge them. This works great for PDF. I was able to generate a report this way of over 1800 pages. Now I try it for print preview, generating 10 pages at a time, calling the render process again when a new size or a page from a different block is requested. The memory use never drops on these calls but in 2 places I see it jumping up, whenever it calls Transformer.transform with a PNGRenderer, and whenever I try to create a new image for a page even though I'm only generating 10 pages and on each render I replace the previous array of images. If I do something as simple as this the memory jumps up sometimes on the create. myBufferedImage = myImageArray.get(pageNum); myBufferedImage.flush(); myBufferedImage = new BufferedImage(sameWidthAsOldImage,sameHeightAsOldImage); It does this if I call PNGRenderer.getPageImage(pageNum) (from Java2DRenderer), so I tried creating the BufferedImage (or VolatileImage same issue) myself and using the Java2DRenderer.print method to draw the page on it. Memory use keeps increasing on the image create and on the transform. The transform is all done in one method, with only the FopFactory and TransformerFactory remaining (created static final to the class) after the method so a second call shouldn't use any more than the first. The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify First Data immediately by replying to this message and deleting it from your computer.
Fop Memory Leak
I could use some help tracking this down. I created a Print Preview program but it just runs out of memory after a dozen or two pages. I started with a program which generates XML data, transforms it into FO data, then transforms that. There are 2 transforms in it, one to create a PDF and one for Print Preview. When I tried to create a large FO and generate a PDF of 1000+ pages it crashed, out of memory. I changed it to break the output. It now stores an array of FO files for no more than 10 pages each, using XSL's intiial-page-number to store a starting point for each document fragment. To create a PDF I call the transformer to get the PDF from the Fop output stream in byte format for each FO. Then I load them into PDF objects using the pfdbox project, create a new PDF with it and copy in the pages to merge them. This works great for PDF. I was able to generate a report this way of over 1800 pages. Now I try it for print preview, generating 10 pages at a time, calling the render process again when a new size or a page from a different block is requested. The memory use never drops on these calls but in 2 places I see it jumping up, whenever it calls Transformer.transform with a PNGRenderer, and whenever I try to create a new image for a page even though I'm only generating 10 pages and on each render I replace the previous array of images. If I do something as simple as this the memory jumps up sometimes on the create. myBufferedImage = myImageArray.get(pageNum); myBufferedImage.flush(); myBufferedImage = new BufferedImage(sameWidthAsOldImage,sameHeightAsOldImage); It does this if I call PNGRenderer.getPageImage(pageNum) (from Java2DRenderer), so I tried creating the BufferedImage (or VolatileImage same issue) myself and using the Java2DRenderer.print method to draw the page on it. Memory use keeps increasing on the image create and on the transform. The transform is all done in one method, with only the FopFactory and TransformerFactory remaining (created static final to the class) after the method so a second call shouldn't use any more than the first.