Re: [Freevo-users] freevo 1.8 memory usage when viewing images
On 19/02/2009 10:16, Olivier Sessink said the following: Attached is a patch that should reduce the memory and speed up the display of images, it may not be 100% correct yet (lack of rotation and using the thumbnails for the full image). The thumbnails can be generated with the helper mkimagemrss and the size of the thumbnail can be controlled with: WWW_IMAGE_THUMBNAIL_SIZE. The patch applied to 1.8.3 with some fuzzy offsets. it is a good improvement.I don't get into problem with the out-of-memory killer anymore on my 512Mb box. Memory usage is now around 80% when listing a directory with photo's. Do you want a comparison with the 1.7 series (where I never had any memory problems?) No don't need the comparison with 1.7 as kaa.imlib2, which generates the thumbnails wasn't used then. I did notice that all the images are cached for later use but didn't check when they are freed from the cache. Of course caching big images would cause a memory problem. It should be a lot faster too, it takes quite a long time for imlib2 to process a large photo (3072x2048) about 1.5 secs on my P3 box which is a performance killer. Duncan -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
On Fri, 2009-02-20 at 10:55 +0100, Duncan Webb wrote: I did notice that all the images are cached for later use but didn't check when they are freed from the cache. Of course caching big images would cause a memory problem. kaa.imlib2 has two layers of cache: one for uncompressed data (16MB), and the one built into imlib2 for compressed data (4MB). The uncompressed data cache can be tuned: kaa.imlib2._image_cache['max-size'] = number of bytes Now that I look at the code, max-size is a bit of a lie. It will actually cache one image, even if that image exceeds the max size. So an extremely large image (3072x2048 in your example) will actually use an extra 24MB of memory. You can, however, bypass the uncompressed image cache by using kaa.imlib2.open_without_cache() It should be a lot faster too, it takes quite a long time for imlib2 to process a large photo (3072x2048) about 1.5 secs on my P3 box which is a performance killer. beacon uses epeg (which is nowadays merged with evas, but beacon still has it split out), which is probably the fastest JPEG thumbnailer around. Jason. -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Jason Tackaberry wrote: On Fri, 2009-02-20 at 10:55 +0100, Duncan Webb wrote: It should be a lot faster too, it takes quite a long time for imlib2 to process a large photo (3072x2048) about 1.5 secs on my P3 box which is a performance killer. beacon uses epeg (which is nowadays merged with evas, but beacon still has it split out), which is probably the fastest JPEG thumbnailer around. Beacon's epeg code is around 4 times faster than imlib2 (so my changelog says). And it looks even faster since beacon is doing all the stuff in the background and you do not have to wait. Dischi -- I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone. -- Bjarne Stroustrup -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Attached is a patch that should reduce the memory and speed up the display of images, it may not be 100% correct yet (lack of rotation and using the thumbnails for the full image). The thumbnails can be generated with the helper mkimagemrss and the size of the thumbnail can be controlled with: WWW_IMAGE_THUMBNAIL_SIZE. The patch applied to 1.8.3 with some fuzzy offsets. it is a good improvement.I don't get into problem with the out-of-memory killer anymore on my 512Mb box. Memory usage is now around 80% when listing a directory with photo's. Do you want a comparison with the 1.7 series (where I never had any memory problems?) Olivier -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Duncan Webb wrote:
Francisco Eduardo Álvarez Solano wrote:
A quick test on a directory of 111 photos (2816x2112) the memory goes
up from 260MB to 540MB until the cursor is being moved around to the
next page then memory goes up to 720MB. Moving back a page then the
memory remains at 720MB.
Then going to audio the memory goes down to 715MB.
Not sure quite what the problem is, can you give a bit more detail?
I've made a directory with 63 photos (3264x2448). When I enter the
directory with Freevo 1.8.3, the memory raises to 813-900Mb, and when
I try to go to the next page, Freevo crashes. If I try with a
directory with less photos, changing from one page to another lasts
several seconds.
If I do the same test with Freevo 1.8.2, the memory footprint stands
without change, and I can navigate through the directory quickly and
without any problem.
I'll send several screenshots of Freevo 1.8.3 and Freevo 1.8.2 with
gnome system monitor in which you can see the memory and cpu
footprint.
When I check the code differences between 1.8.2 and 1.8.3 there are
quite a few changes to the osd.py module that deals with images. Will
you try taking the 1.8.2 osd.py module and replace it the 1.8.3
site-packages (remove the osd.pyc) and then see what happens?
Attached is a patch that deletes temporary images, which may help.
Attached is a patch that should reduce the memory and speed up the
display of images, it may not be 100% correct yet (lack of rotation and
using the thumbnails for the full image).
The thumbnails can be generated with the helper mkimagemrss and the
size of the thumbnail can be controlled with: WWW_IMAGE_THUMBNAIL_SIZE.
The patch applied to 1.8.3 with some fuzzy offsets.
Will you try this and provide some information if it helps.
Duncan
Index: src/skins/main/skin_utils.py
===
--- src/skins/main/skin_utils.py (revision 11318)
+++ src/skins/main/skin_utils.py (working copy)
@@ -66,7 +66,8 @@
@benchmark(benchmarking 0x08, benchmarkcall)
def format_image(settings, item, width, height, force=0, anamorphic=0):
-#print 'format_image(settings=%r, item=%r, width=%r, height=%r, force=%r, anamorphic=%r)' % (settings, item, width, height, force, anamorphic)
+_debug_('format_image(settings=%r, item=%r, width=%r, height=%r, force=%r, anamorphic=%r)' % \
+(settings, item, width, height, force, anamorphic), 2)
try:
type = item.display_type
@@ -79,7 +80,7 @@
type = ''
-item_image=Unicode(item.image)
+item_image = Unicode(item.image)
cname = '%s-%s-%s-%s-%s-%s-%s' % (settings.icon_dir, item_image, type, item.type, width, height, force)
@@ -104,15 +105,23 @@
if not os.path.exists(item.image):
return None, 0, 0
+for folder in ('.thumbs', '.images'):
+image_parts = os.path.split(item.image)
+imagefile = os.path.join(image_parts[0], folder, image_parts[1])
+if os.path.exists(imagefile):
+break
+else:
+imagefile = item.image
+print 'thumb://%s' % (item.image,)
image = load_imagecache['thumb://%s' % item.image]
if not image:
-image = osd.loadbitmap(item.image)
+image = osd.loadbitmap(imagefile)
load_imagecache['thumb://%s' % item.image] = image
if not item['rotation']:
try:
-f=open(item.image, 'rb')
-tags=exif.process_file(f)
+f = open(imagefile, 'rb')
+tags = exif.process_file(f)
f.close()
if tags.has_key('Image Orientation'):
orientation = tags['Image Orientation']
--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Francisco Eduardo Álvarez Solano wrote:
A quick test on a directory of 111 photos (2816x2112) the memory goes
up from 260MB to 540MB until the cursor is being moved around to the
next page then memory goes up to 720MB. Moving back a page then the
memory remains at 720MB.
Then going to audio the memory goes down to 715MB.
Not sure quite what the problem is, can you give a bit more detail?
I've made a directory with 63 photos (3264x2448). When I enter the
directory with Freevo 1.8.3, the memory raises to 813-900Mb, and when
I try to go to the next page, Freevo crashes. If I try with a
directory with less photos, changing from one page to another lasts
several seconds.
If I do the same test with Freevo 1.8.2, the memory footprint stands
without change, and I can navigate through the directory quickly and
without any problem.
I'll send several screenshots of Freevo 1.8.3 and Freevo 1.8.2 with
gnome system monitor in which you can see the memory and cpu
footprint.
When I check the code differences between 1.8.2 and 1.8.3 there are
quite a few changes to the osd.py module that deals with images. Will
you try taking the 1.8.2 osd.py module and replace it the 1.8.3
site-packages (remove the osd.pyc) and then see what happens?
Attached is a patch that deletes temporary images, which may help.
Thanks
Duncan
Index: src/osd.py
===
--- src/osd.py (revision 11313)
+++ src/osd.py (working copy)
@@ -790,6 +790,7 @@
except:
data = util.create_thumbnail(filename)
image = pygame.image.fromstring(data[0], data[1], data[2])
+del data
else:
try:
@@ -799,6 +800,7 @@
try:
i = kaa.imlib2.open(filename)
image = pygame.image.fromstring(i.tostring(), i.size, i.mode)
+del i
except IOError, why:
_debug_('imlib2 image load problem: %s' % (why), DERROR)
return None
--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
I've recently tried Freevo 1.8.2 and 1.8.3 in a freshly installed Ubuntu 8.10 (Intel Core 2 6240, 1Gb RAM, ATI Radeon X300/X550/X1050 Series with 256Mb). Freevo 1.8.3 crashes when viewing a directory with 20 images in thumbnail mode: the memory footprint raises continously since you enter the directory unti Freevo crashes. Freevo 1.8.2 has no problem. It seems there is a bug in Freevo 1.8.3 -- Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Francisco Eduardo Álvarez Solano wrote: I've recently tried Freevo 1.8.2 and 1.8.3 in a freshly installed Ubuntu 8.10 (Intel Core 2 6240, 1Gb RAM, ATI Radeon X300/X550/X1050 Series with 256Mb). Freevo 1.8.3 crashes when viewing a directory with 20 images in thumbnail mode: the memory footprint raises continously since you enter the directory unti Freevo crashes. Freevo 1.8.2 has no problem. It seems there is a bug in Freevo 1.8.3 So you are saying that when you enter a directory with photos and just sit with the thumbnails being displayed then the memory usage creeps up? A quick test on a directory of 111 photos (2816x2112) the memory goes up from 260MB to 540MB until the cursor is being moved around to the next page then memory goes up to 720MB. Moving back a page then the memory remains at 720MB. Then going to audio the memory goes down to 715MB. Not sure quite what the problem is, can you give a bit more detail? Duncan -- Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
I had a brief look at this when you reported the problem. Wasn't sure if this was a memory leak or not but noticed that freevo holds onto the images. Of course with modern cameras images can be very large 3072x2048x4 bytes (24MB) per image so a few images will use quite a bit of memory and will take quite a long time to display as they need rescaling to fit a screen. But I have some folders with several hundred images and never run out of memory. freevo 1.7 ran fine with 512Mb of memory, freevo 1.8.3 is hardly usable with 512Mb of memory... what changed? Olivier -- Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
On 04/02/2009 17:05, Olivier Sessink said the following: nobody else having memory issues with photo viewing and freevo 1.8.3 ? regards, Olivier Olivier Sessink wrote: Jason Tackaberry wrote: On Thu, 2009-01-22 at 20:21 +0100, Olivier Sessink wrote: memory.. It's running freevo dedicated, so besides the X server and a ssh daemon there is nothing else using memory, still freevo is killed by the out-of-memory killer. What changed? Can I tune the memory usage Smells like a memory leak. Does freevo's memory usage increase steadily as you flip through images? immediately when opening a directory with images the memory usage rises skyhigh. without any images, directly after startup: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 76084 40m 8088 S1 8.4 0:01.85 python after opening a directory with 114 5megapixel images (9 images shown): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 251m 218m 8152 S3 46.0 0:10.49 python after scrolling down the first page (again 9 images, of which 6 new): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 370m 336m 8152 S2 70.8 0:15.38 python after scrolling down the second page: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 385m 351m 8152 S1 74.0 0:20.14 python after opening a first image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 422m 388m 8160 S 19 82.0 0:21.97 python after opening the second image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 418m 384m 8160 S 54 81.2 0:25.35 python hit escape, escape, choose a directory with 149 images: *crash* (more than 515MB of memory used) for further information: the system is a Linux Debian unstable system, with freevo-1.8.3 installed from source. It runs on a VIA epia board 1.3GHz Via C7 with 512 MB memory. It boots via PXE and runs root-over-NFS (it doesn't have a harddisk). Display is provided by the TV out of the Unichrome X11 driver. I had a brief look at this when you reported the problem. Wasn't sure if this was a memory leak or not but noticed that freevo holds onto the images. Of course with modern cameras images can be very large 3072x2048x4 bytes (24MB) per image so a few images will use quite a bit of memory and will take quite a long time to display as they need rescaling to fit a screen. But I have some folders with several hundred images and never run out of memory. Duncan -- Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
nobody else having memory issues with photo viewing and freevo 1.8.3 ? regards, Olivier Olivier Sessink wrote: Jason Tackaberry wrote: On Thu, 2009-01-22 at 20:21 +0100, Olivier Sessink wrote: memory.. It's running freevo dedicated, so besides the X server and a ssh daemon there is nothing else using memory, still freevo is killed by the out-of-memory killer. What changed? Can I tune the memory usage Smells like a memory leak. Does freevo's memory usage increase steadily as you flip through images? immediately when opening a directory with images the memory usage rises skyhigh. without any images, directly after startup: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 76084 40m 8088 S1 8.4 0:01.85 python after opening a directory with 114 5megapixel images (9 images shown): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 251m 218m 8152 S3 46.0 0:10.49 python after scrolling down the first page (again 9 images, of which 6 new): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 370m 336m 8152 S2 70.8 0:15.38 python after scrolling down the second page: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 385m 351m 8152 S1 74.0 0:20.14 python after opening a first image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 422m 388m 8160 S 19 82.0 0:21.97 python after opening the second image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 418m 384m 8160 S 54 81.2 0:25.35 python hit escape, escape, choose a directory with 149 images: *crash* (more than 515MB of memory used) for further information: the system is a Linux Debian unstable system, with freevo-1.8.3 installed from source. It runs on a VIA epia board 1.3GHz Via C7 with 512 MB memory. It boots via PXE and runs root-over-NFS (it doesn't have a harddisk). Display is provided by the TV out of the Unichrome X11 driver. regards, Olivier -- Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
[Freevo-users] freevo 1.8 memory usage when viewing images
Hi all, I just upgraded the box under my TV from freevo 1.7 to 1.8.3. Everything seems to work fine, but there is one huge regression: the memory usage when viewing images is increased a lot, and my box has only 512MB memory.. It's running freevo dedicated, so besides the X server and a ssh daemon there is nothing else using memory, still freevo is killed by the out-of-memory killer. What changed? Can I tune the memory usage down? Do I need to debug something? thanks, Olivier -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
On Thu, 2009-01-22 at 20:21 +0100, Olivier Sessink wrote: memory.. It's running freevo dedicated, so besides the X server and a ssh daemon there is nothing else using memory, still freevo is killed by the out-of-memory killer. What changed? Can I tune the memory usage Smells like a memory leak. Does freevo's memory usage increase steadily as you flip through images? -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] freevo 1.8 memory usage when viewing images
Jason Tackaberry wrote: On Thu, 2009-01-22 at 20:21 +0100, Olivier Sessink wrote: memory.. It's running freevo dedicated, so besides the X server and a ssh daemon there is nothing else using memory, still freevo is killed by the out-of-memory killer. What changed? Can I tune the memory usage Smells like a memory leak. Does freevo's memory usage increase steadily as you flip through images? immediately when opening a directory with images the memory usage rises skyhigh. without any images, directly after startup: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 76084 40m 8088 S1 8.4 0:01.85 python after opening a directory with 114 5megapixel images (9 images shown): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 251m 218m 8152 S3 46.0 0:10.49 python after scrolling down the first page (again 9 images, of which 6 new): PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 370m 336m 8152 S2 70.8 0:15.38 python after scrolling down the second page: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 385m 351m 8152 S1 74.0 0:20.14 python after opening a first image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 422m 388m 8160 S 19 82.0 0:21.97 python after opening the second image: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31275 olivier 20 0 418m 384m 8160 S 54 81.2 0:25.35 python hit escape, escape, choose a directory with 149 images: *crash* (more than 515MB of memory used) for further information: the system is a Linux Debian unstable system, with freevo-1.8.3 installed from source. It runs on a VIA epia board 1.3GHz Via C7 with 512 MB memory. It boots via PXE and runs root-over-NFS (it doesn't have a harddisk). Display is provided by the TV out of the Unichrome X11 driver. regards, Olivier -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
