Re: [pygame] Numeric Question
On 8 Jul 2006 at 20:51, Kamilche wrote: Peter Shinners wrote: On Sat, 2006-07-08 at 20:04 -0700, Kamilche wrote: Hey Pete, I have another Numeric question for you, if you've got the time. Let's say I have a bunch of data in RGBA format, where is an arbitrary floating point number, and RGBA are unsigned bytes. The following routine will convert that to an RGBA picture, but it takes a long, time. What's the magical incantation to get Numeric to assign the data directly to the pixels of the surface? I'm not sure if that can be done actually. Numeric may have a way to do casting between data types, but I'm not really sure. I was thinking Numeric.array_constructor could bind an arbitrary sized and typed array onto a string buffer of data. But it looks like that does not work. You'll need a function like that in order for this to work. I know in Python 2.5 the struct module can precompile your formatting codes, which would speed this up. I also see that you unpack an a value out of the binary data, but then throw it away. Also, since you are doing such simple handling of the float data, you could probably just treat the whole pixel as a bunch of bytes. All this will help a bit, but won't make it orders of magnitude faster. xspan = range(w) ordfunc = ord setfunc = pic.set_at for y in range(h): for x in xspan: a = (ordfunc(data[ctr + 3]) 0x7f 0x46 and 255 or 0 setfunc((x, y), ( ordfunc(data[ctr + 4], ordfunc(data[ctr + 5], ordfunc(data[ctr + 6], a)) ctr += 8 Sadly, I'm kind of wildly guessing here if that code to compare with 0x46 will match a floating point value with less than 6 digits left of the decimal. But it works in some simplish tests. Darn. Well, that explains why my attempts to get it to work failed, then. Thanks for the info! This is the file format for WorldBuilder ZBF files. There's no way to render an alpha channel, but there IS a way to render with depth information. That's that float number embedded in there. (The alpha channel contains all 0's, even if you specify 'render with alpha channels.') That simple check is in essence saying 'if the pixel is near, it's opaque; if it's far, it's transparent', but that doesn't give you antialiasing on the picture. So, I render it too big by a factor of 3, apply the near/far algorithm, then shrink the result down. Then, FINALLY, it gives me what I want. It would be far easier if WorldBuilder would just render with alpha channels in the first place. :-P Possibly this works. I have no zbf files to test it on. # Array for extracting rgb information rgb = Numeric.fromstring(data, Numeric.UInt8) rgb.shape = (w, h, 8) # Array for extracting distances d = Numeric.fromstring(data, Numeric.Float32) d.shape = (w, h, 2) # Create alpha plane a = ((d[:,:,0] 10.0) * 255).astype(Numeric.UInt8) # Combine rgb and alpha information pic = pygame.Surface((w, h), pygame.SRCALPHA, 32) pygame.surfarray.blit_array(pic, rgb[:,:,4:7]) pygame.surfarray.pixels_alpha(pic)[:,:] = a There may be room for improvement. Possibly pygame.surfarray.make_surface() when passed a (w, h, 4) array will create a surface with alpha. I just don't know. The 3d array routines ignore alpha. Lenard Lindstrom [EMAIL PROTECTED]
Re: [pygame] Frantic memory usage
Awesome catch, Brian. I thought I was removing the artifacts managers from all containers when they were through, but apparently there's a bug in that code. Even so, I still should be caching/removing the parsed filmstrips. I'll rework the image loading/unloading today and hopefully release a better version tonight. Thanks very much, Dave p.s. I told you it was something braindead On Tue, 11 Jul 2006 01:49:50 -0700, Brian Fisher [EMAIL PROTECTED] said: One memory-related thing I noticed right away is that each time I start a new game, another 40MB gets allocated (so it's leaking mem). It looks like each time you load a level you create new collision artifacts managers, but the old managers get kept around. Each of those managers finds the filmstrip it needs and splits it up into surfaces. You don't load more than one copy of the underlying .png's due to your cache, but each manager happily creates it's own surfaces from the film strip (so you end up with lots of copies of the animations). You may want to think about caching the split filmstrips, instead of caching the .pngs, and not storing the loaded pngs at all (what do you need them for after you've split them anyways?) It looks like the explosion animations in particular take like 10MB a piece... which is no suprise at 14000x200 for the filmstrip (14000x200x4bytes per pixel ~ 10MB) In case you are curious, I was able to find where the 40MB for level loading is being allocated by binary search - basically run the game a few times, inserting a bunch of prints and time.sleep(2) calls in order to give me a chance to look at memory usage in the task manager after large blocks of code, before new stuff ran. It showed that the mem was allocated in load_level_objects in game_scene.py: self.collisions = levels.get_collision_table() for o in game['collision_artifacts']: if o['load'] == self.level: manager_name = o['name'] is_collidable = o['collidable'] log.write_debug_msg( Loading %s % manager_name ) manager_class = import_by_name( %s.%s % ( manager_name, manager_name ) ) manager = manager_class() self.collision_artifact_managers[manager_name] = manager self.game_object_managers.append( manager ) if is_collidable: self.collidable_collision_artifact_managers.append( manager ) what happens there, is that even though the old manager is not in the collision_artifact_managers dict anymore, it's never removed from the game_object_managers or collidable_collision_artifact_managers lists, which is why they keep their memory around. (as a test, I made sure to delete the old managers from those lists if they were already in the dict, and then loading the level again didn't leak memory anymore) if (manager_name in self.collision_artifact_managers): old_manager = self.collision_artifact_managers[manager_name] self.game_object_managers.remove(old_manager) if is_collidable: self.collidable_collision_artifact_managers.remove(old_manager) print skipping, manager_name On 7/10/06, David Mikesell [EMAIL PROTECTED] wrote: I found some image filmstrips I was not unloading after they're cut up into image arrays, and it freed up a decent amount of memory, but my game is still a hog. I totalled the amount of disk space for all images used by the starship (different weapons, explosions, thruster, etc.), and it's 600Kb. However, when I comment out the line that loads the starship_manager, the memory usage drops by a whopping 17Mb. It's even worse for the first enemy, which is loaded when the first level begins. 355Kb of images, but the memory usage skyrockets another 40Mb when it loads on the first level. I'm doing something braindead somewhere, just haven't tracked it down yet. The code is spread across dozens of source files, but if anyone's interested it's at http://medievalelks.com/code.zip. I honestly don't expect anyone to take the time to look at such a volume, but I've been requested several times to post it so there it is. I didn't include any data files, just code. Anyway, I'm making some progress. Oh, and I scaled down the game to 640x480, which also helps the performance. Removed some redundant-sounding music loops, too. Download size is now a svelte 13Mb :-).
Re: [pygame] Frantic memory usage
Down to 5Mb of memory! I refactored what Brian suggested with the images, as well as some other messy code. More testing to do, but I'll post the new version later tonight or tomorrow. Thanks, Brian! On Tue, 11 Jul 2006 06:51:53 -0400, David Mikesell [EMAIL PROTECTED] said: Awesome catch, Brian. I thought I was removing the artifacts managers from all containers when they were through, but apparently there's a bug in that code. Even so, I still should be caching/removing the parsed filmstrips. I'll rework the image loading/unloading today and hopefully release a better version tonight. Thanks very much, Dave p.s. I told you it was something braindead On Tue, 11 Jul 2006 01:49:50 -0700, Brian Fisher [EMAIL PROTECTED] said: One memory-related thing I noticed right away is that each time I start a new game, another 40MB gets allocated (so it's leaking mem). It looks like each time you load a level you create new collision artifacts managers, but the old managers get kept around. Each of those managers finds the filmstrip it needs and splits it up into surfaces. You don't load more than one copy of the underlying .png's due to your cache, but each manager happily creates it's own surfaces from the film strip (so you end up with lots of copies of the animations). You may want to think about caching the split filmstrips, instead of caching the .pngs, and not storing the loaded pngs at all (what do you need them for after you've split them anyways?) It looks like the explosion animations in particular take like 10MB a piece... which is no suprise at 14000x200 for the filmstrip (14000x200x4bytes per pixel ~ 10MB) In case you are curious, I was able to find where the 40MB for level loading is being allocated by binary search - basically run the game a few times, inserting a bunch of prints and time.sleep(2) calls in order to give me a chance to look at memory usage in the task manager after large blocks of code, before new stuff ran. It showed that the mem was allocated in load_level_objects in game_scene.py: self.collisions = levels.get_collision_table() for o in game['collision_artifacts']: if o['load'] == self.level: manager_name = o['name'] is_collidable = o['collidable'] log.write_debug_msg( Loading %s % manager_name ) manager_class = import_by_name( %s.%s % ( manager_name, manager_name ) ) manager = manager_class() self.collision_artifact_managers[manager_name] = manager self.game_object_managers.append( manager ) if is_collidable: self.collidable_collision_artifact_managers.append( manager ) what happens there, is that even though the old manager is not in the collision_artifact_managers dict anymore, it's never removed from the game_object_managers or collidable_collision_artifact_managers lists, which is why they keep their memory around. (as a test, I made sure to delete the old managers from those lists if they were already in the dict, and then loading the level again didn't leak memory anymore) if (manager_name in self.collision_artifact_managers): old_manager = self.collision_artifact_managers[manager_name] self.game_object_managers.remove(old_manager) if is_collidable: self.collidable_collision_artifact_managers.remove(old_manager) print skipping, manager_name On 7/10/06, David Mikesell [EMAIL PROTECTED] wrote: I found some image filmstrips I was not unloading after they're cut up into image arrays, and it freed up a decent amount of memory, but my game is still a hog. I totalled the amount of disk space for all images used by the starship (different weapons, explosions, thruster, etc.), and it's 600Kb. However, when I comment out the line that loads the starship_manager, the memory usage drops by a whopping 17Mb. It's even worse for the first enemy, which is loaded when the first level begins. 355Kb of images, but the memory usage skyrockets another 40Mb when it loads on the first level. I'm doing something braindead somewhere, just haven't tracked it down yet. The code is spread across dozens of source files, but if anyone's interested it's at http://medievalelks.com/code.zip. I honestly don't expect anyone to take the time to look at such a volume, but I've been requested several times to post it so there it is. I didn't include any data files, just code. Anyway, I'm
Re: [pygame] Frantic memory usage
Rats - looked at the wrong process table entry. Had two python processes going - mine still hogging. Back to the drawing board. On Tue, 11 Jul 2006 12:05:07 -0400, David Mikesell [EMAIL PROTECTED] said: Down to 5Mb of memory! I refactored what Brian suggested with the images, as well as some other messy code. More testing to do, but I'll post the new version later tonight or tomorrow. Thanks, Brian! On Tue, 11 Jul 2006 06:51:53 -0400, David Mikesell [EMAIL PROTECTED] said: Awesome catch, Brian. I thought I was removing the artifacts managers from all containers when they were through, but apparently there's a bug in that code. Even so, I still should be caching/removing the parsed filmstrips. I'll rework the image loading/unloading today and hopefully release a better version tonight. Thanks very much, Dave p.s. I told you it was something braindead On Tue, 11 Jul 2006 01:49:50 -0700, Brian Fisher [EMAIL PROTECTED] said: One memory-related thing I noticed right away is that each time I start a new game, another 40MB gets allocated (so it's leaking mem). It looks like each time you load a level you create new collision artifacts managers, but the old managers get kept around. Each of those managers finds the filmstrip it needs and splits it up into surfaces. You don't load more than one copy of the underlying .png's due to your cache, but each manager happily creates it's own surfaces from the film strip (so you end up with lots of copies of the animations). You may want to think about caching the split filmstrips, instead of caching the .pngs, and not storing the loaded pngs at all (what do you need them for after you've split them anyways?) It looks like the explosion animations in particular take like 10MB a piece... which is no suprise at 14000x200 for the filmstrip (14000x200x4bytes per pixel ~ 10MB) In case you are curious, I was able to find where the 40MB for level loading is being allocated by binary search - basically run the game a few times, inserting a bunch of prints and time.sleep(2) calls in order to give me a chance to look at memory usage in the task manager after large blocks of code, before new stuff ran. It showed that the mem was allocated in load_level_objects in game_scene.py: self.collisions = levels.get_collision_table() for o in game['collision_artifacts']: if o['load'] == self.level: manager_name = o['name'] is_collidable = o['collidable'] log.write_debug_msg( Loading %s % manager_name ) manager_class = import_by_name( %s.%s % ( manager_name, manager_name ) ) manager = manager_class() self.collision_artifact_managers[manager_name] = manager self.game_object_managers.append( manager ) if is_collidable: self.collidable_collision_artifact_managers.append( manager ) what happens there, is that even though the old manager is not in the collision_artifact_managers dict anymore, it's never removed from the game_object_managers or collidable_collision_artifact_managers lists, which is why they keep their memory around. (as a test, I made sure to delete the old managers from those lists if they were already in the dict, and then loading the level again didn't leak memory anymore) if (manager_name in self.collision_artifact_managers): old_manager = self.collision_artifact_managers[manager_name] self.game_object_managers.remove(old_manager) if is_collidable: self.collidable_collision_artifact_managers.remove(old_manager) print skipping, manager_name On 7/10/06, David Mikesell [EMAIL PROTECTED] wrote: I found some image filmstrips I was not unloading after they're cut up into image arrays, and it freed up a decent amount of memory, but my game is still a hog. I totalled the amount of disk space for all images used by the starship (different weapons, explosions, thruster, etc.), and it's 600Kb. However, when I comment out the line that loads the starship_manager, the memory usage drops by a whopping 17Mb. It's even worse for the first enemy, which is loaded when the first level begins. 355Kb of images, but the memory usage skyrockets another 40Mb when it loads on the first level. I'm doing something braindead somewhere, just haven't tracked it down yet. The code is spread across dozens of source files, but if anyone's
Re: [pygame] Best way to access data
I had a question as to which way would be better to access data after loading it into memory. your cache looks fine On 7/11/06, Lenard Lindstrom [EMAIL PROTECTED] wrote: Leave out the unnecessary cache and assign directly to global variables. I wouldn't do things that way. Having an image cache object can be a very good thing if you want to call functions to see what is loaded, check the size of loaded images or unload all cached images. If they are just sitting in globals, it can be awkward to do those things.
[pygame] Releasing My Code
I'd like to open-source a lot of the game-related code I've been doing, in hopes of getting useful critiques like Bob's. My code is a mess, and horribly inefficient, but I've at least divided it into subsystems that might be useful: -Conch: Sound/Music -Driftwood: UI -Pearl: Isometric graphics -Aquablue: Tiled game world w/ unlimited size For the moment I've uploaded just Pearl and some graphics. This code lets you move a sprite around in a little world, but doesn't check for things like going outside the zone, overlapping other sprites, or having the proper height for the terrain. I've deliberately separated the graphics engine from the Aquablue world code, which handles all that. Anyway, the demo is at: http://kschnee.xepher.net/pearl060711.zip Sample screenshot using Pearl and the other systems together: http://kschnee.xepher.net/pics/shiningsea060630-0s.jpg I'd appreciate any advice on improving the thing and my coding in general, as I post more of the code. In return you can use the code if you find it helpful. One thing I'd love to see is an open-source Pygame tactical RPG like Disgaea ( http://hg101.classicgaming.gamespy.com/nipponichi/nipponichi2.htm ) where you store and trade characters in XML! Kris
Re: [pygame] pygame website - request for suggestions
Hey, I suppose one improvement in the website is to add some spamming control. Just a few minutes ago, I saw this game that looked new, so when I clicked it it took me to http://www.pygame.org/projects/22/266/?release_id=411 It looks like someone tried to spam some links, cause the description of the game doesnt seem to go with the picture and the links lead to random websites. great site otherwise, thanks for all your hardwork! I would be really lost without it. -spotter.
Re: [pygame] Best way to access data
On 11 Jul 2006 at 10:27, Brian Fisher wrote: I had a question as to which way would be better to access data after loading it into memory. your cache looks fine On 7/11/06, Lenard Lindstrom [EMAIL PROTECTED] wrote: Leave out the unnecessary cache and assign directly to global variables. I wouldn't do things that way. Having an image cache object can be a very good thing if you want to call functions to see what is loaded, check the size of loaded images or unload all cached images. If they are just sitting in globals, it can be awkward to do those things. Good points.The same benefit of using globals can be achieved with a: from play import cache Actually it is faster. The only other change I would suggest is making the cache key independent of the file type by dropping the '.png' suffix. Lenard Lindstrom [EMAIL PROTECTED]
Re: [pygame] pygame website - request for suggestions
Weird. I think that person just copied in a link to a DSL provider, and to lance-tech which is Phils company (I think). I've removed the links for now, but left the image. I'm not sure how to email the owner though. On 7/12/06, spotter . [EMAIL PROTECTED] wrote: Hey, I suppose one improvement in the website is to add some spamming control. Just a few minutes ago, I saw this game that looked new, so when I clicked it it took me to http://www.pygame.org/projects/22/266/?release_id=411 It looks like someone tried to spam some links, cause the description of the game doesnt seem to go with the picture and the links lead to random websites. great site otherwise, thanks for all your hardwork! I would be really lost without it. -spotter.
