Thanks for the reply and information, had no idea. That is unfortunate though, I guess the speedup would have to be done in an alternative for findall.
I tried the pypng decoder but that was over 10 times slower. Just for fun I decided to test a 'ReverseGDIPlusDecoder' where it's the same but doesn't reverse the pitch, and I just save the images upside down that way they are imported right side up. normal loader - 32x32 size - 1000 count - Took: 1.3884263192 normal loader - 2560x2560 size - 100 count - Took: 10.7336410624 reverse loader - 32x32 size - 1000 count - Took: 1.35458577926 reverse loader - 2560x2560 size - 100 count - Took: 4.4178891004 For smaller images almost no difference, but for the larger ones, it is more than a 50% speedup. Pretty interesting, not sure I'll reverse my images as it's kind of a hassle, but maybe as its just a handful. On Tuesday, July 4, 2017 at 2:44:40 AM UTC-5, Benjamin Moran wrote: > > I just made a script that only loads up a 3840 x 1080 png file. After > running it through vmprof, we can confirm your cProfile results: > http://vmprof.com/#/c0d74a65-dc84-4923-987d-4f451fadc3a6 > > I did quick print statements at the top of the _convert method, and can > see that my loaded PNG file has reversed pitch compared to what is needed: > print(format, pitch) > print(self._current_format, self._current_pitch) > >>> RGBA 15360 > >>> RGBA -15360 > > That lead me here: > https://blog.nobel-joergensen.com/2010/11/07/loading-a-png-as-texture-in-opengl-using-libpng/ > OpenGL and PNG use a different pitch by default, so it looks like the > conversion step is unavoidable. > > > > On Tuesday, July 4, 2017 at 3:49:51 PM UTC+9, Charles wrote: >> >> I am on Windows platform, I haven't tested Mac or Linux. >> >> And Benjamin that's correct, however I notice this issue just on the >> resource load. Although get_region does get slow, that's expected since the >> atlas can contain up to 4500 different regions which definitely ends up >> taking a while. (0.4 seconds) >> >> On Monday, July 3, 2017 at 10:14:03 PM UTC-5, swiftcoder wrote: >>> >>> What platform are you on? >>> >>> On some platforms the native format is BGRA, *not* RGBA (this is a >>> common problem when porting between Mac and Windows, for example). >>> >>> On Mon, 3 Jul 2017 at 18:45 Benjamin Moran <[email protected]> wrote: >>> >>>> Thanks for the code Charles, >>>> >>>> If I read it correctly, we can distill it down to something like this >>>> for benchmarking: >>>> import pyglet >>>> >>>> imageFile = pyglet.resource.image(filename) >>>> >>>> >>>> def load(): >>>> # with different x, y, w, h values: >>>> atlas1 = imageFile.get_region(x, y, w, h) >>>> atlas2 = imageFile.get_region(x, y, w, h) >>>> atlas3 = imageFile.get_region(x, y, w, h) >>>> atlas4 = imageFile.get_region(x, y, w, h) >>>> >>>> Since you tested all of the possible formats, and at least the RGBA one >>>> should be OK, maybe the _convert method is unavoidable. We'll have to dig >>>> in a little more. >>>> >>>> >>>> On Tuesday, July 4, 2017 at 7:29:27 AM UTC+9, Charles wrote: >>>>> >>>>> It should be RGBA, the image has alpha, from the settings the format >>>>> is PNG-32 and Pixel format is RGBA8888. I did some tests since the >>>>> texture >>>>> packer allows different types of formats. >>>>> >>>>> I tried a POT texture, same result. NPOT texture, same result. The >>>>> file was an indexed PNG file (to save space and size), I tried unindexed >>>>> with the same result. I can't seem to not trigger this _convert findall >>>>> function. >>>>> >>>>> As far as code this is what I am doing. >>>>> #import xml.etree.ElementTree as ET >>>>> from lxml import etree as ET >>>>> import pyglet >>>>> >>>>> class Atlas(object): >>>>> def __init__(self, filename, default=None): >>>>> tree = ET.parse(pyglet.resource.file(filename +".xml")) >>>>> self.xml = tree.getroot().findall("sprite") >>>>> self.imageFile = pyglet.resource.image(filename+".png") >>>>> self.defaultValue = self.getFile(default) if default else None >>>>> >>>>> def getFile(self, name): >>>>> for sprite in self.xml: >>>>> >>>>> if sprite.attrib['n'] == name: >>>>> region = >>>>> self.imageFile.get_region(int(sprite.attrib['x']), self.imageFile.height >>>>> - >>>>> int(sprite.attrib['y']) - int(sprite.attrib['h']), >>>>> int(sprite.attrib['w']), >>>>> int(sprite.attrib['h'])) >>>>> return region >>>>> >>>>> return self.defaultValue >>>>> >>>>> >>>>> def load(): >>>>> atlas1 = Atlas('image0') >>>>> atlas2 = Atlas('image1') >>>>> atlas3 = Atlas('image2') >>>>> atlas4 = Atlas('image3') >>>>> >>>>> import cProfile >>>>> cProfile.run('load()', 'pyglet_load_test') >>>>> >>>>> >>>>> Basically the XML has data on the regions in the atlas where the >>>>> actual sprites are, then we extract them using getFile. However, just the >>>>> loading of it takes a while, and I'm only loading 4 atlases (in the above >>>>> example) >>>>> >>>>> >>>>> On Sunday, July 2, 2017 at 11:42:56 PM UTC-5, Benjamin Moran wrote: >>>>>> >>>>>> Hey Charles, >>>>>> >>>>>> The internal format is RGBA, so you might start by seeing if your >>>>>> PNGs have an alpha channel or not. I took a quick glance at the module, >>>>>> and >>>>>> it might be possible to avoid the re.findall step altogether if the >>>>>> format >>>>>> is already the same. >>>>>> >>>>>> I'm not super familar with this module, but maybe the code can be >>>>>> rewritten to avoid using the `re` module altogether. It's not really >>>>>> doing >>>>>> very sophisticated matches anyway. This might be a nice project for >>>>>> someone >>>>>> to hack on. >>>>>> >>>>>> If you could post a small example snippet of what you're doing, I'll >>>>>> run it through vmprof and have a look at it as well. >>>>>> >>>>>> >>>>>> On Sunday, July 2, 2017 at 7:59:26 AM UTC+9, Charles wrote: >>>>>>> >>>>>>> I have been profiling my code lately trying to improve performance, >>>>>>> especially at startup. I am not too experienced with the ins and outs >>>>>>> of >>>>>>> pyglet and image data in general, but after profiling it seems a big >>>>>>> chunk >>>>>>> of time is spent on loading my large atlas files. They range anywhere >>>>>>> from >>>>>>> 1024-2048 width or height. >>>>>>> >>>>>>> In my profiling it took 0.818 seconds on a Core i5 processor to load >>>>>>> 5 of them. I can only image how long it takes on a slower machine. >>>>>>> After >>>>>>> digging deeper it seems a majority of the time is spent in >>>>>>> pyglet.image._convert, specifically the re.findall portion (over 90% of >>>>>>> the >>>>>>> time is spent on that). Since I doubt we can improve the speed of a >>>>>>> default >>>>>>> library, I looked at the comment where the findall is found and it >>>>>>> says: >>>>>>> "Pitch is wider than pixel data, need to go row-by-row." which forces >>>>>>> it to >>>>>>> do a findall. >>>>>>> >>>>>>> Is this because of my image format (PNG) or size? Would a different >>>>>>> format produce better results or a way around needing for it to >>>>>>> findall? >>>>>>> Any input is appreciated, thanks. >>>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "pyglet-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/pyglet-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- You received this message because you are subscribed to the Google Groups "pyglet-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/pyglet-users. For more options, visit https://groups.google.com/d/optout.
