It doesn't look like it would be too difficult to make an alternative _convert method without the re module. It's basically just figuring out the number of bytes in a pixel, the current order, and rearranging them. Basically just a bunch of slicing. I suppose the question is how ugly would it look compared to what we have now, and how much of a difference it would actually make. If I wasn't so busy at the moment I might try it out of curiosity. If you feel like giving it a go, It would be interesting to see.
On Saturday, July 15, 2017 at 1:33:26 AM UTC+9, Charles wrote: > > Thanks for the reply and information, had no idea. That is unfortunate > though, I guess the speedup would have to be done in an alternative for > findall. > > I tried the pypng decoder but that was over 10 times slower. > > Just for fun I decided to test a 'ReverseGDIPlusDecoder' where it's the > same but doesn't reverse the pitch, and I just save the images upside down > that way they are imported right side up. > > normal loader - 32x32 size - 1000 count - Took: 1.3884263192 > normal loader - 2560x2560 size - 100 count - Took: 10.7336410624 > > reverse loader - 32x32 size - 1000 count - Took: 1.35458577926 > reverse loader - 2560x2560 size - 100 count - Took: 4.4178891004 > > For smaller images almost no difference, but for the larger ones, it is > more than a 50% speedup. Pretty interesting, not sure I'll reverse my > images as it's kind of a hassle, but maybe as its just a handful. > > On Tuesday, July 4, 2017 at 2:44:40 AM UTC-5, Benjamin Moran wrote: >> >> I just made a script that only loads up a 3840 x 1080 png file. After >> running it through vmprof, we can confirm your cProfile results: >> http://vmprof.com/#/c0d74a65-dc84-4923-987d-4f451fadc3a6 >> >> I did quick print statements at the top of the _convert method, and can >> see that my loaded PNG file has reversed pitch compared to what is needed: >> print(format, pitch) >> print(self._current_format, self._current_pitch) >> >>> RGBA 15360 >> >>> RGBA -15360 >> >> That lead me here: >> https://blog.nobel-joergensen.com/2010/11/07/loading-a-png-as-texture-in-opengl-using-libpng/ >> OpenGL and PNG use a different pitch by default, so it looks like the >> conversion step is unavoidable. >> >> >> >> On Tuesday, July 4, 2017 at 3:49:51 PM UTC+9, Charles wrote: >>> >>> I am on Windows platform, I haven't tested Mac or Linux. >>> >>> And Benjamin that's correct, however I notice this issue just on the >>> resource load. Although get_region does get slow, that's expected since the >>> atlas can contain up to 4500 different regions which definitely ends up >>> taking a while. (0.4 seconds) >>> >>> On Monday, July 3, 2017 at 10:14:03 PM UTC-5, swiftcoder wrote: >>>> >>>> What platform are you on? >>>> >>>> On some platforms the native format is BGRA, *not* RGBA (this is a >>>> common problem when porting between Mac and Windows, for example). >>>> >>>> On Mon, 3 Jul 2017 at 18:45 Benjamin Moran <[email protected]> wrote: >>>> >>>>> Thanks for the code Charles, >>>>> >>>>> If I read it correctly, we can distill it down to something like this >>>>> for benchmarking: >>>>> import pyglet >>>>> >>>>> imageFile = pyglet.resource.image(filename) >>>>> >>>>> >>>>> def load(): >>>>> # with different x, y, w, h values: >>>>> atlas1 = imageFile.get_region(x, y, w, h) >>>>> atlas2 = imageFile.get_region(x, y, w, h) >>>>> atlas3 = imageFile.get_region(x, y, w, h) >>>>> atlas4 = imageFile.get_region(x, y, w, h) >>>>> >>>>> Since you tested all of the possible formats, and at least the RGBA >>>>> one should be OK, maybe the _convert method is unavoidable. We'll have to >>>>> dig in a little more. >>>>> >>>>> >>>>> On Tuesday, July 4, 2017 at 7:29:27 AM UTC+9, Charles wrote: >>>>>> >>>>>> It should be RGBA, the image has alpha, from the settings the format >>>>>> is PNG-32 and Pixel format is RGBA8888. I did some tests since the >>>>>> texture >>>>>> packer allows different types of formats. >>>>>> >>>>>> I tried a POT texture, same result. NPOT texture, same result. The >>>>>> file was an indexed PNG file (to save space and size), I tried unindexed >>>>>> with the same result. I can't seem to not trigger this _convert findall >>>>>> function. >>>>>> >>>>>> As far as code this is what I am doing. >>>>>> #import xml.etree.ElementTree as ET >>>>>> from lxml import etree as ET >>>>>> import pyglet >>>>>> >>>>>> class Atlas(object): >>>>>> def __init__(self, filename, default=None): >>>>>> tree = ET.parse(pyglet.resource.file(filename +".xml")) >>>>>> self.xml = tree.getroot().findall("sprite") >>>>>> self.imageFile = pyglet.resource.image(filename+".png") >>>>>> self.defaultValue = self.getFile(default) if default else None >>>>>> >>>>>> def getFile(self, name): >>>>>> for sprite in self.xml: >>>>>> >>>>>> if sprite.attrib['n'] == name: >>>>>> region = >>>>>> self.imageFile.get_region(int(sprite.attrib['x']), self.imageFile.height >>>>>> - >>>>>> int(sprite.attrib['y']) - int(sprite.attrib['h']), >>>>>> int(sprite.attrib['w']), >>>>>> int(sprite.attrib['h'])) >>>>>> return region >>>>>> >>>>>> return self.defaultValue >>>>>> >>>>>> >>>>>> def load(): >>>>>> atlas1 = Atlas('image0') >>>>>> atlas2 = Atlas('image1') >>>>>> atlas3 = Atlas('image2') >>>>>> atlas4 = Atlas('image3') >>>>>> >>>>>> import cProfile >>>>>> cProfile.run('load()', 'pyglet_load_test') >>>>>> >>>>>> >>>>>> Basically the XML has data on the regions in the atlas where the >>>>>> actual sprites are, then we extract them using getFile. However, just >>>>>> the >>>>>> loading of it takes a while, and I'm only loading 4 atlases (in the >>>>>> above >>>>>> example) >>>>>> >>>>>> >>>>>> On Sunday, July 2, 2017 at 11:42:56 PM UTC-5, Benjamin Moran wrote: >>>>>>> >>>>>>> Hey Charles, >>>>>>> >>>>>>> The internal format is RGBA, so you might start by seeing if your >>>>>>> PNGs have an alpha channel or not. I took a quick glance at the module, >>>>>>> and >>>>>>> it might be possible to avoid the re.findall step altogether if the >>>>>>> format >>>>>>> is already the same. >>>>>>> >>>>>>> I'm not super familar with this module, but maybe the code can be >>>>>>> rewritten to avoid using the `re` module altogether. It's not really >>>>>>> doing >>>>>>> very sophisticated matches anyway. This might be a nice project for >>>>>>> someone >>>>>>> to hack on. >>>>>>> >>>>>>> If you could post a small example snippet of what you're doing, I'll >>>>>>> run it through vmprof and have a look at it as well. >>>>>>> >>>>>>> >>>>>>> On Sunday, July 2, 2017 at 7:59:26 AM UTC+9, Charles wrote: >>>>>>>> >>>>>>>> I have been profiling my code lately trying to improve performance, >>>>>>>> especially at startup. I am not too experienced with the ins and outs >>>>>>>> of >>>>>>>> pyglet and image data in general, but after profiling it seems a big >>>>>>>> chunk >>>>>>>> of time is spent on loading my large atlas files. They range anywhere >>>>>>>> from >>>>>>>> 1024-2048 width or height. >>>>>>>> >>>>>>>> In my profiling it took 0.818 seconds on a Core i5 processor to >>>>>>>> load 5 of them. I can only image how long it takes on a slower >>>>>>>> machine. >>>>>>>> After digging deeper it seems a majority of the time is spent in >>>>>>>> pyglet.image._convert, specifically the re.findall portion (over 90% >>>>>>>> of the >>>>>>>> time is spent on that). Since I doubt we can improve the speed of a >>>>>>>> default >>>>>>>> library, I looked at the comment where the findall is found and it >>>>>>>> says: >>>>>>>> "Pitch is wider than pixel data, need to go row-by-row." which forces >>>>>>>> it to >>>>>>>> do a findall. >>>>>>>> >>>>>>>> Is this because of my image format (PNG) or size? Would a different >>>>>>>> format produce better results or a way around needing for it to >>>>>>>> findall? >>>>>>>> Any input is appreciated, thanks. >>>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "pyglet-users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/pyglet-users. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "pyglet-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/pyglet-users. For more options, visit https://groups.google.com/d/optout.
