Brian Thomason schrieb:
We've been using PyQT/PyKDE here at Linspire for some time now to develop a handful of applications. One of these is Lsongs, and it has grown quite large and has a very large memory footprint. We're trying to reduce this a bit in various ways (switching to gstreamer from libxine for instance) and I was curious if there was a way to reduce the footprint size of pyqt/pykde. The shared object files themselves are quite large - Larger than their C++ counterparts.
I'm totally ignorant on python bindings so this may just be "the way it is" and that's fine. Just curious if there's a way to "shrink" these down a bit.
First, the size of a shared object file has no direct impact on the memory needed when the shared object gets loaded. Because (all?) systems map shared objects into memory via mmap page by page, even a very large shared object can have a small memory footprint.
Second, take a look into the ChangeLog of SIP v4.2rc2 or the latest SIP snapshot. By default, only the modules init function is exported now and the modules are stripped on installation. This reduces the size of the shared objects significantly and gives a better load performance.
The following platforms are supported (until now):
AIX: Tested on AIX 4.3.3 with VisualAge C++ 6.0. gcc should work, but has not been tested yet.
HP-UX: Tested on HP-UX B.11.00 with HP aC++ Compiler C.03.50.
Linux: Tested on SuSE Linux 8.2 with gcc 3.3 20030226 (prerelease). Tested on SuSE Linux 9.2 with 3.3.4 (pre 3.3.5 20040809).
IRIX: Unfortunately this doesn't work with recent gcc releases. To get test results, I have linked the modules by hand. When gcc 4 is available, it should work. With the MipsPro compiler it should work.
In general, it should work on all platforms where the linker supports explicit export of symbols, but it requires a few changes to SIP's build system to support new platforms.
On Windows, exporting only the modules init function is the default. There is nothing more that you can do on this platform to reduce the shared object size. You can get slightly better load performance by proper rebasing all the .dll and .pyd files that make up your Python/PyQt/Qt runtime environment.
I have attached my test results from the supporte platforms, so you can see the effect on module size and load performace. The absolute timing value is the real time needed to import all of the modules listed in the table. The relative timing is the time needed if the machine would give me a PyStone value of 30000, so the relative timing values are comparable across platforms.
I have also attached a test result for Windows that shows the performance effect after rebasing.
NOTE: To have the modules stripped on installation, either QMAKE_STRIP has to be defined in the qmake specs, or you can use
python configure.py STRIP=strip
Of course, you can also simply call strip on all modules after installation.
Ulli
#==================================================================================# # Platform: AIX # # PyStone Value: 5632.73 # #----------------------------------------------------------------------------------# # Module size is in bytes. Timing values are in seconds. Relative timing is based # # on a PyStone value of 30000.00 # #==================================================================================#
+-----------+----------+-------------------+-------------------+-------------------+ | | Normal | Strip | Export | Both | +-----------+----------+-------------------+-------------------+-------------------+ | Size: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | sip | 133903 | 80825 (-39.6%) | 130157 ( -2.8%) | 78087 (-41.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qt | 15580542 | 8390451 (-46.1%) | 13874765 (-10.9%) | 6964553 (-55.3%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtcanvas | 1036057 | 537083 (-48.2%) | 636142 (-38.6%) | 305009 (-70.6%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtnetwork | 960539 | 499447 (-48.0%) | 552481 (-42.5%) | 269676 (-71.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | qttable | 921178 | 470397 (-48.9%) | 540129 (-41.4%) | 250560 (-72.8%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtxml | 1046034 | 573489 (-45.2%) | 651208 (-37.7%) | 342234 (-67.3%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtsql | 1788639 | 936737 (-47.6%) | 1324913 (-25.9%) | 645693 (-63.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtui | 378549 | 197984 (-47.7%) | 39885 (-89.5%) | 20069 (-94.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtgl | 637091 | 323856 (-49.2%) | 283706 (-55.5%) | 128122 (-79.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtext | 3439229 | 2183923 (-36.5%) | 2940923 (-14.5%) | 1855771 (-46.0%) | +-----------+----------+-------------------+-------------------+-------------------+ | qwt | 5248463 | 2964315 (-43.5%) | 4483736 (-14.6%) | 2421150 (-53.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | Timing: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | Absolute | 0.73 | 0.72 ( -1.4%) | 0.63 (-14.2%) | 0.64 (-13.1%) | +-----------+----------+-------------------+-------------------+-------------------+ | Relative | 0.14 | 0.14 | 0.12 | 0.12 | +-----------+----------+-------------------+-------------------+-------------------+
#==================================================================================# # Platform: HP-UX # # PyStone Value: 9863.88 # #----------------------------------------------------------------------------------# # Module size is in bytes. Timing values are in seconds. Relative timing is based # # on a PyStone value of 30000.00 # #==================================================================================# +-----------+----------+-------------------+-------------------+-------------------+ | | Normal | Strip | Export | Both | +-----------+----------+-------------------+-------------------+-------------------+ | Size: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | sip | 155648 | 114688 (-26.3%) | 143360 ( -7.9%) | 102400 (-34.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qt | 14938112 | 11227136 (-24.8%) | 12550144 (-16.0%) | 9080832 (-39.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtcanvas | 1183744 | 843776 (-28.7%) | 872448 (-26.3%) | 540672 (-54.3%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtnetwork | 1110016 | 786432 (-29.2%) | 811008 (-26.9%) | 495616 (-55.4%) | +-----------+----------+-------------------+-------------------+-------------------+ | qttable | 1052672 | 749568 (-28.8%) | 761856 (-27.6%) | 466944 (-55.6%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtxml | 1208320 | 851968 (-29.5%) | 909312 (-24.7%) | 557056 (-53.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtsql | 1863680 | 1355776 (-27.3%) | 1441792 (-22.6%) | 950272 (-49.0%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtui | 540672 | 376832 (-30.3%) | 323584 (-40.2%) | 163840 (-69.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtgl | 782336 | 557056 (-28.8%) | 536576 (-31.4%) | 315392 (-59.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtext | 3993600 | 3026944 (-24.2%) | 3272704 (-18.1%) | 2363392 (-40.8%) | +-----------+----------+-------------------+-------------------+-------------------+ | qwt | 6217728 | 4657152 (-25.1%) | 5013504 (-19.4%) | 3584000 (-42.4%) | +-----------+----------+-------------------+-------------------+-------------------+ | Timing: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | Absolute | 0.61 | 0.61 ( +0.0%) | 0.46 (-24.6%) | 0.46 (-24.4%) | +-----------+----------+-------------------+-------------------+-------------------+ | Relative | 0.20 | 0.20 | 0.15 | 0.15 | +-----------+----------+-------------------+-------------------+-------------------+
#==================================================================================# # Platform: IRIX # # PyStone Value: 3240.03 # #----------------------------------------------------------------------------------# # Module size is in bytes. Timing values are in seconds. Relative timing is based # # on a PyStone value of 30000.00 # #==================================================================================# +-----------+----------+-------------------+-------------------+-------------------+ | | Normal | Strip | Export | Both | +-----------+----------+-------------------+-------------------+-------------------+ | Size: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | sip | 89468 | 88400 ( -1.2%) | 89468 ( +0.0%) | 88400 ( -1.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qt | 8060360 | 8051008 ( -0.1%) | 8056264 ( -0.1%) | 8046912 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtcanvas | 545288 | 544060 ( -0.2%) | 545288 ( +0.0%) | 544060 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtnetwork | 505524 | 504292 ( -0.2%) | 505524 ( +0.0%) | 504292 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qttable | 484572 | 483412 ( -0.2%) | 484572 ( +0.0%) | 483412 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtxml | 550804 | 549576 ( -0.2%) | 550804 ( +0.0%) | 549576 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtsql | 900612 | 899432 ( -0.1%) | 900612 ( -0.0%) | 899432 ( -0.1%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtui | 241644 | 240600 ( -0.4%) | 241644 ( +0.0%) | 240600 ( -0.4%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtgl | 350652 | 349504 ( -0.3%) | 350652 ( +0.0%) | 349504 ( -0.3%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtext | 2074120 | 2070216 ( -0.2%) | 2074120 ( +0.0%) | 2070216 ( -0.2%) | +-----------+----------+-------------------+-------------------+-------------------+ | qwt | 3002384 | 2998840 ( -0.1%) | 3002384 ( +0.0%) | 2998840 ( -0.1%) | +-----------+----------+-------------------+-------------------+-------------------+ | Timing: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | Absolute | 1.45 | 1.41 ( -2.8%) | 0.92 (-36.7%) | 0.91 (-37.3%) | +-----------+----------+-------------------+-------------------+-------------------+ | Relative | 0.16 | 0.15 | 0.10 | 0.10 | +-----------+----------+-------------------+-------------------+-------------------+
#==================================================================================# # Platform: Linux # # PyStone Value: 30625.04 # #----------------------------------------------------------------------------------# # Module size is in bytes. Timing values are in seconds. Relative timing is based # # on a PyStone value of 30000.00 # #==================================================================================# +-----------+----------+-------------------+-------------------+-------------------+ | | Normal | Strip | Export | Both | +-----------+----------+-------------------+-------------------+-------------------+ | Size: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | sip | 71626 | 54636 (-23.7%) | 70010 ( -2.3%) | 53024 (-26.0%) | +-----------+----------+-------------------+-------------------+-------------------+ | qt | 8335177 | 6242564 (-25.1%) | 6863721 (-17.7%) | 4771108 (-42.8%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtcanvas | 630758 | 425140 (-32.6%) | 439406 (-30.3%) | 233788 (-62.9%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtnetwork | 590296 | 394084 (-33.2%) | 405400 (-31.3%) | 209188 (-64.6%) | +-----------+----------+-------------------+-------------------+-------------------+ | qttable | 569048 | 382448 (-32.8%) | 392312 (-31.1%) | 205712 (-63.8%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtxml | 612744 | 423548 (-30.9%) | 424112 (-30.8%) | 234916 (-61.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtsql | 1009981 | 717364 (-29.0%) | 755773 (-25.2%) | 463156 (-54.1%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtui | 288295 | 176416 (-38.8%) | 156343 (-45.8%) | 44464 (-84.6%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtgl | 422830 | 275624 (-34.8%) | 270942 (-35.9%) | 123736 (-70.7%) | +-----------+----------+-------------------+-------------------+-------------------+ | qtext | 2173235 | 1696716 (-21.9%) | 1750739 (-19.4%) | 1274220 (-41.4%) | +-----------+----------+-------------------+-------------------+-------------------+ | qwt | 3002345 | 2276504 (-24.2%) | 2321961 (-22.7%) | 1596120 (-46.8%) | +-----------+----------+-------------------+-------------------+-------------------+ | Timing: | | | | | +-----------+----------+-------------------+-------------------+-------------------+ | Absolute | 0.34 | 0.34 ( +1.8%) | 0.19 (-43.5%) | 0.20 (-40.5%) | +-----------+----------+-------------------+-------------------+-------------------+ | Relative | 0.34 | 0.35 | 0.19 | 0.20 | +-----------+----------+-------------------+-------------------+-------------------+
#==========================================# # Platform: win32 # # PyStone Value: 36769.04 # #------------------------------------------# # Module size is in bytes. Timing values # # in seconds. Relative timing is base on a # # PyStone value of 30000.00 # #==========================================# +-----------+----------+-------------------+ | | Normal | Rebase | +-----------+----------+-------------------+ | Absolute | 0.14 | 0.11 (-26.9%) | +-----------+----------+-------------------+ | Relative | 0.18 | 0.13 | +-----------+----------+-------------------+
_______________________________________________ PyKDE mailing list PyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
