> We have started picking at this again, trying to improve the > performance, and in the process stumbled on this feature: > http://www.swig.org/Doc2.0/Python.html#Python_builtin_types
> I don’t know if you’ve already come across this, but promises > of “significant performance improvement” are obviously quite > alluring, and looking at various user benchmarks suggest this > could just save the day. There is hope! I did not know about this. It appears to be new in Swig 2.0.4. Though it is 2.5 years old, I stopped using swig before 2.0 came out, and the wrapper classes were one of the big reasons. > The issue we have is that this option does not seem to play > very well with the current gem5/python integration. The first > thing that comes crashing down is the custom importer. The custom importer itself, or just the fact that the custom importer tries to package up the python wrapper files? > Do you think you could: 1) have a look (I can send you > the patch for the swig flags and gcc warn flags that are > needed), or 2) bring us up to speed on why it is needed, > what it does, what assumptions it makes, what the > code is based on etc. I can help, let me first explain why the importer exists. Because we were running gem5 on a simulation farm, we only wanted to worry about a distributing a single executable file (because the software handled that stuff for us) and not worry about dealing with both the executable and a whole bunch of python code which could easily get out of sync. So the build system takes every python file (both those in the tree and those generated), compiles it, marshals it, compresses it, and then turns it into a const global C++ byte array which is then compiled into the binary. Think embedded .pyc file and see embedPyFile in src/SConscript. We also build a manifest of all of these embedded py files. This is done with the EmbeddedPython object which is a static global variable. During initialization of these static globals, we build the manifest dynamically at runtime. (see embedPyFile in src/SConscript and init.cc) When gem5 starts up, after all of the global variables are constructed, it calls initM5Python (init.cc). Which basically does two things. It enables the custom importer by finding importer.py in the manfest and running it, and then it goes through the manifest and adds all modules to the importer to that it knows about everything. >From here on out, all embedded python files can be imported normally by the custom importer which really isn't all that fancy. It just checks to see if it can satisfy an import, and if it can, it just does the import manually. So, I think that all you have to do is edit the SwigSource class in src/SConscrupt to remove the .py_source line. It would also be nice to clean things up by removing the makeEmbeddedSwigInit function and associated code (SConscript) and the EmbeddedSwig class (init.(cc|hh)). If this still breaks after you make these changes, why don't you send the diffs and the error messages. I'd like to help, but as you might expect, I have hours, not days, so me doing lots of builds and testing is not going to happen. I could certainly do several builds trying to fix bugs, but I might not do exhaustive compiling of everything. > It would be really great if you could help us get this working > so we could at least see if it solves the performance problems. Certainly. Either way, it seems to be a great improvement. I have other ideas to improve the stats performance significantly, but this should hopefully go a long way. Nate _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
