Hi Richard, I agree with almost everything you write. I will try clarify some points of our decisions.
On Thu, Oct 1, 2009 at 8:53 AM, Richard Dale <[email protected]> wrote: > Thanks to the release of PySide last month I have put a lot of thought into > Python bindings, started a QtScript bindings project, and have even become > something of a Python fan! I thought I'd write a brain dump of my thoughts on > using the Smoke libraries for a Python binding. > > Smoke was originally designed by Ashley Winters and the PerlQt team in 2002. > Since then it has been used for QtRuby, Qyoto C#, PHP, Common Lisp and PerlQt4 > bindings. The idea is very simple and I would call it a 'moc on steriods' as > it works just like slots and signals are implemented in Qt, but for the entire > library rather than only some methods, and has features like virtual method > override callback handling, and caters for multiple inheritance, which the moc > lacks. We know a little about smoke, in the past we took a look in ruby and Qyoto bindings, they have different use of smoke, and both are a good examples of use smoke. But our first concern about smoke is the performance. We can continue discuss about this. And I would like to see how smoke handle with caters for multiple inheritance, because this is a big problem for us. > > I am currently working on a project to implement a Smoke based binding for > QtScript with Ian Monroe of the Amarok team. There is a an existing QtScript > bindings project, but the Amarok guys had found the libs were too large, and > start up time was too slow. The Smoke library to wrap for QtCore, QtGui, > QtNetworking, QtSql, QtSvg, QtXml and QtOpenGL is only 4.3 Mb for the 613 > classes it wraps. The existing QtScript bindings initialize all the classes, > and all their methods at startup which is slow, and it is about 16.3Mb for the > Qt libraries. Maybe we can integrate PySide project tools apiextractor and generators with your project ant together change experience, and work in the same core. What do you think? > > Since the release of PySide I have studied the Python C api and looked at the > code of several Python bindings projects, in order to get an idea of what > would be involved. > > I started looking at the Boost::Python code generated for the current version > of PySide, and it certainly is very human readable, and I especially how > operator methods are defined. But I think it was designed for relatively small > projects where you might have 20-30 C++ classes, and you write the > Boost::Python code at the same time as you are writing the C++ classes, using > Python for prototyping. However, I don't think it was intended to be machine > generated, and used on the scale 600+ classes libraries like Qt. The total > size of PySide for wrapping just the Qt libraries, let alone any extra KDE > classes or whatever, is 30Mb, and that makes it unusable for small devices > like Maemo based ones. It is also very much all or nothing - it looks quite > hard to customize it to use less memory, or add more runtime dynamism. I don't > think the PySide team should have done a first release based on Boost::Python > and I have no idea why they have persisted for so long using something which > is so obviously not suitable. Yes the size generated by boost::pythons is bigger then others but is faster too, but unfortunately for devices this size can be a problem, because of this we start a new phase in our project and start a new back end with all experience already acquired for the past project and all tools this will be faster and easier then the first one. > > Next I looked at how the Python C api works by playing with the 'Noddy' > example in the docs, and reading up on how the descriptor protocol is used > with '__getattribute__', and also how metaclasses work. Here is a summary in > Python of how Smoke would be used: > > class SmokeMeta(type): > def __new__(cls, ...): > # Construct new Qt C++ instances here > > def __getattribute__(self, name): > # Intercept class method calls here, > # return a callable to handle calls to > # static C++ methods in the Smoke library > > class QWidget(object): > __metaclass__ = SmokeMeta > > def __getattribute__(self, name): > # Intercept instance method calls here, > # return a callable to handle calls to > # C++ methods in the Smoke library > > So an actual implementation would be the same as the code above, but written > in C. I think the Python C api would be a good fit to use with Smoke. Maybe this is not possible with python because how works python introspection but we can continue study this because this is the smarter way to create a binding with smoke, and avoid a lot of maintenance work, but cost how much of performance? > > After understanding the C api better, I studied the Gnome pygobject project, > which is what I would call a 'dynamic binding' like Smoke, which looks up > method calls and classes at runtime, instead of them being hard wired into the > bindings library at code generation time. It uses both GObject itself, and > gobject-introspection libraries at runtime. I think it is very impressive and > it uses custom versions of the tp_getattro() C function on the Python class > structs to intercept calls to __getattribute__ just like I thought could be > done with Smoke. The code is LGPL'd and so it could either be used directly, > or at least you could get ideas from it for a dynamic Python binding. Maybe this is not possible with python because how works python introspection but we can continue study this because this is the smarter way to create a binding with smoke, and avoid a lot of maintenance work, but cost how much of performance? > > I had read about an experimental branch in PySide called 'Shiboken' that uses > pretty much the vanilla Python C api, and so I checked it out of gitorious and > had a look. In gitorious it didn't have any Qt classes wrapped, and didn't > have any Qt marshalling either, and so it wasn't possible to tell how large it > might be. I spoke with 'hugpol' on IRC and he told me that they had a version > that wrapped QtCore on an internal git server, and it was about 2.2 Mb. In > PyQt, QtCore is about 2Mb and in the Boost::Python version of PySide is it > 4.4Mb. I think the Smoke version would be less than 1Mb. So I think with > enough work, it might be possible to produce a Shiboken version of PySide that > was about the same size or slightly bigger than PyQt. Yes shoboken produce a smaller size, and we can reduce this with smoke off course, but we need continue take care of performance. > > How much work is 'enough work' though to match PyQt? I studied the PyQt code > last and it really is very impressive indeed. It looks exactly as though a > Python expert has worked fulltime for over 10 years on it with help from the > community that uses it. It starts up fast, as it loads methods lazily only > when they are needed. In fact it uses about the same mill to start up as they > Smoke based QtScript bindings do, and about half the mill of QtRuby does (I > think that is because Ruby is slow, rather than Smoke being slow). I need to > study it more to see what it does, but there really doesn't seem to be much to > fault at all. I agree with you in this point > > I think the bindings generators based on the QtJambi one that PySide, Smoke > and QtScript all use are really good, and they do match the SIP code > generation approach parsing '.sip' files, instead of parsing the C++ headers > directly and adding XML metadata from config files. However, because we didn't > know about the PySide project Arno Rehn developed a bindings generator for > Smoke in a Google Summer of Code project this year, which is based on the > QtScript one just like the PySide team did. Maybe their code bases can be > merged. This will be a great step, and can join enforce to produce good tools. > > The most important advantage of dynamic language bindings is that they are > language independent and not Python only, and are also smaller than > conventional approaches. I think a dynamic Python binding like the gobject- > introspection based pygobject, or a smoke based one is sufficiently > interesting > technically and different enough from PyQt to be worthwhile. On the other > hand, > although I think Shiboken can be made to work, at best it would be much the > same as PyQt and even that would take a pretty heroic effort as far as I can > see. > > To me the only justification I can see for implementing another Python binding > (apart from the GPL vs LGPL license issue which I personally don't care about > much), would be to implement a Maemo based development environment that > combined Python, Ruby and QtScript using common bindings libs with a lower > memory footprint than other approaches. Instead of just doing a "Let's kill > PyQt on all possible platforms", it could be "Let's develop a great multi- > language RAD environment for Maemo". For instance, I can't see the KDE project > switching from PyQt/PyKDE to PySide anytime soon, no matter what approach > PySide takes. And pretending that it would be easy in that area or anywhere > PyQt is already entrenched isn't being realistic IMHO. > > -- Richard > > > _______________________________________________ > PySide mailing list > [email protected] > http://lists.openbossa.org/listinfo/pyside > -- Renato Araujo Oliveira Filho Instituto Nokia de Tecnologia - INdT Mobile: +55 (81) 8704-2144 _______________________________________________ PySide mailing list [email protected] http://lists.openbossa.org/listinfo/pyside
