Hi Igor, I have commited the refactoring that you have suggested. Namely, using the indirect loader as a wrapper of another loader. I have renamed the class to NBSTIndirectArgument (I think it better reflects its purpose)....
I would appreciate though if you can have a look at the "bound checking" that I have implemented for the #emitLoad: since I'm not really an expert user of nativeboost ;). Cheers, Ciprian On Thu, Oct 25, 2012 at 4:15 AM, Igor Stasenko <siguc...@gmail.com> wrote: > On 24 October 2012 21:55, Ciprian Teodorov <ciprian.teodo...@gmail.com> > wrote: > > Hi Igor, > > > > I finally managed to commit the stuff. > > Now I looked a little bit into the idea of generalizing it to work with > any > > loader, which seems to work pretty much. Now I have two questions: > > First of all, what about #argName:type:arity? Should we remove it and > > replace it everywhere with #argName:indirectIndex:type:arity ? Or we > should > > keep it? I don't quite grasp the impact such a change will have on the > whole > > machinery (Ex. NBNativeFunctionGen, and NativeArgument). > > > This is a contract between parser and its requestor. Since we changing > contract, > all requestors should use new protocol. > NBNativeFunctionGen can simply ignore that, because they don't handle > smalltalk objects. > NativeArgument is encapsulation of parsed argument. But since index > transformed into different loader, > there no need to change anything for it. > > > Secondly, i was thinking about ways to check the indirect bounds... and > I'm > > quite dry of ideas... Is there any generic way to check the bounds of an > > object, or we should do that on a case by case basis? Typically I would > > think that we need something that gives us the size of an object in > bytes, > > however I'm not sure that all our objects are contiguous in memory (or if > > they are that they will be like that forever), moreover what about > > smallintegers? Should we restrict the approach to Arrays only? > > > Look at InterpreterProxy, it has such functionality. And sure thing > smallints should no be allowed. > But if you find it hard to implement, just leave 'self halt' there, > and meanwhile use callouts with option to check bounds turned off. I > will implement that later. > > > well that's it for now > > Cheers, > > Ciprian > > > > > > On Wed, Oct 24, 2012 at 8:47 PM, Stéphane Ducasse > > <stephane.duca...@inria.fr> wrote: > >> > >> May be we should ask dave mason because he wrote a C header parser or > >> something like that. > >> > >> On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote: > >> > >> > Hi Igor, > >> > > >> > There seems to be an issue with the squeaksource repo... I don't see > >> > myself listed amongst the developers nor can I commit > >> > > >> > I'm using this connection string: > >> > MCHttpRepository > >> > location: 'http://www.squeaksource.com/NativeBoost' > >> > > >> > > >> > user: 'cip.t' > >> > > >> > password: '*******' > >> > > >> > Cheers > >> > Ciprian > >> > > >> > On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguc...@gmail.com> > >> > wrote: > >> > HI, Ciprian. > >> > I added you to developers. > >> > > >> > Just single comment about auto-generated wrapper(s)/bindings: > >> > You need to do it only once. Then it will not be needed. > >> > > >> > Yes, you can do it automated or manually (automated is preferred), but > >> > since you do it only once, even if manually, this is not going to be > >> > too big bottleneck. > >> > Sure things, having a tool(s) which can automate that would be nice. > >> > But according to my experience, you can automate it only partially, > >> > but not fully. > >> > There , of course, exceptions like OpenGL library which API design > >> > allows to automate wrapping. > >> > > >> > And sure thing it would be nice to have some code generators to import > >> > from C header(s). > >> > > >> > > >> > On 22 October 2012 19:52, Ciprian Teodorov < > ciprian.teodo...@gmail.com> > >> > wrote: > >> > > Hi guys, > >> > > > >> > > First of all sorry for not saying anything the whole day ... but I > >> > > don't > >> > > have internet connection at work. I hope I'll be excused though ;) > >> > > > >> > > Now it will be a difficult for me to address all the issues raised > ... > >> > > however I will try > >> > > > >> > > First of all, IMHO any FFI should facilitate the task of the person > >> > > writing > >> > > the library wrapper. > >> > > This comes the issue of (hopefully automatically) parsing C headers > >> > > and > >> > > generating the calls. > >> > > Why?... well because if I take for instance the BLAS/LAPACK case > >> > > (which was > >> > > cited today) have over 1000 functions exported. > >> > > Even if it is not 1000 lets say that you have a smaller library from > >> > > which > >> > > you want to use only one function X but in order to use that precise > >> > > function you will maybe have to initialize some C context data, or a > >> > > specific data structure. > >> > > Of course you can do that by hand, but what if you really want to > >> > > benefit > >> > > from the library without diving into the details... well in that > case > >> > > you > >> > > are a little bit stuck, either you write by hand a bunch of wrappers > >> > > or you > >> > > quit. > >> > > > >> > > The problem is that nice libraries that you want to use they usually > >> > > have > >> > > horrible data structures behind that you cannot set up easily. This > >> > > happened > >> > > to me several times in the future while trying to use different > >> > > external > >> > > library calls (graphviz, metis, abc, vpr to name a few) in Pharo > (with > >> > > FFI, > >> > > with Alien, etc). In almost all cases I ended up generating text > files > >> > > and > >> > > calling some bloody c program using these files as input. The > problem > >> > > is > >> > > that has a huge impact on performances... > >> > > > >> > > Now with nativeboost I found it pretty easy to automatically > generate > >> > > some > >> > > usable binding... I parsed the C headers with srcml, the I have > parsed > >> > > the > >> > > XML looking for function definitions... For these definitions I've > >> > > generated > >> > > the wrappers. Only that for over 15 argument functions that did not > >> > > work. > >> > > > >> > > Now, I do not like the idea of hacking the compiler, or even > >> > > subclassing it > >> > > like Nicolas did for Smallapack. And luckily we do not need to do > that > >> > > with > >> > > the arguments in an array trick... Moreover, I completely agree with > >> > > Igor on > >> > > the fact that 15 arguments are to many, however Pharo really needs > to > >> > > have > >> > > an automatable FFI generation. > >> > > > >> > > Igor, I have seen that you have mentioned some work-arounds using > >> > > instance > >> > > variables, and/or NBExternalStructures. Both these ideas are great, > >> > > especially the use of the NBExternalStructure, however in my opinion > >> > > if you > >> > > want to generate them automatically you've got yourself an even > bigger > >> > > problem. > >> > > Will you generate a new class for each function call that you have? > >> > > What about the FFI being a wormhole to an ugly and mean world? > >> > > I think we should try to reduce the size of that hole ;) > >> > > Though, you have a point with using named arguments, and I think > that > >> > > is a > >> > > cool solution too. That is why I was speaking about directly > accessing > >> > > the > >> > > instance variables by name. > >> > > I simply didn't know enough about the hidden powers of > >> > > NBExternalStructure. > >> > > ;) > >> > > > >> > > As for extending the nativeboost signatures and accessing object > >> > > fields by > >> > > name, I completely agree with you that it is not a good idea. > >> > > > >> > > However, using the indices of arrays I think is only a small > addition > >> > > that > >> > > can have a huge impact on the way people use NativeBoost FFI without > >> > > adding > >> > > to much extra-overhead. > >> > > By the way, I think we definitely have to do the bound-checking > trick > >> > > by > >> > > default. > >> > > Moreover, having the NBSTInObjectArgument as a wrapper over another > >> > > loader > >> > > is a great idea, giving us another degree of "controlled" freedom. > ;) > >> > > > >> > > Thanks Henrik for the good joke (I didn't see it coming :)), It > makes > >> > > me > >> > > hate this idea now. :P > >> > > > >> > > self callFn: {x . y} > >> > > Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) > >> > > I for one welcome our new syntactic overlords! > >> > > > >> > > As for supporting or not more than 15 arguments for typical method > >> > > calls... > >> > > I don't know... maybe is good maybe is bad, but I think there are > >> > > other > >> > > things that need to be done before making a fuss about it. By the > way > >> > > is it > >> > > an arbitrary limit, or it is imposed by the use of 4 bytes for > storing > >> > > the > >> > > number of arguments? > >> > > > >> > > Now let me come back to practical issues. My squeaksource id is > cip.t > >> > > (you > >> > > can find me searching "ciprian" in the members list). > >> > > > >> > > Cheers, > >> > > Ciprian Teodorov > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguc...@gmail.com> > >> > > wrote: > >> > >> > >> > >> On 22 October 2012 12:00, Henrik Sperre Johansen > >> > >> <henrik.s.johan...@veloxit.no> wrote: > >> > >> > On 22.10.2012 02:37, Igor Stasenko wrote: > >> > >> >> > >> > >> >> On 22 October 2012 01:59, Nicolas Cellier > >> > >> >> <nicolas.cellier.aka.n...@gmail.com> wrote: > >> > >> >>> > >> > >> >>> 2012/10/22 Igor Stasenko <siguc...@gmail.com>: > >> > >> >>>> > >> > >> >>>> On 22 October 2012 01:20, Nicolas Cellier > >> > >> >>>> <nicolas.cellier.aka.n...@gmail.com> wrote: > >> > >> >>>>> > >> > >> >>>>> 5 parameters???? Igor, you're a dictator ;) > >> > >> >>>>> Those theories are nice, but unhelpfull when applied to FFI > >> > >> >>>>> Pragmatically there's not any chance I rewrite LAPACK+BLAS > for > >> > >> >>>>> the > >> > >> >>>>> sake of purity. > >> > >> >>>>> > >> > >> >>>>> And your workaround (creating a class) is very poor, because > >> > >> >>>>> maybe > >> > >> >>>>> classes themselves should not have more than 5 instance > >> > >> >>>>> variables ;) > >> > >> >>>>> > >> > >> >>>> Hehe. > >> > >> >>>> This is same thing like increasing number of literals for > >> > >> >>>> methods > >> > >> >>>> (and > >> > >> >>>> max distances between jumps). > >> > >> >>>> It just makes sense where you deal with external chaotic > world. > >> > >> >>>> My ideology is simple: prevent that chaos from entering our > >> > >> >>>> little > >> > >> >>>> peaceful bay. > >> > >> >>>> > >> > >> >>> That's not exactly the philosophy behind FFI. > >> > >> >>> FFI is here to let the user manage the external chaotic world. > >> > >> >>> OK, external peels of the onion should have 5 parameters or > less. > >> > >> >>> Near the sprout, you can't raise such barriers, or there is no > >> > >> >>> onion > >> > >> >>> at > >> > >> >>> all. > >> > >> >>> > >> > >> >> Well, that's part of developer's responsibility, how to prevent > >> > >> >> chaos. > >> > >> >> Needless to say, nobody wants to deal with so many arguments at > >> > >> >> once > >> > >> >> (too much space for mistakes). > >> > >> >> As for my workaround: this mainly, how you tame the complexity > in > >> > >> >> case > >> > >> >> it is inevitable? > >> > >> >> Look how code to call that function will look like: > >> > >> >> > >> > >> >> 1. passing as array > >> > >> >> > >> > >> >> args := Array new: 100. > >> > >> >> args at:1 put: x; > >> > >> >> at:2 put: y; > >> > >> >> ... > >> > >> >> at: 100 put: zork > >> > >> >> > >> > >> >> self callFn: args. > >> > >> > > >> > >> > self callFn: {x . y} > >> > >> > Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) > >> > >> > I for one welcome our new syntactic overlords! > >> > >> > > >> > >> > >> > >> Yes, it looks familiar, but cannot tell where i seen it. > >> > >> Gah.. how i could forget about it? > >> > >> > >> > >> > >> > >> >> > >> > >> >> 2. passing as instance of class, or external structure: > >> > >> >> > >> > >> >> args := MyFunctionArgs new. > >> > >> >> args > >> > >> >> firstArgumentName: x; > >> > >> >> secondArgumentName: y; > >> > >> >> ... > >> > >> >> hundrethArgumentName: zork. > >> > >> >> self callFn: args. > >> > >> >> > >> > >> >> admit that dealing with names instead of numbers leaves much > less > >> > >> >> space for mistakes > >> > >> >> and serves for better clarity at same time. > >> > >> >> So, even if it is more cumbersome because requires defining > extra > >> > >> >> class, at the end you win much more. > >> > >> >> > >> > >> >> Anyways, if people think it is worth adding indirect argument > >> > >> >> loader ( > >> > >> >> in form of param@<index>, but not param@ivar), we can introduce > >> > >> >> that. > >> > >> >> > >> > >> > While I often find this a good idea for maintainability, it sorta > >> > >> > flies > >> > >> > in > >> > >> > the face of another of ST's strengths, iterative/exporatory > >> > >> > programming. > >> > >> > If you are forced to think up front about which parameter classes > >> > >> > you > >> > >> > need > >> > >> > due to a small limit, rather than introduce them ad-hoc when the > >> > >> > code > >> > >> > really > >> > >> > needs the refactoring to remain legible, it slows you down. > >> > >> > > >> > >> > Not thinking of FFI specifically, but I have seen lots of evolved > >> > >> > mathematical models where a 5-parameter limit upfront would > >> > >> > probably > >> > >> > lead to > >> > >> > either: > >> > >> > a) *Really* bad code, ie. making the calculation object stateful > by > >> > >> > storing > >> > >> > in instvars instead. (and in the process, make it really hard to > >> > >> > know > >> > >> > which > >> > >> > instvars are actually part of object state and not temp vars of > >> > >> > some > >> > >> > calculation) > >> > >> > b) Switching to another programming language out of frustration. > >> > >> > > >> > >> > >> > >> keyword message syntax is bad for many arguments.. > >> > >> for such cases i find a positional argument notation more appeal > >> > >> because it is more compact. > >> > >> In any case, a complex math formulas smell equally bad in any > >> > >> programming language > >> > >> (sometime even if written by hand on paper using math notation(s) > ;) > >> > >> > >> > >> > Cheers, > >> > >> > Henry > >> > >> > > >> > >> > >> > >> > >> > >> > >> > >> -- > >> > >> Best regards, > >> > >> Igor Stasenko. > >> > >> > >> > > > >> > > > >> > > > >> > > -- > >> > > Dr. Ciprian TEODOROV > >> > > Ingénieur Développement CAO > >> > > > >> > > tél : 06 08 54 73 48 > >> > > mail : ciprian.teodo...@gmail.com > >> > > www.teodorov.ro > >> > > >> > > >> > > >> > -- > >> > Best regards, > >> > Igor Stasenko. > >> > > >> > > >> > > >> > > >> > -- > >> > Dr. Ciprian TEODOROV > >> > Ingénieur Développement CAO > >> > > >> > tél : 06 08 54 73 48 > >> > mail : ciprian.teodo...@gmail.com > >> > www.teodorov.ro > >> > >> > > > > > > > > -- > > Dr. Ciprian TEODOROV > > Ingénieur Développement CAO > > > > tél : 06 08 54 73 48 > > mail : ciprian.teodo...@gmail.com > > www.teodorov.ro > > > > -- > Best regards, > Igor Stasenko. > > -- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodo...@gmail.com www.teodorov.ro
