Re: Zcode interpreter release
On Aug 24, 2005, at 7:42 PM, Andrew Rodland wrote: On Wednesday 24 August 2005 04:26 pm, Amir Karger wrote: Several people pointed out that I didn't perldoc -f system. Sorry! Btw, even after reading the docs, I still don't understand why Perl would pass a cd command to a piece of the shell that can't understand it. Granted, I shouldn't do it anyway, because then Perl will exit the shell it created for the system() and my cd will be useless. It's not "a piece of the shell"; it's no shell at all. Perl is trying to execute /bin/cd (or rather, 'cd' somewhere in your search path), as provided for by execvp. As to why perl does it; perl doesn't know or care that the thing you're naming is a shell builtin; it simply tries to run what you told it to. If you wanted to force it to work as a builtin, you could use system('sh', '-c', 'cd /foo'); -- but as you've already noted, it wouldn't be good for anything anyway. Another way to execute a shell builtin is to append a semicolon: system( "cd /foo;" ). Josh
Re: Python PMC's
Leopold Toetsch wrote: > > Above Parrot interface function tries to locate Sub objects in the > namespace of the invocant's class and in the MRO of it. When you go back > to the PIR translation of your code there is e.g. > > class Foo: > def g(self,y): > > I have translated this to: > > .namespace ["Foo"] > .sub g > .param pmc self > > Therefore all the static / default / common "methods" inside Python code > would be callable with the plain Parrot method call syntax. > > The classname contributes the namespace. The sub declaration creates an > entry in that namespace, which is retrievable as: > > func = findglobal "Foo", "g" Honestly, I don't believe that that is workable for Python. Modules are global in Python. Classes are lexically scoped. All subs emitted by Pirate are @anon. Modules are namespaces and can contain classes, functions, and variables. The way to retrieve a method from a Python class defined in module __main__ would be: $P1 = findglobal "__main__", "Foo" getattribute $P2, $P1, 'g' - Sam Ruby
Re: Zcode interpreter release
On Wednesday 24 August 2005 04:26 pm, Amir Karger wrote: > Several people pointed out that I didn't perldoc -f system. Sorry! > Btw, even after reading the docs, I still don't understand why Perl > would pass a cd command to a piece of the shell that can't understand > it. Granted, I shouldn't do it anyway, because then Perl will exit the > shell it created for the system() and my cd will be useless. It's not "a piece of the shell"; it's no shell at all. Perl is trying to execute /bin/cd (or rather, 'cd' somewhere in your search path), as provided for by execvp. As to why perl does it; perl doesn't know or care that the thing you're naming is a shell builtin; it simply tries to run what you told it to. If you wanted to force it to work as a builtin, you could use system('sh', '-c', 'cd /foo'); -- but as you've already noted, it wouldn't be good for anything anyway. Andrew
Re: Zcode interpreter release
On 8/23/05, I wrote: > I adopted the Zcode interpreter that leo posted in February. > > The only bad news is there's something wrong with my make test. > I managed to narrow this down to a very > weird Perl behavior I don't understand at all: > > testportal:~>mkdir z > testportal:~>cd z > testportal:~/z>mkdir y > testportal:~/z>touch y/foo > testportal:~/z>perl -we 'system("cd y")' > Can't exec "cd": No such file or directory at -e line 1. > testportal:~/z>perl -we 'system("cd y && ls")' > foo Several people pointed out that I didn't perldoc -f system. Sorry! Btw, even after reading the docs, I still don't understand why Perl would pass a cd command to a piece of the shell that can't understand it. Granted, I shouldn't do it anyway, because then Perl will exit the shell it created for the system() and my cd will be useless. I changed my tests to chdir to the right directory before doing anything, and now it seems to work no matter where you run it. So now, v0.2 can officially be released. Hooray! -Amir
Re: [pirate] Re: Python PMC's
On Wed, 24 Aug 2005, Sam Ruby wrote: [huge cut] > Below is from the sample that Leo provided. > > > # print foo.f(2) > > > > # emulate python find_name, which checks attributes too > > push_eh m_nf > > $P0 = foo."f"(2) > > clear_eh > > goto m_f > > m_nf: > > # getattribute would also check if __get__ is there > > $P1 = getattribute foo, "f" > > $P0 = foo.$P1(2) > > m_f: > > print_item $P0 > > print_newline > > Note that the code above would need to be emitted for *every* method > call, as there is no way of knowing at compile time what property you > will get and how it is defined. Hey Sam, I agree with what you're saying in this thread. This is slightly off topic, but I wanted to point something out. In general, python has so many places where things have to be dynamic that you really can't know this kind of thing at compile time, especially if you allow for eval/exec, or if you allow the code to be used as a module. However if you treat the code as a closed system, *and* you have access to python at compile time, then we can optimize away a lot of these questions. For example, in your original code: def f(x,y): return y class Foo: f = f def g(self,y): return y foo = Foo() g=foo.g print f(0,1) print foo.f(2) print g(3) Once you know how python works, it's *obvious* that this prints 1,2,3. I see no reason why the compiler couldn't figure this out up front just by walking the tree. In fact, a good optimizing compiler would see the "return y" lines and just get rid of those methods completely. I'd like to allow for the ability to do certain optimizations like this up front, sacrificing flexibility for speed. I know there are many programs that won't allow for this, but for the ones that do, I'd like to be able to do a sort of static compile like this. In other words, sometimes a python-like language is a desirable thing. (But of course this should all be optional so that we can also be 100% python compatible) Sincerely, Michal J Wallace Sabren Enterprises, Inc. - contact: [EMAIL PROTECTED] hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -
Re: Python PMC's
On Aug 24, 2005, at 23:34, Sam Ruby wrote: Leopold Toetsch wrote: Note that you would then be caching the results of a curried function call. This result depends not only on the method string, but also on the particular object upon which it was invoked. No the "inner" Parrot_find_method_with_cache just caches the method for a specific class (obeying C3 method resolution order as in Python). There is no curried function at that stage. Where does Parrot_find_method_with_cache get this data? Remember, in Python, there are no methods, there are only properties. Above Parrot interface function tries to locate Sub objects in the namespace of the invocant's class and in the MRO of it. When you go back to the PIR translation of your code there is e.g. class Foo: def g(self,y): I have translated this to: .namespace ["Foo"] .sub g .param pmc self Therefore all the static / default / common "methods" inside Python code would be callable with the plain Parrot method call syntax. The classname contributes the namespace. The sub declaration creates an entry in that namespace, which is retrievable as: func = findglobal "Foo", "g" And as the namespace is exactly the classname this is exactly what find_method for a "Foo" object does, when trying to locate a method named "g" (I'm omitting here further lookups according to MRO due to other parents). The actual namespace that the classes find_method looks into can further be finetuned with the vtable function VTABLE_namespace_name(interp, class). Other methods, like the one defined by "f = f", would of course need a fallback to attribute lookup (please can we keep the term attribute, which is also used in CPython) - and not property). Therefore the 'find_method' inside Py code should also consult the attributes of the object (and yes properties for per object overrides). Well, this is at least my understanding how it could work. Of course, there is a find_method VTABLE entry, and the implementation of this function calls __get__ which performs the curry function, per the Python specifications. Yes. But currying isn't needed for a plain method call, when the __get__ isn't user defined. Why first curry the function in the first place, create a new curried sub PMC, then during invocation shift arguments, and eventually run it. This isn't the common case. Even if CPython does it like this now (and Python folks are discussing an optimized opcode on pthon-dev), it's not better or more correct - just the result is important. And that works as is now for Parrot method calls. Does Parrot_find_method_with_cache cache the results of the previous call to find_method? It does one dynamic lookup and caches per class/method the result. The cache is invalidated by a store_global (which could influence the dynamic search result). - Sam Ruby leo
Re: Python PMC's
Leopold Toetsch wrote: > >> Note that you would then be caching the results of a curried function >> call. This result depends not only on the method string, but also on >> the particular object upon which it was invoked. > > No the "inner" Parrot_find_method_with_cache just caches the method for > a specific class (obeying C3 method resolution order as in Python). > There is no curried function at that stage. Where does Parrot_find_method_with_cache get this data? Remember, in Python, there are no methods, there are only properties. Of course, there is a find_method VTABLE entry, and the implementation of this function calls __get__ which performs the curry function, per the Python specifications. Does Parrot_find_method_with_cache cache the results of the previous call to find_method? - Sam Ruby
Re: Python PMC's
On Aug 24, 2005, at 19:45, Sam Ruby wrote: Leopold Toetsch wrote: Sam Ruby wrote: The return value is a callable sub. More precisely: a curried function call. This is an important distinction; to see why, see below. A callable sub may be of course a curried one - yes. The interesting optimizations are: - cache the find_method in the global method cache. This happens already, if the method string is constant. Note that you would then be caching the results of a curried function call. This result depends not only on the method string, but also on the particular object upon which it was invoked. No the "inner" Parrot_find_method_with_cache just caches the method for a specific class (obeying C3 method resolution order as in Python). There is no curried function at that stage. - use a PIC [1] (which is tied to the opcode) and cache the final resut of find_method (or any similar lookup) Again, the results will depends on the object. Yes. The nice thing with a PIC is that it is per bytecode location. You have a different PIC and a different cache for every call site. The prologue of PIC opcode is basically: if cache.version == interpreter.version: (cache.function)(args) else: # do dynamic lookup # update cache then repeat The 'version' compare depends on the cached thingy, and is more explicit in individual implementations. But the principle remains always the same: you create a unique id that depends on the variables of the lookup and remember it. Before invoking the cached result you compare actual with cached ids. If there is a cache miss, there are 2 or 3 more cache slots to consult before doing just the dynamic original scheme again (and maybe rewrite the opcode again to just the dynamic one in case of too many cache misses). src/pic.c and ops/pic.ops have an implementation for sub Px, Py - just for fun and profit to show the performance in the MOPS benchmark ;-) The important part in ops/pic.ops is: lr_types = (left->vtable->base_type << 16) | right->vtable->base_type; if (lru->lr_type == lr_types) { runit_v_pp: ((mmd_f_v_pp)lru->f.real_function)(interpreter, left, right); (the lru is part of the cache structure, above code will be only run, when both types are <= 0x) MMD depends on the two involved types. This is compared before calling the cached function directly. As the cache is per bytecode location, there is a fair chance (>95 %) that the involved types for this very code location are matching. The same is true for plain method calls. The callee depends on the invocant and the method name, which is usually a constant. Therefore you can compare the cached version with the actual invocant type and normally, with a match, just run the cached function immediately. Currying is only important for placing the 'self' into the arguments - the actual lookup was already done earlier and doesn't influence the called function. Actually a curried subroutine isa 'Sub' object already and invoked directly withouth any further method lookup. There is no caching involved in the call of a curried sub. - run the invoked Sub inside the same run loop - specifically not within Parrot_run_meth_fromc_args I don't understand this. (Note: it may not be important to this discussion that I do understand this - all that is important to me is that it works, and somehow I doubt that Parrot_run_meth_fromc_args cares whether a given function is curried or not). There are two points to be considered: - currying: the effect is that some call arguments (in this special case the object) are already fixed. The argument passing code has therefore the duty to insert these known (and remembered) arguments into the params for the callee. For the BoundMethod this is of course, shift all arguments up by one, and make the object 'self' the first param of he sub. - the second point is only related to call speed aka optimization. It's just faster to run a PIR sub in the same run loop, then to create a new run loop. The above works because PyInt is a constant. It probably can be extended to handle things that seem unlikely to change very rapidly. Yes. That's the 'trick' behind PIC. It works best the more constant the items are. But as said above, method names and invocants usually don't vary *per byecode location*. Literature states ~95 % of method calls are monomorphic (one type of invocant), and 99,5 % are cached within 4 cache slots. Look at some typical code a.'foo'(x) ... b.'bar'(y) Both method calls have a distinct cache. The method names are constant. Therefore the callee depends only on the invocant. The types of 'a' or 'b' are typically the same (except maybe inside compilers AST visit methods or some such). The same schme applies to plain method or attribute lookups. But the combination of decisions on how to handle the passing of the "self" parameter to a method, keeping find_
Re: Calling positionals by name in presence of a slurpy hash
On Tue, Aug 23, 2005 at 10:11:37AM -0700, Larry Wall wrote: > setting up the proxy hash. It's possible that COW hashes can be made > to work efficiently. We'll need to copy hashes if we want to modify > them to pass to subfunctions, just as when you change your environment > it doesn't affect your parent process's environment variables. I would assume that for parrot, with vtables, a "simple" COW hash would be very efficient. By simple, I mean one that does a complete copy and separation the first time someone tries to write to one of the "copies", rather than the more complex concept of maintaining state on partial copies. My hunch would be that a "simple" system of COW plus an overlay hash would work very well for the case of adding to (or deleting from) a hash passed in as default arguments, because 95% of the time that hash is only directly manipulated by functions back up the call frame, so for the duration of the call would be unchanged. But this is all arm-wavy, and needs real code to analyse before committing to it as a strategy. Nicholas Clark
Re: Demagicalizing pairs
On Wed, Aug 24, 2005 at 10:12:39AM -0700, Chip Salzenberg wrote: > On Wed, Aug 24, 2005 at 08:38:39AM -0400, John Macdonald wrote: > > When calling a function, I would like to be able to have a > > mixture of named and positional arguments. The named argument > > acts as a tab into the argument list and subsequent unnamed > > arguments continue on. > > I see a main point of named parameters to free the caller from the > tyranny of argument order (and vice versa). It seems to me you're > asking for the worst of both worlds. Perhaps I didn't make it clear in my original message - I agree that arbitrary mixing of named and positional is usually a bad thing. The only place where I find it useful is with a group of arguments that are always provided in the same order, used one or more times each by a number of functions, with additional arguments for some/all of those functions. So, a function that takes position and/or vector values would provide a name for each vector/position, but expect each to have an x, a y, and (possibly) a z argument following the name. I saw this in the DO system - a shell written at CDC back in the late 70's. The provided scripts were designed so that all programming scripts used the same sequence of arguments after the OPT keyword, the LINK keywork, etc. As I said originally, the value is diluted in a language with structured data types - you can use a single argument for a position that is a hash or array which contains the x/y/z components within it. The named group helps especially if you generally want to provide separate-but-related arguments. This tends to be things like an optional sub-action that requires multiple parameters if it is used at all. So, I'm mostly saying that a mixture of named and positional arguments is not ALWAYS bad, and that there may be some value in permitting such a mixture in certain circumstances. --
Re: Demagicalizing pairs
I've been trying to thing about how to make this read right without too much line noise. I think Lukes keyword approach ("named") is on the right track. If we want named params at both start and end, then its bound to be a bit confusing. But perhaps we can say that they're always at the end -- but either at the end of the invocant section or the end of the args. Also, "named" is a bit of a clumsy name. "Where" and "given" are taken, so I'll use "with": I think something like these read nicely, without too much line noise: draw_polygon $canvas: @verticies with color => "red"; draw_polygon $canvas with color => "red": @vertices; Dave.
Re: Python PMC's
Leopold Toetsch wrote: > Sam Ruby wrote: > >> Leopold Toetsch wrote: > >>> A stripped down PIR-only, pythonless translation is below. > >> (example: classes aren't global in Python), > > Yes, of course. The stripped down means essential the absence of any > lexical handlings. But as you say, this doesn't matter for these sub and > method calls. Agreed. >> Note that Python provides __get__ methods for you on every function and >> method, i.e., it is *not* optional, getattribute will always succeed. > > That's fine. But if it's not overriden by user code, you exactly know > what it is doing. Therefore you can just emulate it, I think. I'm not as sure as you appear to be, but we can worry about this later. >>> # print foo.f(2) >>> >>># emulate python find_name, which checks attributes too >>>push_eh m_nf >>>$P0 = foo."f"(2) >>>clear_eh >>>goto m_f >>> m_nf: >>># getattribute would also check if __get__ is there >>>$P1 = getattribute foo, "f" >>>$P0 = foo.$P1(2) >>> m_f: > >> Note that the code above would need to be emitted for *every* method >> call, > > Above snippet should not be emitted for a method call, it just emulates > it in PIR. It should demonstrate how Python's find_method() could be > implemented: With that clarification, I agree in principle. > - try to find a method, else > - check attributes > - call __get__, if it is user provided > > (or whatever order CPython actually uses). What's in dynclass/pyclass.pmc matches Python's semantics closer. > The return value is a callable sub. More precisely: a curried function call. This is an important distinction; to see why, see below. >> Ideally, callmethodcc would result in one vtable call to enable >> interesting optimizations for those that care to provide it. The >> default implementation for this vtable entry could be a call to >> find_method and invoke. > > The interesting optimizations are: > - cache the find_method in the global method cache. > This happens already, if the method string is constant. Note that you would then be caching the results of a curried function call. This result depends not only on the method string, but also on the particular object upon which it was invoked. > - use a PIC [1] (which is tied to the opcode) and cache > the final resut of find_method (or any similar lookup) Again, the results will depends on the object. > - run the invoked Sub inside the same run loop - specifically > not within Parrot_run_meth_fromc_args I don't understand this. (Note: it may not be important to this discussion that I do understand this - all that is important to me is that it works, and somehow I doubt that Parrot_run_meth_fromc_args cares whether a given function is curried or not). > [1] polymorphic inline cache > > The important thing is that the method lookup and the subroutine > invocation happens from inside the runloop. Run cores that allow > rewriting of opcodes (prederefed and JIT) can insert faster equivalent > opcodes in these cases. > > Have a look at src/pic.c (which doesn't implement a lot - it's more a > proof of concept now). Anyway here is an example that is implemented: > > new P0, "PyInt"# new_p_sc > > When this opcodes gets execute the first time, the type number of the > class is looked up and then the opcode is replaced with the faster variant: > > new P0, 89 # new_p_ic - arbitrary number for PyInt > > The same scheme can be applied for all these opcodes that consist of any > kind of a lookup (MMD, methods, attributes, ...) and some further > action. The lookup is done once at runtime (and again only after cache > invalidation). The above works because PyInt is a constant. It probably can be extended to handle things that seem unlikely to change very rapidly. But the combination of decisions on how to handle the passing of the "self" parameter to a method, keeping find_method and invoke separated at the VTABLE level, and the semantics of Python make the notion of caching the results of find_method problematic. >> - Sam Ruby > > leo - Sam Ruby
Re: Demagicalizing pairs
On Wed, Aug 24, 2005 at 08:38:39AM -0400, John Macdonald wrote: > When calling a function, I would like to be able to have a > mixture of named and positional arguments. The named argument > acts as a tab into the argument list and subsequent unnamed > arguments continue on. I see a main point of named parameters to free the caller from the tyranny of argument order (and vice versa). It seems to me you're asking for the worst of both worlds. -- Chip Salzenberg <[EMAIL PROTECTED]>
Re: Zcode interpreter release
On Tue, Aug 23, 2005 at 05:09:26PM -0400, Amir Karger wrote: > testportal:~/z>perl -we 'system("cd y")' The right thing to do (tm) here is chdir("y"), but if 'cd' is just an example and not the actual command, the right thing is system LIST form: system command => @args; -- wolverian signature.asc Description: Digital signature
Re: Demagicalizing pairs
On Wed, 24 Aug 2005, Damian Conway wrote: > Larry wrote: > > > Plus I still think it's a really bad idea to allow intermixing of > > positionals and named. We could allow named at the beginning or end > > but still keep a constraint that all positionals must occur together > > in one zone. > > If losing the magic from =>'d pairs isn't buying us named args wherever we > like, why are we contemplating it? I've lost track of the score in this thread, but I thought I would throw a couple pennies into the fountain. I really dread the thought of losing C< name => value > for named parameters. Off the top of my head, ADA, PL/SQL, and php all use that syntax, so it would be a shame to lose something that newbies might find familiar. On the other hand I would like it if the adverbial named parameter style C< :name(value) > were allowed at the begining as well as the end of the parameter list. I think adverbs read better when they are next to the verbs they modify, and I would be nice if I didn't have to resort to macro magic to get them there. Mixing named and positionals is bad though. ~ John Williams
Re: Python PMC's
Sam Ruby wrote: Leopold Toetsch wrote: A stripped down PIR-only, pythonless translation is below. (example: classes aren't global in Python), Yes, of course. The stripped down means essential the absence of any lexical handlings. But as you say, this doesn't matter for these sub and method calls. Note that Python provides __get__ methods for you on every function and method, i.e., it is *not* optional, getattribute will always succeed. That's fine. But if it's not overriden by user code, you exactly know what it is doing. Therefore you can just emulate it, I think. # print foo.f(2) # emulate python find_name, which checks attributes too push_eh m_nf $P0 = foo."f"(2) clear_eh goto m_f m_nf: # getattribute would also check if __get__ is there $P1 = getattribute foo, "f" $P0 = foo.$P1(2) m_f: Note that the code above would need to be emitted for *every* method call, Above snippet should not be emitted for a method call, it just emulates it in PIR. It should demonstrate how Python's find_method() could be implemented: - try to find a method, else - check attributes - call __get__, if it is user provided (or whatever order CPython actually uses). The return value is a callable sub. Ideally, callmethodcc would result in one vtable call to enable interesting optimizations for those that care to provide it. The default implementation for this vtable entry could be a call to find_method and invoke. The interesting optimizations are: - cache the find_method in the global method cache. This happens already, if the method string is constant. - use a PIC [1] (which is tied to the opcode) and cache the final resut of find_method (or any similar lookup) - run the invoked Sub inside the same run loop - specifically not within Parrot_run_meth_fromc_args [1] polymorphic inline cache The important thing is that the method lookup and the subroutine invocation happens from inside the runloop. Run cores that allow rewriting of opcodes (prederefed and JIT) can insert faster equivalent opcodes in these cases. Have a look at src/pic.c (which doesn't implement a lot - it's more a proof of concept now). Anyway here is an example that is implemented: new P0, "PyInt"# new_p_sc When this opcodes gets execute the first time, the type number of the class is looked up and then the opcode is replaced with the faster variant: new P0, 89 # new_p_ic - arbitrary number for PyInt The same scheme can be applied for all these opcodes that consist of any kind of a lookup (MMD, methods, attributes, ...) and some further action. The lookup is done once at runtime (and again only after cache invalidation). - Sam Ruby leo
Re: [pirate] Re: Python PMC's
I agree this following would be cool. However in the general case this type of code inference is HARD to do. I believe that the optimizations you are looking for would require a combination of type inference and graph reduction. PyPy may be the eventual answer. Don't get me wrong, I think it is great and the long term goal. As soon as we compile and run correctly, I'm all about optimizations. Good Insight Michal, Kevin, Michal Wallace wrote: > Hey Sam, >I agree with what you're saying in this >thread. This is slightly off topic, but >I wanted to point something out. > >In general, python has so many places >where things have to be dynamic that you >really can't know this kind of thing at >compile time, especially if you allow for >eval/exec, or if you allow the code to be >used as a module. > >However if you treat the code as a closed >system, *and* you have access to python at >compile time, then we can optimize away a >lot of these questions. > >For example, in your original code: > > def f(x,y): >return y > > class Foo: >f = f >def g(self,y): > return y > > foo = Foo() > > g=foo.g > > print f(0,1) > print foo.f(2) > print g(3) > > >Once you know how python works, it's *obvious* >that this prints 1,2,3. I see no reason why the >compiler couldn't figure this out up front just >by walking the tree. > >In fact, a good optimizing compiler would see the >"return y" lines and just get rid of those methods >completely. > >I'd like to allow for the ability to do certain >optimizations like this up front, sacrificing >flexibility for speed. I know there are many >programs that won't allow for this, but for the >ones that do, I'd like to be able to do a sort >of static compile like this. > >In other words, sometimes a python-like language >is a desirable thing. (But of course this should >all be optional so that we can also be 100% >python compatible) > >Sincerely, > >Michal J Wallace >Sabren Enterprises, Inc. >- >contact: [EMAIL PROTECTED] >hosting: http://www.cornerhost.com/ >my site: http://www.withoutane.com/ >- > >___ >pirate mailing list >[EMAIL PROTECTED] >http://cornerhost.com/mailman/listinfo/pirate >
Re: Demagicalizing pairs
I don't think this example reads very clearly. Visually you have to parse until you see the next => and then back track one word to figure out the key. > move( from=> $x, $y, delta=> $up, $right ); Personally I'd write that as either move(from => [$x, $y], delta => [$up, $right]); OR assuming I has a Position object and a vector object move(from => $pos1, delta => $vec1); The original example just seems difficult to parse. Paul
Re: Python PMC's
Leopold Toetsch wrote: > Sam Ruby wrote: > >> Let me try again to move the discussion from subjective adjectives to >> objective code. Consider: > > [ example code ] > >> If you run this, you will get 1,2,3. >> >> When called as a function, f will return the value of the second >> parameter. When called as a method, the same code will need to return >> the value of the first parameter. > > The calls to f() work more or less w/o problems in branches/leo-ctx5. > >> I'm open to suggestions as to what PIR code should be emitted by Pirate >> to correspond to the above code. > > A stripped down PIR-only, pythonless translation is below. Thanks for doing this. Keeping the conversation anchored with code is very helpful. I have a number of unimportant to the topic at hand comments on that code (example: classes aren't global in Python), but for now, I'll focus on comments that are relevant to the topic at hand. >> The way this currently works with Pirate and the existing Py*.pmc >> closely follows the Python definition, which you can find here: >> >> http://users.rcn.com/python/download/Descriptor.htm#functions-and-methods > > Good explanation, thanks. > >> To illustrate how this works today, lets consider the call to foo.f(2) >> above. This involves both a find_method and an invoke. Lets consider >> each in turn: >> >> * pyclass.find_method first calls getprop on "f". >> * pyclass.find_method then calls "__get__" on the pyfunc returned, >>passing the object on which this method was called on. >> * pyfunc.__get__ creates a new PyBoundMeth object, saving both the >>original function and the object. >> * This PyBoundMeth object is then returned >> >> then: >> >> * pyboundmeth.invoke shifts all registers right, inserts the original >>object as the first parameter, and then calls invoke on the original >>function. >> >> Needless to say, this doesn't necessarily show off Parrot to its best >> advantage from a performance perspective. I tried a number of times to >> argue for a combined "find_and_invoke" VTABLE entry as much of the above >> can be optimized if you know that result of the find_method will only be >> used a single time for an invoke. If you like, you can scan the >> archives to see what reception this suggestion received. > > Well I think that the find_and_invoke is just the callmethodcc opcode as > used in my translation below. The call to __get__ and the BoundMethod > shouldn't be necessary, *if* their is no userdefined descriptor. That > is, when getattribute checks, if the attribute provides __get__ then > call it. Take a look at the definition of callmethodcc in ops/object.ops. It is a find_method followed by an invoke. It is the VTABLE operations that I would like to see combined. Note that Python provides __get__ methods for you on every function and method, i.e., it is *not* optional, getattribute will always succeed. Observe: [EMAIL PROTECTED]:~$ python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> def f(): pass ... >>> f.__get__ >>> class c: ... def m(self): pass ... >>> c.m.__get__ >>> > The yet unsopported thing is > > g = foo.g > ... > g(3) > > which really needs a BoundSub object. But this is just a special case of > currying, or a special case of Perl6's .assuming(). Therefore I think > that Parrot should support this natively. I agree that BoundSub is a special case of currying, or what Perl6 calls assuming. I also agree that Parrot should support this natively. >> - Sam Ruby [snip] Below is from the sample that Leo provided. > # print foo.f(2) > > # emulate python find_name, which checks attributes too > push_eh m_nf > $P0 = foo."f"(2) > clear_eh > goto m_f > m_nf: > # getattribute would also check if __get__ is there > $P1 = getattribute foo, "f" > $P0 = foo.$P1(2) > m_f: > print_item $P0 > print_newline Note that the code above would need to be emitted for *every* method call, as there is no way of knowing at compile time what property you will get and how it is defined. In addition to increasing code size, it is error prone: for example, the above only appears to check for __get__ if the call fails, so the pie-thon b0 test will fail as hooking __get__ is integral to the way it provides a trace. Also, there are other reasons that exceptions may be thrown, and we are incurring the overhead of setting up exception blocks, etc again, this would apply to *every* method call. An alternate solution without these problems would be > # print foo.f(2) > $P0 = foo."f"(2) > print_item $P0 > print_newline Ideally, callmethodcc would result in one vtable call to enable interesting optimizations for those that care to provide it. The default implementation for this vtable entry could be a call to find_method and invoke. - Sam Ruby
Re: Python PMC's
Sam Ruby wrote: Let me try again to move the discussion from subjective adjectives to objective code. Consider: [ example code ] If you run this, you will get 1,2,3. When called as a function, f will return the value of the second parameter. When called as a method, the same code will need to return the value of the first parameter. The calls to f() work more or less w/o problems in branches/leo-ctx5. I'm open to suggestions as to what PIR code should be emitted by Pirate to correspond to the above code. A stripped down PIR-only, pythonless translation is below. The way this currently works with Pirate and the existing Py*.pmc closely follows the Python definition, which you can find here: http://users.rcn.com/python/download/Descriptor.htm#functions-and-methods Good explanation, thanks. To illustrate how this works today, lets consider the call to foo.f(2) above. This involves both a find_method and an invoke. Lets consider each in turn: * pyclass.find_method first calls getprop on "f". * pyclass.find_method then calls "__get__" on the pyfunc returned, passing the object on which this method was called on. * pyfunc.__get__ creates a new PyBoundMeth object, saving both the original function and the object. * This PyBoundMeth object is then returned then: * pyboundmeth.invoke shifts all registers right, inserts the original object as the first parameter, and then calls invoke on the original function. Needless to say, this doesn't necessarily show off Parrot to its best advantage from a performance perspective. I tried a number of times to argue for a combined "find_and_invoke" VTABLE entry as much of the above can be optimized if you know that result of the find_method will only be used a single time for an invoke. If you like, you can scan the archives to see what reception this suggestion received. Well I think that the find_and_invoke is just the callmethodcc opcode as used in my translation below. The call to __get__ and the BoundMethod shouldn't be necessary, *if* their is no userdefined descriptor. That is, when getattribute checks, if the attribute provides __get__ then call it. The yet unsopported thing is g = foo.g ... g(3) which really needs a BoundSub object. But this is just a special case of currying, or a special case of Perl6's .assuming(). Therefore I think that Parrot should support this natively. - Sam Ruby leo # # def f(x,y): #return y .sub f .param pmc x .param pmc y .return (y) .end # # class Foo: .sub create_Foo .local pmc self, Foo self = subclass "Py", "Foo" addattribute self, "f" addattribute self, "g" Foo = find_global "Foo", "Foo" store_global "Foo", Foo .end .namespace ["Foo"] .sub Foo .local pmc o o = new "Foo" .return (o) .end #f = f #def g(self,y): # return y .sub __init method .local pmc f, g f = find_name "f" setattribute self, "f", f g = find_name "g" setattribute self, "g", g .end .sub g .param pmc self .param pmc y .return (y) .end .namespace [""] .sub main @MAIN .local pmc foo, g init_python() create_Foo() # # foo = Foo() # foo = Foo() # g=foo.g # g = getattribute foo, "g" # TODO create bound Sub inside gettattribute like so ## $I0 = isa g, "Sub" ## unless $I0 goto no_sub ## $P0 = new .BoundSub, g ## assign $P0, foo ## g = $P0 ## no_sub: # print f(0,1) $P0 = f(0, 1) print_item $P0 print_newline # print foo.f(2) # emulate python find_name, which checks attributes too push_eh m_nf $P0 = foo."f"(2) clear_eh goto m_f m_nf: # getattribute would also check if __get__ is there $P1 = getattribute foo, "f" $P0 = foo.$P1(2) m_f: print_item $P0 print_newline # print g(3) #$P0 = g(3) #print_item $P0 #print_newline .end # .sub init_python .local pmc py, bs py = newclass "Py" .end
Perl 6 code - a possible compile, link, run cycle
WRT to PIL and compilation and all that, I think it's time to think about how the linker might look. As I see it the compilation chain with the user typing this in the prompt: perl6 foo.pl perl6 is a compiled perl 6 script that takes an input file, and compiles it, and then passes the compiled unit to the default runtime (e.g. parrot). perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. The compiler produces an object representing a linkable unit, which will be discussed later. At this point the runtime kicks in. The runtime really runs compiled byte code for the runtime linker, which takes the compiled unit that the compiler emitted and prepares it for execution. The runtime linker checks if any inclusions of outside code have been made, and if so, invokes a search routine with the foreign module plugin responsible. For example 'use python:Numerical' will use the pyhon module plugin to produce a linkable unit. A given module will normally traverse a search path, find some source code, check to see if there is a valid cached version of the source code, and if needed, recompile the source code into another linkable unit. As the linker gets new linkable units it checks if they have any dependencies of their own, and eventually resolves all the data and code that modules take from one another. The resulting mess has only one requirement: that it can be run by the runtime - that is, byte code can be extracted out of it. If the modules expose more than just byte code with resolved dependencies in the modules, for example type annotations, serialized PIL, serialized perl 6 code, and so forth, it may, at this point, do any amount of static analysis as it pleases, recompiling, relinking, optimizing, inlining, performing early resolution (especially of MMD) and otherwise modifying code (provided it was asked to do this by the user). The optimization premise is: by the time it's linked it probably won't change too much. Link time isa magictime for resolving calls, inlining values, folding newly discovered constants, and so forth. Furthermore, a linker may cache the link between two modules, regardless of the calling script, so that optimization does not have to be repeated. The result is still the same: code that can be executed by the runtime. It just might be more efficient. The linker also must always be able to get the original version of the linked byte code back, either by reversing some changes, or keeping the original. At this point the runtime's runloop kicks in, starting at the start point in the byte code, and doing it's thing. Runtime loading of code (e.g. eval 'sub foo { }') simply reiterates the above procedure: 'sub foo { }' is compiled by the compiler, creating a linkable unit (that can give 'sub foo { }' to the world). The runtime linker gets a fault saying "byte code state is changing, $compiled_code_from_eval is being ammended to $linked_byte_code_in_runtime_loop". The linker must then link the running code to the result of eval. To do this it may need to undo it's optimizations that assumed there was no sub foo. For example, if there is a call to 'foo()' somewhere in foo.pl, the linker may have just inlined a 'die "no such sub foo()"' error instead of the call. Another linker may have put in code to do a runtime search for the '&foo' symbol and apply it. The linker that did a runtime search that may fail doesn't need to change anything, but the linker which inlined a fatal error must undo that optimization now that things have changed. The behavior of the linker WRT to such things is dependant on the deployment setting. In a long running mod_perl application there may even be a linker that optimizes code as time goes by, slowly changing things to be more and more static. As the process progresses through time, the probability of new code being introduced is lower, so the CPU time is invested better. Furthermore, latency is not hindered, and startup is fast because the linker doesn't do any optimizations in the begining. This is part of the proposed optimizer chain, as brought up on p6l a month or so ago. Anyway, back to runtime linking. Once the code is consistent again, e.g. calls to foo() will now work as expected, eval gets the compiled code, and runs it. It just happens that 'sub foo { }' has no runtime effects, so evak returns, and normal execution is resumed. To get the semantics of 'perl -c' you force the linker to resolve everything, but don't actually go to the runloop. Linkable units are first class objects, and may be of a different class. This has merits when, for example, a linkable unit is implemented by an FFI wrapper. The FFI wrapper determines at link time what the foreign interface looks like, and then behaves like the linkable unit one might expect if it were a native interface. It can generate bytecode to cal
Re: Demagicalizing pairs
On Wed, Aug 24, 2005 at 04:27:03PM +1000, Damian Conway wrote: > Larry wrote: > > >Plus I still think it's a really bad idea to allow intermixing of > >positionals and named. We could allow named at the beginning or end > >but still keep a constraint that all positionals must occur together > >in one zone. > > If losing the magic from =>'d pairs isn't buying us named args wherever we > like, why are we contemplating it? When calling a function, I would like to be able to have a mixture of named and positional arguments. The named argument acts as a tab into the argument list and subsequent unnamed arguments continue on. That allows you to use a name for a group of arguments: move( from=> $x, $y, delta=> $up, $right ); In this case, there could even be an optional z-coordinate argument for each of the from and delta groups. The named group concept works well for interfaces that use the same groups in many different functions. It is especially powerful in languages which do not have structured types, which means it is not so necessary in Perl, but even here, you often are computing the components (like $up and $right above) separately, rather than always computing a single structured value (which would mean writing delta=>(x=>$up, y=>$right) instead). --
Re: Zcode interpreter release
Perl's systen() tries to execute an external command directly when doesn't find any any shell metacharacters in the command line. Otherwise, it defaults to the shell. -- avva "There's nothing simply good, nor ill alone" -- John Donne
Re: ~ and + vs. generic eq
On Tue, Aug 23, 2005 at 16:32:37 -0700, Larry Wall wrote: > Hmm, well, I don't think >>&op<< is valid syntax, but you did say > "semantics", so I can't criticize that part. :-) What is >><<, btw? Is it &circumfix:{'>>','<<'} (Code &op --> Code); # takes some code, returns a listop or &precircumfix:{'>>','<<'} (Code &op, [EMAIL PROTECTED] --> List); > I don't know how close ~~ and eqv will end up. There are some > differences in emphasis, and when two operators get too much like each > other, I tend to add more differences to make them inhabit different > parts of the solution space. One current difference is that, despite > the symmetry of ~~, it's not actually a symmetrical operator much of > the time, such as when matching values to rules. ~~ is intended to > be heavily dwimmical, so it's allowed to do various kinds of abstract > coercions to figure out some mystical "good enough" quotient. But eqv > on the other hand should probably be false in asymmetrical situations. > The implementation of ~~ may delegate to eqv in certain symmetrical > situations, of course. Right... Magic is defined in the "base" definitions of ~~: &infix:<~~> ($x, Rule $r) { ... } &infix:<~~> ($x, Code &test) { code($x) } And so on and so forth, and then it is extended by the extender to make cool aggregate operations, but even this doesn't have to be the same for ~~ and eqv, it's just that eqv should have good builtins for collections, is all. > should say "true", since those are the same values, and they can't > change. However, in Perl-5-Think, [1,2,3] produces mutable arrays, > so unless we come up with some kind of fancy COW for [1,2,3] to be > considered immutable until someone, er, mutes it, I think eqv would > have to return false, and consider two such objects to be different > values (potentially different if not actually different). Well, when I use it as if (@array eqv [1, 2, 3]) { } I think it's obvious that I'm checking "what is the value right now", and ditto when I say my $str = "foo"; $hash{$str} = 1; $str ~= "bar"; $hash{$str}; # not the same Arguably use of an array as a hash key is using a reference to a container, and use of a scalar as a hash key is using the value inside a container, so in a sense the hash key didn't change when I appended the string, but this distinction is subtle and mostly an implementation detail. > It would be possible to declare %hash some way that forces a "snapshot" > via some kind of serialization or other, but then it gets hard to keep > the identity around. Then the question arises how we doctor > > if [1,2,3] eqv [1,2,3] { say "true" } else { say "false" } eqvs! it pronounces very well: 'eekyoovies'. Try it: if 1 2 3 eekyooviesses 1 2 3 say true, otherwise say false It's fun, but I hate it even more than eqv ;-) > to do the same snapshot comparison. Arguably ~~ could do it, since it's > explicitly *not* about identity. > > if [1,2,3] ~~ [1,2,3] { say "true" } else { say "false" } But ~~ is not "is the same" it's "matches". This is also true, and not what I want eqv for: if [1, 2, 3] ~~ [code { 1 }, rx/\d+/, Num] { say "true" } else { say "false" } > Or we could have a more explicit way of doing whatever it is that the > snapshot hash does to each argument. > > if [1,2,3].snap eqv [1,2,3].snap { say "true" } else { say "false" } I think the opposite is better, make snapshotting by default, and mutable value equality false by saying [1, 2, 3] eqv [1, 2, 3] :always # or :forever -- () Yuval Kogman <[EMAIL PROTECTED]> 0xEBD27418 perl hacker & /\ kung foo master: *shu*rik*en*sh*u*rik*en*s*hur*i*ke*n*: neeyah pgpqB6BHxX5TY.pgp Description: PGP signature
Re: ~ and + vs. generic eq
Larry wrote: Or we could have a different operator that coerces like == and eq, only via .snap: if [1,2,3] equals [1,2,3] { say "true" } else { say "false" } (Actual name negotiable, of course). The advantage of the latter approach is that you can say @foo >>equals<< @bar and the .snaps are automatically distributed. Otherwise you'd have to say @foo<<.snap >>eqv<< @bar<<.snap which is a pain. On top of which, equals doesn't actually have to implemented in terms of .snap--it could just compare the current values of the mutable objects directly. (Just as =:= doesn't have to be implemented in terms of .id.) Just a meta-point...one thing we really do need to be careful of is not ending up with 17 different "equality" operators (like certain languages I shall refrain from naming). So far we're contemplating: =:= ~~ == eq eqv equals Do we really need even that many??? Damian
Zcode interpreter release
Hi! I adopted the Zcode interpreter that leo posted in February. Luckily, he did the hard parts. I added some opcodes, README/CHANGES/TODO files, and testing, and, finally, did my first Parrot checkin. How exciting! There's much much more to do - 41 more opcodes just for version 3 of the Z-machine. And then of course there's Dan's Holy Grail of making the Zops into native parrot ops... The only bad news is there's something wrong with my make test. The following work: - perl t/harness in languages/Zcode - perl t/whatever.t in languages/Zcode - make test in languages/Zcode - make Zcode.test in languages/ However, I couldn't put Zcode into languages/testall. It breaks when it tries to run my tests. I managed to narrow this down to a very weird Perl behavior I don't understand at all: testportal:~>mkdir z testportal:~>cd z testportal:~/z>mkdir y testportal:~/z>touch y/foo testportal:~/z>perl -we 'system("cd y")' Can't exec "cd": No such file or directory at -e line 1. testportal:~/z>perl -we 'system("cd y && ls")' foo Because of the above, my tests break when they do: run_command("$parrot z3.pir $test_file", CD=>"Zcode" (I need to cd into Zcode so that I can run z3.pir and find the z*.pir files it depends on.) Any thoughts on why this is happening? -Amir Karger
Re: Demagicalizing pairs
Larry mused: On the other hand, I'm not all that attached to colon itself. I *am*!!! If, as proposed elsewhere, we get rid of the %Foo:: notation in favor of some Foo<> variant, then trailing :: becomes available (ignoring ??/:: for the moment), and new Dog:: tail => 'long' almost makes sense, insofar as it kinda looks like it's marking Dog as a type name, even though it isn't. But new Dog:: :tail doesn't look so good. Nor do object methods: wag $dog:: 'tail'; say $fh:: $whatever; On the other hand, looking at it from the other end, the MMD notation tiebreaking notation is a little hard to spot, since colon is easy to miss. Is it??? I've been writing quite a bit of MMD notation, and I think the colon is very obvious...and exactly the right visual "weight". Maybe there's something that shows up better in a signature that also works as the invocant marker and, by extension, the indirect object marker. Since it's an ordering kind of thing, you'd kind of like to work > into it somehow, since the left side is of "greater" importance than the left. Unfortunately, though, "the good ones are all taken". Maybe some digraph like method new ($what*> $:tail) {...} method new ($what+> $:tail) {...} method new ($what.> $:tail) {...} method new ($what|> $:tail) {...} method new ($what>> $:tail) {...} giving new Dog*> :tail new Dog+> :tail new Dog.> :tail new Dog|> :tail new Dog>> :tail I guess that last one is eqivalent to: method new ($what» $:tail) {...} new Dog» :tail which I could maybe get used to. It kind of looks like a prompt to me. Not one of these is anything close to as readable as: new Dog: :tail name $dog: 'Rover'; say fh: @whatever *Please* don't give up on the colon there. It's much more readable. I especially like it for setting up objects: $person = Contact.new; first_name $person: "George"; family_name $person: "Bush"; title $person: "President"; email $person: "[EMAIL PROTECTED]"; spouse $person: search $contacts: "Laura"; The ordinary MMD might look like multi foo ($a, $b, $c» $d) And Lisp-like MMD fallback on every argument would look like multi foo ($a» $b» $c» $d») I suppose that particular use of » could be construed as encouraging people not to do that. :-) I truly believe that using the French quotes or (shudder!) their Texan equivalents here would be a dire step backwards. They're already overloaded for word lists and hyperoperators. I think using them for an invocant marker as well would simply be too much. The colon really was (and still is) the right choice here. Damian
Re: Demagicalizing pairs
Larry wrote: Plus I still think it's a really bad idea to allow intermixing of positionals and named. We could allow named at the beginning or end but still keep a constraint that all positionals must occur together in one zone. If losing the magic from =>'d pairs isn't buying us named args wherever we like, why are we contemplating it? I suspect a lot of people would still prefer to write named args with =>, I'd say so. so we should put some thought into making it syntactically trivial, if not automatic like it is now. Even making named() a listop would help. I'd say that's the best alternative. I'd certainly prefer that to repurposing :(...) I hate to say it, but the named args should probably be marked with : instead of + in the signature. One other idle thought is that, if we don't mind blowing a different kind of consistency, and if we s/+/:/ in sigs, a sig containing :$foo could instead be written $:foo (presuming we take : away from privates as we've postulated), Yes please. Underscore is much better in that role. Damian