Geoffrey Broadwell a écrit :
On Tue, 2008-07-22 at 09:03 +0200, François Perrad wrote:
Ok, talking about libraries :
Lua compiler & Lua Standard Libraries are complete (as far as the
current Parrot supports it).
So, since April 2008, I wrote some extension libraries for Lua
Since mid-June 2008, I tried to write extension libraries for Pipp
(PHP supplies more than 2500 functions !!!, huge work)
>From a couple comments you make later, it sounds like you're aiming to
be perfectly API compatible with the original library implementations
for Lua and PHP, so that moving to Parrot is a "drop-in" replacement as
far as the user's source code is concerned; am I correct in this
assumption?
Yes, fully compatible. For me, a Lua-like or PHP-like is just a toy.
I believe that PHP could be the killer application for Parrot
Why PHP in particular? Is it because of what Parrot can do for PHP
(performance, stability (we hope), interop, ...), or because of what PHP
can do for Parrot (bring lots of eyeballs to Parrot, as Tim Bunce
mentioned for embedding Parrot in Java)?
- lua/src/lib/md5.pir & sha1.pir : wrapper over PMC MD5 & SHA1 (I wrote
them on the outside of Lua)
- lua/src/lib/random.pir : wrapper over
library/Math/Random/mt19937ar.pir (I wrote it on the outside of Lua)
- lua/src/lib/uuid.pir : wrapper over library/uuid.pir (I wrote it on
the outside of Lua)
Meaning, you wrote the PMCs / Parrot library PIR before working on Lua,
without any Lua dependencies?
Yes, they depend only on Parrot.
- lua/src/lib/lfs.pir : Lua File System library, over the PMC OS
(still incomplete)
- pipp/src/common/php_base64.pir : wrapper over library/MIME/Base64.pir
(but incomplete because PHP API has more options than MIME/Base64 Perl API)
- pipp/src/common/php_pcre.pir : wrapper over library/pcre.pir
(incomplete because no PhpArray support, and the NCI PCRE is incomplete
and seems very old)
Can you please create tickets for the stuff that's missing that you
need? Maybe we can get someone to take on improvements of these, so you
don't have to ....
As I need to write a NCI wrapper over gmp, I begin my study of NCI by
using the OpenGL one (with Lua).
Makes sense.
If I try to summarize my experiment with libraries :
Parrot will supply 3 kinds of common libraries for HLL
- written in Perl6, but currently not available (I don't know if NQP is
suitable for writing library)
In theory perhaps, but NQP has a lot of limitations (intentionally).
- written in PIR, but only for bootstrap, because as Bernhard
Schmalhofer tell "PIR is not a decent language"
+ good for libraries not common (in fact equivalent to builtins)
- lua/src/lib/bitlib.pir
- lua/src/lib/lfs.pir
- pipp/src/common/php_ctype.pir
- pipp/src/common/php_math.pir
- pipp/src/common/php_type.pir
+ good when no native library available (but a full test suite is
needed)
- library/Math/Random/mt19937ar.pir (Mersenne Twisted)
I'm not sure what you mean by "only for bootstrap" here. Do you mean
because of potential performance problems implementing heavy math code
in PIR as opposed to C?
Currently, the whole parrot/perl6 project is in bootstrap stage, ie.
Perl6 is not available.
So, the choice is :
- waiting for Perl6 or other magic tools
- writing PIR today, and doing the job
- binding over native (C/C++) shared libraries
+ with native PMC (C compile/link)
- sometime, for security reason, a static linkage is mandatory
(libssl is shared lib, but its subset libcrypto is static lib)
- other advantage, PMC allows direct OO interface
And for security libraries, being able to precisely control buffer
copies is important ....
+ with NCI : the best way (no C compile/link)
- but only procedural interface (no direct OO)
How many system libraries provide a true OO interface only, with no
procedural or procedural-pretending-to-be-OO interface (like GNOME, for
instance)? What functionality do we actually *need* to support those
stragglers?
2 designs choices :
- For long term maintenance, I write PIR close to original C. For
example, I start Lua on Parrot aligned with version 5.0.2 and now it's
5.1.3. And in most of case, the original C is the only valid (updated)
user & requirements documentation.
Like the old "Perl 5 (the language) *is* what perl5 (the VM) *does*"
problem ....
- I try to emit the same (as possible) error or warning messages than
the original implementation and to have the same interface. Rule of
Least Surprise for the end user. And I could run the test suite against
the original implementation.
Makes sense.
So, in the init function of a library, I wrote (or generate) some boring
code like :
.const .Sub _mod_funct = 'func'
_mod_func.'setfenv'(_lua__GLOBAL)
set $P1, 'func'
_mod[$P1] = _mod_func
I wait for a IMCC improvement (hi kjs) in order to support :
.macro register(tname, fname)
.const .Sub $fname = .fname
$fname.'setfenv'(_lua__GLOBAL)
set $P1, .fname
.tname[$P1] = $fname
.endm
Again, what's missing here? Can you write up an RT ticket for the
missing functionality? Or is there one already?
- Currently, the LuaTable implementation don't support the Parrot
namespaces API. But I think it's doable.
That would probably be very useful!
- Since my first implementation, libraries & generated code use the 2
same patterns for function preamble :
.sub 'func_with_vararg' : anon
.param pmc arg1 :optional
.param pmc argn :optional
.param pmc vararg :slurpy
...
.sub 'func_without_vararg' :anon
.param pmc arg1 :optional
.param pmc argn :optional
.param pmc extra :slurpy # ignore silently extra parameters
...
Why do you silently ignore extra arguments? Is that part of the
standard Lua design?
It's the standard Lua behavior. So, it's a requirement.
After parameters are checked by helpers that could perform some type
conversion (to_number, to_string, ...
) and eventually a default value support.
I could do the job with :optional, and without any :opt_flag.
It's a old design choice, perhaps now :opt_flag is better.
Gotcha.
All that I do in a binding for Lua, I already did it in the whole Lua.
Now, I expect two things of Parrot :
1) for language interoperability :
an automatic conversion between LuaString & PhpString (if they are
defined with "does string maps String").
problems come with LuaTable (with array & hash part) and Perl6Hash
We've been discussing this in #parrot the last couple days. Evolving
notes by donaldh++ at:
http://www.perlfoundation.org/parrot/index.cgi?inter_hll_mapping_notes
2) thin NCI binding for all native shared libraries (gmp, GD, OpenGL,
MySQL, ...)
I think this layer doesn't add functionality or behavior.
With generated code from .h
Specific OSes path search hidden
Nodnod.
A way to import all methods in my namespace, without get_global or
get_hll_global for each symbol.
This should be part of the Namespace API. Unfortunately, it's rather
underspecced right now, which is part of why I had to do it myself for
OpenGL.pir. It's pretty easy once you know how to grab a caller's
namespace. In any case, this is a perfect example of why LuaTable
should probably support the Namespace API.
If NCI binding is thin, it could be common & reusable for each HLL.
But in other hand, each HLL needs a specific layer with its specific
argument checking.
When you study wrappers in C interpreters, there are thin. It's just
glue (generated or not) for building a bridge between languages. And
this glue depends more on target interpreter than on library wrapped.
True, but I think there is more in common between dynamic languages than
we get from just a thin condom over the C lib. Most dynamic languages
support with ease things like hashes, optional args, and slurpy args
that are enough of a pain in the C world that many APIs don't use them.
Or worse, force C-isms like filling up an options structure (either
directly or through some painful one-option-at-a-time API), or a
NULL-terminated packed array of key-value pair structures (differently
implemented by every API, of course), or what have you. We should
provide optional helper routines for this. More below.
In early version, wxPython allows only calling convention close to C++
one, now current versions are more pythonic (named parameter, ...). It's
done in Python.
In the same way, when you write a Perl 5 extension, the XS part is
minimum, the perlish interface or OO interface is written in a .pm file.
Sure. But I'm not so sure about your next assertion:
I think argument type checking & conversion are HLL specific.
As I said above, I think most dynamic languages share some common ways
of doing this. I'm all in favor of providing a thin, raw NCI wrapper
that an HLL can use directly. But I think that for each library where
it makes sense, Parrot should *also* provide a thicker wrapper on top of
the thin one, which provides a less verbose and hackish version of the
API. HLLs can then choose to use any of the following:
1. the raw thin wrapper directly
2. the medium-weight cross-HLL wrapper directly
3. a hand-rolled HLL-specific API on top of #1 (your Lua design)
4. a hand-rolled HLL-specific API on top of #2
I submit that #2 is the easiest way for a HLL to make use of a new
Parrot library, and #4 can be relatively light coding work for a nice
boost in end-user satisfaction. #3 should only be needed to provide
strict compatibility with an existing implementation that does things so
differently from #2 that simple impedence matching is not sufficient to
mask the different world view.
This layer
could be written in PIR or in HLL (when checking subroutines are
callable from HLL).
I don't quite understand what you mean by that parenthetical comment.
Currently, in Pipp, PIR is the only way because the compiler is in early
stage (but in progress). But this way is only valid for binding over
shared libraries. PHP has many specific extension written in pure C, I
think these extension must be rewrite in PHP (not a job for PIR).
Why were they written in C instead of PHP in the existing PHP
implementation? Is there any reason we wouldn't be providing a nice
service to the PHP community by doing the rewrite, totally aside from
our needs for the Parrot project?
I think they use C for performance reason.
Finally, improvement in runtime/parrot/library/OpenGL.pir :
at this time, just split the function '_export_all_functions' into
'_export_GL_functions' & '_export_GLUT_functions'.
Should be easy. However, the OpenGL header parser will try to detect
and wrap any GL-related API it can find. So we can do this two ways:
1. Have a single function that takes one or more API names
(GL, GLU, GLE, AGL, GLX ...) and exports them. This gets
closer to acting like I believe the Namespace API intended.
2. Have an export function per API, as you suggested; we'll need
on the order of a dozen of these to handle the GL APIs seen on
each of our platforms. It's a fair amount of copied boilerplate,
but may be mildly more efficient at run time (though I certainly
hope that symbol export isn't a bottleneck for anyone ...).
In either case, we need to decide what happens when the user tries to
export an API that hasn't been wrapped. I'm thinking exception ....
-'f
