Re: pyvm -- faster python

[Stelios Xanthakis]

Hi Michael

[...]
> 
> Releasing open source means that people *may* fix their own bugs, or
> abandon the code.
[...]

I agree with all the points made.

Moreover let me add that "code is one expression of a set of good
ideas", and ideas want to be free!  ;)

I've decided to release the source code of pyvm as soon as it's ready.

Right now *it doesn't* make much sense to give the source because it is
still at an early development stage.  Even if I did and people sent
patches they wouldn't apply because I still make big changes to the
architecture of it.  I'd like to keep it in this status where I can
modify the structure of the program until it becomes really
developer-friendly.  And IMO it doesn't make sense to release incomplete
open source projects: either give something that's good and people can
happily hack, or don't do it at all.  Giving out the source of an
unstable project will most likely harm it (see CherryOS incidents).

The bottomline is that I estimate that pyvm will be ready within
the summer.


Thanks,

Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Michael Sparks]

Stelios Xanthakis wrote:
...
>- It's incompatible with CPython. Not all programs run.
...
> - The demo is an x86/linux binary only. You shouldn't trust binaries,
>run it in a chrooted environment not as root!
> 
> Hope it works!

Whatever the merits of a system like this, a closed system with bugs
(read: incompatibility with the standard python) will be considered to
"not work" unfortunately - unless people can fix the (or get fixed)
bugs they encounter. 

Releasing as closed source guarantees people will come back to you to
get the code fixed, or abandon using your code in favour of CPython,
Jython or IronPython if they want speed now. (Or PyPy if they want
speed later :-)

Releasing open source means that people *may* fix their own bugs, or
abandon the code.

In your release notes you state:
> WHERE IS THE SOURCE?:
> The source code of pyvm is not yet released.  Whether it will and
> 'when' depends on the interest of the community.  Right now I cannot
> afford the maintainance costs (fix the source, remove/insert comments,
> write docs, fix known but harmless bugs, process bug reports, etc). 
> I'm not making any money from pyvm, but at least I'd like to avoid
> paying for it too! 

Whilst that's fair enough - it's your code, your decision - it might be
worth considering that if these are your only objections consider this:
   * Interest from the community is likely to be low unless you release
 the source. It will be an interesting curio, but no more than that.
   * Releasing as open source does NOT imply you have to support (or
 market) the code - it's simply releasing.
   * If the code isn't stable, bear in mind that the existing python
 test suite can largely be used to test your VM and improve it - if
 you have something that works and you intend at *some* point to
 release the code as open source the sooner you do so, the faster
 your project *may* mature.

At the end of the day though, it's your code, you choose what to do with
it. Personally I find your project curious, and if you had fun creating
the project (or its useful in some other way), then it strikes me as a
positive thing (your release URL implies you're a student!).

Best Regards,


Michael.
-- 
[EMAIL PROTECTED]
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Susan A. Smith]

I have read about parrot.  How is that progressing?

Stelios Xanthakis wrote:
> Hi.
> 
> pyvm is a program which can run python 2.4 bytecode (the .pyc files).
> A demo pre-release is available at:
> http://students.ceid.upatras.gr/~sxanth/pyvm/
> 
> 
> Facts about pyvm:
> - It's FAST. According to the "cooked-bench" benchmark suite it finishes
>   in 55% of the time python takes;)
> - It's SMALL. Currently the source code is under 15k lines with the
>   builtin modules.
> - It's new. Uses no code from CPython.
> - It's incomplete. Not even near the stability and quality of python.
>   It needs A LOT of work before it can be compared to CPython.
>   Moreover, at the time it lacks many things like closures, long numbers
>   new style classes, etc.
> - It's incompatible with CPython. Not all programs run.
> - The C API is incompatible. You can't run C modules (a thin wrapper to
>   make pyvm appear as libpython *could* be possible but not a goal
>   AFAIC)
> - The demo is an x86/linux binary only. You shouldn't trust binaries,
>   run it in a chrooted environment not as root!
> 
> Hope it works!
> 
> Cheers,
> 
> Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Terry Reedy]

"Stelios Xanthakis" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]

> Also, for the other part of the thread, I think that bytecode may
> be in fact faster than machine code JIT.  Here is a theory:

>Suppose that for each algorithm there is the "ideal implementation"
>which executes at the speed limit where it can't be done any faster.
...
>Now for such very
> big programs bytecode has the advantage that it achieves very good
> code re-use;  everything is those 400kB of the libpython core and that
> does not increase with the size of the program.  In this case, bytecode
> is the driver that operates the buldozer (the buldozer being C).

This, of course, is exactly what CPython does.  Bytecodes index into a 
switch table that jumps to interface code that calls C routines, most of 
which have been optimized, on and off, for about 15 years.

>  So the
> theory is that for very complex programs a bytecode+core-library is
> closer to the ideal implementation than a JIT which produces/compiles
> megabytes of machine code.
>
> Evidence for that may be that all those JIT efforts don't get any
> great speed ups (psyco is different as it exposes static-ness).
>
> Of course, if we care about speed we'll use Pyrex to convert some
> heavy routines to C.  For example if we need FFT, it's madness to
> do it in the HLL.  Make it part of the buldozer.

And of course, it already is (replacing 'bulldozer' with 'set of 
construction equipment'), under the name NumPy or Numarray.  And both of 
these can take advantage of an Atlas implementation of the BLAS, when 
available, that is tuned to a specific platform with the intention of being 
the 'ideal implementation' of the Basic Linear Algebra Subroutines for that 
platform.

So I congratulate you on conceptually re-inventing Python + C extensions. 
But I haven't understood yet what conceptual variations you have or plan to 
make on the current implementation.

Current CPyVM simulates a stack machine.  This has the virtual of 
simplicity and clarity even for someone like me who has never seriously 
written assembler. People have made proposals to switch to simulating a 
(more complex) register machine, with a new set of bytecodes, that 
theoretically would be faster.  Do you have any opinion on this issue?  You 
seem to like the current byte code set while wanting to run faster.

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Armin Steinhoff wrote:

>>> Is there a working version of lwc ???
>>>
>>
>> pyvm is written in lwc-2.0 which is not yet released because
>> nobody's using it.
> 
> 
> As you mentioned it ... lwc-2.0 is used for pyvm. So it is used :)
> 
> Do you have an idea when lwc-2.0 will be releast ?
> 
> Everyone who are interested in pyvm will need it ...
> 

It will be included together with pyvm.  Normally if you want to just
compile pyvm you do not need lwc, it's a preprocessor that generates C
from C++ and the C for pyvm will be pre-generated.  You'd need lwc for
modifying/hacking pyvm.


Stelios

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Armin Steinhoff]

Stelios Xanthakis wrote:
> Armin Steinhoff wrote:
> 
>>>
>>> pyvm has that. A big part of it is written in "lightweight C++" [1].
>>
>>
>>
>> Realy ?  I have downloaded the lwc distribution and checked it out.
>> It was a surprise that none of the examples are working.
>> I'm using SuSE 9.0 with  gcc 3.3.1 ...
>>
> 
> :(
> 
>> Is there a working version of lwc ???
>>
> 
> pyvm is written in lwc-2.0 which is not yet released because
> nobody's using it.

As you mentioned it ... lwc-2.0 is used for pyvm. So it is used :)

Do you have an idea when lwc-2.0 will be releast ?

Everyone who are interested in pyvm will need it ...


-- Armin


> 
> 
> Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Stelios Xanthakis wrote:

> 
> Also, for the other part of the thread, I think that bytecode may
> be in fact faster than machine code JIT. 

Forgot to add: It depends of course on how good is the bytecode.
IMO Python's bytecode is pretty good for its purpose which is
executing a dynamic language with dynamic types and namespaces.

Also, inside pyvm some things are done with internal bytecode
objects.  One such thing is "list(generator)".  It has been
proved that doing this in bytecode is faster than C.

Stelios


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Armin Steinhoff wrote:
>>
>> pyvm has that. A big part of it is written in "lightweight C++" [1].
> 
> 
> Realy ?  I have downloaded the lwc distribution and checked it out.
> It was a surprise that none of the examples are working.
> I'm using SuSE 9.0 with  gcc 3.3.1 ...
> 

:(

> Is there a working version of lwc ???
> 

pyvm is written in lwc-2.0 which is not yet released because
nobody's using it.


Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Armin Steinhoff]

Stelios Xanthakis wrote:
> Kay Schluehr wrote:
> 
>>
>> Yes. What we are seeking for and this may be the meaning of Armins
>> intentiously provocative statement about the speed of running HLLs is a
>> successor of the C-language and not just another VM interpreter that is
>> written in C and limits all efforts to extend it in a flexible and
>> OO-manner. Python is just the most promising dynamic OO-language to
>> follow this target.
> 
> 
> Bytecode engine is the best method for dynamic code execution
> ("exec", eval, etc).  A low level OOP language would be very suitable
> for a python VM.
> 
> pyvm has that. A big part of it is written in "lightweight C++" [1].

Realy ?  I have downloaded the lwc distribution and checked it out.
It was a surprise that none of the examples are working.
I'm using SuSE 9.0 with  gcc 3.3.1 ...

Is there a working version of lwc ???

Regards

   Armin




> That makes it less portable as the lwc preprocessor is using GNU-C
> extensions.  However, it's the same extensions also used by the linux
> kernel and AFAIK the intel compiler supports them too.
> 
> So probably the bigger "competitor" of pyvm is boost-python.
> And that's one reason the release of the source is stalled until it
> gets better.
> 
> 
> Stelios
> 
> [1] http://students.ceid.upatras.gr/~sxanth/lwc/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Hi,

Kay Schluehr wrote:
> 
> Why this? eval() consumes a string, produces a code object and executes
> it. Wether the code-object is bytecode or a chunk of machine code makes
> a difference in the runtime but does not alter the high level
> behavioural description of eval(). In either way the compile() function
> behind eval is a JIT.
> 

It makes a difference because
1) You depend on a compiler. Especially if you want to optimize the
machine code you depend on a big program (no tcc for example)
2) Optimizing machine code, needs a lot of code/time.
3) What about debugging info? Dwarf?
4) What if there's a new architecture?

The bytecode compiler is *much* faster to produce the bytecode assembly.


Also, for the other part of the thread, I think that bytecode may
be in fact faster than machine code JIT.  Here is a theory:

Suppose that for each algorithm there is the "ideal implementation"
which executes at the speed limit where it can't be done any faster.
For small algorithms the speed limit may be known but for more
complex programs, it's just a theoretical limit.  Now for such very
big programs bytecode has the advantage that it achieves very good
code re-use;  everything is those 400kB of the libpython core and that
does not increase with the size of the program.  In this case, bytecode
is the driver that operates the buldozer (the buldozer being C).  So the
theory is that for very complex programs a bytecode+core-library is
closer to the ideal implementation than a JIT which produces/compiles
megabytes of machine code.

Evidence for that may be that all those JIT efforts don't get any
great speed ups (psyco is different as it exposes static-ness).

Of course, if we care about speed we'll use Pyrex to convert some
heavy routines to C.  For example if we need FFT, it's madness to
do it in the HLL.  Make it part of the buldozer.



Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Mike Meyer]

Paul Rubin  writes:

> Andrew Dalke <[EMAIL PROTECTED]> writes:
>
>> Years ago, presented at one of the Python conferences, was a program
>> to generate C code from the byte code  The conclusion I recall
>> was that it wasn't faster - at best a few percent - and there was a
>> big memory hit because of all the duplicated code.  One thought was
>> that the cache miss caused some of the performance problems.  Does
>> that count as a compiler?
>
> I would say it counts as a compiler and that other languages have
> used a similar compilation approach and gotten much better speedups.
> For example, Java JIT compilers.  The DEC Scheme-to-C translator
> and Kyoto Common Lisp also produced C output from their compilers
> and got really significant speedups.  Part of the problem may be
> with the actual Python language.

The DEC Scheme->C compiler (yes, it was an honest to gods compiler -
it just generated C instead of machine code) doesn't sound very
similar to generating C code from byte code. The latter sounds like a
straightforward translation of byte code to C.  The Scheme->C
compiler, on the other hand, used all the optimizations known to LISP
compiler writers. It would, with the right settings, produce code that
was comparable to hand-coded C. So unless the Python compiler that
produced the byte code in the first place does all those optimizations
(which I don't know), you're not going to get results that compare
with the Scheme->C system.

> Despite the shrieks of the "Python is not Lisp!" crowd, Python
> semantics and Lisp semantics aren't THAT different, and yet compiled
> Lisp implementations com completely beat the pants off of interpreted
> Python in terms of performance.  I don't think Python can ever beat
> carefully coded C for running speed, but it can and should aim for
> parity with compiled Lisp.

There are people out there who claim that compiling carefully coded
LISP can compete with compiling carefully coded C. They use such
systems in real-world, time-critical applications. The thing is, to
get that performance, they had to go back and tell the compiler what
the types of all the variables are, and disable all the runtime type
checking.

So if this is a real goal, you now have some idea of what to look
forward to in Python's future.

  http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

Rocco Moretti <[EMAIL PROTECTED]> writes:
> Python, as a procedural language, makes extensive use of globals &
> mutable variables  IIUC, in Lisp, as a functional language, "all
> politics is local."  Global-like variables are much rarer, and
> mutability is severely limited.

Some people write Lisp code in a functional style, but not everyone.
Lisp provides mutable objects as much as Python does--maybe more,
for example, Lisp strings are mutable.  Overall I'd say there's not
much difference between Lisp and Python in this regard.  Lisp lets
the programmer supply static type declarations that the compiler
can use, but those are optional.  When you use them, you get faster
output code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Rocco Moretti]

Paul Rubin wrote:

> Despite the shrieks of the "Python is not Lisp!" crowd, Python
> semantics and Lisp semantics aren't THAT different, and yet compiled
> Lisp implementations com completely beat the pants off of interpreted
> Python in terms of performance.  

I know little about Lisp compilation, so I could be mistaken, but I was 
under the impression that one of the differences between Python & Lisp 
is directly relevant to compilation issues.

Python, as a procedural language, makes extensive use of globals & 
mutable variables. Not only can the contents of a variable change, but 
that change can non-explicitly affect a function in a "remote" part of 
the program, hence the requirement for the Global Interpreter Lock. 
Tracking how changes propagate in Python is non-trivial, as evidenced by 
the difficulty of replacing the GIL with a less "obtrusive" alternative.

IIUC, in Lisp, as a functional language, "all politics is local." 
Global-like variables are much rarer, and mutability is severely 
limited. In this case it is much easier to track how a piece of code 
alters the program's state - the propagation of those changes are 
handled explicitly, and once you have a reference to a piece of data, 
you don't have to worry about some other function changing the value on 
you. As such, it's a lot easier to write an optimizing compiler - you 
can place much greater limits on what is known at compile time.

It's been quite a while since I've looked at Lisp/functional languages, 
though, so I could be misunderstanding what I remember.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[François Pinard]

[Paul Rubin]

> François Pinard <[EMAIL PROTECTED]> writes:
>
> > Deep down, why or how not having a [traditional, to-native-code]
> > compiler is a deficiency for CPython?  We already know that such a
> > beast would not increase speed so significantly, while using much
> > more memory.

> I'd say the opposite.  The 4x speedup from Psyco is quite significant.
> The speedup would be even greater if the language itself were more
> compiler-friendly.

Psyco is not a traditional compiler.  Compilation occurs at run-time, as
typing information is not available without observation.

> > So far, it seems that the only way to get speed is to attach static
> > type information to some variables.

> No of course not, there's quite a bit of overhead in interpretation in
> general.  Plus, having to do a dictionary lookup for 'bar' in every
> call like foo.bar() adds overhead and I don't think I'd call fixing
> that as similar to adding static type info.

The dictionary lookup does not occur for local variables.  Changes are
planned in Python so lookups may be avoided in a few cases of global
variables.  But until (and despite!) such changes occur, compilation
does not buy you much, here.  This has been much studied and debated.

> > Some compilation avenues do it through information added either in
> > Python source code or in extraneous declarative files, other
> > approaches do it by delaying compilation until such information is
> > discovered at run-time.

> Also, lots of times one can do type inference at compile time.

Yes, particularily in simple or small programs.  I vaguely remember
that the unborn Viper (or Vyper?) was promising compile time type
inference.  It seems that so far, in many years, no one really succeeded
in demonstrating a definite advantage in this direction, at least, not
enough for the community to follow in.

> > The former taints the purity of real CPython as the only source.

> I don't understand this.  What purity?  Why is real CPython the
> only source?  There are dozens of C compilers and none of them is
> the "only source".  Why should Python be different?

Because it is.  There are not dozens of Python compilers.  A few, maybe.

Pyrex gives me a lot of speed, given I taint my Python sources with
cdefs, ctypes, etc.  Oh, I'm absolutely happy with Pyrex when I starve
for speed, but I know and you know that we are not writing pure Python
then.  And if, for not touching sources, one supplements them with
separate declarative files, my Python code may look pure, but I have to
maintain those files as well, and Python is not my only source anymore.

So, to get back to the origin of our argument, I'm still not tempted
to think that Python not having a compiler makes it deficient, because
having a compiler (at least in the traditional acceptation of the term)
would not buy it much enough.

-- 
François Pinard   http://pinard.progiciels-bpi.ca
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[david . tolpin]

> I don't think Python can ever beat
> carefully coded C for running speed, but it can and should aim for
> parity with compiled Lisp.

But common lisp compilers often beat C compilers in speed for similar
tasks
of moderate complexity. In particular, CMUCL beats GCC in numerical
computations.

David

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

Andrew Dalke <[EMAIL PROTECTED]> writes:

> Years ago, presented at one of the Python conferences, was a program
> to generate C code from the byte code  The conclusion I recall
> was that it wasn't faster - at best a few percent - and there was a
> big memory hit because of all the duplicated code.  One thought was
> that the cache miss caused some of the performance problems.  Does
> that count as a compiler?

I would say it counts as a compiler and that other languages have
used a similar compilation approach and gotten much better speedups.
For example, Java JIT compilers.  The DEC Scheme-to-C translator
and Kyoto Common Lisp also produced C output from their compilers
and got really significant speedups.  Part of the problem may be
with the actual Python language.  One of the reasons for wanting
real compilation as a high priority, is that the presence of a
compiler will affect future evolution of the language, in a way
that makes it more conducive to compilation.

Despite the shrieks of the "Python is not Lisp!" crowd, Python
semantics and Lisp semantics aren't THAT different, and yet compiled
Lisp implementations com completely beat the pants off of interpreted
Python in terms of performance.  I don't think Python can ever beat
carefully coded C for running speed, but it can and should aim for
parity with compiled Lisp.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

François Pinard <[EMAIL PROTECTED]> writes:
> Deep down, why or how not having a [traditional, to-native-code]
> compiler is a deficiency for CPython?  We already know that such a beast
> would not increase speed so significantly, while using much more memory.

I'd say the opposite.  The 4x speedup from Psyco is quite significant.
The speedup would be even greater if the language itself were more
compiler-friendly.

> So far, it seems that the only way to get speed is to attach static
> type information to some variables.  

No of course not, there's quite a bit of overhead in interpretation in
general.  Plus, having to do a dictionary lookup for 'bar' in every
call like foo.bar() adds overhead and I don't think I'd call fixing
that as similar to adding static type info.

> Some compilation avenues do it through information added either in
> Python source code or in extraneous declarative files, other
> approaches do it by delaying compilation until such information is
> discovered at run-time.

Also, lots of times one can do type inference at compile time.

> The former taints the purity of real CPython as the only source.

I don't understand this.  What purity?  Why is real CPython the
only source?  There are dozens of C compilers and none of them is
the "only source".  Why should Python be different?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Kay Schluehr]

Stelios Xanthakis wrote:
> Kay Schluehr wrote:
>
> >
> > Yes. What we are seeking for and this may be the meaning of Armins
> > intentiously provocative statement about the speed of running HLLs
is a
> > successor of the C-language and not just another VM interpreter
that is
> > written in C and limits all efforts to extend it in a flexible and
> > OO-manner. Python is just the most promising dynamic OO-language to
> > follow this target.
> >
>
> Bytecode engine is the best method for dynamic code execution
> ("exec", eval, etc).  A low level OOP language would be very suitable
> for a python VM.

Why this? eval() consumes a string, produces a code object and executes
it. Wether the code-object is bytecode or a chunk of machine code makes
a difference in the runtime but does not alter the high level
behavioural description of eval(). In either way the compile() function
behind eval is a JIT.

> pyvm has that. A big part of it is written in "lightweight C++" [1].
> That makes it less portable as the lwc preprocessor is using GNU-C
> extensions.

Hmmm... I'm not amazingly happy that it is not ANSI-C++. To be honest I
don't want to learn lw-C++ and the peculiaritys of the translator and
debuging through translated code. This may be an interesting study on
it's own but it is just not me who is interested in it. Using an
experimental language to write an experimental VM is clearly out of my
motivation ( only as a remark for the quest of contribution not as a
general criticism ).

> However, it's the same extensions also used by the linux
> kernel and AFAIK the intel compiler supports them too.
>
> So probably the bigger "competitor" of pyvm is boost-python.

How could boost-Python be a "competitor"? Isn't it just an ANSI
C++-binding that relates heavily on templates that are not even
supported by lw-C++?

Regards,
Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Andrew Dalke]

Paul Rubin wrote:
> Yes, there are several Python compilers already
 ...
> It's true that CPython doesn't have a compiler and that's a serious
> deficiency.  A lot of Python language features don't play that well
> with compilation, and that's often unnecessary.  So I hope the baseline
> implementation changes to a compiled one before the language evolves
> too much more.

Years ago, presented at one of the Python conferences, was a
program to generate C code from the byte code.  It would still
make calls to the Python run-time library (just as C does to
its run-time library).

The presenter did some optimizations, like not decref at the
end of one instruction when the next immediately does an incref
to it.  The conclusion I recall was that it wasn't faster -
at best a few percent - and there was a big memory hit because
of all the duplicated code.  One thought was that the cache miss
caused some of the performance problems.

Does that count as a compiler?

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[François Pinard]

[Paul Rubin]

> It's true that CPython doesn't have a compiler and that's
> a serious deficiency.

Hi, Paul.  I did not closely follow all of the thread, so maybe my
remark below, only repeats what others might have said and I missed?

Deep down, why or how not having a [traditional, to-native-code]
compiler is a deficiency for CPython?  We already know that such a beast
would not increase speed so significantly, while using much more memory.

It is true that a standard traditional compiler for CPython would allow
one would be to check the box:

[x] has a compiler

in the fashion of the day language information sheet, and for some
readers, not having that box checked is a deficiency in itself. :-)

So far, it seems that the only way to get speed is to attach static
type information to some variables.  Some compilation avenues do it
through information added either in Python source code or in extraneous
declarative files, other approaches do it by delaying compilation
until such information is discovered at run-time.  The former taints
the purity of real CPython as the only source.  The later often shows
spectacular speed gain, but not always, and may bloat size unboudedly.

-- 
François Pinard   http://pinard.progiciels-bpi.ca
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Skip Montanaro]

>> - hacking SWIG.  Shouldn't be too hard and will instantly give
>> us access to wx, qt, etc.

Mike> You can't assume that because some package is a C/C++ library
Mike> wrapped for Python that it uses SWIG. pyqt, for example, doesn't
Mike> use SWIG at all. It uses SIP, which is considerably more
Mike> complicated than SWIG.

PyGTK uses its own lisp-y thing as the wrapper IDL. Actually, it's specific
to GTK and is AFAIK is used to wrap GTK by languages besides Python.

Skip

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Kay Schluehr wrote:

> 
> Yes. What we are seeking for and this may be the meaning of Armins
> intentiously provocative statement about the speed of running HLLs is a
> successor of the C-language and not just another VM interpreter that is
> written in C and limits all efforts to extend it in a flexible and
> OO-manner. Python is just the most promising dynamic OO-language to
> follow this target. 
> 

Bytecode engine is the best method for dynamic code execution
("exec", eval, etc).  A low level OOP language would be very suitable
for a python VM.

pyvm has that. A big part of it is written in "lightweight C++" [1].
That makes it less portable as the lwc preprocessor is using GNU-C
extensions.  However, it's the same extensions also used by the linux
kernel and AFAIK the intel compiler supports them too.

So probably the bigger "competitor" of pyvm is boost-python.
And that's one reason the release of the source is stalled until it
gets better.


Stelios

[1] http://students.ceid.upatras.gr/~sxanth/lwc/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Bengt Richter]

On 11 May 2005 19:48:42 -0700, Paul Rubin  wrote:

>Stelios Xanthakis <[EMAIL PROTECTED]> writes:
>> I didn't know much about PyPy.  It seems that pyvm is *exactly* what
>> pypy needs to boost its performance.  Does pypy has the vm in python
>> as well?  Does pypy have a compiler that produces 2.4 bytecodes?
>
>Pypy makes native machine code, not bytecode.
I thought they hoped to experiment with many targets, which would presumably
include generating code for different CPUs and VMs in various representations,
not necessarily just machine code via low level C or ASM, but I haven't checked
current status re that.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Kay Schluehr]

Paul Rubin wrote:
> "Kay Schluehr" <[EMAIL PROTECTED]> writes:
> > Delete the "standard" and You still obtain huge librarys for .Net,
Java
> > and Python. I also regret that Prothon starved in infancy but it
might
> > be exeggerated to demand that each language designer or one of his
> > apostels should manage a huge community that enjoys doing redundant
> > stuff like writing Tk-bindings, regexp-engines and all that.
>
> Maybe there needs to be some kind of standard FFI (foreign function
> interface) like Lisp implementations have.
>
> > Sooner or later Python will go the LISP way of having a
standardized
> > "Common-Python" ( std_objectspace) and a number of dialects and
DSLs
> > running in their own derived object spaces. Maybe Python 3000 is an
> > illusion and will fade away like a Fata Morgana the closer we seem
come.
>
> Right now Python feels about like Maclisp in the 1970's must have
felt
> (that was before my time so I can't know for certain).  Lots of
> hackerly excitement, lots of cruft.  It needs to ascend to the next
> level.  PyPy looks like the best vehicle for that so far.  See
>
>http://catb.org/~esr/jargon/html/M/MFTL.html
>
> for the canonical remark about languages that can't be used to
> implement their own compilers.  Python is fun and useful, but it
> really isn't mature until PyPy is released for production use.

Yes. What we are seeking for and this may be the meaning of Armins
intentiously provocative statement about the speed of running HLLs is a
successor of the C-language and not just another VM interpreter that is
written in C and limits all efforts to extend it in a flexible and
OO-manner. Python is just the most promising dynamic OO-language to
follow this target. 

Ciao,
Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

Greg Ewing <[EMAIL PROTECTED]> writes:
> > PyPy looks like the best vehicle for that so far.  See
> >http://catb.org/~esr/jargon/html/M/MFTL.html
> > for the canonical remark about languages that can't be used to
> > implement their own compilers.
> 
> Which makes it clear that the remark is only intended to apply to
> *compiled* languages.

Yes, there are several Python compilers already, (Psyco, Jython followed
up by your favorite JIT compiler, IronPython (similarly), Pyrex (not
quite the same input language but I think it should be counted), etc.
It's true that CPython doesn't have a compiler and that's a serious
deficiency.  A lot of Python language features don't play that well
with compilation, and that's often unnecessary.  So I hope the baseline
implementation changes to a compiled one before the language evolves
too much more.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Greg Ewing]

Paul Rubin wrote:
> PyPy looks like the best vehicle for that so far.  See
> 
>http://catb.org/~esr/jargon/html/M/MFTL.html
> 
> for the canonical remark about languages that can't be used to
> implement their own compilers.

Which makes it clear that the remark is only intended
to apply to *compiled* languages.

-- 
Greg Ewing, Computer Science Dept,
University of Canterbury,   
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Mike Meyer]

Stelios Xanthakis <[EMAIL PROTECTED]> writes:
>   - hacking SWIG.  Shouldn't be too hard and will instantly give
>   us access to wx, qt, etc.

You can't assume that because some package is a C/C++ library wrapped
for Python that it uses SWIG. pyqt, for example, doesn't use SWIG at
all. It uses SIP, which is considerably more complicated than SWIG.

   http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

Stelios Xanthakis <[EMAIL PROTECTED]> writes:
> I didn't know much about PyPy.  It seems that pyvm is *exactly* what
> pypy needs to boost its performance.  Does pypy has the vm in python
> as well?  Does pypy have a compiler that produces 2.4 bytecodes?

Pypy makes native machine code, not bytecode.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Robert Kern]

Stelios Xanthakis wrote:

> There are two kinds of C modules: those that do have a knowledge
> of the C API (like sre, tkinter, etc) and those that are just C/C++
> libraries which are simply wrapped as modules.  For the latter there
> are two solutions besides adding a wrapper which makes pyvm appear
> as libpython:
>   - an advanced ctypes module which will make dlopening libraries
>   and wrapping their symbols behind python functions, a matter of
>   python code.  I'm considering this approach to provide things
>   like 'raw_input'.
>   - hacking SWIG.  Shouldn't be too hard and will instantly give
>   us access to wx, qt, etc.

No, writing a pyvm modules for SWIG won't give you compatibility with 
most existing SWIG wrappers. Such wrappers are very, very rarely "pure 
SWIG". Almost all nontrivial wrappers include ad hoc typemaps that use 
the Python C API.

-- 
Robert Kern
[EMAIL PROTECTED]

"In the fields of hell where the grass grows high
  Are the graves of dreams allowed to die."
   -- Richard Harter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Roger Binns]

 "Stelios Xanthakis" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
>> - hacking SWIG.  Shouldn't be too hard and will instantly give
>> us access to wx, qt, etc.

 Have you ever written a non-trivial extension using Swig?  It isn't
as simple as you would think.  There are a lot of little things to
deal with, usually because because of the mismatch between C/C++
semantics and convention and Python library expectations.  You have
to do a fair amount of work to fix all that using C code, Python
code and sometimes parts of the internals of Swig-Python. wxPython
has used a modified version of Swig for most of its lifetime.

Some people also have issues with Swig staying compatible with itself:

  http://bob.pythonmac.org/archives/2005/03/11/swig-hate/

PyQT uses another tool to do the wrapping - SIP.

Tools like Swig have been in the business of exposing C/C++ to Python
for years and are still suboptimal.  I believe you will find "hacking
SWIG" and actually porting an existing project that uses it to be
hard.

I would suggest looking in the lib/python directory of Swig to get an
idea of what you have to start with.  Note how there is minimal overlap
with the Java directory to pick one example.

>> The thing is that the C API of pyvm is IMHO superior and much more fun.

That is not under dispute.  The suggestion is that you *also* provide a
way of running existing extensions, even if it is slow and inferior,
ideally without even having to recompile them.  (If you do require a
recompile then you'll also have to port/invent an equivalent to distutils).

>> You can wrap the entire sockets module in a couple of hours and also
>> enjoy it.

sockets are easy.  There are no structures, plain integer handles, no pointers,
about 8 methods and a few constants.  About the only thing to remember is
to deal with threading correctly.

>> I wish I could clone myself to port the entire std library
>> to pyvm -- so much fun it is:)

Well, if you don't provide an easy way to move existing stuff across, you'll
end up doing just that yourself!

Roger 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Cameron Laird]

In article <[EMAIL PROTECTED]>,
Roger Binns <[EMAIL PROTECTED]> wrote:
>> could You tell us a bit more about Your motivation to create an
>> alternative C-Python interpreter?
>
>I'd also be curious to know if the performance gains would remain
>once it gets fleshed out with things like closures, long numbers,
>new style classes and a C library compatibility shim.
>
>Roger 
>
>

And Unicode.  And ...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Paul Rubin wrote:
> 
> I hope that PyPy will replace CPython once it's solid enough.  Trying
> to stay backwards compatible with the legacy C API doesn't seem to me
> to be that important a goal.  Redoing the library may take more work
> than the Prothon guy was willing to do for Prothon, but PyPy has more
> community interest and maybe can attract more resources.

I didn't know much about PyPy.  It seems that pyvm is *exactly* what
pypy needs to boost its performance.  Does pypy has the vm in python
as well?  Does pypy have a compiler that produces 2.4 bytecodes?

I think the next step in speeding up python (not that it's slow, I'm
in the group of people that don't think Python is slow), is the AST
compiler.  An AST compiler for CPython is in the works AFAIK, but it
would be more powerful in written in python.  It would be also easy
to have 'alternative' compilers that, for example are not whitespace
sensitive, or the code looks like perl :), but they all produce python
bytecode.


Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Roger Binns wrote:
>>I am not very interested on C compatibility.
> 
> 
> That will rule out all the gui frameworks, SSL, cryptography
> and numerous other packages.  Have a look at what happened to
> Prothon.  What ultimately killed it was the problem of having
> a decent library.  You don't have to make the C library
> compatibility totally high performance, but having any form of
> it there will make adoption by others easier.
> 

There are two kinds of C modules: those that do have a knowledge
of the C API (like sre, tkinter, etc) and those that are just C/C++
libraries which are simply wrapped as modules.  For the latter there
are two solutions besides adding a wrapper which makes pyvm appear
as libpython:
- an advanced ctypes module which will make dlopening libraries
and wrapping their symbols behind python functions, a matter of
python code.  I'm considering this approach to provide things
like 'raw_input'.
- hacking SWIG.  Shouldn't be too hard and will instantly give
us access to wx, qt, etc.

The thing is that the C API of pyvm is IMHO superior and much more fun.
You can wrap the entire sockets module in a couple of hours and also
enjoy it.  I wish I could clone myself to port the entire std library
to pyvm -- so much fun it is:)


thanks,

Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

"Roger Binns" <[EMAIL PROTECTED]> writes:
> Some examples are gui toolkits (eg wxPython), SSL (eg M2Crypto, pyopenssl)
> and database (pysqlite, APSW).  These aren't in the shipped with Python
> library but are widely used.

M2Crypto is a straightforward SWIG wrapper around OpenSSL, I thought.
I don't know about wxPython or pysqlite.  It seems to me that some
kind of SQL client should be part of the stdlib.  But why isn't a SWIG
wrapper enough?  The CPython stdlib has for possibly good reasons
avoided SWIG but a new implementation doesn't need to.

> You don't have to stay backwards compatible.  It is best to provide
> some sort of way of using the old extensions even if it is suboptimal
> (eg some sort of mapping shim).

Yeah, there's something to be said for that, if it doesn't cause too
much pain.

> I already get burnt out on the matrix of CPython versions and different
> platforms.  Adding another interpretter would make life even harder
> for extension authors.

There's already Jython and Python.net is on its way.  This is
something Lisp users have dealt with for decades and they've developed
workable machinery for it.  Maybe Python can adopt some of those
methods.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

"Kay Schluehr" <[EMAIL PROTECTED]> writes:
> Delete the "standard" and You still obtain huge librarys for .Net, Java
> and Python. I also regret that Prothon starved in infancy but it might
> be exeggerated to demand that each language designer or one of his
> apostels should manage a huge community that enjoys doing redundant
> stuff like writing Tk-bindings, regexp-engines and all that.

Maybe there needs to be some kind of standard FFI (foreign function
interface) like Lisp implementations have.

> Sooner or later Python will go the LISP way of having a standardized
> "Common-Python" ( std_objectspace) and a number of dialects and DSLs
> running in their own derived object spaces. Maybe Python 3000 is an
> illusion and will fade away like a Fata Morgana the closer we seem come.

Right now Python feels about like Maclisp in the 1970's must have felt
(that was before my time so I can't know for certain).  Lots of
hackerly excitement, lots of cruft.  It needs to ascend to the next
level.  PyPy looks like the best vehicle for that so far.  See

   http://catb.org/~esr/jargon/html/M/MFTL.html

for the canonical remark about languages that can't be used to
implement their own compilers.  Python is fun and useful, but it
really isn't mature until PyPy is released for production use.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Roger Binns]

"Paul Rubin"  wrote in message news:[EMAIL PROTECTED]
> "Roger Binns" <[EMAIL PROTECTED]> writes:
> What kind of stuff is in the existing Python C library that couldn't
> be reimplemented or retargeted pretty easily?  Most of it is either
> wrappers for standard C functions (system calls), stuff that shouldn't
> be in C in the first place (the Python compiler and interpreter), etc.

Some examples are gui toolkits (eg wxPython), SSL (eg M2Crypto, pyopenssl)
and database (pysqlite, APSW).  These aren't in the shipped with Python
library but are widely used.

> I hope that PyPy will replace CPython once it's solid enough.  Trying
> to stay backwards compatible with the legacy C API doesn't seem to me
> to be that important a goal.  Redoing the library may take more work
> than the Prothon guy was willing to do for Prothon, but PyPy has more
> community interest and maybe can attract more resources.

You don't have to stay backwards compatible.  It is best to provide
some sort of way of using the old extensions even if it is suboptimal
(eg some sort of mapping shim).  The extensions can later be improved,
but if there is no simple way of recompiling and running then it will
take a long time for all the extensions to get ported.

I already get burnt out on the matrix of CPython versions and different
platforms.  Adding another interpretter would make life even harder
for extension authors.

Roger 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Kay Schluehr]

Paul Rubin wrote:
> "Roger Binns" <[EMAIL PROTECTED]> writes:
> > Err, you proved my point!  Prothon was fine at the VM level.  The
> > author couldn't figure out how to get a decent sized "standard"
> > library, and ultimately ended up abandoning his VM for .Net since
> > that gets you a large standard library.  Jython also gets a large
> > standard library from the Java ones.
>
> What kind of stuff is in the existing Python C library that couldn't
> be reimplemented or retargeted pretty easily?  Most of it is either
> wrappers for standard C functions (system calls), stuff that
shouldn't
> be in C in the first place (the Python compiler and interpreter),
etc.

Delete the "standard" and You still obtain huge librarys for .Net, Java
and Python. I also regret that Prothon starved in infancy but it might
be exeggerated to demand that each language designer or one of his
apostels should manage a huge community that enjoys doing redundant
stuff like writing Tk-bindings, regexp-engines and all that.

> I hope that PyPy will replace CPython once it's solid enough.  Trying
> to stay backwards compatible with the legacy C API doesn't seem to me
> to be that important a goal.  Redoing the library may take more work
> than the Prothon guy was willing to do for Prothon, but PyPy has more
> community interest and maybe can attract more resources.

PyPy is a beast on it's own and provides interesting new ideas to
Python as well as runtime design in general. But I'm not quite shure
that it will stay in the boundarys of the current Python community
process and will subordinate to Guidos will. As a Marxist would say:
who controls the runtime also controls the language. Sooner or later
Python will go the LISP way of having a standardized "Common-Python" (
std_objectspace) and a number of dialects and DSLs running in their own
derived object spaces. Maybe Python 3000 is an illusion and will fade
away like a Fata Morgana the closer we seem come.

Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

"Roger Binns" <[EMAIL PROTECTED]> writes:
> Err, you proved my point!  Prothon was fine at the VM level.  The
> author couldn't figure out how to get a decent sized "standard"
> library, and ultimately ended up abandoning his VM for .Net since
> that gets you a large standard library.  Jython also gets a large
> standard library from the Java ones.

What kind of stuff is in the existing Python C library that couldn't
be reimplemented or retargeted pretty easily?  Most of it is either
wrappers for standard C functions (system calls), stuff that shouldn't
be in C in the first place (the Python compiler and interpreter), etc.

I hope that PyPy will replace CPython once it's solid enough.  Trying
to stay backwards compatible with the legacy C API doesn't seem to me
to be that important a goal.  Redoing the library may take more work
than the Prothon guy was willing to do for Prothon, but PyPy has more
community interest and maybe can attract more resources.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Roger Binns]

"Paul Rubin"  wrote in message news:[EMAIL PROTECTED]
> "Roger Binns" <[EMAIL PROTECTED]> writes:
>> That will rule out all the gui frameworks, SSL, cryptography
>> and numerous other packages.  Have a look at what happened to
>> Prothon.
>
> I think it would be enough to retarget SWIG.

That is also a lot of work, and a lot of toolkits that use Swig
for Python bindings are very strongly tied to the Swig-Python
internals.  For example I doubt there are many where you could
tell Swig to output Java and it would just work.  (I know none
of mine would work, nor would wxPython).

>> What ultimately killed it was the problem of having a decent
>> library.
>
> Nah, I think people just lost interest.  Otherwise, why isn't Jython
> dead?

Err, you proved my point!  Prothon was fine at the VM level.  The
author couldn't figure out how to get a decent sized "standard"
library, and ultimately ended up abandoning his VM for .Net since
that gets you a large standard library.  Jython also gets a large
standard library from the Java ones.

Roger 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Peter Hansen]

Paul Rubin wrote:
> "Roger Binns" <[EMAIL PROTECTED]> writes:
> 
>>That will rule out all the gui frameworks, SSL, cryptography
>>and numerous other packages.  Have a look at what happened to
>>Prothon.  
> 
> 
> I think it would be enough to retarget SWIG.
> 
> 
>>What ultimately killed it was the problem of having a decent
>>library.
> 
> 
> Nah, I think people just lost interest.  Otherwise, why isn't Jython
> dead?

I thought Jython had full access to the entire library of standard Java, 
plus at least the Python parts of the Python standard library...

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

"Roger Binns" <[EMAIL PROTECTED]> writes:
> That will rule out all the gui frameworks, SSL, cryptography
> and numerous other packages.  Have a look at what happened to
> Prothon.  

I think it would be enough to retarget SWIG.

> What ultimately killed it was the problem of having a decent
> library.

Nah, I think people just lost interest.  Otherwise, why isn't Jython
dead?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Roger Binns]

> I am not very interested on C compatibility.

That will rule out all the gui frameworks, SSL, cryptography
and numerous other packages.  Have a look at what happened to
Prothon.  What ultimately killed it was the problem of having
a decent library.  You don't have to make the C library
compatibility totally high performance, but having any form of
it there will make adoption by others easier.

Roger


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Terry Reedy]

"Stelios Xanthakis" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>> Maybe you can explain us why it is so fast, and/or maybe you can work
>> with the other developers to improve the speed of the normal CPython,
>> this can require equal or less work for you, and it can produce more
>> long-lasting results/consequences for your work.
>>
>
> The reason is that it's built from the scratch.
> Guido would disagree with that, see py-dev thread:

Guido, like me, believes in being correct before being fast.

> http://www.mail-archive.com/python-dev@python.org/msg01225.html
>
>
> There *are* alternative ways to do some things in the vm and Joel
> is simply wrong:)

The smiley doesn't negate your silliness.  Joel explicitly talked about 
complex, mature, debugged, in-use systems, with actual or potential 
competitive replacements, not laboratory development toys and prototypes. 
Three years between releases *is* a long time.  And the Python codebase 
*does* embody a lot of hard-won knowledge, hardly available elsewhere, that 
should not be tossed.  A particular example is how to get certain things to 
work across numerous platforms in the face of ambuiguity and 'not defined' 
behavior in the C standard and implementations.

Its possible that some future reference Python will be based on a new, 
built-from-scratch core developed *in parallel* with current Python.  But 
that will only happen after it has been tested on a hundred different 
platforms with a hundred different applications and packages (as happens, 
more or less, with each new CPython release).

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[François Pinard]

[Stelios Xanthakis]

> I'm afraid this may end up dead before unborn too.  So it depends what
> people want. If nobody cares, [...]

People might not care so much about what could be done about your
project, unless you give them proper and complete means for evaluating
the state of affairs.  Your project is very likely to die if you keep
your sources closed, and yourself loose interest in the project.

Opening your sources is no guarantee either that the community will
adopt your project.  But this might give your project a better chance.

-- 
François Pinard   http://pinard.progiciels-bpi.ca
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Roger Binns wrote:

>>could You tell us a bit more about Your motivation to create an
>>alternative C-Python interpreter?
> 
> 
> I'd also be curious to know if the performance gains would remain
> once it gets fleshed out with things like closures, long numbers,
> new style classes and a C library compatibility shim.
> 

I guess it will.  There are several places with /*XXX:speed up*/
and a couple more ideas to gain exrta speed.  On the other hand
pyvm does not check for Bad Internal Calls and other checks Cpython
does to be OK in all cases (for example if __cmp__ modifies a list
while it's being sorted, in pyvm you *can* get a crash).

There is no plan to implement closures; unless there is a way to
turn closures to default arguments with bytecode hacking.  But yes,
closures are interesting, especially in generator expressions and
lambdas.

I am not very interested on C compatibility.  Once the source is
open people can implement it if they want but I'd rather go with
an advanced ctypes module which will make loading dynamic libraries
a matter of python code.


Stelios

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

[EMAIL PROTECTED] wrote:

> This project is probably a LOT of work; maybe people can tell us about
> such efforts *before* doing so much work, so we can discuss it, and
> avoid wasting time.
> 

It is a lot of work indeed.
Usually, when people announce "we shall create X", it doesn't happen.
And you don't know if you can do it at all *before*.


> Maybe you can explain us why it is so fast, and/or maybe you can work
> with the other developers to improve the speed of the normal CPython,
> this can require equal or less work for you, and it can produce more
> long-lasting results/consequences for your work.
> 

The reason is that it's built from the scratch.
Guido would disagree with that, see py-dev thread:

http://www.mail-archive.com/python-dev@python.org/msg01225.html


There *are* alternative ways to do some things in the vm and Joel
is simply wrong:)
Could this be  Python 3000!!!? :)


Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Stelios Xanthakis]

Kay Schluehr wrote:
> 
> could You tell us a bit more about Your motivation to create an
> alternative C-Python interpreter? There is AFAIK no such ambitious
> project that has ever survived. The last one I remember died shortly
> after it was born:
> 

The motivation is that I just needed some bytecode engine. I deciced
to go with python bytecode, because well, IMO it's the best!
Eventually it turned out what it is.

I'm afraid this may end up dead before unborn too.
So it depends what people want. If nobody cares, I just learned python
the hard way and the project does to the abandonware cemetary.


Stelios



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Dan Christensen]

djw <[EMAIL PROTECTED]> writes:

> Paul Rubin wrote:
>> Stelios Xanthakis <[EMAIL PROTECTED]> writes:
>> 
>>>- The demo is an x86/linux binary only. You shouldn't trust binaries,
>>>   run it in a chrooted environment not as root!
>> 
>> 
>> Are you going to release the source?  If not, it's a lot less interesting.
>
>  From the website:
>
> "...the source will be released when it becomes more complete and stable."

If he releases the source now, it'll probably become more complete and
stable sooner...

Remember, release early, release often!

Dan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[djw]

Paul Rubin wrote:
> Stelios Xanthakis <[EMAIL PROTECTED]> writes:
> 
>>- The demo is an x86/linux binary only. You shouldn't trust binaries,
>>   run it in a chrooted environment not as root!
> 
> 
> Are you going to release the source?  If not, it's a lot less interesting.

 From the website:

"...the source will be released when it becomes more complete and stable."
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Paul Rubin]

Stelios Xanthakis <[EMAIL PROTECTED]> writes:
> - The demo is an x86/linux binary only. You shouldn't trust binaries,
>run it in a chrooted environment not as root!

Are you going to release the source?  If not, it's a lot less interesting.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Roger Binns]

> could You tell us a bit more about Your motivation to create an
> alternative C-Python interpreter?

I'd also be curious to know if the performance gains would remain
once it gets fleshed out with things like closures, long numbers,
new style classes and a C library compatibility shim.

Roger 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Peter Hansen]

Kay Schluehr wrote:
> Stelios Xanthakis wrote:
>>pyvm is a program which can run python 2.4 bytecode (the .pyc files).
>>A demo pre-release is available at:
>>  http://students.ceid.upatras.gr/~sxanth/pyvm/
> 
> could You tell us a bit more about Your motivation to create an
> alternative C-Python interpreter? There is AFAIK no such ambitious
> project that has ever survived. ..

If you check the URL more closely, you'll notice that this is a 
university site.

It seem likely this being done at least partly as a school project, 
which is certainly motivation enough for such a thing, *even* if it 
doesn't survive (since it will pay for itself in learning).

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[bearophileHUGS]

I've seen the benchmarks, they look quite interesting.

This project is probably a LOT of work; maybe people can tell us about
such efforts *before* doing so much work, so we can discuss it, and
avoid wasting time.

Maybe you can explain us why it is so fast, and/or maybe you can work
with the other developers to improve the speed of the normal CPython,
this can require equal or less work for you, and it can produce more
long-lasting results/consequences for your work.

Bye,
Bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyvm -- faster python

[Kay Schluehr]

Stelios Xanthakis wrote:
> Hi.
>
> pyvm is a program which can run python 2.4 bytecode (the .pyc files).
> A demo pre-release is available at:
>   http://students.ceid.upatras.gr/~sxanth/pyvm/
>
>
> Facts about pyvm:
> - It's FAST. According to the "cooked-bench" benchmark suite it
finishes
>in 55% of the time python takes;)
> - It's SMALL. Currently the source code is under 15k lines with the
>builtin modules.
> - It's new. Uses no code from CPython.
> - It's incomplete. Not even near the stability and quality of python.
>It needs A LOT of work before it can be compared to CPython.
>Moreover, at the time it lacks many things like closures, long
numbers
>new style classes, etc.
> - It's incompatible with CPython. Not all programs run.
> - The C API is incompatible. You can't run C modules (a thin wrapper
to
>make pyvm appear as libpython *could* be possible but not a goal
>AFAIC)
> - The demo is an x86/linux binary only. You shouldn't trust binaries,
>run it in a chrooted environment not as root!
>
> Hope it works!
>
> Cheers,
>
> Stelios

Hi Stelios,

could You tell us a bit more about Your motivation to create an
alternative C-Python interpreter? There is AFAIK no such ambitious
project that has ever survived. The last one I remember died shortly
after it was born:

http://www.python.org/pycon/papers/pymite/

This is sad because it is still challenging to offer a tiny interpreter
of a dynamic language for glue code/RAD on tiny hardware. A lot of
effort was spent to provide Java for microcontrollers especially for
SmartCards. I think a lot of people would show interest in Your project
if it gets somehow focussed and does not seem to be redundant.

Ciao,
Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

pyvm -- faster python

[Stelios Xanthakis]

Hi.

pyvm is a program which can run python 2.4 bytecode (the .pyc files).
A demo pre-release is available at:
http://students.ceid.upatras.gr/~sxanth/pyvm/


Facts about pyvm:
- It's FAST. According to the "cooked-bench" benchmark suite it finishes
   in 55% of the time python takes;)
- It's SMALL. Currently the source code is under 15k lines with the
   builtin modules.
- It's new. Uses no code from CPython.
- It's incomplete. Not even near the stability and quality of python.
   It needs A LOT of work before it can be compared to CPython.
   Moreover, at the time it lacks many things like closures, long numbers
   new style classes, etc.
- It's incompatible with CPython. Not all programs run.
- The C API is incompatible. You can't run C modules (a thin wrapper to
   make pyvm appear as libpython *could* be possible but not a goal
   AFAIC)
- The demo is an x86/linux binary only. You shouldn't trust binaries,
   run it in a chrooted environment not as root!

Hope it works!

Cheers,

Stelios
-- 
http://mail.python.org/mailman/listinfo/python-list