Re: Why Python is not both an interpreter and a compiler?

[Jussi Piitulainen]

Steven D'Aprano writes:
> On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:
>> Steven D'Aprano:
>> 
>>> I believe that Marko is wrong. It is not so easy to compile Python
>>> to machine language for real machines. That's why the compiler
>>> targets a virtual machine instead.
>> 
>> Somehow Guile manages it even though Scheme is at least as dynamic a
>> language as Python.
>
> It's not about the dynamicism precisely, it's about what counts as
> primitive data types and operations.
>
> What are the primitives in Scheme? In Python, the primitives are
> pretty complex. I don't know how accurate this page is:
>
> https://en.wikibooks.org/wiki/Scheme_Programming/Scheme_Datatypes
>
> but it suggests that Scheme primitives are quite close to the
> machine. That may keep the runtime small.

I think I spotted a tiny inaccuracy, but the real problem with that page
is that it's *very* incomplete: no mention of exact integers and
rationals, strings appear in text but not as an entry, were vectors even
mentioned?, nothing about procedures!, let alone reified continuations,
nothing about input/output ports. Also records (definable types with
named fields) are now officially in for some time already, so that's
another kind of structured types. Many implementations have object
systems, possibly modules as objects.

It's quite analogous to Python's objects.

Eval in Scheme is more restricted. Procedure internals are not
accessible at all, and implementations typically offer ways to declare
that standard names in a compilation unit indeed have their standard
meaning and variables will not be assigned outside the given compilation
unit, so the compiler can propagate information about expected argument
types around, infer more types, inline code, and lot's of things that
are way beyond me.

Some Scheme implementations use quite aggressive compilation techniques.
Guile is one, I think. Gambit and Larceny are two others.

[- -]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Laura Creighton]

In a message of Wed, 02 Sep 2015 10:50:15 +1000, Chris Angelico writes:
>And compiled C programs are notoriously hard to distribute. Can you
>pick up a Guile binary and carry it to another computer? Do you have
>to absolutely perfectly match the libguile version, architecture,
>build settings, etc?

This is the problem that docker has set out to solve.  
https://www.docker.com/whatisdocker
I think it is pretty neat stuff, myself.

Laura
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[eric]

On Tue, Sep 1, 2015 at 12:20 PM, Marko Rauhamaa  
wrote:

Steven D'Aprano :

 I believe that Marko is wrong. It is not so easy to compile Python 
to
 machine language for real machines. That's why the compiler targets 
a

 virtual machine instead.


Somehow Guile manages it even though Scheme is at least as dynamic a
language as Python.


I'm wondering if any of the VM lessons learned with forth environments 
would help?


https://en.wikipedia.org/wiki/Threaded_code




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Chris Angelico]

On Wed, Sep 2, 2015 at 2:20 AM, Marko Rauhamaa  wrote:
> In fact, the shebang notation turns any single .py file into such an
> executable. The problem is if you break your program into modules. Java,
> of course, solved a similar problem with .jar files (but still wouldn't
> jump over the final hurdle of making the .jar files executable).

The time machine strikes again! Python supports zip file execution,
which gives you the same benefit as a .jar file, plus you can slap on
a shebang and run the whole thing!

Anyone for some pyzza?

https://www.python.org/dev/peps/pep-0441/

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Ian Kelly]

On Mon, Aug 31, 2015 at 11:45 PM, Luca Menegotto
 wrote:
> Il 31/08/2015 19:48, Mahan Marwat ha scritto:
>
>> If it hasn't been considered all that useful, then why
>
>> the tools like cx_freeze, pytoexe are doing very hard!
>
> Well, I consider those tools useless at all!
> I appreciate Python because, taken one or two precautions, I can easily port
> my code from one OS to another with no pain.
> So, why should I loose this wonderful freedom?

You don't. You can still take your unbundled code and port it just as
easily as before.

What is gained from those tools is the ability to easily distribute
your code to (Windows) users who aren't knowledgable or interested in
maintaining a Python installation on their system. It's something that
you don't likely use unless you have a specific need to do that,
however.

At my previous job where IT had everybody on Windows, we published our
code with batch file launchers onto the internal file server and
maintained several Python installations there to run them with, rather
than maintain them on the systems of 200+ users plus lab computers.
With that setup we didn't require cx_freeze, but we used it in some
cases for better network performance (it's typically faster to
download one large file from the network than hundreds of small
files).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Laura Creighton]

>But are Guile programs small?  I the OP made a categorisation error,
>confusing correlation with causation.  (i.e. the presence of
>feathers makes a animal able to fly).

s/I the/I think the/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Laura Creighton]

In a message of Tue, 01 Sep 2015 19:20:51 +0300, Marko Rauhamaa writes:
>Somehow Guile manages it even though Scheme is at least as dynamic a
>language as Python.

But are Guile programs small?  I the OP made a categorisation error,
confusing correlation with causation.  (i.e. the presence of
feathers makes a animal able to fly).

Laura

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Chris Angelico]

On Wed, Sep 2, 2015 at 6:08 AM, Marko Rauhamaa  wrote:
> Laura Creighton :
>
>> But are Guile programs small?
>
> They can be tiny because libguile-2.0.so, the interpreter, is a dynamic
> library and is installed on the computer. It's barely different from how
> compiled C programs can be a few kilobytes in size because libc.so is
> dynamic.

And compiled C programs are notoriously hard to distribute. Can you
pick up a Guile binary and carry it to another computer? Do you have
to absolutely perfectly match the libguile version, architecture,
build settings, etc?

Also... how is this different from distributing .pyc files and
expecting people to have a Python interpreter?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Steven D'Aprano]

On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:

> Steven D'Aprano :
> 
>> I believe that Marko is wrong. It is not so easy to compile Python to
>> machine language for real machines. That's why the compiler targets a
>> virtual machine instead.
> 
> Somehow Guile manages it even though Scheme is at least as dynamic a
> language as Python.

It's not about the dynamicism precisely, it's about what counts as primitive
data types and operations.

What are the primitives in Scheme? In Python, the primitives are pretty
complex. I don't know how accurate this page is:

https://en.wikibooks.org/wiki/Scheme_Programming/Scheme_Datatypes

but it suggests that Scheme primitives are quite close to the machine. That
may keep the runtime small.


> I never said a compiler would translate Python to (analogous) machine
> language. I said you could easily turn CPython into a dynamic library
> (run-time environment) and write a small bootstrapper that you package
> into an executable archive with the Python code (whether .py or .pyc).
> What results is a single executable that you can run analogously to any
> other command.

Provided you have the correct version of the dynamic library installed in
the correct place. 

But this doesn't solve the problem of being able to distribute a single
executable file that requires no pre-installed libraries, which is the
problem cx_freeze and pytoexe are made to solve. They solve the case when
you can't assume that there is a Python run-time environment available.

If you are going to require a Python run-time environment, let's call
it "pylib", then you might as well require the python compiler and standard
library be installed. (In the case of C, that's not the case, a distinct
run-time environment makes sense, as the compile-time and run-time
environments are sharply delineated in C. One certainly wouldn't want to
have to re-compile the average C application each and every time you run
it.)

Maybe you could separate the REPL and remove it from pylib, but that's
probably quite small, it might not make that much of a difference to the
total size.

Maybe you could build a pylib that was significantly smaller than the Python
interpreter plus stdlib. That's fine, I'm not arguing it can't be done. I'm
arguing that it's not *easy*, if it were easy somebody likely would have
done it by now.

I don't know the state of the art here. Does Cython work in this space? How
about Nuitka?


> In fact, the shebang notation turns any single .py file into such an
> executable.

I trust that you're not actually arguing that distributing .py files meets
the requirement for a standalone application.


> The problem is if you break your program into modules. Java, 
> of course, solved a similar problem with .jar files (but still wouldn't
> jump over the final hurdle of making the .jar files executable).

You can distribute your Python app as a zip file, except of course you still
need the Python interpreter to be installed.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Marko Rauhamaa]

Laura Creighton :

> But are Guile programs small?

They can be tiny because libguile-2.0.so, the interpreter, is a dynamic
library and is installed on the computer. It's barely different from how
compiled C programs can be a few kilobytes in size because libc.so is
dynamic.

Emacs is a lisp interpreter. As part of its installation, emacs dumps
core, and the executable core file is installed as the editor we know
and love. That way, the most useful lisp modules are already preloaded
in the binary, executable image of the editor.

On my machine, the emacs executable is 15 megabytes in size.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Laura Creighton]

If I understand what you are saying, then I think what you are
looking for is not a compiler, but docker.
see: https://www.docker.com/
in particular https://www.docker.com/whatisdocker

PyPy used this to produce portable PyPy binaries.  See:
https://github.com/squeaky-pl/portable-pypy/blob/master/README.rst

Why did we need to make portable PyPy binaries?

>From the PyPy downloads page:

Linux binaries and common distributions

Linux binaries are dynamically linked, as is usual, and thus might
not be usable due to the sad story of linux binary
compatibility. This means that Linux binaries are only usable on
the distributions where they were created unless you're ready to hack
your system by adding symlinks to the libraries it tries to open.

We had to make versions for debian stable and unstable, and all the
different versions of RHEL, and Centos, and Suse and  every time
we turned around somebody made a new linux distribution, with new and
different versions of shared libraries, located in some new creative
idea of where the best place was to put such things.  And we had to make 32
bit and 64 bit versions, and versions for x86 and for ARM and the list
went on, and on, and on.  This made releases far harder than they
needed to be, and, from the user end meant that people had to work
rather hard just to figure out what version of pypy they should
download from our site.   Lots of people never could quite figure
it out, though we tried to help them on the pypy mailing list and
over irc ...

So finally, we used docker to make a portable version of pypy, which
you can use to biuld your own version of pypy from the nightly
sources, for instance.

I think that this is what you are looking for, as well.  But go read
up on docker and see if it suits.

Laura
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Michael Torrie]

On 08/31/2015 02:35 AM, Mahan Marwat wrote:
> What I know about an interpreter and a compiler is: they both convert
> source code to machine code and the only difference is, an
> interpreter convert it, line by line while compiler convert the whole
> source file. Now if we compile a C source file on C compiler, it will
> produce a small executable file. But if we compile (freeze) a Python
> source file on Python interpreter (using cx_freeze), it will produce
> a big executable file. Now the thing is C compiler does not embed a C
> compiler inside our program, while the Python interpreter (cx_freeze,
> pytoexe) embed Python interpreter inside our program, what is the
> reason? The question is, why cx_freeze etc... embed Python
> interpreter inside our programs, they are unable to produce machine
> code like C compiler do? Cant we program a language, which is both
> interpreted and compiled? The core developer cant add the compiling
> functionality to Python?

It think your questions have been well answered by others.  But there
are several attempts at making an actual python compiler.  Often this
involve less-dynamic subset of Python.  For example, pypy has a dialect
called rpython which compiles straight to C++ code, and then to machine
code.

Another subset compiler is cython, which is somewhat of a specialized
compiler. It compiles a subset of Python to a binary shared library that
can be imported into a Python program running in the normal interpreter.

The dynamic nature of Python means that probably your best route to
speed is going to be through just-in-time compilation. The pypy project
is an attempt to do JIT with Python. So far the results are very
promising.  Pretty cool since pypy is written in Python and bootstraps
from the standard python interpreter.

Lastly, one attempt at a compiler is nuitka (google it).  It produces
self-contained executables.  Nuitka compiles what it can, and interprets
the rest (if I understand it correctly) by embedding libpython itself in
the executable.  At this time, nuitka isn't focusing on performance,
more correctness.  GvR doesn't really think much of nuitka, but I think
it's a cool project and the developer is a nice guy.  Maybe have its uses.

So far I haven't had a use for nuikta; cPython is enough for me, with
cython for compiling functions that need some more raw speed.  I tend to
use more conventional optimization techniques that work just fine with
the interpreter.  And often the results are fast enough.  For example I
implemented a simple memoization wrapper for a particularly expensive
function that was called a lot, often over the same inputs several
times.  Runtime went from 10 seconds to less than 1.  Enough for me!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Marko Rauhamaa]

Steven D'Aprano :

> On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:
>> I never said a compiler would translate Python to (analogous) machine
>> language. I said you could easily turn CPython into a dynamic library
>> (run-time environment) and write a small bootstrapper that you
>> package into an executable archive with the Python code (whether .py
>> or .pyc). What results is a single executable that you can run
>> analogously to any other command.
>
> Provided you have the correct version of the dynamic library installed
> in the correct place.

Yes, virtually all Linux software builds upon dynamic libraries that
have been introduced to the linker via ldconfig.

> But this doesn't solve the problem of being able to distribute a single
> executable file that requires no pre-installed libraries, which is the
> problem cx_freeze and pytoexe are made to solve.

I wasn't trying to solve that particular problem. I was simply stating
you could compile (translate, turn) a Python program into a single,
executable file.

> I trust that you're not actually arguing that distributing .py files
> meets the requirement for a standalone application.

If your application consists of a single .py file, why not?

>> The problem is if you break your program into modules. Java, of
>> course, solved a similar problem with .jar files (but still wouldn't
>> jump over the final hurdle of making the .jar files executable).
>
> You can distribute your Python app as a zip file, except of course you
> still need the Python interpreter to be installed.

Again, having a Python interpreter around is not the issue I'm talking
about. I'm talking about the possibility to compile (translate, turn) a
Python program into a single, executable file.

Now, even C programs can suffer from the module problem: you sometimes
need to ship extra dynamic libraries ("modules") with your binary.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Marko Rauhamaa]

Steven D'Aprano :

> I believe that Marko is wrong. It is not so easy to compile Python to
> machine language for real machines. That's why the compiler targets a
> virtual machine instead.

Somehow Guile manages it even though Scheme is at least as dynamic a
language as Python.

I never said a compiler would translate Python to (analogous) machine
language. I said you could easily turn CPython into a dynamic library
(run-time environment) and write a small bootstrapper that you package
into an executable archive with the Python code (whether .py or .pyc).
What results is a single executable that you can run analogously to any
other command.

In fact, the shebang notation turns any single .py file into such an
executable. The problem is if you break your program into modules. Java,
of course, solved a similar problem with .jar files (but still wouldn't
jump over the final hurdle of making the .jar files executable).


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Steven D'Aprano]

On Tue, 1 Sep 2015 03:48 am, Mahan Marwat wrote:

>> Python programs *could* easily be compiled the same way, but it generally
>> hasn't been considered all that useful.
> 
> If it hasn't been considered all that useful, then why the tools like
> cx_freeze, pytoexe are doing very hard! And if it is really easy, then why
> cx_freeze, pytoexe developer are doing it in such a rubbish way instead of
> creating one (compiler)?

I believe that Marko is wrong. It is not so easy to compile Python to
machine language for real machines. That's why the compiler targets a
virtual machine instead.

Let's take the simple case of adding two numbers:

x + y

With a real machine, this will probably compile down to a handful of
assembly language statements. But, x will have to be an integer of a
certain type, say, 32 bits, and y likewise will also have to be the same
size. And what happens if the addition overflows? Some machines may
overflow to a negative value, some might return the largest positive value.

Python addition does not do that. With Python, it doesn't matter whether x
and y are the same size, or different. In fact, Python integers are BigNums
which support numbers as big as you like, limited only by the amount of
memory. Or they could be floats, complex numbers, or fractions. Or x might
be a string, and y a list, and the + operator has to safely raise an
exception, not dump core, or worse.

So to compile Python into assembly language for a real CPU would be very
hard. Even simple instructions would require a large, complex chunk of
machine code. If only we could design our own machine with a CPU that
understood commands like "add two values together" without caring that the
values are compatible (real) machine types.

And that's what we do. Python runs in a virtual machine with "assembly
language" commands that are far, far more complex than real assembly for
real CPUs. Those commands, of course, eventually end up executing machine
code in the real CPU, but it usually does so by calling the Python runtime
engine.

There is another reason why frozen Python code is so large. It has to
include a full Python interpreter. The Python language includes a few
commands for compiling and interpreting Python source code:

- eval
- exec
- compile


so the Python runtime has to include the Python compiler and interpreter,
which effectively means that the Python runtime has to be *all* of Python,
or at least nearly all.

There may be projects that will compile Python down to machine code for a
real machine (perhaps Cython?) but doing so isn't easy, which is why most
compilers don't do it.




-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Mahan Marwat]

> Python programs *could* easily be compiled the same way, but it generally  
> hasn't been considered all that useful.

If it hasn't been considered all that useful, then why the tools like 
cx_freeze, pytoexe are doing very hard!
And if it is really easy, then why cx_freeze, pytoexe developer are doing it in 
such a rubbish way instead of creating one (compiler)?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Emile van Sebille]

On 8/31/2015 10:48 AM, Mahan Marwat wrote:

Python programs *could* easily be compiled the same way, but it generally
hasn't been considered all that useful.


If it hasn't been considered all that useful, then why the tools like 
cx_freeze, pytoexe are doing very hard!


People scratching their own itch.


And if it is really easy, then why cx_freeze, pytoexe developer are doing it in 
such a rubbish way instead of creating one (compiler)?


You'd have to ask them.

Emile




--
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Ben Finney]

Mahan Marwat  writes:

> > Python programs *could* easily be compiled the same way, but it generally  
> > hasn't been considered all that useful.
>
> If it hasn't been considered all that useful, then why the tools like
> cx_freeze, pytoexe are doing very hard!

Thay don't compile to machine code for the host CPU. That's the step
that “compiled the same way” refers to above, as not having been
considered all that useful.

What those tools do – compile to Python virtual machine code, and then
bundle that machine code with the virtual machine into a single
executable file – is quite useful.
-- 
 \“Some people, when confronted with a problem, think ‘I know, |
  `\   I'll use regular expressions’. Now they have two problems.” |
_o__)   —Jamie Zawinski, in alt.religion.emacs |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Cameron Simpson]

On 31Aug2015 01:35, Mahan Marwat  wrote:

What I know about an interpreter and a compiler is: they both convert source 
code to machine code and the only difference is, an interpreter convert it, 
line by line while compiler convert the whole source file.


This is simplistic and misleading.

Compilation is the process of taking a program and converting it into an 
efficient binary form, attaching whatever else is needed (such a references to 
required libraries). Languages like C are usually compiled to machine code, and 
executed directly by hardware. Languages like Python are usually compiled to 
"byte code", effectively a machine code for a logical machine.


When hardware acts on machine code it is said to execute it. When a piece of 
software acts on byte code, it is said to "interpret" it. The main difference 
is the notional level of execution, and the corresponding level of abstraction 
in the machine code: machine code for a hardware CPU is normally very direct 
and contains instructions directly related to the hardware facilities exposed 
by the CPU (registers, memory, etc). Conversely, byte code (machine code for a 
more abstract interpreter) is more abstract: the basic elements will be higher 
level entities like strings or integers , and memory will be described by more 
abstract items like slots in call frames or names to look up in symbol tables.



Now if we compile a C source file on C compiler, it will produce a small 
executable file. But if we compile (freeze) a Python source file on Python 
interpreter (using cx_freeze), it will produce a big executable file.


If you compile a C program with "static" linking you will get a much large 
program. Modern machines support dynamic library linking, where the needed 
libraries are attached to the executing program on the fly, largely when the 
program is started. That relies on the libraries being available in the 
filesystem when the program is invoked. By contrast a "staticly" linked program 
will get copies of the libraries installed into the executable when compiled.



Now the thing is C compiler does not embed a C compiler inside our program, 
while the Python interpreter (cx_freeze, pytoexe) embed Python interpreter 
inside our program, what is the reason?


Chris Angelico has described why a Python interpreter is needed at runtime: 
Python can accept arbitrary Python code at runtime, so it needs to be able to 
compile it (to byte code) and interpret that byte code.



The question is, why cx_freeze etc... embed Python interpreter inside our 
programs, they are unable to produce machine code like C compiler do?
Cant we program a language, which is both interpreted and compiled? The core 
developer cant add the compiling functionality to Python?


You can go a long way there. There are alternatives to CPython which try to do 
this, such as Cython. A major obstacle to "complete" compilation is that Python 
is a dynamic language. Every variable may be bound to an object of any type at 
any time, so a lot of behaviour is dependent on the current binding, which may 
change.


Languages like C are staticly typed: their operations convert far more directly 
to machine operations (in fact, this is an explicit target of their design).  
Languages like Python aim for a more freindly and flexible programming syntax, 
and some of the price of that is more indirection between "byte code" and the 
underlying machine operations.


Cheers,
Cameron Simpson 

remember, information is not knowledge,
knowledge is not wisdom
wisdom is not truth
truth is not beauty
beauty is not love
love is not music
music is the best   - frank zappa
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Chris Angelico]

On Mon, Aug 31, 2015 at 6:35 PM, Mahan Marwat  wrote:
> What I know about an interpreter and a compiler is: they both convert source 
> code to machine code and the only difference is, an interpreter convert it, 
> line by line while compiler convert the whole source file.
> Now if we compile a C source file on C compiler, it will produce a small 
> executable file. But if we compile (freeze) a Python source file on Python 
> interpreter (using cx_freeze), it will produce a big executable file.
> Now the thing is C compiler does not embed a C compiler inside our program, 
> while the Python interpreter (cx_freeze, pytoexe) embed Python interpreter 
> inside our program, what is the reason?
> The question is, why cx_freeze etc... embed Python interpreter inside our 
> programs, they are unable to produce machine code like C compiler do?
> Cant we program a language, which is both interpreted and compiled? The core 
> developer cant add the compiling functionality to Python?

Compiling is trivially easy for trivial programs, and becomes
virtually impossible for arbitrarily complex and dynamic programs. In
an extreme example, Python includes the ability to execute arbitrary
code:

>>> exec("print('Hello, world!')")
Hello, world!

You basically can't compile that without having the full capabilities
of the Python language. If you want something that makes Python into
an executable binary, you either want a wrapper (like cx_freeze and
pytoexe and so on), or else something like the PyPy project. Check it
out - it's pretty cool!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Ben Finney]

Mahan Marwat  writes:

> What I know about an interpreter and a compiler is: they both convert
> source code to machine code

Yes, Python implementations always compile the source code to machine
code. The target machine for the Python compiler, though, is a virtual
machine which then gets emulated by a Python runtime interpreter.

You can prove this by doing *only* the compile step, and later do
*only* the interpret step::

$ python -m compileall foo.py
Compiling foo.py ...

$ python foo.pyc
Hello, world!

So anything which is going to run your Python source code first needs
that source code to be compiled to machine code, and then later use an
interpreter to run that machine code.

That's true of pretty much every programming language today: To run it,
the source code is first compiled to a machine code, which is then
interpreted at run-time by some 

You might want to target some other machine code; ultimately, you might
want to the machine code of the CPU you intend to run the program. That
is a special case, though, and is by far not the only case of compiling
code.

> and the only difference is, an interpreter convert it, line by line
> while compiler convert the whole source file.

Hmm, by that definition, there are almost no language interpreters in
use today for running a program – not Python, not Ruby, not JavaScript,
not Perl – that meets your definition. They all need a compiler first to
produce machine code from the source code file.

> Now if we compile a C source file on C compiler, it will produce a
> small executable file. But if we compile (freeze) a Python source file
> on Python interpreter (using cx_freeze), it will produce a big
> executable file.

Yes, that's because the C language is low-level enough that a compiler
can target directly the host CPU's machine code. A Python program,
though, is written in a dynamic language, which is compiled to a virtual
machine code, which needs the Python virtual machine as well to
interpret that machine code.

> Now the thing is C compiler does not embed a C compiler inside our
> program, while the Python interpreter (cx_freeze, pytoexe) embed
> Python interpreter inside our program, what is the reason?

Because Python is a dynamic language. A great many facts about the
run-time state are deferred to be figured out at run-time, so can't be
decided at compile time. The resulting machine code assumes a very
powerful, dynamic machine language, which is not the same as the host
CPU's machine language.

> Cant we program a language, which is both interpreted and compiled?

I hope you see now that Python is both interpreted and compiled, by
necessity.

What you seem to want is a Python compiler that targets the host CPU;
but your target host CPU can't execute the dynamic runtime that Python
machine code needs.

In other words: The very dynamic nature that makes Python so expressive,
succinct, and efficient to write, makes it that much more distant from
the extremely low-level machine operations of the host CPU. That power
comes at the cost of a level of abstraction, in the Python virtual
machine interpreter.

-- 
 \   “Pray, v. To ask that the laws of the universe be annulled in |
  `\ behalf of a single petitioner confessedly unworthy.” —Ambrose |
_o__)   Bierce, _The Devil's Dictionary_, 1906 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Luca Menegotto]

Il 31/08/2015 19:48, Mahan Marwat ha scritto:


If it hasn't been considered all that useful, then why

> the tools like cx_freeze, pytoexe are doing very hard!

Well, I consider those tools useless at all!
I appreciate Python because, taken one or two precautions, I can easily 
port my code from one OS to another with no pain.

So, why should I loose this wonderful freedom?

--
Ciao!
Luca

--
https://mail.python.org/mailman/listinfo/python-list

Re: Why Python is not both an interpreter and a compiler?

[Marko Rauhamaa]

Ben Finney :

> Yes, that's because the C language is low-level enough that a compiler
> can target directly the host CPU's machine code. A Python program,
> though, is written in a dynamic language, which is compiled to a
> virtual machine code, which needs the Python virtual machine as well
> to interpret that machine code.

More generally, both C and Python programs are run with the assistance
of a run-time environment. C programs are compiled in such a way that a
single compiled file is enough to start up the program. Python programs
*could* easily be compiled the same way, but it generally hasn't been
considered all that useful.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list