Re: [IronPython] My takes on C Extensions for IronPython
I'll just reply here and try to summarize some things. On 10/18/07 Seo Sanghyeon wrote: It's certainly more than 200 lines, and it's not completely trivial because some parts of Python/C API are C macros, not C functions. (Reference counting is the most important, but also getting type(=vtable) of Python objects and like.) The good news is that this part is already developed, debugged, and pretty much finished -- for Python 2.3 to 2.6, and for wide and narrow unicode builds of Python. Get it here (~ 1600 lines): That code has each dllimport method spanning 5 lines of code. I don't consider a cut and pasted [DllImport()] attribute as something that is hard to write:) So this complete work that covers 4 major versions of python is less than 200 p/invokes. It's missing a few structure definitions. Nothing an intern can't write in an afternoon. My point still stands: interoperating with the C code is not an issue. I understand MC++ or Managed C++ is used to refer related technologies, but in the narrow sense it is a compiler that shipped with .NET 1.x which produced mix of managed and x86 code. I believe .NET 2.x compiler, i.e. C++/CLI is what is currently discussed. Yes, Managed C++ referes to the old broken and already abandoned technology, but it's sometimes used also to refer to the new C++/CLI lanaguage, likely because C++/CLI is not a really good name:) It would be nice to be able to compile without Microsoft tools, but I don't think it's essential. If assembly compiled with C++/CLI / clr:pure can be made to run on Mono that's fine. First, someone should investigate if clr:pure actually provides seamless integration with C code. I doubt it works, but I'll wait for someone using windows to confirm this. The issue then is that, even if the above would work, the MS C++/CLI compiler will emit code that depends on the windows ABI (specifically always considering long to be 32 bits, they even put this in the standard, sigh). What this means for people not familiar with low-level details, is that such code is not portable and hence won't work on Linux 64 bit systems (this is always assuming the interoperability with C code works at all even on 32 bit systems). So, while it's not essential that this code be built with non-Microsoft tools, only the MS tools are able to compile that code and they compile it incorrectly (and as the C++/CLI standard requires, sadly, so there is no chance this will be ever fixed). Also Mono page doesn't mention this, but GCC backend to emit CLI code *is* complete and working. It's available right now from GCC SVN. But it's yet another project, not Google Summer of Code attempts. This project is backed by STMicroelectronics, a big corporation, probably for its own use. http://gcc.gnu.org/projects/cli.html Remember, Mono doesn't need to be the sole provider of non-Microsoft CLI tools. :) I believe GCC/CLI can be made to help this project. See above for the portability issues: they'd have to either not follow the standard or to produce non-working code. Also the gcc CLI backend is not really complete and even if it were it doesn't mean that C++/CLI would be any easier to implement on top of it. My estimate is that C++/CLI on top of gcc will require at least 4-5 years, if someone implements it at all (there is not much demand for it, there are the portability and standard issues I mentioned above and by that time Mono will have an extension to semalessly dllimport C++ code, lowering demand for C++/CLI even more). In 4-5 years Microsoft itself will pull the plug on C++/CLI, so I think basing a new design on it is a mistake. Anyway, I'm not the one who's going to write the code, so more power to people that will, I just wanted to make people aware that using managed C++ or C++/CLI will prevent the sw to work on anything but windows with the .net runtime. Happy hacking! lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Announcement: Project to get some CPython C extensions running under IronPython
On 10/17/07 Tony Djordjevski wrote: You do need to write C code as the API is a C API. I didn't list in the last mail because the C implementation of these functions is not a difference between the C# impl and the C++ impl of the rest of the code. The dllimport declarations I mentioned above would pinvoke to these C functions. Those C functions you speak about will need to execute managed code. So, either way, what needs to be done is to allow Reverse P/Invoke or some form of DllExport. Assuming this project is successful, native C/C++ will have to, at some point, execute managed code. There are only two ways that I know of that you can accomplish this in .NET: 1. C++ Interop 2. Modifying a compiled assembly or 3, the easiest of all: using function pointers in the C code which the runtimes will generate automatically when passing a delegate to a p/invoked method. Besides being trivial compared to both 1 and 2, it works on Mono and it works on the MS CLR. Happy hacking! lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Announcement: Project to get some CPython C extensions running under IronPython
On 10/15/07 Giles Thomas wrote: Curt Hagenlocher wrote: My two cents would be this: using Managed C++, try for source compatibility first. As python extensions use a C API, I don't see how Managed C++ would provide source compatibility. Managed C++ would be an ill-advided method, IMHO. Binary compatibility would be a great thing in the long term - after all, we don't want to run the risk of branching the target library's codebase! - but my gut feeling is that you're right, Curt: it would be much easier, at least as an exploratory first stage, to target source compatibility only. So perhaps we should start off in that direction. I agree you should aim for source compatibility first, it's not like people commonly use binary proprietary python modules (and if they do it's the same people that wrote them and so can recompile). Binary compatibility is harder to prove correct, anyway and you'll have already your share of issues to overcome:) Does anyone know how this would impact the potential for Mono compatibility? A Managed C++ implementation would be pretty useless for a mono port, but I think it would be pretty useless for the .net runtime, as you need to expose a C API anyway. You'd have two components: the C API headers and some C code (this is required in any case) is the first. Then you need an assembly that translates from the C API to the IronPython model. You can write this in nice C# + a few dllimports or with the ugly managed C++ (assuming managed C++ can actually consume the python headers). lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Announcement: Project to get some CPython C extensions running under IronPython
On 10/17/07 Curt Hagenlocher wrote: On 10/17/07, Paolo Molaro [EMAIL PROTECTED] wrote: As python extensions use a C API, I don't see how Managed C++ would provide source compatibility. Managed C++ would be an ill-advided method, IMHO. [...] You'd have two components: the C API headers and some C code (this is required in any case) is the first. Then you need an assembly that translates from the C API to the IronPython model. You can write this in nice C# + a few dllimports or with the ugly managed C++ (assuming managed C++ can actually consume the python headers). MC++ lets you create a single module that contains both normal machine code and MSIL, and takes care of the transitions between the two. There's no reason you can't compile the existing C code and link it directly with the MC++ wrapper -- which is exactly what I originally meant. The python API requires a couple dozen structure definitions plus a few dozen dllimport declarations. This can be about 200 lines of trivial to write declarative stuff. The rest of the code is likely in the order of 50 K lines, so now ask yourself: would you rather write a) 50 K lines of nice C# code + 200 lines of trivial declarations or b) 50 K lines of orrible C++ code? I know what I would answer:) The managed C++ discussion is focused on avoiding the 200 lines of trivial declarations, when the issue is the 50 KLOC. Managed C++ might make sense if you have lots of legacy C++ code you need to integrate (at least until MS itself will pull the plug on it). For new code like this case, managed C++ is the wrong tool, IMHO. lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Announcement: Project to get some CPython C extensions running under IronPython
On 10/17/07 Giles Thomas wrote: I must admit I'd also misunderstood how a MC++ option would work, in the same way as Paolo Molaro, but this sounds really useful. Is there any I don't think I misunderstood anything:) I said that there are two modules, the C one and the one that bridges the C code to the ironpython world. That the two modules can be combined into the same binary image doesn't change the fact that they are two different things, it's just an implementation detail. Thanks! lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Announcement: Project to get some CPython C extensions running under IronPython
On 10/17/07 Curt Hagenlocher wrote:
On 10/17/07, Paolo Molaro [EMAIL PROTECTED] wrote:
The python API requires a couple dozen structure definitions plus a few
dozen dllimport declarations. This can be about 200 lines of trivial
to write declarative stuff.
I'm having trouble understanding what you mean by this. Let's say I've got
an extension method that looks like this:
PyObject * ModifyColumns(PyObject * self, PyObject * args)
{
[...]
In order to use this extension from IronPython, I need C implementations of
each of those API functions. The PySequence_ methods should to be able to
understand CLR arrays or any other IEnumerable-like object. I'm not sure
what PyTuple_New should return, but whatever it is will need to have
PyObject*-like semantics.
How does DllImport fit into this picture? How can I avoid implementing all
these functions in C or C++?
You do need to write C code as the API is a C API. I didn't list in the
last mail because the C implementation of these functions is not a
difference between the C# impl and the C++ impl of the rest of the code.
The dllimport declarations I mentioned above would pinvoke to these C
functions.
lupus
--
-
[EMAIL PROTECTED] debian/rules
[EMAIL PROTECTED] Monkeys do it better
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] IronPython now runs faster on Mono
On 06/05/07 Sanghyeon Seo wrote: This is because we JIT many more methods in 2.0 and the JIT time is accounted in the results. Adding a warmup call to pystone.py will get the 1.1 and 2.0 results much closer than they appear to be with a default run (58k vs 46k in the default and 59k vs 55k with the warmup on my pentium M 1.6). How do I add a warmup call to pystone.py? Add: pystones(1) at the end of the file just before main(loops). It should be 1 instead of 1, but the code is buggy and will result in a division by zero error (and 1 may not be enough on a fast box, so if you get an error you'll need to increase it...). lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] IronPython now runs faster on Mono
On 06/03/07 Sanghyeon Seo wrote: Thanks to optimization works on delegate invocation, IronPython now runs faster on Mono. Conservatively, you can expect 5% speedup for 1.1, and 10% speedup for 2.0a1. Current svn mono should be a few percent faster as well (optimized string.CompareOrdinal). I got a few more insights into the performace issues with version 2.0 as well: if you try to run pystone with more passes (200, 300, 500 K) you'll notice the reported performance is significantly higher. This is because we JIT many more methods in 2.0 and the JIT time is accounted in the results. Adding a warmup call to pystone.py will get the 1.1 and 2.0 results much closer than they appear to be with a default run (58k vs 46k in the default and 59k vs 55k with the warmup on my pentium M 1.6). The good news is that most of the additional methods compiled are just because we haven't implemented in mono code sharing for generic methods with reference generic arguments. The bad news is that we haven't scheduled the time yet to implement code sharing. I guess these data points will help us to prioritize it. lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] os.popen() + Mono == segfault
On 12/09/06 Anthony Baxter wrote: On 12/8/06, Paolo Molaro [EMAIL PROTECTED] wrote: The bug is fixed in svn (at least on non-win32): Process didn't handle the case when only the filename is set in ProcessStartInfo. Fabulous! Thanks for the quick response. This will be in the 1.2.3 release? Yes. Note the workaround I mentioned: use an argument to the program to get it to work on earlier versions. lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] IPCE and Mono 1.2 interactive interpreter sadness
On 12/13/06 Miguel de Icaza wrote: IPCE on Mono 1.1.17 works really nicely in interactive mode. Backspace works, ^D to exit works, the lot. On 1.2.2, however, it's a world of broken. Backspace is just insane, ^D stopped working again... it's pretty much unusable. I'm not sure whether this is a Mono bug (in which case it should be logged) or a problem with the IPCE code. Can you provide a more informed opinion? I wonder if IPCE has any patches in the Console area in addition to standard IronPython, because we recently fixed the Console to work properly with IronPython (stock). I used stock ipy.exe with svn mono and it fails, at least on a gnome-terminal (TERM=xterm) see: http://bugzilla.ximian.com/show_bug.cgi?id=80208 lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] os.popen() + Mono == segfault
On 12/07/06 Anthony Baxter wrote:
On both IronPython 1.0.1 and IPCE release 4, os.popen() segfaults
under Mono 1.17.1 (on Ubuntu edgy).
To reproduce:
ipy.exe -c import os; print os.popen('/bin/ls', 'r').read()
Stacktrace follows, for whatever value it is... I can't tell
immediately whether it's an IronPython or Mono problem, although it
_appears_ to be in Mono. If other people agree, I'll log a Mono bug
tomorrow.
The bug is fixed in svn (at least on non-win32): Process didn't handle
the case when only the filename is set in ProcessStartInfo.
lupus
--
-
[EMAIL PROTECTED] debian/rules
[EMAIL PROTECTED] Monkeys do it better
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
