[IronPython] Ironclad v2.6.0rc1 released

[William Reade]

Hi all

I'm very happy to announce the latest release (candidate) of Ironclad,
the 120-proof home-brewed CPython compatibility layer, now available
for IronPython 2.6!

No longer need .NET pythonistas toil thanklessly without the benefits
of bz2, csv, numpy and scipy: with a simple 'import ironclad', (most
parts of) the above packages -- and many more -- will transparently
Just Work.

Get the package from:
http://code.google.com/p/ironclad/

...and get support from:
http://groups.google.com/group/c-extensions-for-ironpython

...or just ask me directly.

I'm very keen to hear your experiences, both positive and negative; I
haven't been able to test it on as many machines as I have in the
past, so your feedback is especially important this time round*.

Cheers
William

* I'd be especially grateful if someone with a newish multicore
machine would run the numpy and scipy test scripts (included in the
source distrbution) a few times to check for consistent results and
absence of weird crashes; if someone volunteers, I'll help however I
can.
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] IronPython and PIL?

[William Reade]

Hi Markus

Parts of PIL should work with IronPython if you're willing to import 
ironclad first ( http://code.google.com/p/ironclad ). If you're still 
using ipy 2, the latest binary release should work for you.


(If you're using 2.6, it definitely won't work right now, because ipy 
2.6 can't parse PIL.Image)


Incidentally, the reason I specify 'parts of' PIL is that it doesn't 
come with tests. I do have a few homebrew test cases that check for 
identical output when cpy and ipy do the same things -- and they do work 
-- but I can't claim any serious coverage.


If you try Ironclad, please let me know how it works for you.

Cheers
William


Markus Törnqvist wrote:

Hi!

Clearly PIL doesn't exist for IronPython, right?

I figured I'd debug the weird 404 behaviour with the debug server, but
bumped into PIL not existing.

I'm very surprised NWSGI/IIS allows the program to even start, as it
doesn't start with runserver :/

Maybe this is related to the 404s in some very roundabout way?

Anyway, to the point:
My application uses ImageFields in the code, so although I exchanged
all my own PIL stuff for System.Drawing stuff through clr, I'm still
using ImageField. Which clearly does not work.

Does anyone know if there exists a clr/ironpython ImageField implementation
that doesn't use PIL? A quick Google didn't turn up anything...

Thanks!



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] SystemError while compiling PIL.Image

[William Reade]

Hi Dino

If I comment out 2 methods, the file parses fine. Uncommenting either is 
enough to cause it to fail.


def convert(self, mode=None, data=None, dither=None, palette=WEB, 
colors=256):


def rotate(self, angle, resample=NEAREST, expand=0):

WEB and NEAREST are both set to 0 at module level; convert and rotate 
are both methods on the module-level old-style class Image. The only 
obvious common feature is that both have a non-name default parameter 
following a name default parameter, but I haven't managed to produce a 
minimal repro.


If you're allowed to just look at other people's code willy-nilly, the 
file is Image.py in PIL 1.16 :).


Cheers
william




Dino Viehland wrote:

Is there a parameter with a default value that is a name?  If so what is the
scope of the name in relation to the function definition?


-Original Message-
From: users-boun...@lists.ironpython.com [mailto:users-
boun...@lists.ironpython.com] On Behalf Of William Reade
Sent: Thursday, October 22, 2009 6:37 AM
To: Discussion of IronPython
Subject: [IronPython] SystemError while compiling PIL.Image

Transcript follows. The contents of the file don't look obviously
weird.
Does anyone have any idea what the problem might be?

Cheers
William

---
--

C:\dev\ironclad>ipy -X:ExceptionDetail
C:\Python26\Lib\site-packages\PIL\Image.py
Object reference not set to an instance of an object.
at IronPython.Compiler.Ast.GlobalAllocator.GetVariable(AstGenerator
ag, PythonVariable variable)
at IronPython.Compiler.Ast.NameExpression.Transform(AstGenerator
ag,
Type type)
at
IronPython.Compiler.Ast.FunctionDefinition.TransformParameters(AstGener
ator
outer, AstGenerator inner, List`1 defaults, List`1 names, Boolean
needsWrapperMethod, List`1 init)
at
IronPython.Compiler.Ast.FunctionDefinition.TransformToFunctionExpressio
n(AstGenerator
ag)
at
IronPython.Compiler.Ast.FunctionDefinition.Transform(AstGenerator ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Stat
ement
body, SourceLocation prevStart)
at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator
ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Stat
ement
body, SourceLocation prevStart)
at IronPython.Compiler.Ast.IfStatement.Transform(AstGenerator ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at
IronPython.Compiler.Ast.FunctionDefinition.TryTransformBody(AstGenerato
r
ag, List`1 statements)
at
IronPython.Compiler.Ast.FunctionDefinition.TransformToFunctionExpressio
n(AstGenerator
ag)
at
IronPython.Compiler.Ast.FunctionDefinition.Transform(AstGenerator ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Stat
ement
body, SourceLocation prevStart)
at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator
ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at IronPython.Compiler.Ast.ClassDefinition.Transform(AstGenerator
ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Stat
ement
body, SourceLocation prevStart)
at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator
ag)
at
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Stat
ement
fromStmt)
at IronPython.Compiler.Ast.PythonAst.Transform(AstGenerator ag)
at IronPython.Compiler.Ast.PythonAst.TransformToAst(CompilationMode
mode, CompilerContext context)
at IronPython.Runtime.PythonContext.CompilePythonCode(Nullable`1
compilationMode, SourceUnit sourceUnit, CompilerOptions options,
ErrorSink errorSink)
at IronPython.Runtime.PythonContext.GetScriptCode(SourceUnit
sourceCode, String moduleName, ModuleOptions options, Nullable`1 mode)
at IronPython.Runtime.PythonContext.GetScriptCode(SourceUnit
sourceCode, String moduleName, ModuleOptions options)
at IronPython.Runtime.PythonContext.CompileModule(String fileName,
String moduleName, SourceUnit sourceCode, ModuleOptions options,
ScriptCode& scriptCode)
at IronPython.Hosting.PythonCommandLine.RunFileWorker(String
fileName)
at IronPython.Hosting.PythonCommandLine.RunFile(String fileName)
SystemError: Object reference not set 

[IronPython] SystemError while compiling PIL.Image

[William Reade]

Transcript follows. The contents of the file don't look obviously weird. 
Does anyone have any idea what the problem might be?


Cheers
William

-

C:\dev\ironclad>ipy -X:ExceptionDetail 
C:\Python26\Lib\site-packages\PIL\Image.py

Object reference not set to an instance of an object.
   at IronPython.Compiler.Ast.GlobalAllocator.GetVariable(AstGenerator 
ag, PythonVariable variable)
   at IronPython.Compiler.Ast.NameExpression.Transform(AstGenerator ag, 
Type type)
   at 
IronPython.Compiler.Ast.FunctionDefinition.TransformParameters(AstGenerator 
outer, AstGenerator inner, List`1 defaults, List`1 names, Boolean 
needsWrapperMethod, List`1 init)
   at 
IronPython.Compiler.Ast.FunctionDefinition.TransformToFunctionExpression(AstGenerator 
ag)

   at IronPython.Compiler.Ast.FunctionDefinition.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Statement 
body, SourceLocation prevStart)

   at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
   at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Statement 
body, SourceLocation prevStart)

   at IronPython.Compiler.Ast.IfStatement.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)
   at 
IronPython.Compiler.Ast.FunctionDefinition.TryTransformBody(AstGenerator 
ag, List`1 statements)
   at 
IronPython.Compiler.Ast.FunctionDefinition.TransformToFunctionExpression(AstGenerator 
ag)

   at IronPython.Compiler.Ast.FunctionDefinition.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Statement 
body, SourceLocation prevStart)

   at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
   at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)

   at IronPython.Compiler.Ast.ClassDefinition.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformMaybeSingleLineSuite(Statement 
body, SourceLocation prevStart)

   at IronPython.Compiler.Ast.AstGenerator.Transform(Statement[] from)
   at IronPython.Compiler.Ast.SuiteStatement.Transform(AstGenerator ag)
   at 
IronPython.Compiler.Ast.AstGenerator.TransformWithLineNumberUpdate(Statement 
fromStmt)

   at IronPython.Compiler.Ast.PythonAst.Transform(AstGenerator ag)
   at IronPython.Compiler.Ast.PythonAst.TransformToAst(CompilationMode 
mode, CompilerContext context)
   at IronPython.Runtime.PythonContext.CompilePythonCode(Nullable`1 
compilationMode, SourceUnit sourceUnit, CompilerOptions options, 
ErrorSink errorSink)
   at IronPython.Runtime.PythonContext.GetScriptCode(SourceUnit 
sourceCode, String moduleName, ModuleOptions options, Nullable`1 mode)
   at IronPython.Runtime.PythonContext.GetScriptCode(SourceUnit 
sourceCode, String moduleName, ModuleOptions options)
   at IronPython.Runtime.PythonContext.CompileModule(String fileName, 
String moduleName, SourceUnit sourceCode, ModuleOptions options, 
ScriptCode& scriptCode)

   at IronPython.Hosting.PythonCommandLine.RunFileWorker(String fileName)
   at IronPython.Hosting.PythonCommandLine.RunFile(String fileName)
SystemError: Object reference not set to an instance of an object.

-
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Hosted IronPython "cannot import" error

[William Reade]

Packetlogic does indeed use compiled CPython extensions, and so won't 
normally work with IronPython. However, it appears to import cleanly if 
you use Ironclad:


C:\dev\ironclad-v0.8.5-src\build>ipy
IronPython 2.0.2 (2.0.0.0) on .NET 2.0.50727.3074
Type "help", "copyright", "credits" or "license" for more information.
>>> import ironclad
>>> import packetlogic2.v11.plapi
>>>

Get it from http://code.google.com/p/ironclad/

Ask questions at http://groups.google.com/group/c-extensions-for-ironpython

Please let me know if you find bugs :-).

Cheers
William


Dave Fugate wrote:
Just curious; what happens if you cd to site-packages and try to run the 
same import directly from ipy.exe (with the -X:ExceptionDetail flag)?  
Based on the fact you can get this to work with IDLE, I'd guess 
packetlogic is C-based and hence subject to 
http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=11333.
 
My best,
 
Dave
 

*From:* users-boun...@lists.ironpython.com 
[users-boun...@lists.ironpython.com] on behalf of Blosser, Peter 
[peterblos...@letu.edu]

*Sent:* Thursday, September 10, 2009 3:19 PM
*To:* users@lists.ironpython.com
*Subject:* [IronPython] Hosted IronPython "cannot import" error

Hi.

I have been trying to implement a hosted IronPython solution in C#.  I 
have been successful at executing basic python code, but  I am having 
problems when trying to import a module.


 

The module I am trying to call is the packetlogic API for a traffic 
shaping device, and I have been able to use it with IDLE, but not 
IronPython.  (http://www.proceranetworks.com/pythonapi.html)


 

I was origionally getting the error of  “name ‘packetlogic2’ not 
defined, but I noticed that the sys.path was empty when I ran the 
program.  So I set the path to the folder where the module is located 
using  AddToPath() in the PythonEngine, and now I am getting the current 
error “cannot import PLv11 from packetlogic2.v11.plapi”


 

v11 and plapi are both folders that are under the packetlogic2 folder in 
“site-packages”, but it acts like it cannot find them.  Is there some 
other  path that I need to setup?  Any ideas?


 


Thanks

-

Peter




___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] object lifecycle issues

[William Reade]

Damn, it's my problem -- sorry for the noise. It seems that 
IronPython.NewTypes.BlahBlah.ExtensibleBlah types don't handle equality 
quite how I expected... clearly, I need to learn to fire up windbg 
*before* I post. At least we rediscovered the __del__ bug ;-).


Incidentally, congratulations on 2.0.2!

Dino Viehland wrote:

The first 2 things that occur to me is that either there's an exception
happening when we run WeakRefTracker's finalizer (it swallows exceptions)
or the object simply isn't being collected.

But it definitely sounds like we're creating the object which will
ultimately call __del__ when it gets finalized.

Have you tried windbg + SOS and done a !DumpHeap to see if the object
could still be alive?  Another thought would be to look at the finalizer
queue and make sure it's not blocked on something (like a transition
into an STA).


  

-Original Message-
From: users-boun...@lists.ironpython.com [mailto:users-
boun...@lists.ironpython.com] On Behalf Of William Reade
Sent: Tuesday, July 21, 2009 9:20 AM
To: Discussion of IronPython
Subject: Re: [IronPython] object lifecycle issues

Dino Viehland wrote:


#2 I'd guess could be either that for some reason we're failing to
detect the presence of __del__ or that when we run the finalizer
  

cleanup


that we believe it's part of cyclic trash and don't think we should
  

cleanup.


Of those I'd think it'd be the cyclic trash detection that'd be most
likely to go wrong.

I'd suggest putting a break point or some logging in
PythonOps.InitializeForFinalization and in WeakRefTracker's finalizer
and see what's happening there.

  

In PythonOps.InitializeForFinalization, I see 2 InstanceFinalizers
created for each object; one after the stub __new__, and one after the
real __new__. Each IF creates a WeakRefTracker as expected; in both
cases, the second WRT (and only the second one) is subsequently
GC.SuppressFinalized in SetFinalizerWorker. However, in the broken
cases, the WRT finalizer just never gets called.

I'll keep hunting, but it seemed worth posting this in case it rang any
bells.

Cheers
william


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] object lifecycle issues

[William Reade]

Dino Viehland wrote:

#2 I'd guess could be either that for some reason we're failing to
detect the presence of __del__ or that when we run the finalizer cleanup
that we believe it's part of cyclic trash and don't think we should cleanup.
Of those I'd think it'd be the cyclic trash detection that'd be most
likely to go wrong.

I'd suggest putting a break point or some logging in
PythonOps.InitializeForFinalization and in WeakRefTracker's finalizer
and see what's happening there.
  


In PythonOps.InitializeForFinalization, I see 2 InstanceFinalizers 
created for each object; one after the stub __new__, and one after the 
real __new__. Each IF creates a WeakRefTracker as expected; in both 
cases, the second WRT (and only the second one) is subsequently 
GC.SuppressFinalized in SetFinalizerWorker. However, in the broken 
cases, the WRT finalizer just never gets called.


I'll keep hunting, but it seemed worth posting this in case it rang any 
bells.


Cheers
william


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] object lifecycle issues

[William Reade]

 }


BTW: As it happens, the stub class is now a sibling of the real class 
rather than a subclass, because it feels cleaner. Regardless, the 
behaviour is identical in each case.


Cheers
william

Dino Viehland wrote:

Could this be issue 2?

class Real(object):
def __new__(cls, *args, **kwargs):
print 'real new'
return object.__new__(Stub)
#def __del__(self): pass # uncomment me and this works as expected

class Stub(Real):
def __del__(self):
    print "I've been finalized"

f = Real(1.0)
del f

import sys
if sys.platform == 'clr':
import clr
from System import GC
for _ in range(4):
GC.Collect()
GC.WaitForPendingFinalizers()


  

-Original Message-
From: users-boun...@lists.ironpython.com [mailto:users-
boun...@lists.ironpython.com] On Behalf Of William Reade
Sent: Monday, July 20, 2009 9:38 AM
To: Discussion of IronPython
Subject: [IronPython] object lifecycle issues

Hi all

I have two problems that are at least somewhat related:

+++ Issue 1 (probably your bug):

---
C:\dev\ironclad - Copy>ipy y.py
real new
stub new
real init
real del

C:\dev\ironclad - Copy>python y.py
real new
stub new
stub init
real del
---

I freely admit that the attached code is pretty weird, but I really do
need to do stuff like this in Ironclad. The difference in behaviour may
or may not be responsible for certain failing numpy/scipy tests -- I'm
not sure how to tell -- but I'd enormously appreciate a fix.

I'd report the issue on Codeplex, but trying to visit the issue tracker
just leaves my browser spinning forever. Speaking of which: is there
any alternative way of reporting bugs that doesn't make me feel as if
I'm spamming the list with out-of-band noise? I'm pretty sure that a
few bugs have just dropped off my stack in the last few months, just
because I got tired of waiting for Codeplex to start working.

+++ Issue 2 (almost certainly my bug):

In a nearly identical* situation -- close enough that I can't say how
it's actually different -- f will never get its __del__ method called
(the object is destroyed -- a WeakReference to it knows it's dead --
but the __del__ call never happens).

For context: I have *very* similar classes, whose instances are
constructed in exactly the weird way demonstrated in the attached file,
and which work fine. The only difference between the cases that work
and the cases that don't is that the broken cases multiply inherit from
ipy types which wrap CLR types (int and float (and maybe str, although
I haven't tested that one)), while the working cases have simple chains
of single inheritance from user-defined types all the way up to object.
However, the attached repro doesn't show my problem, so it's clearly
not
*just* to do with multiply inheriting from CLR types.

Does anyone have any idea what I might be doing wrong? I know this is a
vague request, but I'm running out of ideas, and I'd really appreciate
some input from someone who understands precisely what all those ipy
MetaFoo classes are doing.

Cheers
william


* the attached file started life as an attempt to repro the __del__
issue, and I incidentally noticed the __init__ issue.


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  



import sys
sys.path.append('build')

import ironclad
import numpy
ironclad.gcwait()

dtypes = (numpy.int8, numpy.int32, numpy.float32, numpy.float64)
for d in dtypes:
print
print '='*40
print
print 'start', d
print
f = d(3)
print
print 'constructed; id is', id(f)
print

del f
ironclad.gcwait()

print
print 'finished', d

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] object lifecycle issues

[William Reade]

Clearing cookies worked -- thanks guys.

http://codeplex.codeplex.com/WorkItem/View.aspx?WorkItemId=23563

Jimmy Schementi wrote:

WRT the Codeplex issues, http://codeplex.codeplex.com is the place to post 
issues about codeplex itself: http://codeplex.codeplex.com/WorkItem/List.aspx. 
Let us know when you open a bug so we can reinforce the complaints.

~js

  

-Original Message-
From: users-boun...@lists.ironpython.com [mailto:users-
boun...@lists.ironpython.com] On Behalf Of Dino Viehland
Sent: Monday, July 20, 2009 9:58 AM
To: Discussion of IronPython
Subject: Re: [IronPython] object lifecycle issues

#1's definitely our bug.  We're caching the lookup of __init__ from Real and
invoking that instead of doing the lookup each time.  It should be easy
enough to fix.  I've opened a bug (23555 -
http://ironpython.codeplex.com/WorkItem/View.aspx?WorkItemId=23555)

#2 I'd guess could be either that for some reason we're failing to detect the
presence of __del__ or that when we run the finalizer cleanup that we
believe it's part of cyclic trash and don't think we should cleanup.
Of those I'd think it'd be the cyclic trash detection that'd be most likely to 
go
wrong.

I'd suggest putting a break point or some logging in
PythonOps.InitializeForFinalization and in WeakRefTracker's finalizer and see
what's happening there.

As for CodePlex I guess you can't even open a bug to complain to CodePlex
:(.  If you want to send us your browser/OS info we could open a bug for you
against CodePlex.  You could also try the old heavy hammer of clearing
cookies as maybe CodePlex has built up some bad browser state.  You could
also try using TeamExplorer to open the bugs but I'm not sure if you'll have
permissions there to do that or not.




-Original Message-
From: users-boun...@lists.ironpython.com [mailto:users-
boun...@lists.ironpython.com] On Behalf Of William Reade
Sent: Monday, July 20, 2009 9:38 AM
To: Discussion of IronPython
Subject: [IronPython] object lifecycle issues

Hi all

I have two problems that are at least somewhat related:

+++ Issue 1 (probably your bug):

---
C:\dev\ironclad - Copy>ipy y.py
real new
stub new
real init
real del

C:\dev\ironclad - Copy>python y.py
real new
stub new
stub init
real del
---

I freely admit that the attached code is pretty weird, but I really do
need to do stuff like this in Ironclad. The difference in behaviour
may or may not be responsible for certain failing numpy/scipy tests --
I'm not sure how to tell -- but I'd enormously appreciate a fix.

I'd report the issue on Codeplex, but trying to visit the issue
tracker just leaves my browser spinning forever. Speaking of which: is
there any alternative way of reporting bugs that doesn't make me feel
as if I'm spamming the list with out-of-band noise? I'm pretty sure
that a few bugs have just dropped off my stack in the last few months,
just because I got tired of waiting for Codeplex to start working.

+++ Issue 2 (almost certainly my bug):

In a nearly identical* situation -- close enough that I can't say how
it's actually different -- f will never get its __del__ method called
(the object is destroyed -- a WeakReference to it knows it's dead --
but the __del__ call never happens).

For context: I have *very* similar classes, whose instances are
constructed in exactly the weird way demonstrated in the attached
file, and which work fine. The only difference between the cases that
work and the cases that don't is that the broken cases multiply
inherit from ipy types which wrap CLR types (int and float (and maybe
str, although I haven't tested that one)), while the working cases
have simple chains of single inheritance from user-defined types all the
  

way up to object.


However, the attached repro doesn't show my problem, so it's clearly
not
*just* to do with multiply inheriting from CLR types.

Does anyone have any idea what I might be doing wrong? I know this is
a vague request, but I'm running out of ideas, and I'd really
appreciate some input from someone who understands precisely what all
those ipy MetaFoo classes are doing.

Cheers
william


* the attached file started life as an attempt to repro the __del__
issue, and I incidentally noticed the __init__ issue.
  

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] object lifecycle issues

[William Reade]

Hi all

I have two problems that are at least somewhat related:

+++ Issue 1 (probably your bug):

---
C:\dev\ironclad - Copy>ipy y.py
real new
stub new
real init
real del

C:\dev\ironclad - Copy>python y.py
real new
stub new
stub init
real del
---

I freely admit that the attached code is pretty weird, but I really do 
need to do stuff like this in Ironclad. The difference in behaviour may 
or may not be responsible for certain failing numpy/scipy tests -- I'm 
not sure how to tell -- but I'd enormously appreciate a fix.


I'd report the issue on Codeplex, but trying to visit the issue tracker 
just leaves my browser spinning forever. Speaking of which: is there any 
alternative way of reporting bugs that doesn't make me feel as if I'm 
spamming the list with out-of-band noise? I'm pretty sure that a few 
bugs have just dropped off my stack in the last few months, just because 
I got tired of waiting for Codeplex to start working.


+++ Issue 2 (almost certainly my bug):

In a nearly identical* situation -- close enough that I can't say how 
it's actually different -- f will never get its __del__ method called 
(the object is destroyed -- a WeakReference to it knows it's dead -- but 
the __del__ call never happens).


For context: I have *very* similar classes, whose instances are 
constructed in exactly the weird way demonstrated in the attached file, 
and which work fine. The only difference between the cases that work and 
the cases that don't is that the broken cases multiply inherit from ipy 
types which wrap CLR types (int and float (and maybe str, although I 
haven't tested that one)), while the working cases have simple chains of 
single inheritance from user-defined types all the way up to object. 
However, the attached repro doesn't show my problem, so it's clearly not 
*just* to do with multiply inheriting from CLR types.


Does anyone have any idea what I might be doing wrong? I know this is a 
vague request, but I'm running out of ideas, and I'd really appreciate 
some input from someone who understands precisely what all those ipy 
MetaFoo classes are doing.


Cheers
william


* the attached file started life as an attempt to repro the __del__ 
issue, and I incidentally noticed the __init__ issue.

class Base(object):
pass

class Real(Base, float):
def __new__(cls, *args, **kwargs):
print 'real new'
result = Stub.__new__(cls, *args, **kwargs)
return result
def __init__(self, *args, **kwargs):
print 'real init'
def __del__(self):
print 'real del'

class Stub(Real):
def __new__(cls, *args, **kwargs):
print 'stub new'
return float.__new__(Stub, args[0])
def __init__(self, *args, **kwargs):
print 'stub init'
def __del__(self):
print "this should never happen; it's just here to ensure I get 
registered for GC"

def ConstructReal(x):
f = Real(x)
f.__class__ = Real
return f

f = ConstructReal(1.0)
del f

import sys
if sys.platform == 'clr':
import clr
from System import GC
for _ in range(4):
GC.Collect()
GC.WaitForPendingFinalizers()

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] mmap, csv, others

[William Reade]

Thanks Dino.

Incidentally, for some reason, mmap objects are constructed with filenos 
rather than files, which may complicate matters for you... in Ironclad 
I'm planning to get around this by forcing users to use real 
PyFileObjects if they want to mmap them. In the general case it'll be an 
annoying departure from the intended transparency, but at least I can 
patch known-mmaping modules like numpy.core.memmap so that they, at 
least, silently use the correct file type.


(Yes, it's evil, but it seems to be the least evil option; if anyone has 
a better suggestion (or is writing code which assumes uniqueness of ipy 
filenos), please let me know.)


Dino Viehland wrote:

Harry had started implementing _csv but I think he's stalled out on
playing w/ __clrtype__.  No one here has touched mmap - I believe
.NET 4.0 will have a managed wrapper around memory mapped files
which would give us a chance to do it in the future but until then
we wouldn't even think about it.

-Original Message-
From: users-boun...@lists.ironpython.com 
[mailto:users-boun...@lists.ironpython.com] On Behalf Of William Reade
Sent: Tuesday, May 19, 2009 2:37 AM
To: Discussion of IronPython
Subject: [IronPython] mmap, csv, others

Hi all

Life permitting, the next version of Ironclad should have a few of the
built-in C extensions included in it. I have vague plans that the first
two will be mmap and _csv (because I've had to fake both of those out
for numpy); however, I just wanted to check whether anybody else was
doing the same thing.

Cheers
william
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] mmap, csv, others

[William Reade]

Hi all

Life permitting, the next version of Ironclad should have a few of the 
built-in C extensions included in it. I have vague plans that the first 
two will be mmap and _csv (because I've had to fake both of those out 
for numpy); however, I just wanted to check whether anybody else was 
doing the same thing.


Cheers
william
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Access to current Python engine in C# (Silverlight)

[William Reade]

I had a similar problem in Ironclad -- needing to get the engine which 
called a C# method. Try adding a CodeContext parameter to GetModule 
(which gets automagically inserted; no need to change the call from 
Python), and then using PythonContext.GetContext to get a PythonContext 
from the CodeContext. See the constructors in 
http://code.google.com/p/ironclad/source/browse/trunk/src/Python25Mapper.cs


...if your goggles can stand it.

Michael Foord wrote:

Hello guys,

I have a second use case for embedding IronPython in Silverlight. This 
is actually a dynamic application with a C# component that needs to 
programattically build a Python module.


Again I have the same problem - imports in Python code fail. I would 
have expected that accessing the current runtime and fetching a Python 
engine would fetch the current Python engine, with the browser host 
correctly setup. Unfortunately that seems not to be the case. Can 
anyone spot problems with the following code:



using Microsoft.Scripting.Silverlight;
using IronPython;
using IronPython.Hosting;
using Microsoft.Scripting;
using Microsoft.Scripting.Hosting;

namespace EmbeddedSLModule
{
   public class EmbeddedSLModule
   {
   private static string source = @"
import something
";
   public static ScriptScope GetModule(){
   ScriptRuntime runtime = DynamicApplication.Current.Runtime;
   ScriptEngine engine = runtime.GetEngine("Python");
   ScriptScope scope = engine.CreateScope();
   ScriptSource script = 
engine.CreateScriptSourceFromString(source, SourceCodeKind.Statements);

   script.Execute(scope);

   return scope;

   }
   }
}


It works fine for code that doesn't import anything - but imports from 
within the xap file fail.


Thanks

Michael Foord



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] .exe is not pre-compiled

[William Reade]

Generally, CPython modules have the .pyd extension -- and, in fact, 
Ironclad will only detect and handle attempts to import .pyd files. So, 
sadly, it won't help you here. It wouldn't be hard to make Ironclad 
handle .exe files, but the likelihood of it doing anything useful with 
them is minimal.


Is it the case that you've somehow compiled an IronPython project to an 
.exe, when you really wanted to use Pyc (included with ipy2, I think) to 
turn it into a .dll that you can import from? If you search these lists, 
you should find more help on that topic.


Curt Hagenlocher wrote:

On Fri, May 1, 2009 at 9:12 PM, Vadim Khaskel  wrote:
  

I'm trying to import C-python module and getting the following ecxeption:

C:\Documents and Settings\vkhask\My Documents\IronPython
Studio\pygnu\pygnu\bin\Debug\pygnu.exe is not pre-compiled module; try again
after deleting it.



If this is a Python extension written in C, then you might be able to
use it with the Ironclad project ( http://code.google.com/p/ironclad/
).  IronPython itself doesn't support loading C extensions directly.

--
Curt Hagenlocher
c...@hagenlocher.org
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Control event ids

[William Reade]

Curt Hagenlocher wrote:

iour makes it extremely cumbersome to detach from events
(some events, anyway; I'm pretty certain that I've seen events which
don't exhibit this behaviour, but I can't remember what they are).



An example of this would be useful.
  

Sorry for the noise -- it seems I was hallucinating.

--
Curt Hagenlocher
c...@hagenlocher.org
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Control event ids

[William Reade]

Hi List


>>> import clr
>>> clr.AddReference('System.Windows.Forms')
>>> from System.Windows.Forms import TabControl
>>> t = TabControl()
>>> t.MouseDown == t.MouseDown
False
>>> t.MouseDown is t.MouseDown
False
>>>


This behaviour makes it extremely cumbersome to detach from events (some 
events, anyway; I'm pretty certain that I've seen events which don't 
exhibit this behaviour, but I can't remember what they are).


Is this an IronPython bug, or have I got a broken mental model of what 
should be happening?



Cheers
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] pywin32 on Iron Python?

[William Reade]

Dave Fugate wrote:
That said, there is something extremely useful the community can do for IronPython that our team simply cannot:  get 3rd party Python applications such as Django, pywin32, NumPy, etc running under IronPython.  This could mean adapting something like adodbapi.py to utilize IronPython APIs similar to what Vernon Cole did, or re-implementing NumPy's C-based modules in C#.  
NumPy works pretty well already (under ironclad*). I suspect that in 
circumstances where managedness is so important that the CPython numpy 
can't be used, one would be better off using a preexisting .NET library 
than trying to rewrite parts of NumPy.


Also, I just performed a quick test on pywin32 under ironclad and, once 
I added the 'pythonwin', 'win32', and 'win32\lib' folders from 
site-packages to sys.path, was able to call win32api.GetComputerName 
without problems. Not all of it will work, but see the numpy-related 
footnote below.


Cheers
william

* and if it doesn't do what you need, or if what you need is too slow, 
please let me know and I'll do my best to fix it :-)



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Please, help to get started with .Net and IronPython.

[William Reade]

I think he's rewriting a C++ component in IronPython, and wants to know 
how he can minimise the impact on the C# application which hosts it.


Curt Hagenlocher wrote:
It's great that you want to improve your skills.  But I feel obliged 
to point out that IronPython -- and CPython, for that matter -- are 
about equally susceptible to memory leaks as C#.


On Thu, Apr 23, 2009 at 2:00 AM, Vladimir Eremeev > wrote:




Dody Gunawinata wrote:
>
> Can I ask the reason why you want to rewrite portion of this
application
> in
> IronPython?
>

I am tired fighting with memory leaks, data type conversions, etc.

I don't like the C# language, its subjective. Just don't like.
However, I
have a basic understanding of it.
And I want to improve my programming skills.
--
View this message in context:

http://www.nabble.com/Please%2C-help-to-get-started-with-.Net-and-IronPython.-tp23192785p23193038.html
Sent from the IronPython mailing list archive at Nabble.com.

___
Users mailing list
Users@lists.ironpython.com 
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com




___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Ironclad v0.8.1 released

[William Reade]

Hi all

I'm fairly pleased to announce the release of Ironclad v0.8.1; it's not 
an enormous technical leap above v0.8, but it does now enable you to 
import and use SciPy and Matplotlib with IronPython on Win32 (with some 
restrictions; see project page) . Downloads, and more details, are 
available at http://code.google.com/p/ironclad -- please do play with 
it, and let me know if you have any problems.


Cheers
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Resolver One 1.4 beta - with IronPython 2.0

[William Reade]

I should point out ahead of time that there's no mmap module in 
IronPython at the moment, and so memory-mapped ndarrays don't work yet 
-- although most of the usual numpy save/load bits do work, so you'll 
probably be fine (unless they're too big to fit in memory).


Dan Shechter wrote:

Here!

I can't wait to use it with NumPy.

I actually have a .NET C# assembly that does some really nasty 
low-level stuff like loading data from memory mapped

files and wrapping them with IList :)

I can't wait to make Ironclad/Numpy blow up :)



On Tue, Feb 3, 2009 at 8:50 PM, Giles Thomas 
> wrote:


Hi all,

Version 1.4 of Resolver One, our Pythonic spreadsheet, is based on
IronPython 2.0, and includes (alpha-level) support for numpy using
our Ironclad project.
We're releasing the beta tomorrow: this has a few performance
problems (which are being addressed - many thanks to Dino Viehland
for helping with this!) but is otherwise functionally complete.
 If you're interested in trying it out, drop me a line!


Best regards,

Giles
-- 


Giles Thomas
MD & CTO, Resolver Systems Ltd.
giles.tho...@resolversystems.com

+44 (0) 20 7253 6372

Win up to $17,000 for a spreadsheet:


17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number 5467329.
Registered address: 843 Finchley Road, London NW11 8NA, UK


___
Users mailing list
Users@lists.ironpython.com 
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com




--

   Shechter.


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Ironclad 0.8 released

[William Reade]

Still no x64, although the necessary supporting changes are gradually 
creeping in. I can't give you a timescale, I'm afraid, but every person 
who asks me about it pushes it up my priority list :-).


Dan Shechter wrote:

Congratulations...

Is x64 fully supported?

  Shechter.

On 29/01/2009, at 13:43, William Reade  
wrote:



Hi all

I'm delighted to announce the release of Ironclad v0.8 -- the 
all-singing, all-dancing CPython API compatibility layer for 
IronPython -- available now from http://code.google.com/p/ironclad/ . 
Notable improvements over the last release include:


* Ironclad is now a neatly self-contained package -- just copy to 
your site-packages and 'import ironclad'.
* No more silly requirement to call ironclad.shutdown() when you're 
finished.

* A few performance improvements.
* Over 900 NumPy tests now pass: in fact, almost all the tests from 
the core, fft, lib, linalg, ma, oldnumeric and random subpackages.
* Over half the .pyds distributed with CPython 2.5 now import 
cleanly; some of them appear to actually work, including _hashlib and 
_elementtree.


Ironclad grows more stable and mature with every release, and I urge 
IronPython users to try it out and share their impressions: feedback, 
whether positive or negative, is always welcomed.


Cheers
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Ironclad 0.8 released

[William Reade]

Hi all

I'm delighted to announce the release of Ironclad v0.8 -- the 
all-singing, all-dancing CPython API compatibility layer for IronPython 
-- available now from http://code.google.com/p/ironclad/ . Notable 
improvements over the last release include:


* Ironclad is now a neatly self-contained package -- just copy to your 
site-packages and 'import ironclad'.
* No more silly requirement to call ironclad.shutdown() when you're 
finished.

* A few performance improvements.
* Over 900 NumPy tests now pass: in fact, almost all the tests from the 
core, fft, lib, linalg, ma, oldnumeric and random subpackages.
* Over half the .pyds distributed with CPython 2.5 now import cleanly; 
some of them appear to actually work, including _hashlib and _elementtree.


Ironclad grows more stable and mature with every release, and I urge 
IronPython users to try it out and share their impressions: feedback, 
whether positive or negative, is always welcomed.


Cheers
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] IronPython and file descriptors

[William Reade]

Just a quick note: you can't necessarily just P/invoke out to msvcrt, 
because CPython was built with msvcr71. Sorry :(.


Tom Wright wrote:
I'm not sure about sockets. The only example I have found in the wild 
so far is with files in PIL.


The python implementation seems to have socket.fileno() returning a 
file descriptor which comes from 's socketpair function 
in the standard C library (see socketmodule.c if you are interested)


So, again, it would be possible for people to pass this identifier as 
an integer directly to C. Whether people do this in practice is a 
different question...


Tom



Michael Foord wrote:

Dino Viehland wrote:


(now I’ve replied to the correct thread…)

Can you open a feature request on CodePlex? It's certainly an 
interesting idea to ponder and I'm leaning towards it but there's 
lots of details to be gotten right.


Do you know if this needs to work w/ sockets as well? (There's also 
the question of can we make it work with sockets? :))


There'll be a bunch of places we need to update (nt, socket, file, 
select, etc...) so I think it'll have to wait until 2.1 instead of 
coming in a minor update like 2.0.1.




Tom will have to comment on whether it is needed for sockets as well. 
Not yet I guess. :-)


Tom created an issue on Codeplex - I've added a comment to that:

http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=20242

Michael


*From:* users-boun...@lists.ironpython.com 
[mailto:users-boun...@lists.ironpython.com] *On Behalf Of *Michael 
Foord

*Sent:* Monday, December 15, 2008 12:53 PM
*To:* Discussion of IronPython
*Subject:* Re: [IronPython] IronPython and file descriptors

2008/12/15 Dino Viehland >


Presumably CPython is linking against msvcrt and is using the C 
abstraction rather than calling the Win32 APIs which return handles 
directly.


As Curt said the biggest problem for us is that .NET does not expose 
C file descriptors. Therefore we could fix this by P/Invoking out to 
msvcrt. For users on other platforms we'd need to either add support 
for their platform or hope that their handles == C runtime file 
descriptors. For something like fileno maybe this is ok because it's 
an interop point only - or are there any non-interop uses of fileno 
in the world?


What do people think of this? This would be the 1st place where we 
would add a P/Invoke so I'd want to tread lightly and make sure this 
is really the right thing to do.




Well, some way of doing this is needed if certain Python C 
extensions are to work with Ironclad. Perhaps a global or per engine 
setting that allows file handles to be associated with a C 
descriptor. Users of Ironclad could switch it on if they wished and 
take the consequences.


Michael Foord


-Original Message-
From: users-boun...@lists.ironpython.com

[mailto:users-boun...@lists.ironpython.com
] On Behalf Of Slide
Sent: Monday, December 15, 2008 10:21 AM
To: Discussion of IronPython
Subject: Re: [IronPython] IronPython and file descriptors

On Mon, Dec 15, 2008 at 11:18 AM, Tom Wright
mailto:tom.wri...@resolversystems.com>> wrote:
> Agreed. It is certainly possible with some work to get a file
descriptor and
> pass it to C code.
>
> This is not the problem, however. Ironclad's aim (eventually) is
to allow
> *arbitrary* C extensions to work with Ironpython without
changing the
> extensions. Ctypes aim is to allow arbitrary C code to be run by
python
> (possibly in other python libraries which you really don't want
to modify).
>
> In CPython .fileno() is a file descriptor and some modules use
this fact
> (specifically PIL) by passing file descriptors as integers into
C code. I do
> not know how one can make this code work in IronPython unless
.fileno()
> returns a file descriptor.
>
> The game here is not making a particular C library work with
IronPython by
> modifying this library but making as many libraries as possible
work without
> modification. Of course whether this is worthwhile depends on
how many
> libraries rely on this fact.
>
> Sorry - I hope this is a little clearer.
>
> Tom


How does CPython get the file descriptor for fileno()? Does it just
return the HANDLE on Windows?

slide
___
Users mailing list
Users@lists.ironpython.com 
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com 
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com




--
http://www.ironpythoninaction.com/

--

[IronPython] Ironclad v0.7 released

[William Reade]

Hi all

I'm delighted to announce the release of Ironclad v0.7, which is now 
available from http://code.google.com/p/ironclad/downloads/list . This 
release is a major step forward:


* Runs transparently on vanilla IronPython 2.0RC2, without creating 
extra PythonEngines or breaking .NET namespace imports
* Many numpy 1.2 tests (from the core, fft, lib, linalg and random 
subpackages) now reliably pass (run "ipy numpytests.py" from the build 
directory)
* Significant performance improvements (by several orders of magnitude 
in some places :D)


So... if you want to use numpy (or other C extension modules) with 
IronPython on Win32, please download it and try it out; I'm very keen to 
hear your experiences, and to know which neglected features will be most 
useful to you.


Cheers
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] (ironclad) is it possible to get the current ScriptEngine?

[William Reade]

Thanks again Dino: it's all working and checked in, and it was all much 
easier than I expected. No problems with SourceUnits, but I don't try to 
do anything remotely complex with them, so others' mileage may vary.


Dino Viehland wrote:

Oh, and instead of ScriptSource you can use SourceUnit.  You might find there's 
too much stuff marked internal w/ SourceUnit so any issues you run into here 
would be interesting to hear.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 20, 2008 9:50 AM
To: Discussion of IronPython
Subject: Re: [IronPython] (ironclad) is it possible to get the current 
ScriptEngine?

It seems I don't actually use the engine for very much, so this sounds
pretty plausible -- context.SystemState can replace Python.GetSysModule,
and HostingHelpers.GetLanguageContext goes away entirely :).

However, I can't see any obvious way to create a ScriptScope
(engine.CreateScope) or a ScriptSource
(engine.CreateScriptSourceFromString) -- even if I just dupe IronPython
code, it seems I'll still need an actual PythonEngine to construct them.
Am I missing something obvious?

William Reade wrote:
  

Thanks Dino -- I'll see what I can do with that :)

Dino Viehland wrote:


It's not really possible to get back to the ScriptEngine - but you
also probably don't need to.  You probably want to get back to the
LanguageContext(PythonContext)/ScriptDomainManager and work with
those directly.  ScriptEngine/ScriptRuntime are wrappers around those
which expose the friendly API and provide the easy remoting features.

You can get to the LanguageContext via a CodeContext.  You can get a
CodeContext by just defining it as the 1st parameter on a .NET method
which we'll be calling at some point.

For example our ModuleLoader class for pre-compiled code has a
load_module method which receives a CodeContext.  It then uses it to
call back into the current PythonContext and create a new module.

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 20, 2008 5:36 AM
To: Discussion of IronPython
Subject: [IronPython] (ironclad) is it possible to get the current
ScriptEngine?

Hi all

At the moment, when a user types 'import ironclad', I create a new
ScriptEngine and set that up to allow .pyd imports; I then abuse
path_hooks to use the new engine to do the imports, and copy them into
the original engine's sys.modules. Clearly, this is evil and wrong on
any number of levels, but so far it's been (just barely) good enough.

However, if I can find out which ScriptEngine is executing the code (so
I can pass that into the Python25Mapper), I can greatly simplify
ironclad.py and, hopefully, solve another user's problem (which, I
believe, is related to sys.path not being shared between the two engines
(quite rightly so, ofc ;))).

So: is there any way I can get a reference to the executing engine from
within IronPython code? It feels as if it should be possible, but
whenever I look into it I end up chasing my tail...

William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


  

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com




___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] (ironclad) is it possible to get the current ScriptEngine?

[William Reade]

It seems I don't actually use the engine for very much, so this sounds 
pretty plausible -- context.SystemState can replace Python.GetSysModule, 
and HostingHelpers.GetLanguageContext goes away entirely :).


However, I can't see any obvious way to create a ScriptScope 
(engine.CreateScope) or a ScriptSource 
(engine.CreateScriptSourceFromString) -- even if I just dupe IronPython 
code, it seems I'll still need an actual PythonEngine to construct them. 
Am I missing something obvious?


William Reade wrote:

Thanks Dino -- I'll see what I can do with that :)

Dino Viehland wrote:
It's not really possible to get back to the ScriptEngine - but you 
also probably don't need to.  You probably want to get back to the 
LanguageContext(PythonContext)/ScriptDomainManager and work with 
those directly.  ScriptEngine/ScriptRuntime are wrappers around those 
which expose the friendly API and provide the easy remoting features.


You can get to the LanguageContext via a CodeContext.  You can get a 
CodeContext by just defining it as the 1st parameter on a .NET method 
which we'll be calling at some point.


For example our ModuleLoader class for pre-compiled code has a 
load_module method which receives a CodeContext.  It then uses it to 
call back into the current PythonContext and create a new module.


-Original Message-
From: [EMAIL PROTECTED] 
[mailto:[EMAIL PROTECTED] On Behalf Of William Reade

Sent: Thursday, November 20, 2008 5:36 AM
To: Discussion of IronPython
Subject: [IronPython] (ironclad) is it possible to get the current 
ScriptEngine?


Hi all

At the moment, when a user types 'import ironclad', I create a new
ScriptEngine and set that up to allow .pyd imports; I then abuse
path_hooks to use the new engine to do the imports, and copy them into
the original engine's sys.modules. Clearly, this is evil and wrong on
any number of levels, but so far it's been (just barely) good enough.

However, if I can find out which ScriptEngine is executing the code (so
I can pass that into the Python25Mapper), I can greatly simplify
ironclad.py and, hopefully, solve another user's problem (which, I
believe, is related to sys.path not being shared between the two engines
(quite rightly so, ofc ;))).

So: is there any way I can get a reference to the executing engine from
within IronPython code? It feels as if it should be possible, but
whenever I look into it I end up chasing my tail...

William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] (ironclad) is it possible to get the current ScriptEngine?

[William Reade]

Thanks Dino -- I'll see what I can do with that :)

Dino Viehland wrote:

It's not really possible to get back to the ScriptEngine - but you also 
probably don't need to.  You probably want to get back to the 
LanguageContext(PythonContext)/ScriptDomainManager and work with those 
directly.  ScriptEngine/ScriptRuntime are wrappers around those which expose 
the friendly API and provide the easy remoting features.

You can get to the LanguageContext via a CodeContext.  You can get a 
CodeContext by just defining it as the 1st parameter on a .NET method which 
we'll be calling at some point.

For example our ModuleLoader class for pre-compiled code has a load_module 
method which receives a CodeContext.  It then uses it to call back into the 
current PythonContext and create a new module.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 20, 2008 5:36 AM
To: Discussion of IronPython
Subject: [IronPython] (ironclad) is it possible to get the current ScriptEngine?

Hi all

At the moment, when a user types 'import ironclad', I create a new
ScriptEngine and set that up to allow .pyd imports; I then abuse
path_hooks to use the new engine to do the imports, and copy them into
the original engine's sys.modules. Clearly, this is evil and wrong on
any number of levels, but so far it's been (just barely) good enough.

However, if I can find out which ScriptEngine is executing the code (so
I can pass that into the Python25Mapper), I can greatly simplify
ironclad.py and, hopefully, solve another user's problem (which, I
believe, is related to sys.path not being shared between the two engines
(quite rightly so, ofc ;))).

So: is there any way I can get a reference to the executing engine from
within IronPython code? It feels as if it should be possible, but
whenever I look into it I end up chasing my tail...

William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] (ironclad) is it possible to get the current ScriptEngine?

[William Reade]

Hi all

At the moment, when a user types 'import ironclad', I create a new 
ScriptEngine and set that up to allow .pyd imports; I then abuse 
path_hooks to use the new engine to do the imports, and copy them into 
the original engine's sys.modules. Clearly, this is evil and wrong on 
any number of levels, but so far it's been (just barely) good enough.


However, if I can find out which ScriptEngine is executing the code (so 
I can pass that into the Python25Mapper), I can greatly simplify 
ironclad.py and, hopefully, solve another user's problem (which, I 
believe, is related to sys.path not being shared between the two engines 
(quite rightly so, ofc ;))).


So: is there any way I can get a reference to the executing engine from 
within IronPython code? It feels as if it should be possible, but 
whenever I look into it I end up chasing my tail...


William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

So far as I can tell, CPython ctypes and installers (neither of which 
are currently relevant) use COM, and numpy doesn't use it at all. 
However, my search wasn't very sophisticated -- I just looked for 
CoInitialize, CoCreateInstance, and lpVtbl, which is all the COM I can 
remember*. Was that actually a sensible way of checking?


Ironclad doesn't use any of the COM-related methods on Marshal, and only 
uses a handful of unmanaged functions (from kernel32 and msvcr71), so if 
it does use COM it's doing so secretly and without my knowledge :).


Running ipy on its own does appear to load ole32, but I haven't worked 
out what, if anything, it's doing with it.


* I had to work with faked-up COM interfaces while working on mac game 
ports a few years ago  -- that's the full extent of my experience.


William Reade wrote:

I'll look into that along with everything else :).

Dino Viehland wrote:
Do you know if numpy is using COM anywhere?  Or does Ironclad use COM 
for any of its interop?


-Original Message-
From: [EMAIL PROTECTED] 
[mailto:[EMAIL PROTECTED] On Behalf Of William Reade

Sent: Thursday, November 06, 2008 6:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here 
may be able to help


Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer
reproduce the problem. However, this bothers me a bit: I'm not competent
to follow all the consequences here, but this situation seems to imply
that Ironclad won't be usable safely from any STA thread. Is this an
intended restriction or a bug?

Incidentally, tracking down the call stacks proved to be hard work: the
timing changed enough that I only got a single failure over dozens of
runs, and it turned out I'd got the logging wrong, so I didn't find out
anything useful :(.

Dino Viehland wrote:
 
I would suggest getting a snap shot of the call stacks when this is 
happening if that's possible.  I can't pin anything down but I 
wonder if you could have an STA object or something that otherwise 
requires message pumping.  That message pumping could happen while 
you're doing a Monitor.Enter call.  If that was happening maybe 
there is some subtle CLR bug or a surprise feature?  It is 
surprising that Monitor.Enter can go re-entrant but it can...


So it'd be interesting to get thread snapshots and see if


   

   EnsureGIL (443) 2
   EnsureGIL (443) 1  <- omg, wtf, bbq, etc.

  
Could be happening because Thread 1 experiences contention, blocks 
and pumps messages, causing the finalizer thread (2) to run, the 
lock is acquired and ... ?


Only other thing I could think of is does it repro on other 
machines?  Maybe it's a hardware bug as unlikely as that seems?


-----Original Message-
From: [EMAIL PROTECTED] 
[mailto:[EMAIL PROTECTED] On Behalf Of William Reade

Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here 
may be able to help


The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.

I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on 
the

main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come 
back as

soon as anyone tries any serious multithreading :).

Curt Hagenlocher wrote:

   

Locking during finalization is often considered to be a bad idea.  In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized.  But clearly, that's not what's happening here.

Other questions that probably don't matter but might be interesting to
know:

Can we assume that the finalization thread isn't the first place where
this lock is required?  That your log starts somewhere in the middle?

Is this under x86 or x64 or both?

Are you creating any additional AppDomains in the process?


On Wed, Nov 5, 2008 at 10:15 AM, William Reade
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> 
wrote:


Hi Curt

I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?

William

Curt Hagenlocher wrote:

...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.

On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
    wrote:

   So, the obvious question for

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

I'll look into that along with everything else :).

Dino Viehland wrote:

Do you know if numpy is using COM anywhere?  Or does Ironclad use COM for any 
of its interop?

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 06, 2008 6:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able 
to help

Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer
reproduce the problem. However, this bothers me a bit: I'm not competent
to follow all the consequences here, but this situation seems to imply
that Ironclad won't be usable safely from any STA thread. Is this an
intended restriction or a bug?

Incidentally, tracking down the call stacks proved to be hard work: the
timing changed enough that I only got a single failure over dozens of
runs, and it turned out I'd got the logging wrong, so I didn't find out
anything useful :(.

Dino Viehland wrote:
  

I would suggest getting a snap shot of the call stacks when this is happening 
if that's possible.  I can't pin anything down but I wonder if you could have 
an STA object or something that otherwise requires message pumping.  That 
message pumping could happen while you're doing a Monitor.Enter call.  If that 
was happening maybe there is some subtle CLR bug or a surprise feature?  It is 
surprising that Monitor.Enter can go re-entrant but it can...

So it'd be interesting to get thread snapshots and see if




   EnsureGIL (443) 2
   EnsureGIL (443) 1  <- omg, wtf, bbq, etc.

  

Could be happening because Thread 1 experiences contention, blocks and pumps 
messages, causing the finalizer thread (2) to run, the lock is acquired and ... 
?

Only other thing I could think of is does it repro on other machines?  Maybe 
it's a hardware bug as unlikely as that seems?

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able 
to help

The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.

I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on the
main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come back as
soon as anyone tries any serious multithreading :).

Curt Hagenlocher wrote:



Locking during finalization is often considered to be a bad idea.  In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized.  But clearly, that's not what's happening here.

Other questions that probably don't matter but might be interesting to
know:

Can we assume that the finalization thread isn't the first place where
this lock is required?  That your log starts somewhere in the middle?

Is this under x86 or x64 or both?

Are you creating any additional AppDomains in the process?


On Wed, Nov 5, 2008 at 10:15 AM, William Reade
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:

Hi Curt

I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?

William

Curt Hagenlocher wrote:

...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.

On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
wrote:

   So, the obvious question for me is whether or not you're
using any
   finalizers.


   On Wed, Nov 5, 2008 at 5:57 AM, William Reade
   <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>>

   wrote:

   Hi all

   While running the numpy tests, I've come across a situation
   which, to the best of my knowledge, is simply
impossible. I'm
   hoping that one of the local .NET gurus will be able to
tell
   me what I'm missing, or point me somewhere I can get
more insight.

   The 4 methods involved are as follows:
   ---
 public int GetThreadId()
 {
 return T

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Thanks Dino -- you've given me plenty to be going on with until you hear 
back from the CLR dev.


Dino Viehland wrote:

In MTA mode we'll create a new thread which is thread 3 - thread 1 will just be 
hanging around waiting for it to exit.  Unfortunately w/o a separate EXE we 
can't avoid the separate thread.

I've pinged the CLR dev who owns interop these days to see if he's ever seen 
anything like this.  Unfortunately he's OOF until the 10th.  My best suggestion 
for the time being is to avoid using Monitor for this lock so that you don't 
pump messages.  That's probably going to mean writing your own lock and then 
P/Invoking to WaitForSingleObject when you experience contention and need to 
block the thread.  Using an AutoResetEvent or ManualResetEvent directly won't 
work because we'll pump messages there too!

I would also suggest looking at what CPython does here.  Dumpbin /imports 
python.exe shows no imports from ole32 so I'm guessing they don't do anything 
to spin up COM - which makes sense.  I would therefore assume when acquiring 
the GIL CPython will never pump messages and therefore programs don't expect 
possible re-entrancy - it'd probably be a good idea to mimic that behavior :).  
Unfortunately there are other places where we use Monitor in IronPython - 
list.append comes to mind.  It'll be interesting to see if that turns out to be 
a problem eventually as well.

Anyway, I'll let you know when I hear back.


-Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 06, 2008 7:29 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able 
to help

Finalization still happens on thread 2, and I see no evidence of thread
1 any more; my main thread now appears to have id 3.

Curt Hagenlocher wrote:
  

When you run with -X:MTA, is the object now being finalized on a
thread other than thread #2?

On Thu, Nov 6, 2008 at 7:01 AM, William Reade
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:

Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no
longer reproduce the problem. However, this bothers me a bit: I'm
not competent to follow all the consequences here, but this
situation seems to imply that Ironclad won't be usable safely from
any STA thread. Is this an intended restriction or a bug?

Incidentally, tracking down the call stacks proved to be hard
work: the timing changed enough that I only got a single failure
over dozens of runs, and it turned out I'd got the logging wrong,
so I didn't find out anything useful :(.


Dino Viehland wrote:

I would suggest getting a snap shot of the call stacks when
this is happening if that's possible.  I can't pin anything
down but I wonder if you could have an STA object or something
that otherwise requires message pumping.  That message pumping
could happen while you're doing a Monitor.Enter call.  If that
was happening maybe there is some subtle CLR bug or a surprise
feature?  It is surprising that Monitor.Enter can go
re-entrant but it can...

So it'd be interesting to get thread snapshots and see if



  EnsureGIL (443) 2
  EnsureGIL (443) 1  <- omg, wtf, bbq, etc.



Could be happening because Thread 1 experiences contention,
blocks and pumps messages, causing the finalizer thread (2) to
run, the lock is acquired and ... ?

Only other thing I could think of is does it repro on other
machines?  Maybe it's a hardware bug as unlikely as that seems?

-Original Message-
From: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
[mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>] On Behalf Of
William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone
here may be able to help

The log starts in the middle (after many lock/unlocks, some
from each
thread); I'm running on x86; and I have no additional AppDomains.

I don't think it would be safe for me to entirely avoid
locking during
finalization, but I could probably cut it down to a quick
lock, on a
separate object, to enqueue pointers for cleanup and
deallocation on the
main thread. However, I'm reluctant to do that until I'm sure
that the
problem is specifically related to GC, otherwise it'll just
come back as
soon as anyone tries any serious multithreading :).

Curt Hagenlocher wrote:


  

[IronPython] trivial long() bug

[William Reade]

CPy:
>>> long('')
Traceback (most recent call last):
 File "", line 1, in 
ValueError: invalid literal for long() with base 10: ''

IPy:
>>> long('')
0L
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Finalization still happens on thread 2, and I see no evidence of thread 
1 any more; my main thread now appears to have id 3.


Curt Hagenlocher wrote:
When you run with -X:MTA, is the object now being finalized on a 
thread other than thread #2?


On Thu, Nov 6, 2008 at 7:01 AM, William Reade 
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:


Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no
longer reproduce the problem. However, this bothers me a bit: I'm
not competent to follow all the consequences here, but this
situation seems to imply that Ironclad won't be usable safely from
any STA thread. Is this an intended restriction or a bug?

Incidentally, tracking down the call stacks proved to be hard
work: the timing changed enough that I only got a single failure
over dozens of runs, and it turned out I'd got the logging wrong,
so I didn't find out anything useful :(.


Dino Viehland wrote:

I would suggest getting a snap shot of the call stacks when
this is happening if that's possible.  I can't pin anything
down but I wonder if you could have an STA object or something
that otherwise requires message pumping.  That message pumping
could happen while you're doing a Monitor.Enter call.  If that
was happening maybe there is some subtle CLR bug or a surprise
feature?  It is surprising that Monitor.Enter can go
re-entrant but it can...

So it'd be interesting to get thread snapshots and see if

 


  EnsureGIL (443) 2
  EnsureGIL (443) 1  <- omg, wtf, bbq, etc.
   



Could be happening because Thread 1 experiences contention,
blocks and pumps messages, causing the finalizer thread (2) to
run, the lock is acquired and ... ?

Only other thing I could think of is does it repro on other
machines?  Maybe it's a hardware bug as unlikely as that seems?

-Original Message-
From: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
[mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>] On Behalf Of
William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone
here may be able to help

The log starts in the middle (after many lock/unlocks, some
from each
thread); I'm running on x86; and I have no additional AppDomains.

I don't think it would be safe for me to entirely avoid
locking during
finalization, but I could probably cut it down to a quick
lock, on a
separate object, to enqueue pointers for cleanup and
deallocation on the
main thread. However, I'm reluctant to do that until I'm sure
that the
problem is specifically related to GC, otherwise it'll just
come back as
soon as anyone tries any serious multithreading :).

Curt Hagenlocher wrote:
 


Locking during finalization is often considered to be a
bad idea.  In
particular, locking without a timeout introduces the
possibility that
you will hang the finalization thread, preventing further
objects from
being finalized.  But clearly, that's not what's happening
here.

Other questions that probably don't matter but might be
interesting to
know:

Can we assume that the finalization thread isn't the first
place where
this lock is required?  That your log starts somewhere in
the middle?

Is this under x86 or x64 or both?

        Are you creating any additional AppDomains in the process?


On Wed, Nov 5, 2008 at 10:15 AM, William Reade
<[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>> wrote:

   Hi Curt

   I am indeed; that's how I know thread 2 is the GC
thread. Is
   locking during GC forbidden?

   William

   Curt Hagenlocher wrote:

   ...or, for that matter, any __del__ methods from
within Python
   -- which ultimately are handled by finalization.

   On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
   <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
   <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]> <mailt

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer 
reproduce the problem. However, this bothers me a bit: I'm not competent 
to follow all the consequences here, but this situation seems to imply 
that Ironclad won't be usable safely from any STA thread. Is this an 
intended restriction or a bug?


Incidentally, tracking down the call stacks proved to be hard work: the 
timing changed enough that I only got a single failure over dozens of 
runs, and it turned out I'd got the logging wrong, so I didn't find out 
anything useful :(.


Dino Viehland wrote:

I would suggest getting a snap shot of the call stacks when this is happening 
if that's possible.  I can't pin anything down but I wonder if you could have 
an STA object or something that otherwise requires message pumping.  That 
message pumping could happen while you're doing a Monitor.Enter call.  If that 
was happening maybe there is some subtle CLR bug or a surprise feature?  It is 
surprising that Monitor.Enter can go re-entrant but it can...

So it'd be interesting to get thread snapshots and see if

  

   EnsureGIL (443) 2
   EnsureGIL (443) 1  <- omg, wtf, bbq, etc.



Could be happening because Thread 1 experiences contention, blocks and pumps 
messages, causing the finalizer thread (2) to run, the lock is acquired and ... 
?

Only other thing I could think of is does it repro on other machines?  Maybe 
it's a hardware bug as unlikely as that seems?

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able 
to help

The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.

I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on the
main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come back as
soon as anyone tries any serious multithreading :).

Curt Hagenlocher wrote:
  

Locking during finalization is often considered to be a bad idea.  In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized.  But clearly, that's not what's happening here.

Other questions that probably don't matter but might be interesting to
know:

Can we assume that the finalization thread isn't the first place where
this lock is required?  That your log starts somewhere in the middle?

Is this under x86 or x64 or both?

Are you creating any additional AppDomains in the process?


On Wed, Nov 5, 2008 at 10:15 AM, William Reade
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:

Hi Curt

I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?

William

Curt Hagenlocher wrote:

...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.

On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
wrote:

   So, the obvious question for me is whether or not you're
using any
   finalizers.


   On Wed, Nov 5, 2008 at 5:57 AM, William Reade
   <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>>

   wrote:

   Hi all

   While running the numpy tests, I've come across a situation
   which, to the best of my knowledge, is simply
impossible. I'm
   hoping that one of the local .NET gurus will be able to
tell
   me what I'm missing, or point me somewhere I can get
more insight.

   The 4 methods involved are as follows:
   ---
 public int GetThreadId()
 {
 return Thread.CurrentThread.ManagedThreadId;
 }

 public void WriteFlush(string info)
 {
 Console.WriteLine(info);
 Console.Out.Flush();
 }

 public void EnsureGIL()
 {
 Monitor.Enter(this.dispatcherLock);
 this.WriteFlush(String.For

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

The log starts in the middle (after many lock/unlocks, some from each 
thread); I'm running on x86; and I have no additional AppDomains.


I don't think it would be safe for me to entirely avoid locking during 
finalization, but I could probably cut it down to a quick lock, on a 
separate object, to enqueue pointers for cleanup and deallocation on the 
main thread. However, I'm reluctant to do that until I'm sure that the 
problem is specifically related to GC, otherwise it'll just come back as 
soon as anyone tries any serious multithreading :).


Curt Hagenlocher wrote:
Locking during finalization is often considered to be a bad idea.  In 
particular, locking without a timeout introduces the possibility that 
you will hang the finalization thread, preventing further objects from 
being finalized.  But clearly, that's not what's happening here.


Other questions that probably don't matter but might be interesting to 
know:


Can we assume that the finalization thread isn't the first place where 
this lock is required?  That your log starts somewhere in the middle?


Is this under x86 or x64 or both?

Are you creating any additional AppDomains in the process?


On Wed, Nov 5, 2008 at 10:15 AM, William Reade 
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:


Hi Curt

I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?

William

Curt Hagenlocher wrote:

...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.

On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
wrote:

   So, the obvious question for me is whether or not you're
    using any
   finalizers.


   On Wed, Nov 5, 2008 at 5:57 AM, William Reade
   <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>>

   wrote:

   Hi all

   While running the numpy tests, I've come across a situation
   which, to the best of my knowledge, is simply
impossible. I'm
   hoping that one of the local .NET gurus will be able to
tell
   me what I'm missing, or point me somewhere I can get
more insight.

   The 4 methods involved are as follows:
   ---
 public int GetThreadId()
 {
 return Thread.CurrentThread.ManagedThreadId;
 }

 public void WriteFlush(string info)
 {
 Console.WriteLine(info);
 Console.Out.Flush();
 }

 public void EnsureGIL()
 {
 Monitor.Enter(this.dispatcherLock);
 this.WriteFlush(String.Format(
 "EnsureGIL ({1}) {0}", this.GetThreadId(),
   Builtin.id(this.dispatcherLock)));
 }

 public void ReleaseGIL()
 {
 this.WriteFlush(String.Format(
 "ReleaseGIL ({1}) {0}\n", this.GetThreadId(),
   Builtin.id(this.dispatcherLock)));
 Monitor.Exit(this.dispatcherLock);
 }
   ---
   ...and they can, and do, occasionally produce output as
follows:
   ---
   EnsureGIL (443) 2
   EnsureGIL (443) 1  <- omg, wtf, bbq, etc.
   ReleaseGIL (443) 2

   EnsureGIL (443) 2
   ReleaseGIL (443) 1

   ReleaseGIL (443) 2
   ---
   When this happens, the process continues happily for a
short
   time and then falls over in a later call to ReleaseGIL
(after
   successfully calling it several times). The error is "
Object
   synchronization method was called from an
unsynchronized block
   of code", which I understand to mean "you can't release
this
   lock because you don't hold it".

   It doesn't happen very often, but I can usually
reproduce it
   by running
test_multiarray.TestFromToFile.test_malformed a few
   hundred times. It may be relevant to note that thread 2
is the
   GC thread, and thread 1 is the main thread. I have
consider

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Hi Curt

I am indeed; that's how I know thread 2 is the GC thread. Is locking 
during GC forbidden?


William

Curt Hagenlocher wrote:
...or, for that matter, any __del__ methods from within Python -- 
which ultimately are handled by finalization.


On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher <[EMAIL PROTECTED] 
<mailto:[EMAIL PROTECTED]>> wrote:


So, the obvious question for me is whether or not you're using any
finalizers.


On Wed, Nov 5, 2008 at 5:57 AM, William Reade
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
wrote:

Hi all

While running the numpy tests, I've come across a situation
which, to the best of my knowledge, is simply impossible. I'm
hoping that one of the local .NET gurus will be able to tell
me what I'm missing, or point me somewhere I can get more insight.

The 4 methods involved are as follows:
---
  public int GetThreadId()
  {
  return Thread.CurrentThread.ManagedThreadId;
  }

  public void WriteFlush(string info)
  {
  Console.WriteLine(info);
  Console.Out.Flush();
  }

  public void EnsureGIL()
  {
  Monitor.Enter(this.dispatcherLock);
  this.WriteFlush(String.Format(
  "EnsureGIL ({1}) {0}", this.GetThreadId(),
Builtin.id(this.dispatcherLock)));
  }

  public void ReleaseGIL()
  {
  this.WriteFlush(String.Format(
  "ReleaseGIL ({1}) {0}\n", this.GetThreadId(),
Builtin.id(this.dispatcherLock)));
  Monitor.Exit(this.dispatcherLock);
  }
---
...and they can, and do, occasionally produce output as follows:
---
EnsureGIL (443) 2
EnsureGIL (443) 1  <- omg, wtf, bbq, etc.
ReleaseGIL (443) 2

EnsureGIL (443) 2
ReleaseGIL (443) 1

ReleaseGIL (443) 2
---
When this happens, the process continues happily for a short
time and then falls over in a later call to ReleaseGIL (after
successfully calling it several times). The error is " Object
synchronization method was called from an unsynchronized block
of code", which I understand to mean "you can't release this
lock because you don't hold it".

It doesn't happen very often, but I can usually reproduce it
by running test_multiarray.TestFromToFile.test_malformed a few
hundred times. It may be relevant to note that thread 2 is the
GC thread, and thread 1 is the main thread. I have considered
the following possibilities:

(1) That I'm locking on the wrong object. I believe that isn't
the case, because it's constructed only once, as a "new
Object()" (ie, a reference type), and is only subsequently
used for locking; and, because it keeps the same ipy id
throughout.

(2) That Monitor.Enter occasionally allows two different
threads to acquire the same lock. I consider this extremely
unlikely, because... well, how many multithreaded .NET apps
already exist? If Monitor really were broken, I think we'd
probably know about it by now.

(3) That calling Flush() on a SyncTextWriter (the type of
Console.Out) doesn't actually do anything, and the output is
somehow wrongly ordered (although I can't imagine how this
could actually be: if the locking is really working, then my
console writes are strictly sequential). I don't have access
to the code, so I have no idea how it's implemented, but even
if this is the case it doesn't help much with the fundamental
problem (the synchronisation error which follows).

Apart from the above, I'm out of ideas. Can anyone suggest
what I've missed?

William
___
Users mailing list
Users@lists.ironpython.com <mailto:Users@lists.ironpython.com>
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com





___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Incidentally, logging a Stopwatch timestamp in WriteFlush reveals that, 
yes, the calls really are happening in the order they appear to be. So, 
option (3) appears to be a red herring, and options (1) and (2) remain 
unchanged.


William Reade wrote:

Hi all

While running the numpy tests, I've come across a situation which, to 
the best of my knowledge, is simply impossible. I'm hoping that one of 
the local .NET gurus will be able to tell me what I'm missing, or 
point me somewhere I can get more insight.


The 4 methods involved are as follows:
---
   public int GetThreadId()
   {
   return Thread.CurrentThread.ManagedThreadId;
   }

   public void WriteFlush(string info)
   {
   Console.WriteLine(info);
   Console.Out.Flush();
   }

   public void EnsureGIL()
   {
   Monitor.Enter(this.dispatcherLock);
   this.WriteFlush(String.Format(
   "EnsureGIL ({1}) {0}", this.GetThreadId(), 
Builtin.id(this.dispatcherLock)));

   }

   public void ReleaseGIL()
   {
   this.WriteFlush(String.Format(
   "ReleaseGIL ({1}) {0}\n", this.GetThreadId(), 
Builtin.id(this.dispatcherLock)));

   Monitor.Exit(this.dispatcherLock);
   }
---
...and they can, and do, occasionally produce output as follows:
---
EnsureGIL (443) 2
EnsureGIL (443) 1  <- omg, wtf, bbq, etc.
ReleaseGIL (443) 2

EnsureGIL (443) 2
ReleaseGIL (443) 1

ReleaseGIL (443) 2
---
When this happens, the process continues happily for a short time and 
then falls over in a later call to ReleaseGIL (after successfully 
calling it several times). The error is " Object synchronization 
method was called from an unsynchronized block of code", which I 
understand to mean "you can't release this lock because you don't hold 
it".


It doesn't happen very often, but I can usually reproduce it by 
running test_multiarray.TestFromToFile.test_malformed a few hundred 
times. It may be relevant to note that thread 2 is the GC thread, and 
thread 1 is the main thread. I have considered the following 
possibilities:


(1) That I'm locking on the wrong object. I believe that isn't the 
case, because it's constructed only once, as a "new Object()" (ie, a 
reference type), and is only subsequently used for locking; and, 
because it keeps the same ipy id throughout.


(2) That Monitor.Enter occasionally allows two different threads to 
acquire the same lock. I consider this extremely unlikely, because... 
well, how many multithreaded .NET apps already exist? If Monitor 
really were broken, I think we'd probably know about it by now.


(3) That calling Flush() on a SyncTextWriter (the type of Console.Out) 
doesn't actually do anything, and the output is somehow wrongly 
ordered (although I can't imagine how this could actually be: if the 
locking is really working, then my console writes are strictly 
sequential). I don't have access to the code, so I have no idea how 
it's implemented, but even if this is the case it doesn't help much 
with the fundamental problem (the synchronisation error which follows).


Apart from the above, I'm out of ideas. Can anyone suggest what I've 
missed?


William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Ironclad problem, with which someone here may be able to help

[William Reade]

Hi all

While running the numpy tests, I've come across a situation which, to 
the best of my knowledge, is simply impossible. I'm hoping that one of 
the local .NET gurus will be able to tell me what I'm missing, or point 
me somewhere I can get more insight.


The 4 methods involved are as follows:
---
   public int GetThreadId()
   {
   return Thread.CurrentThread.ManagedThreadId;
   }

   public void WriteFlush(string info)
   {
   Console.WriteLine(info);
   Console.Out.Flush();
   }

   public void EnsureGIL()
   {
   Monitor.Enter(this.dispatcherLock);
   this.WriteFlush(String.Format(
   "EnsureGIL ({1}) {0}", this.GetThreadId(), 
Builtin.id(this.dispatcherLock)));

   }

   public void ReleaseGIL()
   {
   this.WriteFlush(String.Format(
   "ReleaseGIL ({1}) {0}\n", this.GetThreadId(), 
Builtin.id(this.dispatcherLock)));

   Monitor.Exit(this.dispatcherLock);
   }
---
...and they can, and do, occasionally produce output as follows:
---
EnsureGIL (443) 2
EnsureGIL (443) 1  <- omg, wtf, bbq, etc.
ReleaseGIL (443) 2

EnsureGIL (443) 2
ReleaseGIL (443) 1

ReleaseGIL (443) 2
---
When this happens, the process continues happily for a short time and 
then falls over in a later call to ReleaseGIL (after successfully 
calling it several times). The error is " Object synchronization method 
was called from an unsynchronized block of code", which I understand to 
mean "you can't release this lock because you don't hold it".


It doesn't happen very often, but I can usually reproduce it by running 
test_multiarray.TestFromToFile.test_malformed a few hundred times. It 
may be relevant to note that thread 2 is the GC thread, and thread 1 is 
the main thread. I have considered the following possibilities:


(1) That I'm locking on the wrong object. I believe that isn't the case, 
because it's constructed only once, as a "new Object()" (ie, a reference 
type), and is only subsequently used for locking; and, because it keeps 
the same ipy id throughout.


(2) That Monitor.Enter occasionally allows two different threads to 
acquire the same lock. I consider this extremely unlikely, because... 
well, how many multithreaded .NET apps already exist? If Monitor really 
were broken, I think we'd probably know about it by now.


(3) That calling Flush() on a SyncTextWriter (the type of Console.Out) 
doesn't actually do anything, and the output is somehow wrongly ordered 
(although I can't imagine how this could actually be: if the locking is 
really working, then my console writes are strictly sequential). I don't 
have access to the code, so I have no idea how it's implemented, but 
even if this is the case it doesn't help much with the fundamental 
problem (the synchronisation error which follows).


Apart from the above, I'm out of ideas. Can anyone suggest what I've missed?

William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] blocker: objects deleted when they shouldn't be

[William Reade]

Great -- thank you very much.

Dino Viehland wrote:

It might be this week, it might be next week - I'm currently doing some perf 
tuning for the DLR team and am queueing up a bunch of bugs to fix in a single 
day or two.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Monday, September 29, 2008 10:50 AM
To: Discussion of IronPython
Subject: Re: [IronPython] blocker: objects deleted when they shouldn't be

Thanks Dino :)

Can you suggest an approximate ETA for a fix? I don't want to hassle
you, but I'm very keen to get numpy matrices working and would really
appreciate a guess -- even if it's wildly wrong, it still gives me a
framework to work within ;).

Cheers
William

Dino Viehland wrote:
  

Very cool bug and great analysis - thanks for reporting this.  BTW That code 
lives in MetaPythonType.Calls now.

It seems like this logic needs to be moved into PythonType.CreateInstance 
instead instead of in the generated rule code.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Monday, September 29, 2008 7:11 AM
To: users@lists.ironpython.com
Subject: [IronPython] blocker: objects deleted when they shouldn't be

Hi all

I'm about to file this on Codeplex, but I think I've tracked down a
subtle and nasty bug, which is blocking Ironclad with numpy (it fatally
breaks matrices and masked arrays), and I feel compelled to shout loudly
about it to ensure it doesn't get lost in the noise of the impending 2.0
release.

In short, the following code:
-
from System import GC

class C(object):
_instance = None
def __new__(cls):
if C._instance is None:
C._instance = object.__new__(cls)
return C._instance

def __del__(self):
print 'deleting object with id', id(self)

c = C()
print 'created a C with id', id(c)
d = C()
print 'created another C with id', id(d)

for _ in range(3):
GC.Collect()
GC.WaitForPendingFinalizers()

print id(c), id(d)
print c
print d
-

...produces the following output:
-
C:\dev\ironclad\build>ipy deltest.py
created a C with id 43
created another C with id 43
deleting object with id 43
43 43


-

...and I don't think that the 'deleting...' message should be printed at
that point.

My suspicion is that c's 'magic_finalization_wossname'* slot is being
overwritten with a new instance after __new__ returns, leaving the
original 'magic_finalization_wossname' unreferenced; in the fullness of
time, that gets finalized and calls c.__del__, even though c itself
still exists.

Cheers
William

* This is probably not the actual member name, but I couldn't find the
relevant code today, even though I remember reading it in the past...
hopefully the implementers know what I'm trying to refer to, even if
nobody else does.
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com





___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] blocker: objects deleted when they shouldn't be

[William Reade]

Thanks Dino :)

Can you suggest an approximate ETA for a fix? I don't want to hassle 
you, but I'm very keen to get numpy matrices working and would really 
appreciate a guess -- even if it's wildly wrong, it still gives me a 
framework to work within ;).


Cheers
William

Dino Viehland wrote:

Very cool bug and great analysis - thanks for reporting this.  BTW That code 
lives in MetaPythonType.Calls now.

It seems like this logic needs to be moved into PythonType.CreateInstance 
instead instead of in the generated rule code.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Monday, September 29, 2008 7:11 AM
To: users@lists.ironpython.com
Subject: [IronPython] blocker: objects deleted when they shouldn't be

Hi all

I'm about to file this on Codeplex, but I think I've tracked down a
subtle and nasty bug, which is blocking Ironclad with numpy (it fatally
breaks matrices and masked arrays), and I feel compelled to shout loudly
about it to ensure it doesn't get lost in the noise of the impending 2.0
release.

In short, the following code:
-
from System import GC

class C(object):
_instance = None
def __new__(cls):
if C._instance is None:
C._instance = object.__new__(cls)
return C._instance

def __del__(self):
print 'deleting object with id', id(self)

c = C()
print 'created a C with id', id(c)
d = C()
print 'created another C with id', id(d)

for _ in range(3):
GC.Collect()
GC.WaitForPendingFinalizers()

print id(c), id(d)
print c
print d
-

...produces the following output:
-
C:\dev\ironclad\build>ipy deltest.py
created a C with id 43
created another C with id 43
deleting object with id 43
43 43


-

...and I don't think that the 'deleting...' message should be printed at
that point.

My suspicion is that c's 'magic_finalization_wossname'* slot is being
overwritten with a new instance after __new__ returns, leaving the
original 'magic_finalization_wossname' unreferenced; in the fullness of
time, that gets finalized and calls c.__del__, even though c itself
still exists.

Cheers
William

* This is probably not the actual member name, but I couldn't find the
relevant code today, even though I remember reading it in the past...
hopefully the implementers know what I'm trying to refer to, even if
nobody else does.
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

  


___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] blocker: objects deleted when they shouldn't be

[William Reade]

Hi all

I'm about to file this on Codeplex, but I think I've tracked down a 
subtle and nasty bug, which is blocking Ironclad with numpy (it fatally 
breaks matrices and masked arrays), and I feel compelled to shout loudly 
about it to ensure it doesn't get lost in the noise of the impending 2.0 
release.


In short, the following code:
-
from System import GC

class C(object):
   _instance = None
   def __new__(cls):
   if C._instance is None:
   C._instance = object.__new__(cls)
   return C._instance
  
   def __del__(self):

   print 'deleting object with id', id(self)

c = C()
print 'created a C with id', id(c)
d = C()
print 'created another C with id', id(d)

for _ in range(3):
   GC.Collect()
   GC.WaitForPendingFinalizers()

print id(c), id(d)
print c
print d
-

...produces the following output:
-
C:\dev\ironclad\build>ipy deltest.py
created a C with id 43
created another C with id 43
deleting object with id 43
43 43


-

...and I don't think that the 'deleting...' message should be printed at 
that point.


My suspicion is that c's 'magic_finalization_wossname'* slot is being 
overwritten with a new instance after __new__ returns, leaving the 
original 'magic_finalization_wossname' unreferenced; in the fullness of 
time, that gets finalized and calls c.__del__, even though c itself 
still exists.


Cheers
William

* This is probably not the actual member name, but I couldn't find the 
relevant code today, even though I remember reading it in the past... 
hopefully the implementers know what I'm trying to refer to, even if 
nobody else does.

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Ironclad v0.5 released

[William Reade]

(Cross-posted from the c-extensions-for-ironpython google group -- I 
thought a few people here might be interested too :))


Hi all

Our original goal for 0.5 was to import numpy, from IronPython, and do 
something with it. With one monstrous caveat, we have reached that goal; 
the problem is that you need to comment out line 532 of 
numpy.core.numerictypes.py before it'll import. That line reads 
"_unicodesize = array('u','U1').itemsize", and fails trying to construct 
a unicode array; however, since _unicodesize appears to be unreferenced, 
we feel reasonably comfortable ignoring it for now.


Once numpy is imported, you can create integer arrays and do a few 
things with them; of course, the number of things you can do is still 
dwarfed by the number of things you can't do. You can add, subtract and 
multiply integer arrays, and raise them to integer powers... and that's 
about it.


We will continue to work on numpy; immediate goals include getting 
floating-point arrays to print out as numbers instead of NaNs, hooking 
up the many missing PyTypeObject fields, and implementing more API 
functions as we encounter them. I don't have a very clear goal in mind 
for v0.6 as yet; making floating-point arrays work properly is obviously 
critical, but further steps are not yet clear. If anyone is
particularly keen to start using numpy with IronPython, please get in 
touch; I'm keen to know what you want to do with it, so I can focus on 
features I know are valuable :).


Best
William
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Ironclad v0.2 released

[William Reade]

Hi all

Please enjoy Ironclad v0.2, which is now available from:

http://code.google.com/p/ironclad/downloads/list

Major changes are as follows:

* Ironclad now uses IronPython 2.0b1 for everything.
* It is now possible to use all the functions and types -- and their 
methods -- from the bz2 module.

Major deficiencies are as follows:

* Lists are not well mapped -- changes made on one side of the 
managed/unmanaged divide will not necessarily be seen on the other.
* Type members and properties don't work yet.
* Most of the CPython API is still not implemented.

The next steps are to make attributes and properties work, and then to 
start work on the multiarray module from numpy.

Best regards
william
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] file.mode property doesn't exist

[William Reade]

Hi

IronPython 1.1.1 (1.1.1) on .NET 2.0.50727.312
Copyright (c) Microsoft Corporation. All rights reserved.
 >>> f = open("text.txt")
 >>> f.mode
Traceback (most recent call last):
  File , line 0, in ##82
AttributeError: 'file' object has no attribute 'mode'
 >>>


Would it be possible to implement this property? I'm trying to implement 
PyFile_AsFile for Ironclad, and it would be enormously helpful to be 
able to retrieve a file's mode (for _fdopen in msvcrt.dll) without 
either (1) depending on a custom IronPython or (2) reflecting the real 
IronPython half to death.

For now, I'm just going to assume that all files have been opened for 
reading, so I can get some useful functionality without it; still, I'd 
be very grateful if you would slip this into a nearish-future release 
(or, ofc, tell me what I failed to notice that makes my request 
unnecessary ;-) ).

Cheers
william
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Announcement: Ironclad v0.1

[William Reade]

Hi all

I'm delighted to announce that Ironclad 0.1, our nascent CPython 
compatibility library, is now available at:

http://code.google.com/p/ironclad/downloads/list

It's not very impressive yet, and it's still win32-only, but it will 
import the bz2.pyd module from CPython 2.5; the compress() and 
decompress() functions should work, as should the BZ2Compressor and 
BZ2Decompressor types. Sadly, BZ2File still needs quite a lot of work. 
Nonetheless, please download it and have a play :-).

Be aware that you can't just reference the DLLs and have everything Just 
Work -- have a look in 'tests/functionalitytest.py' to see how to make 
it work; have a look at 'doc/details.txt' if you're interested in what's 
going on under the hood.

Bug reports, complaints, advice and patches are all very welcome; of 
course, bug reports with integrated patches and test cases will receive 
the maximum possible brownie points ;-).

Cheers
William

-- 
William Reade
Resolver Systems
[EMAIL PROTECTED]

We're hiring! http://www.resolversystems.com/jobs/ 

Office address: 17a Clerkenwell Road, London EC1M 5RD, UK
Registered address: 843 Finchley Road, London NW11 8NA, UK

Resolver Systems Limited is registered in England and Wales as company number 
5467329. 

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Patching sys.modules to hide CLR 'modules'

[William Reade]

Hi all

I recently wanted to do as described in the subject line. However, it 
seems that when I import (say) System.Drawing, there is no detectable 
change to sys.modules, and so I cannot temporarily replace it with my 
own module in order to substitute simpler functionality for testing 
purposes*. Is this correct? If so, can I work around it, and what 
exciting pitfalls will I walk into if I try?

William

* That is to say, I cannot monkey-patch anything in sys.modules that 
didn't come from IronPython.

-- 
William Reade
Resolver Systems
[EMAIL PROTECTED]

Office address: 17a Clerkenwell Road, London EC1M 5RD, UK
Registered address: 843 Finchley Road, London NW11 8NA, UK

Resolver Systems Limited is registered in England and Wales as company number 
5467329. 

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Groping in the dark

[William Reade]

Hi Patrick

I haven't been able to see the anomalies you mention -- perhaps the 
following sample will help.
---
import clr
clr.AddReference("System.Windows.Forms")

from System.Windows.Forms import (
Application, DockStyle, Form, Keys, MenuStrip, ToolStripMenuItem, 
ToolStripSeparator
)

def SayHi(*_):
print "hi"

f = Form()
f.Text = "Hello multiverse!"

menuStrip = MenuStrip()
menuStrip.Dock = DockStyle.Top
f.Controls.Add(menuStrip)

menu = ToolStripMenuItem()
menu.Text = "SomeMenu"
menuStrip.Items.Add(menu)

menuItem = ToolStripMenuItem()
menuItem.Text = "SomeMenuItem"
menuItem.Click += SayHi
menuItem.ShortcutKeys = Keys.Control | Keys.Q
menu.DropDownItems.Add(menuItem)
menu.DropDownItems.Add(ToolStripSeparator())

Application.Run(f)


Cheers
William

Patrick O'Brien wrote:

> Am I missing something or is everyone else groping around in the dark 
> as much as I am when it comes to figuring out acceptable syntax in 
> IronPython versus other languages supported by the CLR.  What I'm 
> doing is building a simple app in Visual Studio C# 2005 Express to see 
> what objects are available and what properties they have.  I'm also 
> looking at the generated code.  But my translations into Python don't 
> always work out as I expect.  For example, here are some differences 
> I've come across:
>
> For all of these examples I'm using this bit of code:
>
> import clr
> clr.AddReference('System.Windows.Forms')
> import System.Windows.Forms as SWF
>
>
> To add a separator to a menu you need to do this:
>
> Add('-')
>
> Whereas this raises an exception:
>
> Add(SWF.ToolStripSeparator())
>
> Various attempts at adding shortcut keys failed, but this works:
>
> open_item.Shortcut = SWF.Shortcut.CtrlO
>
> Apparently the .ShortcutKeys property does not exist on menu items in 
> IronPython.
>
> So my basic question is how does one learn all these anomalies other 
> than by trial and error?
>
> -- 
> Patrick K. O'Brien
> Orbtech   http://www.orbtech.com
> Schevohttp://www.schevo.org
> Louie http://www.pylouie.org 
>
>
>
>___
>users mailing list
>users@lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  
>

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Compilation difficulties

[William Reade]

Hi all

I've been playing with the (excellent) Pyc sample, and I've got stuck. I 
have a series of files a bit like the following:

project/
| main.py
+ Cheesemonger/
  | Shopkeeper.py
  | Counter.py

...in which main.py contains statements like:

from Cheesemonger.Counter import Counter

...and Counter.py also contains statements like:

from Cheesemonger.Shopkeeper import Shopkeeper

---

This is all very well, but I can't work out how to compile a 
Cheesemonger.dll in which the above imports still function -- when I 
inspect it with ildasm, the dll contains "Counter" and "Shopkeeper" 
rather than "Cheesemonger.Counter" and "Cheesemonger.Shopkeeper".

I can "fix" the issue by undisambiguating the imports:

clr.AddReference("Cheesemonger")
from Counter import Counter

...and:

from Shopkeeper import Shopkeeper

...but we've been burned by ambiguous imports before and would prefer 
not to reintroduce them (life can get confusing when every directory 
contains its own "UnitTests" directory ;)).

Is there any way of compiling a directory (and subdirectories) into a 
single dll, such that no import statements need to be changed? A quick 
look at the IP source left me none the wiser, I'm afraid.

Cheers
William

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Adding None to a set

[William Reade]

Hi guys

As if you didn't have enough on your plates already...

IronPython beta8:
 >>> st = set([])
 >>> st.add(None)
 >>> st
set([])

CPython:
 >>> st = set([])
 >>> st.add(None)
 >>> st
set([None])

Cheers
William
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Beta 5 bug: False == None?

[William Reade]

---
IronPython 1.0.2280 (Beta) on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
 >>> False == None
True
---

The avove comparison should evaluate to False. It was working correctly 
in IP beta 4.

However, "None == False" still (correctly) evaluates to False.

Cheers
William
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Nitpicking:

[William Reade]

"lambda" appears to be misspelled here:

---
IronPython 1.0.2216 (Beta) on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
 >>> lambda x : x

---

Cheers
William

-- 
We're hiring!


___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Lists of instances of classes

[William Reade]

It appears that tuples have similar problems to lists (tested on 0.9.5 
and 1.0b1):

--
 >>> class C:
...   pass
...
 >>> c1 = C()
 >>> c2 = C()
 >>> (c1, c2) == (c2, c1)
True
--

Cheers
William

Giles Thomas wrote:

>Some more discoveries, following on from the below:
>
>---
> >>> class D:
>...   pass
>...
> >>> d1 = D()
> >>> [c1, c2, c1] == [c1, c2, d1]
>True
> >>> class E:
>...   def __init__(self):
>... pass
>...
> >>> e1 = E()
> >>> [c1, c2, c1] == [c1, c2, e1]
>True
> >>> e1.foo = "bar"
> >>> [c1, c2, c1] == [c1, c2, e1]
>True
>---
>
>Looks like it's just assuming all objects are identical.
>
>
>Regards,
>
>Giles
>
>
>Giles Thomas wrote:
>  
>
>>Hi,
>>
>>We've found what looks like a bug in list comparison:
>>
>>---
>>IronPython 1.0.2190 (Beta) on .NET 2.0.50727.42
>>Copyright (c) Microsoft Corporation. All rights reserved.
>> >>> class C:
>>...   pass
>>...
>> >>> c1 = C()
>> >>> c2 = C()
>> >>> [c1, c2] == [c2, c1]
>>True
>> >>> [c1] == [c2]
>>True
>> >>> c1 == c2
>>False
>> >>>
>>-
>>
>>Confirmed in 0.9.5, 0.9.6, and 1.0.2190
>>
>>There is a horrible hack to work around it:
>>
>>-
>> >>> str([c1, c2]) == str([c2, c1])
>>False
>>-
>>
>>
>>Cheers,
>>
>>Giles
>>
>>
>>
>___
>users mailing list
>users@lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>  
>

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] function/method comparison

[William Reade]

Hi, we have another bug report. This one's a bit longer...

In IronPythonConsole (0.9.6):
--
IronPython 0.9.6 on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
 >>> class C:
...   def f(self):
... pass
...
 >>> c = C()
 >>> c.f == c.f
False
 >>> C.f == C.f
False
--

...and in IronPythonConsole (0.9.5):
--
IronPython 0.9.5 on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
 >>> class C:
...   def f(self):
... pass
...
 >>> c = C()
 >>> c.f == c.f
False
 >>> C.f == C.f
True
--

We believe that both comparisons should evaluate to True.

Merry Christmas ;-)
William
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Comparing enum to None

[William Reade]

Hi, we have a bug report. The following code:

--
import sys
sys.LoadAssemblyByName("System.Windows.Forms")
from System.Windows.Forms import *

if 1 == None:
print "won't happen"

if DialogResult.Cancel == None: # boom!
print "won't happen"
--

...raises a NullReferenceException, as follows, at the marked line.

--
Unhandled Exception: System.NullReferenceException: Object reference not 
set to
an instance of an object.
   at IronPython.Objects.EnumOps.Equal(Object self, Object other)
   at IronPython.Objects.Ops.Equal(Object x, Object y)
   at __main__.Initialize() in H:\dev\current\sandbox\testenumnone.py:line 8
--

We feel the expression should just evaluate to False.

Incidentally, should we be bothering the list with these things, or 
should we put them directly into the bug tracker?

Cheers
William
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] get_Item in subclass

[William Reade]

We've been having some problems subclassing the SyncFusion Grid control 
with IronPython; namely, that:

class TestGridControl(GridControl):
pass

grid = TestGridControl()
value = "foobar"
grid[1, 1].CellValue = "foobar"

fails with the following stack trace:

Unhandled exception: Bad args for the methodat 
IronPython.Objects.ReflectedMethodBase.TryCall(Object[] args, Object& ret)
   at IronPython.Objects.ReflectedMethodBase.Call(Object[] args)
   at IronPython.Objects.Ops.Call(Object func, Object[] args)
   at IronPython.Objects.DynamicType.__getitem__(Object self, Object index)
   at IronPython.Objects.Ops.GetIndex(Object o, Object index)
   at 
FunctionalTests.UnitTests.CellUtilsTest.testSetCellValue$f222(Object self)
 in 
H:\dev\current\resolver-working\trunk\FunctionalTests\UnitTests\CellUtilsTest.py:line
 
41

While:

grid = GridControl()
grid[1, 1].CellValue = "foobar"

works as expected.

Has anyone come across this before, and does the list have any bright 
ideas for working around it? It seems to us that there's some magic for 
dealing with the CLR Item property which isn't working quite right in 
subclasses, but that's as far as we've got...

Cheers
William
___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] NUnit with IronPython

[William Reade]

Sadly, NUnit won't run static methods, so that won't work, but thank you 
anyway.

Are class names similarly mangled? If there's any documentation on this, 
I'd really appreciate a pointer to it, as I haven't been able to find 
much help elsewhere... or is it just a matter of hunting through the source?

William

Martin Maly wrote:

>IronPython mangles names. For example:
>
>def method():
>return 1
>
>Produces:
>
>public static object method$f0()
>
>The test_prefix would get therefore preserved so on a second thought, this may 
>work, provided that nunit executes public static methods and doesn't require 
>specific return value. Ours is always "object" in this case.
>
>Martin
>
>
>-Original Message-
>From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
>Sent: Thursday, December 08, 2005 10:02 AM
>To: Discussion of IronPython
>Subject: Re: [IronPython] NUnit with IronPython
>
>Does this mean that IronPython will emit methods, but their names will be 
>mangled, or that it simply won't produce anything that can be effectively used 
>from outside?
>
>William
>
>Martin Maly wrote:
>
>  
>
>>This is, sadly, true. IronPython will not preserve the name of the method as 
>>we know it...
>>
>>Martin
>>
>>-Original Message-
>>From: [EMAIL PROTECTED] 
>>[mailto:[EMAIL PROTECTED] On Behalf Of Keith J. 
>>Farmer
>>Sent: Thursday, December 08, 2005 9:46 AM
>>To: Discussion of IronPython
>>Subject: Re: [IronPython] NUnit with IronPython
>>
>>For backwards compatibility, NUnit would also accept method names beginning 
>>with "test" ("test_" ?).  However, I don't think IP emits methods as we know 
>>them yet?
>>
>>-
>>Keith J. Farmer // [EMAIL PROTECTED]
>> 
>>
>>
>>
>
>___
>users mailing list
>users@lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>___
>users mailing list
>users@lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>  
>

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] NUnit with IronPython

[William Reade]

Does this mean that IronPython will emit methods, but their names will 
be mangled, or that it simply won't produce anything that can be 
effectively used from outside?

William

Martin Maly wrote:

>This is, sadly, true. IronPython will not preserve the name of the method as 
>we know it...
>
>Martin
>
>-Original Message-
>From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Keith J. Farmer
>Sent: Thursday, December 08, 2005 9:46 AM
>To: Discussion of IronPython
>Subject: Re: [IronPython] NUnit with IronPython
>
>For backwards compatibility, NUnit would also accept method names beginning 
>with "test" ("test_" ?).  However, I don't think IP emits methods as we know 
>them yet?
>
>-
>Keith J. Farmer // [EMAIL PROTECTED]
>  
>

___
users mailing list
users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com