Re: [IronPython] Slow Performance of CPython libs?

[Birsch]

Great, you've got my vote :)

Thanks a lot for your help.

On Fri, Feb 22, 2008 at 9:03 PM, Dino Viehland <[EMAIL PROTECTED]>
wrote:

>  Oh, and I've gone ahead and created a bug for this one for 2.0:
> http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=15289
>
>
>
> *From:* [EMAIL PROTECTED] [mailto:
> [EMAIL PROTECTED] *On Behalf Of *Birsch
> *Sent:* Friday, February 22, 2008 9:40 AM
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Slow Performance of CPython libs?
>
>
>
> Dino - do you want me to register a workitem? Also, if I wanted to patch
> v1.1.1, where should I start?
>
> On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland <
> [EMAIL PROTECTED]> wrote:
>
> After Birsh's last comment that they're calling compile I was thinking we
> might be doing something really stupid.  Turns out that we are.  For our
> supposedly compiled regex's we're not passing the Compiled option.  So
> they're all getting interpretted and of course that's going to suck big
> time.
>
> So the fix for this is probably to just pass RegexOptions.Compiled to the
> Regex object being contructed into RE_Pattern.  Internally the Regex class
> will compile to dynamic methods (much better than compiling to an assembly
> because it'll be collectible).
>
> It's actually a little more complex than that because we create RE_Pattern
> objects for the non-compiled so we need to flow the option in, but it's
> still a simple fix.
>
>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:
> [EMAIL PROTECTED] On Behalf Of Dan Shechter
> Sent: Thursday, February 21, 2008 10:19 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Slow Performance of CPython libs?
>
> Just out of curiosity...
> I'm guessing that there are two levels of caching that could be used.
> One would be to hold a global dictionary of some sort of pattern ->
> Regex that will save the lengthy parsing of the re.
>
> The other would be to actually call Regex.CompileToAssembly to get a
> more efficient reperesentation (in runtime that is) of the re.
>
> Would you (the IP team) favor the former or the latter?
>
>Shechter.
>
>
> Birsch wrote:
> > I checked both sgmllib.py and BeautifulSoup.py - and it seems both are
> > reusing the same regexps (searched for re.compile).
> >
> > I think your suggestion is very relevant in this case. It makes sense to
> > replicate the "compile once use many" behavior that is commonly used
> > with regexp.
> >
> > -Birsch
> >
> > On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> > <[EMAIL PROTECTED] >
> wrote:
> >
> > Do you know if the same reg ex is being used repeatedly?  If so
> > maybe we can cache & compile the regex instead of interpretting it
> > all the time.
> >
> >
> >
> > *From:* [EMAIL PROTECTED]
> > 
> > [mailto:[EMAIL PROTECTED]
> > ] *On Behalf Of *Birsch
> > *Sent:* Thursday, February 21, 2008 8:30 AM
> >
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] Slow Performance of CPython libs?
> >
> >
> >
> > I took on Cooper's advice and profiled with ANTS. Here are the top
> > methods:
> >
> > *Namespace*
> >
> >
> >
> > *Method name*
> >
> >
> >
> > *Time (sec.)*
> >
> >
> >
> > *Time with children (sec.)*
> >
> >
> >
> > *Hit count*
> >
> >
> >
> > *Source file*
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Go()
> >
> >
> >
> > 37.0189
> >
> >
> >
> > 94.4676
> >
> >
> >
> > 13689612
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Operator()
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Advance(int i)
> >
> >
> >
> > 5.9264
> >
> >
> >
> > 8.7202
> >
> >
> >
> > 66000263
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.SetOperator(int op)
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Backtrack()
> >
> >
> >
> > 5.5692
> >
> >
> >
> > 9.4895
> >
> >
> >
> > 37781343
> >
> >
> >
> > IronPython.Runtime.Operations
> >
> >
> >
> > Ops.CallWithContext(ICallerContext context, object func, object
> > arg0, object arg1)
> >
> >
> >
> > 5.5572
> >
> >
> >
> > 114.5245
> >
> >
> >
> > 79754
> >
> >
> >
> > IronPython.Runtime.Calls
> >
> >
> >
> > Method.Call(ICallerContext context, object arg0)
> >
> >
> >
> > 4.9052
> >
> >
> >
> > 114.8251
> >
> >
> >
> > 50886
> >
> >
> >
> > IronPython.Runtime.Calls
> >
> >
> >
> > PythonFunction.CallInstance(ICallerContext context, object arg0,
> > object arg1)
> >
> >
> >
> > 4.8876
> >
> >
> >
> > 114.8059
> >
> >
> >
> > 50886

[IronPython] .NET Arrays

[Herny Johnson]

When I'm trying to add .NET Arrays that are holding one type of object with the 
System.Array.Add function.  I get an error that the says "expected object with 
__len__ function, got Texture2D (whatever objects that I want to merge).  I'm 
using Xna  as you might have guessed.  I just want to add the objects into one 
array, not add the objects together.  How do I do this?___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Announcement: Ironclad v0.1

[William Reade]

Hi all

I'm delighted to announce that Ironclad 0.1, our nascent CPython 
compatibility library, is now available at:

http://code.google.com/p/ironclad/downloads/list

It's not very impressive yet, and it's still win32-only, but it will 
import the bz2.pyd module from CPython 2.5; the compress() and 
decompress() functions should work, as should the BZ2Compressor and 
BZ2Decompressor types. Sadly, BZ2File still needs quite a lot of work. 
Nonetheless, please download it and have a play :-).

Be aware that you can't just reference the DLLs and have everything Just 
Work -- have a look in 'tests/functionalitytest.py' to see how to make 
it work; have a look at 'doc/details.txt' if you're interested in what's 
going on under the hood.

Bug reports, complaints, advice and patches are all very welcome; of 
course, bug reports with integrated patches and test cases will receive 
the maximum possible brownie points ;-).

Cheers
William

-- 
William Reade
Resolver Systems
[EMAIL PROTECTED]

We're hiring! http://www.resolversystems.com/jobs/ 

Office address: 17a Clerkenwell Road, London EC1M 5RD, UK
Registered address: 843 Finchley Road, London NW11 8NA, UK

Resolver Systems Limited is registered in England and Wales as company number 
5467329. 

___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Slow Performance of CPython libs?

[Dino Viehland]

Oh, and I've gone ahead and created a bug for this one for 2.0: 
http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=15289

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Birsch
Sent: Friday, February 22, 2008 9:40 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Slow Performance of CPython libs?

Dino - do you want me to register a workitem? Also, if I wanted to patch 
v1.1.1, where should I start?
On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland <[EMAIL PROTECTED]> wrote:
After Birsh's last comment that they're calling compile I was thinking we might 
be doing something really stupid.  Turns out that we are.  For our supposedly 
compiled regex's we're not passing the Compiled option.  So they're all getting 
interpretted and of course that's going to suck big time.

So the fix for this is probably to just pass RegexOptions.Compiled to the Regex 
object being contructed into RE_Pattern.  Internally the Regex class will 
compile to dynamic methods (much better than compiling to an assembly because 
it'll be collectible).

It's actually a little more complex than that because we create RE_Pattern 
objects for the non-compiled so we need to flow the option in, but it's still a 
simple fix.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL 
PROTECTED]] On Behalf Of Dan Shechter
Sent: Thursday, February 21, 2008 10:19 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Slow Performance of CPython libs?

Just out of curiosity...
I'm guessing that there are two levels of caching that could be used.
One would be to hold a global dictionary of some sort of pattern ->
Regex that will save the lengthy parsing of the re.

The other would be to actually call Regex.CompileToAssembly to get a
more efficient reperesentation (in runtime that is) of the re.

Would you (the IP team) favor the former or the latter?

   Shechter.


Birsch wrote:
> I checked both sgmllib.py and BeautifulSoup.py - and it seems both are
> reusing the same regexps (searched for re.compile).
>
> I think your suggestion is very relevant in this case. It makes sense to
> replicate the "compile once use many" behavior that is commonly used
> with regexp.
>
> -Birsch
>
> On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> <[EMAIL PROTECTED]  PROTECTED]>> wrote:
>
> Do you know if the same reg ex is being used repeatedly?  If so
> maybe we can cache & compile the regex instead of interpretting it
> all the time.
>
>
>
> *From:* [EMAIL PROTECTED]
> >
> [mailto:[EMAIL PROTECTED]
> >] *On Behalf Of 
> *Birsch
> *Sent:* Thursday, February 21, 2008 8:30 AM
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Slow Performance of CPython libs?
>
>
>
> I took on Cooper's advice and profiled with ANTS. Here are the top
> methods:
>
> *Namespace*
>
>
>
> *Method name*
>
>
>
> *Time (sec.)*
>
>
>
> *Time with children (sec.)*
>
>
>
> *Hit count*
>
>
>
> *Source file*
>
> System.Text.RegularExpressions
>
>
>
> RegexInterpreter.Go()
>
>
>
> 37.0189
>
>
>
> 94.4676
>
>
>
> 13689612
>
>
>
> System.Text.RegularExpressions
>
>
>
> RegexInterpreter.Operator()
>
>
>
> 6.2411
>
>
>
> 6.2411
>
>
>
> 131146274
>
>
>
> System.Text.RegularExpressions
>
>
>
> RegexInterpreter.Advance(int i)
>
>
>
> 5.9264
>
>
>
> 8.7202
>
>
>
> 66000263
>
>
>
> System.Text.RegularExpressions
>
>
>
> RegexInterpreter.SetOperator(int op)
>
>
>
> 5.5750
>
>
>
> 5.5750
>
>
>
> 131146274
>
>
>
> System.Text.RegularExpressions
>
>
>
> RegexInterpreter.Backtrack()
>
>
>
> 5.5692
>
>
>
> 9.4895
>
>
>
> 37781343
>
>
>
> IronPython.Runtime.Operations
>
>
>
> Ops.CallWithContext(ICallerContext context, object func, object
> arg0, object arg1)
>
>
>
> 5.5572
>
>
>
> 114.5245
>
>
>
> 79754
>
>
>
> IronPython.Runtime.Calls
>
>
>
> Method.Call(ICallerContext context, object arg0)
>
>
>
> 4.9052
>
>
>
> 114.8251
>
>
>
> 50886
>
>
>
> IronPython.Runtime.Calls
>
>
>
> PythonFunction.CallInstance(ICallerContext context, object arg0,
> object arg1)
>
>
>
> 4.8876
>
>
>
> 114.8059
>
>
>
> 50886
>
>
>
> IronPython.Runtime.Calls
>
>
>
> Function2.Call(ICallerContext context, object arg0, object arg1)
>
>
>
> 4.6400
>
>
>
> 114.5471
>
>
>
> 47486
>
>
>
> IronPython.Runtime.Operations
>
>
>
> Ops.CallWithContext(ICallerContext context, object func, object arg0)
>
>
>
> 4.2344
>
>
>
> 114.1604
>
>
>
> 146658
>
>
>
> System.Text.RegularExpressions
>
>
>
> RegexBoyerMoore

Re: [IronPython] Slow Performance of CPython libs?

[Dino Viehland]

Unfortunately CodePlex doesn't make it very easy to track bugs across multiple 
releases.  If there's ever any bugs that you'd like to see fixed in 1.1.x I'd 
suggest opening a bug with a title of 1.1.x: .  You could also 
include a link to the associated 2.0 bug number if there's one there.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Foord
Sent: Friday, February 22, 2008 10:30 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Slow Performance of CPython libs?

Birsch wrote:
> Dino - do you want me to register a workitem? Also, if I wanted to
> patch v1.1.1, where should I start?

This is a good point. As you fix bugs is it possible to at least mark
which ones might be suitable to fold back into 1.1. This and the socket
bugs recently discussed would be really great to see fixed in a 1.1.2
release...

Thanks

Michael Foord
http://www.manning.com/foord

>
> On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland
> <[EMAIL PROTECTED] >
> wrote:
>
> After Birsh's last comment that they're calling compile I was
> thinking we might be doing something really stupid.  Turns out
> that we are.  For our supposedly compiled regex's we're not
> passing the Compiled option.  So they're all getting interpretted
> and of course that's going to suck big time.
>
> So the fix for this is probably to just pass RegexOptions.Compiled
> to the Regex object being contructed into RE_Pattern.  Internally
> the Regex class will compile to dynamic methods (much better than
> compiling to an assembly because it'll be collectible).
>
> It's actually a little more complex than that because we create
> RE_Pattern objects for the non-compiled so we need to flow the
> option in, but it's still a simple fix.
>
> -Original Message-
> From: [EMAIL PROTECTED]
> 
> [mailto:[EMAIL PROTECTED]
> ] On Behalf Of Dan Shechter
> Sent: Thursday, February 21, 2008 10:19 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Slow Performance of CPython libs?
>
> Just out of curiosity...
> I'm guessing that there are two levels of caching that could be used.
> One would be to hold a global dictionary of some sort of pattern ->
> Regex that will save the lengthy parsing of the re.
>
> The other would be to actually call Regex.CompileToAssembly to get a
> more efficient reperesentation (in runtime that is) of the re.
>
> Would you (the IP team) favor the former or the latter?
>
>Shechter.
>
>
> Birsch wrote:
> > I checked both sgmllib.py and BeautifulSoup.py - and it seems
> both are
> > reusing the same regexps (searched for re.compile).
> >
> > I think your suggestion is very relevant in this case. It makes
> sense to
> > replicate the "compile once use many" behavior that is commonly used
> > with regexp.
> >
> > -Birsch
> >
> > On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> > <[EMAIL PROTECTED]
> 
>  >> wrote:
> >
> > Do you know if the same reg ex is being used repeatedly?  If so
> > maybe we can cache & compile the regex instead of
> interpretting it
> > all the time.
> >
> >
> >
> > *From:* [EMAIL PROTECTED]
> 
> >  >
> > [mailto:[EMAIL PROTECTED]
> 
> >  >] *On Behalf Of *Birsch
> > *Sent:* Thursday, February 21, 2008 8:30 AM
> >
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] Slow Performance of CPython libs?
> >
> >
> >
> > I took on Cooper's advice and profiled with ANTS. Here are
> the top
> > methods:
> >
> > *Namespace*
> >
> >
> >
> > *Method name*
> >
> >
> >
> > *Time (sec.)*
> >
> >
> >
> > *Time with children (sec.)*
> >
> >
> >
> > *Hit count*
> >
> >
> >
> > *Source file*
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Go()
> >
> >
> >
> > 37.0189
> >
> >
> >
> > 94.4676
> >
> >
> >
> > 13689612
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Operator()
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Advance(in

Re: [IronPython] Slow Performance of CPython libs?

[Michael Foord]

Birsch wrote:
> Dino - do you want me to register a workitem? Also, if I wanted to 
> patch v1.1.1, where should I start?

This is a good point. As you fix bugs is it possible to at least mark 
which ones might be suitable to fold back into 1.1. This and the socket 
bugs recently discussed would be really great to see fixed in a 1.1.2 
release...

Thanks

Michael Foord
http://www.manning.com/foord

>
> On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland 
> <[EMAIL PROTECTED] > 
> wrote:
>
> After Birsh's last comment that they're calling compile I was
> thinking we might be doing something really stupid.  Turns out
> that we are.  For our supposedly compiled regex's we're not
> passing the Compiled option.  So they're all getting interpretted
> and of course that's going to suck big time.
>
> So the fix for this is probably to just pass RegexOptions.Compiled
> to the Regex object being contructed into RE_Pattern.  Internally
> the Regex class will compile to dynamic methods (much better than
> compiling to an assembly because it'll be collectible).
>
> It's actually a little more complex than that because we create
> RE_Pattern objects for the non-compiled so we need to flow the
> option in, but it's still a simple fix.
>
> -Original Message-
> From: [EMAIL PROTECTED]
> 
> [mailto:[EMAIL PROTECTED]
> ] On Behalf Of Dan Shechter
> Sent: Thursday, February 21, 2008 10:19 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Slow Performance of CPython libs?
>
> Just out of curiosity...
> I'm guessing that there are two levels of caching that could be used.
> One would be to hold a global dictionary of some sort of pattern ->
> Regex that will save the lengthy parsing of the re.
>
> The other would be to actually call Regex.CompileToAssembly to get a
> more efficient reperesentation (in runtime that is) of the re.
>
> Would you (the IP team) favor the former or the latter?
>
>Shechter.
>
>
> Birsch wrote:
> > I checked both sgmllib.py and BeautifulSoup.py - and it seems
> both are
> > reusing the same regexps (searched for re.compile).
> >
> > I think your suggestion is very relevant in this case. It makes
> sense to
> > replicate the "compile once use many" behavior that is commonly used
> > with regexp.
> >
> > -Birsch
> >
> > On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> > <[EMAIL PROTECTED]
> 
>  >> wrote:
> >
> > Do you know if the same reg ex is being used repeatedly?  If so
> > maybe we can cache & compile the regex instead of
> interpretting it
> > all the time.
> >
> >
> >
> > *From:* [EMAIL PROTECTED]
> 
> >  >
> > [mailto:[EMAIL PROTECTED]
> 
> >  >] *On Behalf Of *Birsch
> > *Sent:* Thursday, February 21, 2008 8:30 AM
> >
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] Slow Performance of CPython libs?
> >
> >
> >
> > I took on Cooper's advice and profiled with ANTS. Here are
> the top
> > methods:
> >
> > *Namespace*
> >
> >
> >
> > *Method name*
> >
> >
> >
> > *Time (sec.)*
> >
> >
> >
> > *Time with children (sec.)*
> >
> >
> >
> > *Hit count*
> >
> >
> >
> > *Source file*
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Go()
> >
> >
> >
> > 37.0189
> >
> >
> >
> > 94.4676
> >
> >
> >
> > 13689612
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Operator()
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Advance(int i)
> >
> >
> >
> > 5.9264
> >
> >
> >
> > 8.7202
> >
> >
> >
> > 66000263
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.SetOperator(int op)
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpr

Re: [IronPython] Slow Performance of CPython libs?

[Michael Foord]

Birsch wrote:
> Dino - do you want me to register a workitem? Also, if I wanted to 
> patch v1.1.1, where should I start?

This is a good point. As you fix bugs is it possible to at least mark 
which ones might be suitable to fold back into 1.1. This and the socket 
bugs recently discussed would be really great to see fixed in a 1.1.2 
release...

Thanks

Michael Foord
http://www.manning.com/foord

>
> On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland 
> <[EMAIL PROTECTED] > 
> wrote:
>
> After Birsh's last comment that they're calling compile I was
> thinking we might be doing something really stupid.  Turns out
> that we are.  For our supposedly compiled regex's we're not
> passing the Compiled option.  So they're all getting interpretted
> and of course that's going to suck big time.
>
> So the fix for this is probably to just pass RegexOptions.Compiled
> to the Regex object being contructed into RE_Pattern.  Internally
> the Regex class will compile to dynamic methods (much better than
> compiling to an assembly because it'll be collectible).
>
> It's actually a little more complex than that because we create
> RE_Pattern objects for the non-compiled so we need to flow the
> option in, but it's still a simple fix.
>
> -Original Message-
> From: [EMAIL PROTECTED]
> 
> [mailto:[EMAIL PROTECTED]
> ] On Behalf Of Dan Shechter
> Sent: Thursday, February 21, 2008 10:19 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Slow Performance of CPython libs?
>
> Just out of curiosity...
> I'm guessing that there are two levels of caching that could be used.
> One would be to hold a global dictionary of some sort of pattern ->
> Regex that will save the lengthy parsing of the re.
>
> The other would be to actually call Regex.CompileToAssembly to get a
> more efficient reperesentation (in runtime that is) of the re.
>
> Would you (the IP team) favor the former or the latter?
>
>Shechter.
>
>
> Birsch wrote:
> > I checked both sgmllib.py and BeautifulSoup.py - and it seems
> both are
> > reusing the same regexps (searched for re.compile).
> >
> > I think your suggestion is very relevant in this case. It makes
> sense to
> > replicate the "compile once use many" behavior that is commonly used
> > with regexp.
> >
> > -Birsch
> >
> > On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> > <[EMAIL PROTECTED]
> 
>  >> wrote:
> >
> > Do you know if the same reg ex is being used repeatedly?  If so
> > maybe we can cache & compile the regex instead of
> interpretting it
> > all the time.
> >
> >
> >
> > *From:* [EMAIL PROTECTED]
> 
> >  >
> > [mailto:[EMAIL PROTECTED]
> 
> >  >] *On Behalf Of *Birsch
> > *Sent:* Thursday, February 21, 2008 8:30 AM
> >
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] Slow Performance of CPython libs?
> >
> >
> >
> > I took on Cooper's advice and profiled with ANTS. Here are
> the top
> > methods:
> >
> > *Namespace*
> >
> >
> >
> > *Method name*
> >
> >
> >
> > *Time (sec.)*
> >
> >
> >
> > *Time with children (sec.)*
> >
> >
> >
> > *Hit count*
> >
> >
> >
> > *Source file*
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Go()
> >
> >
> >
> > 37.0189
> >
> >
> >
> > 94.4676
> >
> >
> >
> > 13689612
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Operator()
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Advance(int i)
> >
> >
> >
> > 5.9264
> >
> >
> >
> > 8.7202
> >
> >
> >
> > 66000263
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.SetOperator(int op)
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpr

Re: [IronPython] Slow Performance of CPython libs?

[Birsch]

Dino - do you want me to register a workitem? Also, if I wanted to patch
v1.1.1, where should I start?

On Thu, Feb 21, 2008 at 8:48 PM, Dino Viehland <[EMAIL PROTECTED]>
wrote:

> After Birsh's last comment that they're calling compile I was thinking we
> might be doing something really stupid.  Turns out that we are.  For our
> supposedly compiled regex's we're not passing the Compiled option.  So
> they're all getting interpretted and of course that's going to suck big
> time.
>
> So the fix for this is probably to just pass RegexOptions.Compiled to the
> Regex object being contructed into RE_Pattern.  Internally the Regex class
> will compile to dynamic methods (much better than compiling to an assembly
> because it'll be collectible).
>
> It's actually a little more complex than that because we create RE_Pattern
> objects for the non-compiled so we need to flow the option in, but it's
> still a simple fix.
>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:
> [EMAIL PROTECTED] On Behalf Of Dan Shechter
> Sent: Thursday, February 21, 2008 10:19 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Slow Performance of CPython libs?
>
> Just out of curiosity...
> I'm guessing that there are two levels of caching that could be used.
> One would be to hold a global dictionary of some sort of pattern ->
> Regex that will save the lengthy parsing of the re.
>
> The other would be to actually call Regex.CompileToAssembly to get a
> more efficient reperesentation (in runtime that is) of the re.
>
> Would you (the IP team) favor the former or the latter?
>
>Shechter.
>
>
> Birsch wrote:
> > I checked both sgmllib.py and BeautifulSoup.py - and it seems both are
> > reusing the same regexps (searched for re.compile).
> >
> > I think your suggestion is very relevant in this case. It makes sense to
> > replicate the "compile once use many" behavior that is commonly used
> > with regexp.
> >
> > -Birsch
> >
> > On Thu, Feb 21, 2008 at 7:30 PM, Dino Viehland
> > <[EMAIL PROTECTED] >
> wrote:
> >
> > Do you know if the same reg ex is being used repeatedly?  If so
> > maybe we can cache & compile the regex instead of interpretting it
> > all the time.
> >
> >
> >
> > *From:* [EMAIL PROTECTED]
> > 
> > [mailto:[EMAIL PROTECTED]
> > ] *On Behalf Of *Birsch
> > *Sent:* Thursday, February 21, 2008 8:30 AM
> >
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] Slow Performance of CPython libs?
> >
> >
> >
> > I took on Cooper's advice and profiled with ANTS. Here are the top
> > methods:
> >
> > *Namespace*
> >
> >
> >
> > *Method name*
> >
> >
> >
> > *Time (sec.)*
> >
> >
> >
> > *Time with children (sec.)*
> >
> >
> >
> > *Hit count*
> >
> >
> >
> > *Source file*
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Go()
> >
> >
> >
> > 37.0189
> >
> >
> >
> > 94.4676
> >
> >
> >
> > 13689612
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Operator()
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 6.2411
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Advance(int i)
> >
> >
> >
> > 5.9264
> >
> >
> >
> > 8.7202
> >
> >
> >
> > 66000263
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.SetOperator(int op)
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 5.5750
> >
> >
> >
> > 131146274
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexInterpreter.Backtrack()
> >
> >
> >
> > 5.5692
> >
> >
> >
> > 9.4895
> >
> >
> >
> > 37781343
> >
> >
> >
> > IronPython.Runtime.Operations
> >
> >
> >
> > Ops.CallWithContext(ICallerContext context, object func, object
> > arg0, object arg1)
> >
> >
> >
> > 5.5572
> >
> >
> >
> > 114.5245
> >
> >
> >
> > 79754
> >
> >
> >
> > IronPython.Runtime.Calls
> >
> >
> >
> > Method.Call(ICallerContext context, object arg0)
> >
> >
> >
> > 4.9052
> >
> >
> >
> > 114.8251
> >
> >
> >
> > 50886
> >
> >
> >
> > IronPython.Runtime.Calls
> >
> >
> >
> > PythonFunction.CallInstance(ICallerContext context, object arg0,
> > object arg1)
> >
> >
> >
> > 4.8876
> >
> >
> >
> > 114.8059
> >
> >
> >
> > 50886
> >
> >
> >
> > IronPython.Runtime.Calls
> >
> >
> >
> > Function2.Call(ICallerContext context, object arg0, object arg1)
> >
> >
> >
> > 4.6400
> >
> >
> >
> > 114.5471
> >
> >
> >
> > 47486
> >
> >
> >
> > IronPython.Runtime.Operations
> >
> >
> >
> > Ops.CallWithContext(ICallerContext context, object func, object
> arg0)
> >
> >
> >
> > 4.2344
> >
> >
> >
> > 114.1604
> >
> >
> >
> > 146658
> >
> >
> >
> > System.Text.RegularExpressions
> >
> >
> >
> > RegexBoyerMo

Re: [IronPython] Hosting IronPython 2.0 Alpha 8

[Dave Fugate]

The tests in the source IronPython distributions should be up to date with 
respect to the releases.  While these aren't as eloquent as the blog entry 
referenced below, they should give you a good idea of what the IronPython 
hosting API is like in the release you're using.  For example, 
IronPythonTest\EngineTest.cs and Tests\test_ipye.py are a couple of our hosting 
tests found in IronPython-2.0A8-Src.zip.

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fernando Correia
Sent: Friday, February 22, 2008 6:52 AM
To: users@lists.ironpython.com
Subject: [IronPython] Hosting IronPython 2.0 Alpha 8

I am trying out IronPython hosting on C#. I found out an article about hosting 
with 2.0 Alpha 6:

http://blogs.msdn.com/rdawson/archive/2007/11/29/hosting-ironpython-2-0-alpha-6-via-the-dlr.aspx

But those instructions don't seem to work with 2.0 Alpha 8. I've searched the 
Web but I couldn't find updated instructions.

If someone can give me a pointer, I'd appreciate.
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Hosting IronPython 2.0 Alpha 8

[Curt Hagenlocher]

Sorry... the type of "theClass" should actually be PythonType and not
object.

On Fri, Feb 22, 2008 at 8:18 AM, Curt Hagenlocher <[EMAIL PROTECTED]>
wrote:

> Here's a snippet that I'm using under A8.  I can't promise that everything
> is "right", but it works correctly for how I'm using it.
>
> // Load Class1.py into a new scope
> SourceUnit source =
> PythonEngine.CurrentEngine.CreateScriptSourceFromFile("Class1.py");
> IScriptScope m = PythonEngine.CurrentEngine.CreateScope();
> PythonEngine.CurrentEngine.Execute(m, source);
>
> // Get the class object for "Class1"
> object value;
> m.TryGetVariable("Class1", out value);
> object theClass = (value as PythonType);
>  // Instantiate "Class1"
> LanguageContext language = LanguageContext.FromEngine(
> PythonEngine.CurrentEngine);
> Scope scope = new Scope(language);
> CodeContext context = new CodeContext(scope, language);
> object theValue = new PythonTypeOps.TypeCaller(theClass).Call(context2,
> new object[0]);
>
> // Call the "Clone function on the new instance
> object theFunction = 
> PythonEngine.CurrentEngine.Operations.GetMember(theValue,
> "Clone");
> object result = PythonEngine.CurrentEngine.Operations.Call(theFunction,
> new object[] { s, i });
> result = PythonEngine.CurrentEngine.Operations.ConvertTo(result);
>
>   On Fri, Feb 22, 2008 at 6:51 AM, Fernando Correia <
> [EMAIL PROTECTED]> wrote:
>
> >  I am trying out IronPython hosting on C#. I found out an article about
> > hosting with 2.0 Alpha 6:
> >
> >
> > http://blogs.msdn.com/rdawson/archive/2007/11/29/hosting-ironpython-2-0-alpha-6-via-the-dlr.aspx
> >
> > But those instructions don't seem to work with 2.0 Alpha 8. I've
> > searched the Web but I couldn't find updated instructions.
> >
> > If someone can give me a pointer, I'd appreciate.
> >
> > ___
> > Users mailing list
> > Users@lists.ironpython.com
> > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> >
> >
>
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] Hosting IronPython 2.0 Alpha 8

[Curt Hagenlocher]

Here's a snippet that I'm using under A8.  I can't promise that everything
is "right", but it works correctly for how I'm using it.

// Load Class1.py into a new scope
SourceUnit source =
PythonEngine.CurrentEngine.CreateScriptSourceFromFile("Class1.py");
IScriptScope m = PythonEngine.CurrentEngine.CreateScope();
PythonEngine.CurrentEngine.Execute(m, source);

// Get the class object for "Class1"
object value;
m.TryGetVariable("Class1", out value);
object theClass = (value as PythonType);
 // Instantiate "Class1"
LanguageContext language = LanguageContext.FromEngine(
PythonEngine.CurrentEngine);
Scope scope = new Scope(language);
CodeContext context = new CodeContext(scope, language);
object theValue = new PythonTypeOps.TypeCaller(theClass).Call(context2,
new object[0]);

// Call the "Clone function on the new instance
object theFunction =
PythonEngine.CurrentEngine.Operations.GetMember(theValue,
"Clone");
object result = PythonEngine.CurrentEngine.Operations.Call(theFunction,
new object[] { s, i });
result = PythonEngine.CurrentEngine.Operations.ConvertTo(result);

On Fri, Feb 22, 2008 at 6:51 AM, Fernando Correia <
[EMAIL PROTECTED]> wrote:

> I am trying out IronPython hosting on C#. I found out an article about
> hosting with 2.0 Alpha 6:
>
>
> http://blogs.msdn.com/rdawson/archive/2007/11/29/hosting-ironpython-2-0-alpha-6-via-the-dlr.aspx
>
> But those instructions don't seem to work with 2.0 Alpha 8. I've searched
> the Web but I couldn't find updated instructions.
>
> If someone can give me a pointer, I'd appreciate.
>
> ___
> Users mailing list
> Users@lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[IronPython] Hosting IronPython 2.0 Alpha 8

[Fernando Correia]

I am trying out IronPython hosting on C#. I found out an article about
hosting with 2.0 Alpha 6:

http://blogs.msdn.com/rdawson/archive/2007/11/29/hosting-ironpython-2-0-alpha-6-via-the-dlr.aspx

But those instructions don't seem to work with 2.0 Alpha 8. I've searched
the Web but I couldn't find updated instructions.

If someone can give me a pointer, I'd appreciate.
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Re: [IronPython] FW: FW: FW: returning inherited classes to .NET

[Curt Hagenlocher]

I built a test executable and was able to reproduce this.  And when I
changed
clr.AddReferenceToFile("WebControlLibrary.dll")
to
clr.AddReference("WebControlLibrary")
it worked as expected.
On Fri, Feb 22, 2008 at 6:02 AM, Curt Hagenlocher <[EMAIL PROTECTED]>
wrote:

> Ah, no.  I was thinking that you would modify your VB code to launch it as
> a console app (for instance) rather than from inside of 
> ASP.NET-- and see if you still get the same error.
>
>
> On Thu, Feb 21, 2008 at 10:07 PM, Howland-Rose, Kyle <
> [EMAIL PROTECTED]> wrote:
>
> >  Thanks for your help so late Curt (not that I need a quick answer at
> > all).
> >
> > I am sure I am missing something big-time here and "talking" as
> > cross-purposes, but, to my mind the reason I don't have a problem when
> > running from the command line is because all I am doing is running the
> > ipy.exe on the file which loads some assemblies, creates a class
> > definition and a function definition and then terminates without even
> > instantiating the class.  Is this what you mean by "running it
> > directly"?
> >
> > Thanks again,
> > Kyle
> >
> >  --
> >  *From:* [EMAIL PROTECTED] [mailto:
> > [EMAIL PROTECTED] *On Behalf Of *Curt Hagenlocher
> > *Sent:* Friday, 22 February 2008 4:55 PM
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] FW: FW: returning inherited classes to .NET
> >
> >   If you're only getting the error under ASP.NET , then
> > it's highly likely that you're loading multiple copies of the same assembly,
> > which is why the CLR can't cast your Python class back to the VB base -- it
> > literally considers them to be two different classes.  This may be related
> > to the way ASP.NET  makes shadow copies of your files,
> > but there could be other reasons as well (none of which, unfortunately, come
> > to mind right now).
> >
> > Using an interface instead of a base class would give you the same
> > results.
> > On Thu, Feb 21, 2008 at 9:44 PM, Howland-Rose, Kyle <
> > [EMAIL PROTECTED]> wrote:
> >
> > >  It runs fine from the command line but then it does not do much ...
> > > yet.  For the moment I am simplifying it by removing IP :(
> > >
> > > I wanted to so some moderately heavy reflection which to my thinking
> > > should be easier from IP but I will use VB for now.
> > >
> > > I assume I *should* be able to return a derived python class to VB?
> > > Alternatively I could implement an interface.  How can IP implement a .NET
> > > interface?
> > > (All pointers to pre-existing doco warmly received :)
> > >
> > > Thanks
> > > Kyle
> > >
> > >  --
> > >  *From:* [EMAIL PROTECTED] [mailto:
> > > [EMAIL PROTECTED] *On Behalf Of *Curt Hagenlocher
> > > *Sent:* Friday, 22 February 2008 4:29 PM
> > > *To:* Discussion of IronPython
> > > *Subject:* Re: [IronPython] FW: returning inherited classes to .NET
> > >
> > >   My apologies; I wasn't reading the code correctly.  That's probably
> > > a sign that I should be going to bed. :)  Have you tried simplifying your
> > > version just a little bit by running it directly rather than through
> > > ASP.NET ?
> > >
> > >
> > > On Thu, Feb 21, 2008 at 8:43 PM, Howland-Rose, Kyle <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Hi Curt,
> > > >
> > > > Thanks for replying.  When I view the link I see the following:
> > > >
> > > > The .NET code creates the "Test" class and then runs the python code
> > > > which imports the "Test" class a subclasses it to create python
> > > > class
> > > > "X".
> > > >
> > > > Class "X" is then called and cast to type "Test" and assigned to the
> > > > value obj.
> > > >
> > > > Is this what you are looking at?
> > > >
> > > > My code is pretty much identical.  The VB looks like:
> > > >
> > > >
> > > > Public MustInherit Class ChildControl
> > > >Public Function Control() As WebControl
> > > >Return Nothing
> > > >End Function
> > > >Public Function GetValue() As String
> > > >Return Nothing
> > > >End Function
> > > >Public Function SetValue(ByVal value As Object) As String
> > > >Return Nothing
> > > >End Function
> > > > End Class
> > > >
> > > > Class IPReflection
> > > >Private Shared Instance As IPReflection = Nothing
> > > >Private PyEngine As PythonEngine
> > > >Private PyModule As EngineModule
> > > >
> > > >Public Shared Function Inst() As IPReflection
> > > >If Instance Is Nothing Then
> > > >Instance = New IPReflection
> > > >End If
> > > >Return Instance
> > > >End Function
> > > >
> > > >Private Sub New()
> > > >Dim path As String = HttpContext.Current.Request.MapPath(".")
> > > >PyEngine = New PythonEngine
> > > >PyEngine.AddToPath(path)
> > > >PyModule = PyEngine.CreateModule
> > > >PyEngine.DefaultModule = PyModule
> > > >
> > > > PyEngine.ExecuteFile(HttpContext.

Re: [IronPython] FW: FW: FW: returning inherited classes to .NET

[Curt Hagenlocher]

Ah, no.  I was thinking that you would modify your VB code to launch it as a
console app (for instance) rather than from inside of ASP.NET -- and see if
you still get the same error.

On Thu, Feb 21, 2008 at 10:07 PM, Howland-Rose, Kyle <
[EMAIL PROTECTED]> wrote:

>  Thanks for your help so late Curt (not that I need a quick answer at
> all).
>
> I am sure I am missing something big-time here and "talking" as
> cross-purposes, but, to my mind the reason I don't have a problem when
> running from the command line is because all I am doing is running the
> ipy.exe on the file which loads some assemblies, creates a class
> definition and a function definition and then terminates without even
> instantiating the class.  Is this what you mean by "running it directly"?
>
>
> Thanks again,
> Kyle
>
>  --
>  *From:* [EMAIL PROTECTED] [mailto:
> [EMAIL PROTECTED] *On Behalf Of *Curt Hagenlocher
> *Sent:* Friday, 22 February 2008 4:55 PM
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] FW: FW: returning inherited classes to .NET
>
>   If you're only getting the error under ASP.NET , then
> it's highly likely that you're loading multiple copies of the same assembly,
> which is why the CLR can't cast your Python class back to the VB base -- it
> literally considers them to be two different classes.  This may be related
> to the way ASP.NET  makes shadow copies of your files,
> but there could be other reasons as well (none of which, unfortunately, come
> to mind right now).
>
> Using an interface instead of a base class would give you the same
> results.
> On Thu, Feb 21, 2008 at 9:44 PM, Howland-Rose, Kyle <
> [EMAIL PROTECTED]> wrote:
>
> >  It runs fine from the command line but then it does not do much ...
> > yet.  For the moment I am simplifying it by removing IP :(
> >
> > I wanted to so some moderately heavy reflection which to my thinking
> > should be easier from IP but I will use VB for now.
> >
> > I assume I *should* be able to return a derived python class to VB?
> > Alternatively I could implement an interface.  How can IP implement a .NET
> > interface?
> > (All pointers to pre-existing doco warmly received :)
> >
> > Thanks
> > Kyle
> >
> >  --
> >  *From:* [EMAIL PROTECTED] [mailto:
> > [EMAIL PROTECTED] *On Behalf Of *Curt Hagenlocher
> > *Sent:* Friday, 22 February 2008 4:29 PM
> > *To:* Discussion of IronPython
> > *Subject:* Re: [IronPython] FW: returning inherited classes to .NET
> >
> >   My apologies; I wasn't reading the code correctly.  That's probably a
> > sign that I should be going to bed. :)  Have you tried simplifying your
> > version just a little bit by running it directly rather than through
> > ASP.NET ?
> >
> >
> > On Thu, Feb 21, 2008 at 8:43 PM, Howland-Rose, Kyle <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi Curt,
> > >
> > > Thanks for replying.  When I view the link I see the following:
> > >
> > > The .NET code creates the "Test" class and then runs the python code
> > > which imports the "Test" class a subclasses it to create python class
> > > "X".
> > >
> > > Class "X" is then called and cast to type "Test" and assigned to the
> > > value obj.
> > >
> > > Is this what you are looking at?
> > >
> > > My code is pretty much identical.  The VB looks like:
> > >
> > >
> > > Public MustInherit Class ChildControl
> > >Public Function Control() As WebControl
> > >Return Nothing
> > >End Function
> > >Public Function GetValue() As String
> > >Return Nothing
> > >End Function
> > >Public Function SetValue(ByVal value As Object) As String
> > >Return Nothing
> > >End Function
> > > End Class
> > >
> > > Class IPReflection
> > >Private Shared Instance As IPReflection = Nothing
> > >Private PyEngine As PythonEngine
> > >Private PyModule As EngineModule
> > >
> > >Public Shared Function Inst() As IPReflection
> > >If Instance Is Nothing Then
> > >Instance = New IPReflection
> > >End If
> > >Return Instance
> > >End Function
> > >
> > >Private Sub New()
> > >Dim path As String = HttpContext.Current.Request.MapPath(".")
> > >PyEngine = New PythonEngine
> > >PyEngine.AddToPath(path)
> > >PyModule = PyEngine.CreateModule
> > >PyEngine.DefaultModule = PyModule
> > >
> > > PyEngine.ExecuteFile(HttpContext.Current.Request.MapPath
> > > ("CustomControls
> > > .py"))
> > >End Sub
> > >
> > >Public Function GetChildControl() As ChildControl
> > >Return
> > > CType(Operations.Ops.Call(PyModule.Globals("GetControl")),
> > > ChildControl)
> > >End Function
> > > End Class
> > >
> > > And the python looks like:
> > >
> > > import clr
> > > import sys
> > >
> > > clr.AddReference('System')
> > > import System
> > >
> > > sys.path.append(sys.path[0] + "/bin")
> > >
> > > clr.AddReferenceToFile("WebControlLibrary.dll"