Re: [Python-Dev] Ordering keyword dicts
On Mon, May 20, 2013 at 6:39 AM, Barry Warsaw wrote: > Or in other words, if dicts are to be ordered, let's make it an explicit > language feature that we can measure compliance against. Guaranteeing a dict order would be tough on Jython - today it's nice that we can just have a thin wrapper around ConcurrentHashMap. In a world with hard ordering guarantees I think we'd need to write our own from scratch. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPy, Jython, & IronPython: Enum convenience function and pickleablity
On Thu, May 2, 2013 at 12:07 PM, Ethan Furman wrote: > In order for the Enum convenience function to be pickleable, we have this > line of code in the metaclass: > > enum_class.__module__ = sys._getframe(1).f_globals['__name__'] > > This works fine for Cpython, but what about the others? This should work for Jython, but I can't say I like it. I believe IronPython has a sort of speedup mode that disallows the use of _getframe, and I'd like to add this to Jython someday. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
On Fri, Mar 29, 2013 at 11:33 PM, Simon Cross wrote: > Having a standalone version of IDLE might be really useful to > alternative Python implementations. I suspect it's too hard. I remember seeing some work on "anygui.py" that looked like an attempt to make these sorts of things work across various windowing platforms, but I don't think it made it very far. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Federated repo model [was: IDLE in the stdlib]
On Thu, Mar 21, 2013 at 8:23 AM, Nick Coghlan wrote: > I think a federated repo model in general is something we need to > consider, it's not something we should consider IDLE specific. I would love to have a federated repo model. I have recently made the attempt to port the devguide for CPython to Jython with some reasonable success. Part of that success has come because the devguide is in its own repo and so forking it and continuing to merge improvements from the original has been very easy. I'd love to be able to do the same for the Doc/ directory at the root of the CPython repo, but currently would have to fork the entire code and doc repository etc. This would mean that merging the Doc/ improvements would be a big pain, with lots and lots of useless merges where it would be hard to pick out the Doc changes. To a lesser extent the same would hold for the Lib/ area - though in that case I don't mind just pushing our changes to the CPython Lib/ (through the tracker and with code reviews of course) in the medium term. Still, a separate repo for Lib would definitely be nice down the road. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
Hi all, I won't be able to make it to the summit and probably not the conference. I have a raging 104F fever (40C for many of you =) The doctor says I have influenza and am highly contagious, so I shouldn't be going anywhere near conference full of people for five days - looks like I'm missing this one :( I may make it down for a couple of sprint days. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Tue, Mar 5, 2013 at 8:55 AM, fwierzbi...@gmail.com wrote: > I've been thinking that this is a bit of a historical mistake on our > part. I'm strongly considering setting os.name properly in Jython3. In fairness to Jython implementers past - it wasn't a mistake but a deliberate design choice at the time since in the old days we where going for the now defunct "100% Java" thing (does anyone remember that?) so this is more of "the design has evolved such that Jython's os.name now feels incorrect". I'll stop having conversations with myself in public now, sorry :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Mon, Mar 4, 2013 at 9:39 PM, Jeff Hardy wrote: > I think you misremembered - there's lots of code that uses > `sys.platform == 'win32'` to detect Windows, but sys.platform is 'cli' > for IronPython. I'm pretty sure `os.name has always been 'nt' (when > running on Windows), and if not, it definitely is now. > > Jython sets os.name to 'java' (IIRC), so there isn't a uniform way to > detect Windows across all implementations. I've been thinking that this is a bit of a historical mistake on our part. I'm strongly considering setting os.name properly in Jython3. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging Jython code into standard Lib [was Re: Python Language Summit at PyCon: Agenda]
On Thu, Feb 28, 2013 at 12:35 PM, Brett Cannon wrote: > > On Thu, Feb 28, 2013 at 3:17 PM, fwierzbi...@gmail.com > wrote: >> >> It would be nice in this particular case if there was a zlib.py that >> imported _zlib -- then it would be easy to shim in Jython's version, >> whether it is written in a .py file or in Java. > > > That should be fine as that is what we already do for accelerator modules > anyway. If you want to work towards having an equivalent of CPython's > Modules/ directory so you can ditch your custom Lib/ modules by treating > your specific code as accelerators I think we can move towards that > solution. Sounds great! I'm betting that implementing PEP 420 on Jython will make mixed Python/Java code easier to deal with, so _zlib.py might just end up living next to our Java code. So deleting Jython's Lib/ may still be an option. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging Jython code into standard Lib [was Re: Python Language Summit at PyCon: Agenda]
On Thu, Feb 28, 2013 at 12:00 PM, Antoine Pitrou wrote: > IMHO, we should remove the plat-* directories, they are completely > unmaintained, undocumented, and serve no useful purpose. Oh I didn't know that - so definitely adding to that is right out :) Really for cases like Jython's zlib.py (no useful code for CPython) I don't have any trouble keeping them entirely in Jython. It just would have been fun to delete our Lib/ :) It would be nice in this particular case if there was a zlib.py that imported _zlib -- then it would be easy to shim in Jython's version, whether it is written in a .py file or in Java. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging Jython code into standard Lib [was Re: Python Language Summit at PyCon: Agenda]
On Thu, Feb 28, 2013 at 11:46 AM, fwierzbi...@gmail.com wrote: >> Agreed on those problems. Would it be possible to use a design >> pattern in these cases so the Jython-only code wouldn't need to be >> part of the CPython repo? A naive example would be refactoring zlib >> to allow subclassing in the way that Jython needs, and then Jython >> could subclass in its own repo. CPython could have tests to check the >> subclass "contract" that Jython needs. > What about a plat-java section to parallel plat-aix4, plat-darwin, > etc? The analogy being that the Java platform is somewhat analogous to > being it's own os? And these areas are not active when on other > operating systems... Oh yeah and this does not preclude the zlib refactoring, and again for the record once this gets serious I'd want to help in bringing up a Jython buildbot to track breakage. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging Jython code into standard Lib [was Re: Python Language Summit at PyCon: Agenda]
On Thu, Feb 28, 2013 at 11:24 AM, Chris Jerdonek wrote: > On Thu, Feb 28, 2013 at 1:30 AM, Antoine Pitrou wrote: >> Le Wed, 27 Feb 2013 11:33:30 -0800, >> "fwierzbi...@gmail.com" a écrit : >>> >>> There are a couple of spots that might be more controversial. For >>> example, Jython has a file Lib/zlib.py that implements zlib in terms >>> of the existing Java support for zlib. I do wonder if such a file is >>> acceptable in CPython's Lib since its 195 lines of code would be >>> entirely skipped by CPython. >> >> That's a bit annoying. How will we know that the code still works, even >> though our buildbots don't exercise it? >> Also, what happens if the code doesn't work anymore? > > Agreed on those problems. Would it be possible to use a design > pattern in these cases so the Jython-only code wouldn't need to be > part of the CPython repo? A naive example would be refactoring zlib > to allow subclassing in the way that Jython needs, and then Jython > could subclass in its own repo. CPython could have tests to check the > subclass "contract" that Jython needs. What about a plat-java section to parallel plat-aix4, plat-darwin, etc? The analogy being that the Java platform is somewhat analogous to being it's own os? And these areas are not active when on other operating systems... -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Thu, Feb 28, 2013 at 1:30 AM, Antoine Pitrou wrote: > Le Wed, 27 Feb 2013 11:33:30 -0800, > "fwierzbi...@gmail.com" a écrit : >> >> There are a couple of spots that might be more controversial. For >> example, Jython has a file Lib/zlib.py that implements zlib in terms >> of the existing Java support for zlib. I do wonder if such a file is >> acceptable in CPython's Lib since its 195 lines of code would be >> entirely skipped by CPython. > > That's a bit annoying. How will we know that the code still works, even > though our buildbots don't exercise it? > Also, what happens if the code doesn't work anymore? That is definitely something to think about. Maybe when this becomes a serious effort and not just talk I can help get a Jython buildbot going. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Thu, Feb 28, 2013 at 1:15 AM, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 1:37 PM, Barry Warsaw wrote: >> On Feb 27, 2013, at 11:33 AM, fwierzbi...@gmail.com wrote: >>>The easy part for Jython is pushing some of our "if is_jython:" stuff >>>into the appropriate spots in CPython's Lib/. >> >> I wonder if there isn't a better way to do this than sprinkling is_jython, >> is_pypy, is_ironpython, is_thenextbigthing all over the code base. I have no >> bright ideas here, but it seems like a feature matrix would be a better way >> to >> go than something that assumes a particular Python implementation has a >> particular feature set (which may change in the future). > > Yes, avoiding that kind of thing is a key motivation for > sys.implementation. Any proposal for "is_jython" blocks should instead > be reformulated as a proposal for new sys.implementation attributes. Ah nice - the merging effort should definitely cause some careful consideration of these. Maybe I'll start a discussion about a "garbage collection type" for sys.implementation. Some way to ask would catch some of these. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 7:37 PM, Barry Warsaw wrote: > On Feb 27, 2013, at 11:33 AM, fwierzbi...@gmail.com wrote: > >>I am suggesting that we push forward on the "shared library" approach to the >>files in the Lib/* directory, so that would certainly include IronPython and >>PyPy as well I hope. > > +1 > >>The easy part for Jython is pushing some of our "if is_jython:" stuff >>into the appropriate spots in CPython's Lib/. > > I wonder if there isn't a better way to do this than sprinkling is_jython, > is_pypy, is_ironpython, is_thenextbigthing all over the code base. I have no > bright ideas here, but it seems like a feature matrix would be a better way to > go than something that assumes a particular Python implementation has a > particular feature set (which may change in the future). Sorry I meant "is_jython" as a sort of shorthand for a case by case check. It would be cool if we had a nice set of checks somewhere like "is_refcounted", etc. Would the sys.implementation area be a good place for such things? On the other hand in some ways Jython is sort of like Python on a weird virtual OS that lets the real OS bleed through some. This may still need to be checked in that way (there's are still checks of right?) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 11:21 AM, Brett Cannon wrote: > Do you mean more generally getting more pure Python implementations of > modules in the stdlib? If so then as a reference there is > http://bugs.python.org/issue16651 which lists the modules in the stdlib w/ > only extension module implementations (although operator and random have > patches; the latter is waiting for Raymond to approve). That is part of it, although my bigger goal is slightly more ambitious. I'm hoping to eventually delete the entire Lib/ directory from Jython and just pull in CPython's. > And if I am right > about what you are suggesting, Frank, this should probably be expanded to a > more concerted effort with IronPython and PyPy. IOW it warrants a > discussion. =) Oh sure sorry - I have some tunnel vision lately :) I am suggesting that we push forward on the "shared library" approach to the files in the Lib/* directory, so that would certainly include IronPython and PyPy as well I hope. The easy part for Jython is pushing some of our "if is_jython:" stuff into the appropriate spots in CPython's Lib/. I'd also like to do something at the top of CPython specific .py files that would fail the import in case it is called from Jython. I suspect that OS packagers would like it if the Lib directory for a particular Python version could be entirely shared between implementations. There are a couple of spots that might be more controversial. For example, Jython has a file Lib/zlib.py that implements zlib in terms of the existing Java support for zlib. I do wonder if such a file is acceptable in CPython's Lib since its 195 lines of code would be entirely skipped by CPython. Anyway I think I might be rambling - it seems like a good thing to discuss :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 8:51 AM, Michael Foord wrote: > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. I'd like to discuss merging Jython's standard Lib (the *.py files). We have in the past had agreement that this would be a good idea - I just want to bring it up since I think this is probably the year that I'm actually going to do it. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Usage of += on strings in loops in stdlib
On Tue, Feb 12, 2013 at 1:03 PM, Maciej Fijalkowski wrote: > Hi > > We recently encountered a performance issue in stdlib for pypy. It > turned out that someone commited a performance "fix" that uses += for > strings instead of "".join() that was there before. > > Now this hurts pypy (we can mitigate it to some degree though) and > possible Jython and IronPython too. Just to confirm Jython does not have optimizations for += String and will do much better with the idiomatic "".join(). -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More compact dictionaries with faster iteration
On Mon, Dec 10, 2012 at 3:13 PM, fwierzbi...@gmail.com wrote: > On Mon, Dec 10, 2012 at 10:01 AM, Armin Rigo wrote: >> Technically, I could see Python switching to ordered dictionaries >> everywhere. Raymond's insight suddenly makes it easy for CPython and >> PyPy, and at least Jython could use the LinkedHashMap class (although >> this would need checking with Jython guys). > I honestly hope this doesn't happen - we use ConcurrentHashMap for our > dictionaries (which lack ordering) and I'm sure getting it to preserve > insertion order would cost us. I just found this http://code.google.com/p/concurrentlinkedhashmap/ so maybe it wouldn't be all that bad. I still personally like the idea of leaving basic dict order undetermined (there is already an OrderedDict if you need it right?) But if ConcurrentLinkedHashMap is as good as is suggested on that page then Jython doesn't need to be the thing that blocks the argument. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More compact dictionaries with faster iteration
On Mon, Dec 10, 2012 at 10:01 AM, Armin Rigo wrote: > Technically, I could see Python switching to ordered dictionaries > everywhere. Raymond's insight suddenly makes it easy for CPython and > PyPy, and at least Jython could use the LinkedHashMap class (although > this would need checking with Jython guys). I honestly hope this doesn't happen - we use ConcurrentHashMap for our dictionaries (which lack ordering) and I'm sure getting it to preserve insertion order would cost us. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
Thanks Brett, that cleared everything up for me! And indeed it is what I'm thinking of doing for Jython (Minimal nodes for the compiler and parallel PyObjects for Python). -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
On Mon, Aug 13, 2012 at 3:10 PM, Brett Cannon wrote: > Direct. There is an AST grammar file that gets compiled into C and Python > objects which are used by the compiler (c version) or exposed to users > (Python version). At the risk of making you repeat yourself, and just to be sure I understand: There are C objects used by the compiler and Python objects that are exposed to the users (written in C though) that are generated by the AST grammar. That at least sounds like they are different. The last I checked the grammar was Python.asdl and the translater was asdl_c.py resulting in /Python/Python-ast.c which looks like it is the implementation of _ast.py Are the AST objects from Python-ast.c used by the compiler? And what is the relationship between Python-ast.c and /Python/ast.c? And what about the CST mentioned at the top of /Python/ast.c? I ask all of this because I want to be sure that separating the internal AST in Jython from the one exposed in ast.py is really a good idea. If CPython does not make this distinction that will be a strike against the idea. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
On Mon, Aug 13, 2012 at 1:46 PM, Guido van Rossum wrote: > On Mon, Aug 13, 2012 at 1:05 PM, fwierzbi...@gmail.com > wrote: >> On Mon, Aug 13, 2012 at 12:06 PM, Brett Cannon wrote: >>>> I see nothing about ast possibly being CPython only. Should there be? >>> >>> >>> Time to ask the other VMs what they are currently doing (the ast module came >>> into existence in Python 2.6 so all the VMs should be answer the question >>> since Jython is in alpha for 2.7 compatibility). > > [Jython] >> 2.5+ contains an ast.py that I obsessively compared to CPython's 2.5 >> ast.py. > > But CPython's ast.py contains very little code -- it's all done in ast.c. What I did was dump a pretty print of the ast from Jython and CPython for every file in Lib/* and diff the results with a script. I got the differences down to a small number of minor variations. > Still, I'm glad you are actually considering this a cross-language > feature, and I will gladly retract my warning. (Still, I don't know if > it is subject to the usual backward compatibility constraints.) I don't know if IronPython does the same though... we might want to wait for them to respond. > It might be pure python for Jython, but it's not for CPython. It's actually Java for us :) -- in fact the internal AST uses the exact Java that is exposed from our _ast.py - which I've come to regard as a mistake (though it was useful at the time). I want to do the same obsessive diff game with 3.x but then probably separate out our internal ast implementation (possibly making ast.py pure Python). BTW - is Python's internal AST exactly exposed by ast.py or is there a separate internal AST implementation? -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
On Mon, Aug 13, 2012 at 1:05 PM, fwierzbi...@gmail.com wrote: > 2.5+ contains I should have said *Jython* 2.5+ -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
On Mon, Aug 13, 2012 at 12:06 PM, Brett Cannon wrote: >> I see nothing about ast possibly being CPython only. Should there be? > > > Time to ask the other VMs what they are currently doing (the ast module came > into existence in Python 2.6 so all the VMs should be answer the question > since Jython is in alpha for 2.7 compatibility). 2.5+ contains an ast.py that I obsessively compared to CPython's 2.5 ast.py. I haven't applied the same obsessiveness to 2.7, but I do intend to look closely at Jython's ast.py results compared to CPython's in the 3.x effort. Also I plan to allow some backwards compatibility compromises between early point releases of our 2.7 series, as I want to apply what I learn in our 3.x effort to 2.7 point releases, so we should be able to keep up with most simple ast.py changes. I'm not so sure that the current discussion are going to be "simple though" :) -- if it's pure python we should hopefully be alright. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What's New in Python 3.3
On Wed, Aug 8, 2012 at 12:54 AM, Raymond Hettinger wrote: > Hello all, > > I'll soon be starting the edits of Whatsnew for 3.3. > > When I did this for 3.2, it took over 150 hours of work to research all the > changes. This time there are many more changes, so my previous process won't > work (reviewing every "new in 3.3" entry in the docs, every entry in the > voluminous Misc/NEWS file, etc). > Thanks for all of this work! And thanks to A.M. Kuchling and everyone else that goes through the effort to make the "What's new in Python" documents so great. They are my high level roadmap for re-implementing in Jython. It would be so much harder without them. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM
On Tue, Jun 12, 2012 at 12:01 PM, Brett Cannon wrote: > > > On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow > wrote: >> >> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon wrote: >> > I should mention another option is to add sys.dont_read_bytecode (I >> > think I >> > have discussed this with Frank at some point). >> >> Or check for "sys.implementation.cache_tag is None"... > > > Perfect! Will that work for Jython (Franke) and IronPython (Jeff)? So Jython does actually emit bytecodes, but they are Java bytecodes instead of Python bytecodes. Right now they end up next to the .py files just like .pyc files. They have the possibly unfortunate naming foo.py -> foo$py.class -- If I understand cache_tag (I may not) I guess Python 3 is putting the .pyc files into hidden subdirectories instead of putting them next to the .py files? If so we may do the same with our $py.class files. Incidentally we also have a mode for reading .pyc files -- though we haven't implementing writing them yet (we probably will eventually) I guess what I'm trying to say is that I don't know exactly how we will handle these new flags, but chances are we will use them (Again provided my guesses about what they do are anywhere near what they really do). > > This does mean, though, that imp.cache_from_source() and > imp.source_from_cache() might need to be updated to raise a reasonable > exception when sys.implementation.cache_tag is set to None as I believe > right now it will raise a TypeError because None isn't a str. But what to > raise instead? TypeError? EnvironmentError? NotImplementedError seems fine for me too if we don't end up using this flag. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM
On Tue, Jun 12, 2012 at 10:04 AM, fwierzbi...@gmail.com wrote: > Jython does support sys.dont_write_bytecode, but doesn't support > sys.dont_read_bytecode yet - do you happen to know when > dont_read_bytecode was added? It should be pretty straightforward, and > so I'll probably add it to our 2.7. Looking around it seems dont_read_bytecode came in sometime in 3.x so I'll wait for 3 (and so I'll probably just use importlib? :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM
On Tue, Jun 12, 2012 at 9:38 AM, Alex Gaynor wrote: > For PyPy: I'm not an expert in our import, but from looking at the source > > 1) imp.cache_from_source is unimplemented, it's an AttributeError. Jython does not (yet) have a cache_from_source. > 2) sys.dont_write_bytecode is always false, we don't respect that flag (we > really > should IMO, but it's not a high priority for me, or anyone else apparently) Jython does support sys.dont_write_bytecode, but doesn't support sys.dont_read_bytecode yet - do you happen to know when dont_read_bytecode was added? It should be pretty straightforward, and so I'll probably add it to our 2.7. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython
On Mon, Jun 11, 2012 at 10:46 PM, Nick Coghlan wrote: > To allow easier transition to a separate list (if that seems necessary > at a later date), my preferred colour for the bikeshed is > [compatibility-sig]. I for one approve of this bikeshed colour :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython
On Sun, Jun 10, 2012 at 11:58 PM, Nick Coghlan wrote: > 1. Asking on python-dev is considered adequate. If an implementation > wants to be consulted on changes, one or more of their developers > *must* follow python-dev sufficiently closely that they don't miss > cross-VM compatibility questions. (My concern is that this isn't > reliable - we know from experience that other VMs can miss such > questions when they're mixed in with the rest of the python-dev > traffic) > 2. As 1, but we adopt a subject line convention to make it easier to > filter out general python-dev traffic for those that are just > interested in cross-vm questions > 3. Create a separate list for cross-VM discussions, *including* > discussions that aren't directly relevant to Python-the-language or > CPython-the-reference-interpreter (e.g. collaborating on a shared > standard library fork). python-dev threads may be advertised on the > new list if cross-VM feedback is considered particularly necessary. (2) and (3) work for me - I try to do (1) but often miss discussions until they have gone stale. I bet (2) would work well enough as long as there are enough interested participants to remember to add the conventional string to the subject of an ongoing discussion. It would be very easy for me to add a filter for such a string. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython
On Fri, Jun 8, 2012 at 11:57 AM, Eric Snow wrote: > This would have been handy for feedback on sys.implementation. FWIW I followed the discussion and am happy with the result :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython
On Fri, Jun 8, 2012 at 10:59 AM, Brett Cannon wrote: > R. David already replied to this, but just to reiterate: tests can always > get updated, and code that fixes a bug (and leaving a file open can be > considered a bug) can also go in. It's just stuff like code refactoring, > speed improvements, etc. that can't go into Python 2.7 at this point. Thanks for the clarification! > If/until the stdlib is made into its own repo, should the various VMs > consider keeping a common Python 2.7 repo that contains nothing but the > stdlib (or at least just modifications to those) so they can modify in ways > that CPython can't accept because of compatibility policy? You could keep it > on hg.python.org (or wherever) and then all push to it. This might also be a > good way to share Python implementations of extension modules for Python 2.7 > instead of everyone maintaining there own for the next few years (although I > think those modules should go into the stdlib directly for Python 3 as > well). Basically this could be a test to see if communication and > collaboration will be high enough among the other VMs to bother with > breaking out the actual stdlib into its own repo or if it would just be a > big waste of time. I'd be up for trying this. I don't think it's easy to fork a subdirectory of CPython though - right now I just keep an unchanged copy of the 2.7 LIb in our repo (PyPy does the same, at least the last time I checked). > P.S. Do we need a python-implementations mailing list or something for > discussing overall VM-related stuff among all VMs instead of always bringing > this up on python-dev? E.g. I wish I had a place where I could get all the > VM stakeholders' attention to make sure that importlib as it stands in > Python 3.3 will skip trying to import Python bytecode properly (or if the > VMs will simply provide their own setup function and that won't be a worry). > And I would have no problem with keeping it like python-committers in terms > of closed subscriptions, open archive in order to keep the noise low. I think a python-implementations list would be a fantastic idea - I sometimes miss multi-implementation discussions in python-dev, or at least come in very late. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython
On Fri, Jun 8, 2012 at 9:22 AM, Jeff Hardy wrote: > On Fri, Jun 8, 2012 at 8:13 AM, Matti Picus wrote: >> >> The windows port of pypy makes special demands on stdlib, specifically that >> files are explicitly closed. There are some other minor issues, in order to >> merge all the changes necessary to get pypy windows up to speed, around 10 >> modules or at least their tests seem to need to be modified. >> I have been doing a bit of work on the stdlib shipped with pypy 1.9 >> (version 2.7.2 unfortunately) to make it compliant. Assuming there is >> interest, >> what would be the best path to get, for instance, a modified version of >> mailbox.py with its tests (test_mailbox.py and test_old_mailbox.py) >> backported >> to cpython? > > These fixes would also be useful for IronPython and possibly Jython as > well. Unclosed files are probably the biggest set of failures when > running CPython's tests on IronPython (along with assuming that > sys.platform == 'win32' means Windows). Whether or not they get > backported to CPython, it might be worth finding a way to share the > 2.7 stdlib between the alternative implementations (changes to 3.x > should go into CPython, obviously). I think it's supposed to be alright to push changes to CPython's 2.7 *tests* (like test_mailbox.py) but not other parts of the standard library (like mailbox.py) -- I'd love to find a way to share the modifications from various implementations though -- and in the 3.x future hopefully it can all just end up in CPython's Lib. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:39 PM, Victor Stinner wrote: > Hi, > > During the Language Summit 2011 (*), it was discussed that PyPy and > Jython don't support non-string key in type dict. An issue was open to > emit a warning on such dict, but the patch has not been commited yet. It should be noted that Jython started supporting non-string dict keys in version 2.5. IIRC this was done to get Django running, but in general we decided to go with compatibility. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 407 / splitting the stdlib
On Wed, Jan 18, 2012 at 9:56 AM, Brett Cannon wrote: > Doing a release every 6 months that includes updates to the stdlib and > bugfixes to the language/VM also benefits other VMs by getting compatibility > fixes in faster. All of the other VM maintainers have told me that keeping > the stdlib non-CPython compliant is the biggest hurdle. This kind of switch > means they could release a VM that supports a release 6 months or a year > after a language change release (e.g. 1 to 2 releases in) so as to get > changes in faster and lower the need to keep their own fork. As one of the other VM maintainers I agree with everything Brett has said here. The proposal sounds very good to me from that perspective. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
On Fri, Sep 9, 2011 at 2:21 PM, Guido van Rossum wrote: > I, for one, am very interested. It sounds like the 'unicode' datatype > in Jython does not in fact have O(1) indexing characteristics if the > string contains any characters in the astral plane. Interesting. I > wonder if you have heard from anyone about this affecting their app's > performance? So far we haven't had any complaints - I'm not really sure how often Jython gets used with astral plane characters at this point, but I expect it will happen more in the future, especially once we put together a Jython 3 and Unicode support becomes a stronger expectation. Personally I'm hoping that in that time frame Java will come under pressure to provide a better answer (or we may need to think in the same direction as Dino was thinking in an earlier part of this thread and make a more Python specific String type for Jython) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
On Fri, Sep 9, 2011 at 10:16 AM, Terry Reedy wrote:
> I am curious how you index by code point rather than code unit with 16-bit
> code units and how it compares with the method I posted. Is there anything I
> can read? Reply off list if you want.
I'll post on-list until someone complains, just in case there are
interested onlookers :)
There aren't docs, but the code is here:
https://bitbucket.org/jython/jython/src/8a8642e45433/src/org/python/core/PyUnicode.java
Here are (I think) the most relevant bits for random access -- note
that getString() returns the internal representation of the PyUnicode
which is a java.lang.String
@Override
protected PyObject pyget(int i) {
if (isBasicPlane()) {
return Py.makeCharacter(getString().charAt(i), true);
}
int k = 0;
while (i > 0) {
int W1 = getString().charAt(k);
if (W1 >= 0xD800 && W1 < 0xDC00) {
k += 2;
} else {
k += 1;
}
i--;
}
int codepoint = getString().codePointAt(k);
return Py.makeCharacter(codepoint, true);
}
public boolean isBasicPlane() {
if (plane == Plane.BASIC) {
return true;
} else if (plane == Plane.UNKNOWN) {
plane = (getString().length() == getCodePointCount()) ?
Plane.BASIC : Plane.ASTRAL;
}
return plane == Plane.BASIC;
}
public int getCodePointCount() {
if (codePointCount >= 0) {
return codePointCount;
}
codePointCount = getString().codePointCount(0, getString().length());
return codePointCount;
}
-Frank
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
On Thu, Sep 8, 2011 at 10:39 PM, Terry Reedy wrote: > On 9/8/2011 6:15 PM, fwierzbi...@gmail.com wrote: >> >> Oops, forgot to add the link for the gory details for Java and> 2 byte >> unicode: >> >> http://java.sun.com/developer/technicalArticles/Intl/Supplementary/ > > This is dated 2004. Basically, they considered several options, tried out 4, > and ended up sticking with char[] (sequences) as UTF-16 with char = 16 bit > code unit and added 32-bit Character(int) class for low-level manipulation > of code points. > > I did not see the indexing problem mentioned. I get the impression that they > encourage sequence forward-backward iteration (cursor-based access) rather > than random-access indexing. Hmmm, sorry for the irrelevant link - my lack of expertise here is showing. What I do know is that we (meaning Jim Baker) are taking great pains to always use codepoints even for random access in our unicode code. I can't speak to the performance implications without some deeper study into what Jim has done. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
On Fri, Aug 26, 2011 at 3:00 PM, Guido van Rossum wrote: > I have a different question about IronPython and Jython now. Do their > regular expression libraries support Unicode better than CPython's? > E.g. does "." match a surrogate pair? Tom C suggests that Java's regex > libraries get this and many other details right despite Java's use of > UTF-16 to represent strings. So hopefully Jython's re library is built > on top of Java's? Even bigger oops - I answered the thread questions and not this specific one. Currently Jython's re is a Jython specific implementation and so is not likely to benefit from the improvements in Java's re implementation. I think in terms of PEP 393 this should probably be considered a bug that we need to fix... -Frank Wierzbicki ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
Oops, forgot to add the link for the gory details for Java and > 2 byte unicode: http://java.sun.com/developer/technicalArticles/Intl/Supplementary/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393 Summer of Code Project
On Fri, Aug 26, 2011 at 3:00 PM, Guido van Rossum wrote: > I have a different question about IronPython and Jython now. Do their > regular expression libraries support Unicode better than CPython's? > E.g. does "." match a surrogate pair? Tom C suggests that Java's regex > libraries get this and many other details right despite Java's use of > UTF-16 to represent strings. So hopefully Jython's re library is built > on top of Java's? > > PS. Is there a better contact for Jython? The best contact for Unicode and Jython is Jim Baker (I added him to the cc) - I'll do my best to answer though: Java 5 added a bunch of methods for dealing with Unicode that doesn't fit into 2 bytes - and looking at our code for our Unicode object, I see that we are using methods like the codePointCount method off of java.lang.String to compute length[1] and using similar methods all through that code to make sure we deal in code points when dealing with unicode. So it looks pretty good for us as far as I can tell. [1] http://download.oracle.com/javase/6/docs/api/java/lang/String.html#codePointCount(int, int) -Frank Wierzbicki ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy for making changes to the AST
On Tue, Apr 5, 2011 at 6:37 AM, Nick Coghlan wrote: > 1. Making "docstring" an attribute of the Function node rather than > leaving it embedded as the first statement in the suite (this avoids > issues where AST-based constant folding could potentially corrupt the > docstring) > 2. Collapsing Num, Str, Bytes, Ellipsis into a single Literal node > type (the handling of those nodes is the same in a lot of cases) > 3. Since they're keywords now, pick up True, False, None at the > parsing stage and turn them into instances of the Literal node type, > allowing the current Name-based special casing to be removed. All of these sound like useful changes to me - I wouldn't want them blocked on Jython's account. We'll just implement them when we catch up to this version as far as I'm concerned. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy for making changes to the AST
As a re-implementor of ast.py that tries to be node for node compatible, I'm fine with #1 but would really like to have tests that will fail in test_ast.py to alert me! -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
