Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Mon, May 31, 2010 at 8:23 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, May 30, 2010 at 5:53 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, May 31, 2010 at 2:06 AM, Charles R Harris charlesr.har...@gmail.com wrote: Hey, I thought that was your job ;) Maybe the best thing is to start by making a branch for the removal and we can argue about who gets the short end of the stick later... Manage the release yes, do the heavy lifting not necessarily:) OK, I'll make the branch tonight. If the removal just involves the same changes that were made for 1.4.1 then I can do it. But I'm not familiar with this code and if it's more work, I probably won't have time for it between the scipy 0.8.0 release and my 'real job', like you call it. I think it may be a bit trickier because Travis made more changes. I think the relevant commits are r8113..r8115 and 8107..8108. After removing those we still need to remove the same stuff as we did for the 1.4.1 release. We will need some way of testing if the removal was successful. That still looks like it's not an insane amount of work. We probably want to make sure there is a documentation update also, maybe before making the branch. I checked and there's not too much to merge, most of the changes ( http://docs.scipy.org/numpy/patch/) don't apply cleanly or at all. The latter because they're docs for constants, lists, etc. The biggest chuck of recent changes is for the polynomial and chebyshev docs, can you give your opinion on those Charles? The OK to Apply is set to True for all of them, but I'm not sure who did that. The patch generation won't work for many of those docs, so if you could check if any docs should be merged manually right now that would be useful. @ David G: there are some conflicts in docs you recently edited, http://docs.scipy.org/numpy/merge/. Would you mind resolving those? Thanks, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Mon, May 31, 2010 at 4:54 AM, Ralf Gommers
ralf.gomm...@googlemail.comwrote:
On Mon, May 31, 2010 at 8:23 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, May 30, 2010 at 5:53 PM, Ralf Gommers
ralf.gomm...@googlemail.com wrote:
On Mon, May 31, 2010 at 2:06 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
Hey, I thought that was your job ;) Maybe the best thing is to start by
making a branch for the removal and we can argue about who gets the short
end of the stick later...
Manage the release yes, do the heavy lifting not necessarily:) OK, I'll
make the branch tonight.
If the removal just involves the same changes that were made for 1.4.1
then I can do it. But I'm not familiar with this code and if it's more work,
I probably won't have time for it between the scipy 0.8.0 release and my
'real job', like you call it.
I think it may be a bit trickier because Travis made more changes. I think
the relevant commits are r8113..r8115 and 8107..8108. After removing those
we still need to remove the same stuff as we did for the 1.4.1 release. We
will need some way of testing if the removal was successful.
That still looks like it's not an insane amount of work.
We probably want to make sure there is a documentation update also, maybe
before making the branch.
I checked and there's not too much to merge, most of the changes (
http://docs.scipy.org/numpy/patch/) don't apply cleanly or at all. The
latter because they're docs for constants, lists, etc.
The biggest chuck of recent changes is for the polynomial and chebyshev
docs, can you give your opinion on those Charles? The OK to Apply is set to
True for all of them, but I'm not sure who did that.
I'll go have a look. David (G) worked with me off-line so I expect they are
generally OK. The markup seems a bit excessive to me, but that is probably a
tradeoff between the appearance of the generated docs and the terminal help.
I don't know what is standard practice there these days.
The patch generation won't work for many of those docs, so if you could
check if any docs should be merged manually right now that would be useful.
The {Chebyshev, Polynomial} documentation that has conflicts can't be edited
on the web, as they reference documents generated from a template at module
load time. I'll fix that at some point, but at the moment it is easier to
maintain one template than to maintain two separate versions. I'll hand
merge anything there that looks appropriate.
@ David G: there are some conflicts in docs you recently edited,
http://docs.scipy.org/numpy/merge/. Would you mind resolving those?
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Mon, May 31, 2010 at 3:54 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Mon, May 31, 2010 at 8:23 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, May 30, 2010 at 5:53 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, May 31, 2010 at 2:06 AM, Charles R Harris charlesr.har...@gmail.com wrote: Hey, I thought that was your job ;) Maybe the best thing is to start by making a branch for the removal and we can argue about who gets the short end of the stick later... Manage the release yes, do the heavy lifting not necessarily:) OK, I'll make the branch tonight. If the removal just involves the same changes that were made for 1.4.1 then I can do it. But I'm not familiar with this code and if it's more work, I probably won't have time for it between the scipy 0.8.0 release and my 'real job', like you call it. I think it may be a bit trickier because Travis made more changes. I think the relevant commits are r8113..r8115 and 8107..8108. After removing those we still need to remove the same stuff as we did for the 1.4.1 release. We will need some way of testing if the removal was successful. That still looks like it's not an insane amount of work. We probably want to make sure there is a documentation update also, maybe before making the branch. I checked and there's not too much to merge, most of the changes ( http://docs.scipy.org/numpy/patch/) don't apply cleanly or at all. The latter because they're docs for constants, lists, etc. The biggest chuck of recent changes is for the polynomial and chebyshev docs, can you give your opinion on those Charles? The OK to Apply is set to True for all of them, but I'm not sure who did that. The patch generation won't work for many of those docs, so if you could check if any docs should be merged manually right now that would be useful. @ David G: there are some conflicts in docs you recently edited, http://docs.scipy.org/numpy/merge/. Would you mind resolving those? Nothing to resolve: those docstrings are generated via a template and should not be modified via the wiki at all - any differences between what's in the wiki and svn are inadvertent and can be ignored. DG Thanks, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Mathematician: noun, someone who disavows certainty when their uncertainty set is non-empty, even if that set has measure zero. Hope: noun, that delusive spirit which escaped Pandora's jar and, with her lies, prevents mankind from committing a general suicide. (As interpreted by Robert Graves) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Sat, May 29, 2010 at 9:05 AM, Pauli Virtanen p...@iki.fi wrote: Fri, 28 May 2010 12:13:57 -0500, Travis Oliphant wrote: [clip] I think this makes sense so that we can reschedule NumPy 2.0 for September and still provide a release with the Python 3k changes (I am assuming these can be done in an ABI-compatible way). As far as I remember, the Py3 changes did not require breaking ABI (on Py2). Good. How much work is it to remove datetime, and who wants to do it? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Sun, May 30, 2010 at 9:52 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Sat, May 29, 2010 at 9:05 AM, Pauli Virtanen p...@iki.fi wrote: Fri, 28 May 2010 12:13:57 -0500, Travis Oliphant wrote: [clip] I think this makes sense so that we can reschedule NumPy 2.0 for September and still provide a release with the Python 3k changes (I am assuming these can be done in an ABI-compatible way). As far as I remember, the Py3 changes did not require breaking ABI (on Py2). Good. How much work is it to remove datetime, and who wants to do it? Hey, I thought that was your job ;) Maybe the best thing is to start by making a branch for the removal and we can argue about who gets the short end of the stick later... Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Mon, May 31, 2010 at 2:06 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, May 30, 2010 at 9:52 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, May 29, 2010 at 9:05 AM, Pauli Virtanen p...@iki.fi wrote: Fri, 28 May 2010 12:13:57 -0500, Travis Oliphant wrote: [clip] I think this makes sense so that we can reschedule NumPy 2.0 for September and still provide a release with the Python 3k changes (I am assuming these can be done in an ABI-compatible way). As far as I remember, the Py3 changes did not require breaking ABI (on Py2). Good. How much work is it to remove datetime, and who wants to do it? Hey, I thought that was your job ;) Maybe the best thing is to start by making a branch for the removal and we can argue about who gets the short end of the stick later... Manage the release yes, do the heavy lifting not necessarily:) OK, I'll make the branch tonight. If the removal just involves the same changes that were made for 1.4.1 then I can do it. But I'm not familiar with this code and if it's more work, I probably won't have time for it between the scipy 0.8.0 release and my 'real job', like you call it. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Sun, May 30, 2010 at 5:53 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Mon, May 31, 2010 at 2:06 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, May 30, 2010 at 9:52 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, May 29, 2010 at 9:05 AM, Pauli Virtanen p...@iki.fi wrote: Fri, 28 May 2010 12:13:57 -0500, Travis Oliphant wrote: [clip] I think this makes sense so that we can reschedule NumPy 2.0 for September and still provide a release with the Python 3k changes (I am assuming these can be done in an ABI-compatible way). As far as I remember, the Py3 changes did not require breaking ABI (on Py2). Good. How much work is it to remove datetime, and who wants to do it? Hey, I thought that was your job ;) Maybe the best thing is to start by making a branch for the removal and we can argue about who gets the short end of the stick later... Manage the release yes, do the heavy lifting not necessarily:) OK, I'll make the branch tonight. If the removal just involves the same changes that were made for 1.4.1 then I can do it. But I'm not familiar with this code and if it's more work, I probably won't have time for it between the scipy 0.8.0 release and my 'real job', like you call it. I think it may be a bit trickier because Travis made more changes. I think the relevant commits are r8113..r8115 and 8107..8108. After removing those we still need to remove the same stuff as we did for the 1.4.1 release. We will need some way of testing if the removal was successful. We probably want to make sure there is a documentation update also, maybe before making the branch. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 27, 2010, at 6:51 AM, Ralf Gommers wrote: On Wed, May 26, 2010 at 12:23 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 5:06 PM, David Cournapeau wrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is an interesting idea and also workable. I am really worried about breaking things once now and once in a few months (or even a year). I am too. That's why this discussion.We will have the NumPy refactor done by end of July at the latest. Numpy 2.0 should be able to come out in August. This thread got a bit side-tracked with the move to git, but I don't see a conclusion about what to release when. Even if the refactoring is done in July, I think a 2.0 release with so many major changes will probably need a longer test/release cycle. So if we say September, do you still want a 1.5 release? I think this makes sense so that we can reschedule NumPy 2.0 for September and still provide a release with the Python 3k changes (I am assuming these can be done in an ABI-compatible way). -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 12:23 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 5:06 PM, David Cournapeau wrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is an interesting idea and also workable. I am really worried about breaking things once now and once in a few months (or even a year). I am too. That's why this discussion.We will have the NumPy refactor done by end of July at the latest. Numpy 2.0 should be able to come out in August. This thread got a bit side-tracked with the move to git, but I don't see a conclusion about what to release when. Even if the refactoring is done in July, I think a 2.0 release with so many major changes will probably need a longer test/release cycle. So if we say September, do you still want a 1.5 release? Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Thu, May 27, 2010 at 8:51 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Wed, May 26, 2010 at 12:23 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 5:06 PM, David Cournapeau wrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is an interesting idea and also workable. I am really worried about breaking things once now and once in a few months (or even a year). I am too. That's why this discussion. We will have the NumPy refactor done by end of July at the latest. Numpy 2.0 should be able to come out in August. This thread got a bit side-tracked with the move to git, but I don't see a conclusion about what to release when. My understanding is that there was a general agreement on splitting the code that breaks the ABI/API (datetime, maybe future refactoring) from everything else (mostly py3k port): - 1.5: everything in the trunk minus API/ABI breaking stuff - 2.0: still in flux, depends on how the refactoring will happen cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 9:22 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 4:49 PM, David Goldsmith wrote: Travis: do you already have a place on the NumPy Development Wikihttp://wiki.numpy.org/where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. This is a great idea of course and we will definitely post progess there. Thanks; specific URL please, when available; plus, prominently feature (a link to) the location on the Development Wiki home page, at the very least (i.e., if not also on the NumPy home page). So far, the code has been reviewed, I.e., the existing code, yes? and several functions identified for re-factoring. Please enumerate on the Wiki Refactoring Log (name tentative - I don't care what we call it, just so long as it exists, its purpose is clear, and we all know where it is). This is taking place in a github branch of numpy called numpy refactor. This = the actual creation/modification of code, yes? DG -Travis DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/37409) many different views on what the new C-API should be were expressed. Naturally, I wonder if the new C-API will be useful for my purposes. So, I'm not so excited about a refactoring log where only the progress is reported. I fear that some (potentially minor) design decisions would render the new C-API useless for me. So my question is: Does this refactoring log also include something like a Numpy Enhancement Proposal? Something that can be discussed beforehand? I.e., will there be a detailed description (i.e. code examples) what the goal of the refactoring is? If there is any interest, I could provide some simple test examples in C that would explain what I'd like to be able to do with the new C-API. Sebastian On Wed, May 26, 2010 at 8:21 AM, David Goldsmith d.l.goldsm...@gmail.com wrote: On Tue, May 25, 2010 at 9:22 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 4:49 PM, David Goldsmith wrote: Travis: do you already have a place on the NumPy Development Wiki where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. This is a great idea of course and we will definitely post progess there. Thanks; specific URL please, when available; plus, prominently feature (a link to) the location on the Development Wiki home page, at the very least (i.e., if not also on the NumPy home page). So far, the code has been reviewed, I.e., the existing code, yes? and several functions identified for re-factoring. Please enumerate on the Wiki Refactoring Log (name tentative - I don't care what we call it, just so long as it exists, its purpose is clear, and we all know where it is). This is taking place in a github branch of numpy called numpy refactor. This = the actual creation/modification of code, yes? DG -Travis DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it. My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility. It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Wed, 26 May 2010 10:50:19 +0200, Sebastian Walter wrote: I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/37409) many different views on what the new C-API should be were expressed. I believe the aim of the refactor is to *not* change the C-API at all, but separate it internally from the routines that do the heavy lifting. Externally, Numpy would still look the same, but be more easy to maintain. The routines that do the heavy lifting could then be good for reuse and be more easy to maintain, but I think how and where they would be exposed hasn't been discussed so far... -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 12:31 PM, Pauli Virtanen p...@iki.fi wrote: Wed, 26 May 2010 10:50:19 +0200, Sebastian Walter wrote: I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/37409) many different views on what the new C-API should be were expressed. I believe the aim of the refactor is to *not* change the C-API at all, but separate it internally from the routines that do the heavy lifting. Externally, Numpy would still look the same, but be more easy to maintain. Sorry for the confusion. By C-API I meant a C-API that would be independent of the CPython API. The routines that do the heavy lifting could then be good for reuse and be more easy to maintain, but I think how and where they would be exposed hasn't been discussed so far... I had the impression that the goal is not only to have code that is easier to maintain but to give developers the possibility to use numpy functionality (broadcasting, ufuncs, ...) within C code without having to use CPython API (refcounts, construction of PyObjects etc.). -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
2010/5/25 Stéfan van der Walt ste...@sun.ac.za: Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! I would love to move numpy to github as well. Almost everything I work on is there now and I am really enjoying using git and the github infrastructure is really nice. This is obviously a separate issue and one that shouldn't deflect the discussion on the proposed refactoring. But given how many of the developers are using git-svn and that you can use an svn client with github, it might be worth having a quick discussion about this in the near future. For instance, I wonder how many of the developer's prefer using git at this point. Also it would be interesting to hear from any of the developer's who would be opposed to git. A few year's ago this was a hot topic for discussion, but it may be that this isn't very controversial at this point. Jarrod ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 1:03 PM, Jarrod Millman mill...@berkeley.eduwrote: 2010/5/25 Stéfan van der Walt ste...@sun.ac.za: Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! I would love to move numpy to github as well. Almost everything I work on is there now and I am really enjoying using git and the github infrastructure is really nice. This is obviously a separate issue and one that shouldn't deflect the discussion on the proposed refactoring. But given how many of the developers are using git-svn and that you can use an svn client with github, it might be worth having a quick discussion about this in the near future. For instance, I wonder how many of the developer's prefer using git at this point. Also it would be interesting to hear from any of the developer's who would be opposed to git. A few year's ago this was a hot topic for discussion, but it may be that this isn't very controversial at this point. I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 1:54 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 26, 2010 at 1:03 PM, Jarrod Millman mill...@berkeley.eduwrote: 2010/5/25 Stéfan van der Walt ste...@sun.ac.za: Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! I would love to move numpy to github as well. Almost everything I work on is there now and I am really enjoying using git and the github infrastructure is really nice. This is obviously a separate issue and one that shouldn't deflect the discussion on the proposed refactoring. But given how many of the developers are using git-svn and that you can use an svn client with github, it might be worth having a quick discussion about this in the near future. For instance, I wonder how many of the developer's prefer using git at this point. Also it would be interesting to hear from any of the developer's who would be opposed to git. A few year's ago this was a hot topic for discussion, but it may be that this isn't very controversial at this point. I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. A newish git windows client I hadn't heard of before is gitextensionshttp://code.google.com/p/gitextensions/ . Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Thu, May 27, 2010 at 4:54 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 26, 2010 at 1:03 PM, Jarrod Millman mill...@berkeley.edu wrote: 2010/5/25 Stéfan van der Walt ste...@sun.ac.za: Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! I would love to move numpy to github as well. Almost everything I work on is there now and I am really enjoying using git and the github infrastructure is really nice. This is obviously a separate issue and one that shouldn't deflect the discussion on the proposed refactoring. But given how many of the developers are using git-svn and that you can use an svn client with github, it might be worth having a quick discussion about this in the near future. For instance, I wonder how many of the developer's prefer using git at this point. Also it would be interesting to hear from any of the developer's who would be opposed to git. A few year's ago this was a hot topic for discussion, but it may be that this isn't very controversial at this point. I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. As I mentioned in a previous post, there is smartgit, which is free for personal use, and is a graphical UI (does *not* depend on the mingw port of git, uses the reimplementation of git jgit in java used in google for android I believe). gitextensions is just a GUI around the mingw tools, and as such is less reliable. github also supports smart http for people behind proxies (although I don't know about the authentification issues if any). Trac and buildbot could use a svn mirror as provided by github for the time being, although there seems to be an issue with the numpy repo ATM (maybe my fault: http://support.github.com/discussions/repos/3155-svn-checkout-error-200-ok-error) David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 26, 2010, at 3:50 AM, Sebastian Walter wrote: I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/37409) many different views on what the new C-API should be were expressed. Naturally, I wonder if the new C-API will be useful for my purposes. So, I'm not so excited about a refactoring log where only the progress is reported. I fear that some (potentially minor) design decisions would render the new C-API useless for me. So my question is: Does this refactoring log also include something like a Numpy Enhancement Proposal? Something that can be discussed beforehand? I.e., will there be a detailed description (i.e. code examples) what the goal of the refactoring is? If there is any interest, I could provide some simple test examples in C that would explain what I'd like to be able to do with the new C-API. Our team would love to see this if possible. -Travis Sebastian On Wed, May 26, 2010 at 8:21 AM, David Goldsmith d.l.goldsm...@gmail.com wrote: On Tue, May 25, 2010 at 9:22 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 4:49 PM, David Goldsmith wrote: Travis: do you already have a place on the NumPy Development Wiki where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. This is a great idea of course and we will definitely post progess there. Thanks; specific URL please, when available; plus, prominently feature (a link to) the location on the Development Wiki home page, at the very least (i.e., if not also on the NumPy home page). So far, the code has been reviewed, I.e., the existing code, yes? and several functions identified for re-factoring. Please enumerate on the Wiki Refactoring Log (name tentative - I don't care what we call it, just so long as it exists, its purpose is clear, and we all know where it is). This is taking place in a github branch of numpy called numpy refactor. This = the actual creation/modification of code, yes? DG -Travis DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re- factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 26, 2010, at 5:31 AM, Pauli Virtanen wrote: Wed, 26 May 2010 10:50:19 +0200, Sebastian Walter wrote: I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/37409) many different views on what the new C-API should be were expressed. I believe the aim of the refactor is to *not* change the C-API at all, but separate it internally from the routines that do the heavy lifting. Externally, Numpy would still look the same, but be more easy to maintain. The routines that do the heavy lifting could then be good for reuse and be more easy to maintain, but I think how and where they would be exposed hasn't been discussed so far... This is correct. In our plans, the current NumPy C-API would not change. Clearly there will be another potential API that could be used by other systems, but exactly what this is should be discussed and decided upon over time. I don't think our goals for the re-factoring depends on how this is done exactly. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 26, 2010, at 6:40 AM, Sebastian Walter wrote: On Wed, May 26, 2010 at 12:31 PM, Pauli Virtanen p...@iki.fi wrote: Wed, 26 May 2010 10:50:19 +0200, Sebastian Walter wrote: I'm a potential user of the C-API and therefore I'm very interested in the outcome. In the previous discussion (http://comments.gmane.org/gmane.comp.python.numeric.general/ 37409) many different views on what the new C-API should be were expressed. I believe the aim of the refactor is to *not* change the C-API at all, but separate it internally from the routines that do the heavy lifting. Externally, Numpy would still look the same, but be more easy to maintain. Sorry for the confusion. By C-API I meant a C-API that would be independent of the CPython API. The routines that do the heavy lifting could then be good for reuse and be more easy to maintain, but I think how and where they would be exposed hasn't been discussed so far... I had the impression that the goal is not only to have code that is easier to maintain but to give developers the possibility to use numpy functionality (broadcasting, ufuncs, ...) within C code without having to use CPython API (refcounts, construction of PyObjects etc.). This is partially correct. There may be a need to have some stub implementation of some aspects of the Python C-API (probably at least reference counting and exception handling for now). We don't need to work all of this out initially. I think getting the separation done in the next several weeks will spawn conversations and ideas that may take several months to work out the new C-level-only API. That interface API is important in the short term, but also one that could change over the next several months. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Hi, I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. For windows - I think honestly this is now not a serious barrier to using git. I've installed msysgit on 4 or 5 machines recently, and it has been very smooth - as well as providing a nice bash shell. I think it would be a huge reduction in the barrier to contributing to numpy if we could change to git. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 7:44 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. For windows - I think honestly this is now not a serious barrier to using git. I've installed msysgit on 4 or 5 machines recently, and it has been very smooth - as well as providing a nice bash shell. there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. Josef I think it would be a huge reduction in the barrier to contributing to numpy if we could change to git. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Hi, there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. It is a nice bash shell. You may not want a nice bash shell ;) I can't imagine you'd object to one though. It's just a useful place to type git commands, with file / directory path autocompletion, git branch autocompletion and so on. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 7:25 PM, josef.p...@gmail.com wrote: On Wed, May 26, 2010 at 7:44 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I think the main problem has been windows compatibility. Git is best from the command line whereas the windows command line is an afterthought. Another box that needs a check-mark is the buildbot. If svn clients are supported then it may be that neither of those are going to be a problem. However, It needs user testing. For windows - I think honestly this is now not a serious barrier to using git. I've installed msysgit on 4 or 5 machines recently, and it has been very smooth - as well as providing a nice bash shell. there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. Heh. Can you to try the svn interface to github using your favorite svn ap. I suppose we need to set up a test account there. Is it possible to have a multiple user git account on github, or is it all push and merge? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 7:34 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. It is a nice bash shell. You may not want a nice bash shell ;) I can't imagine you'd object to one though. It's just a useful place to type git commands, with file / directory path autocompletion, git branch autocompletion and so on. Any shell on windows is a pain, if only because of the spaces in the filenames. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 8:37 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 26, 2010 at 7:34 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. It is a nice bash shell. You may not want a nice bash shell ;) I can't imagine you'd object to one though. It's just a useful place to type git commands, with file / directory path autocompletion, git branch autocompletion and so on. Any shell on windows is a pain, if only because of the spaces in the filenames. Why? Can't you just escape the spaces with backslashes... oh, nevermind... Ben Root Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 9:37 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 26, 2010 at 7:34 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. It is a nice bash shell. You may not want a nice bash shell ;) I can't imagine you'd object to one though. It's just a useful place to type git commands, with file / directory path autocompletion, git branch autocompletion and so on. Any shell on windows is a pain, if only because of the spaces in the filenames. When I'm using git, or bzr or svn I use the windows shell which I'm very familiar with ,and allows standard copy-paste and has quotes. But since, I think, there are no numpy developers on Windows, and I'm the only one for scipy, and occasional commits I can do with anything, I won't argue this time. Josef Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Hi, Any shell on windows is a pain, if only because of the spaces in the filenames. When I'm using git, or bzr or svn I use the windows shell which I'm very familiar with ,and allows standard copy-paste and has quotes. But since, I think, there are no numpy developers on Windows, and I'm the only one for scipy, and occasional commits I can do with anything, I won't argue this time. I've been testing quite a bit on windows recently, and I used to use windows all the time. I've found msysgit to be pretty good. I personally have always hated the windows shell, but you can use msysgit from the windows shell if you prefer... If you're OK with bzr or svn from the windows command line, I am sure git will pose no major problems. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 6:35 PM, Charles R Harris charlesr.har...@gmail.com wrote: Heh. Can you to try the svn interface to github using your favorite svn ap. I suppose we need to set up a test account there. Is it possible to have a multiple user git account on github, or is it all push and merge? Yes, that is possible. Currently it is not officially allowed according to their terms of service, but they are planning to enable that and have granted exceptions to their current policy in a few instances (that I am familiar with). But this is a detail that we can easily address in a NEP, if there seems to be interest (which there currently seems to be). So for now just keep sending emails with issues you want addressed in any actual proposal for this transition. Let's move this discussion to the git thread and use the thread for the discussion related directly to the subject line. Thanks, -- Jarrod Millman Helen Wills Neuroscience Institute 10 Giannini Hall, UC Berkeley http://cirl.berkeley.edu/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 7:47 PM, Benjamin Root ben.r...@ou.edu wrote: On Wed, May 26, 2010 at 8:37 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 26, 2010 at 7:34 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, there is no such thing as a nice bash shell for a windows user. I have no idea how to use one. It is a nice bash shell. You may not want a nice bash shell ;) I can't imagine you'd object to one though. It's just a useful place to type git commands, with file / directory path autocompletion, git branch autocompletion and so on. Any shell on windows is a pain, if only because of the spaces in the filenames. Why? Can't you just escape the spaces with backslashes... oh, nevermind... Sure, no doubt the experience would cleanse my soul. But flagellation would be less painful. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython. There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. -Travis In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliph...@enthought.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Travis: do you already have a place on the NumPy Development Wikihttp://wiki.numpy.org/where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Mathematician: noun, someone
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 4:37 AM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it. My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). This sounds great. As for how this is to be done, what would be the numpy aspects to be worked on first ? The obvious candidates are broadcasting, indexing, and ufunc ? Concerning the goal of making numpy available to other languages, is the strategy already decided ? For example, will the core C API be reference-counted though python, or will this be abstracted ? It may be not feasable in the timeframe that Enthought has in mind, but I have been wondering for some time if we could do something like LUA does. LUA has a stronger story than python in terms of embedding within other languages, and the stack may be one solution for abstracting the reference counting (only the API around the stack and its implementation would need to be aware of a particular implementation of memory management). David While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 4:06 PM, David Cournapeau courn...@gmail.comwrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. If I had to choose between those, I would pick making a 1.5 release, that is, branch the current trunk and then excise datetime and all the related changes. Let me propose a schedule: - Branch 1.5 in late June. The time until then to be spent closing tickets. - Release 1.5 towards the end of July. That should be doable now that the release folks have had some practice. - Release 2.0 next spring. I don't think 3 man months is enough time to redesign/refactor numpy, get it tested, and document the changes. If we hide stuff away it will be even longer before folks who have written extensions can make the needed changes. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, 25 May 2010 15:54:26 -0500, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython. There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. -Travis In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliph...@enthought.com Chuck wants to make 2.0 be the py3k release. I expressed a desire some time ago to hold 2.0 until we had reviewed docs, on the basis that this should have been done for 1.0, and 2.0 is our real 1.0 for this and other reasons. Now Travis wants to refactor the C code in a way that breaks extensions (another real 1.0 item). But Chuck wants to get his project out the door soon, and not without justification. NumPy is not a small project anymore and it would make sense not only to have an open community process (which we sort-of do, except when we don't), but to define that process and to conduct more of it in plain view, on the web site, rather than hiding it here in the list archives where only subscribers are likely to see it. There are many interested parties (users) who are not on the lists. It's fine (even good) not to commit to specific dates until we know them for sure, but the process and rough timeline should be laid out publicly. I hope we can agree on one
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 4:31 PM, David Cournapeau courn...@gmail.comwrote: On Wed, May 26, 2010 at 4:37 AM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). This sounds great. As for how this is to be done, what would be the numpy aspects to be worked on first ? The obvious candidates are broadcasting, indexing, and ufunc ? If it was ufuncs alone it could be broken out into a separate project where ufuncs operated on objects that exposed the buffer interface. This would keep it separate from from numpy and at some point we could drop it into numpy trunk. If things went that way it would also make sense to drop support for versions of python that don't support the new buffer protocol. Speaking of which, can the datetime types be supported with the buffer interface? If not, there is some design work to be done there as to how to use the buffer interface for new types. snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On 25 May 2010 15:06, David Cournapeau courn...@gmail.com wrote: Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is a good suggestion. Release 1.5 without ABI breakage and then leave enough time to discuss an optimal API, refactor the C code and include datetime functionality for 2.0. We don't stand anything to gain by rushing. If I'm not mistaken, David did warn that this kind of situation may occur the last time around :) Cheers Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
2010/5/25 Stéfan van der Walt ste...@sun.ac.za On 25 May 2010 15:06, David Cournapeau courn...@gmail.com wrote: Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is a good suggestion. Release 1.5 without ABI breakage and then leave enough time to discuss an optimal API, refactor the C code and include datetime functionality for 2.0. We don't stand anything to gain by rushing. If I'm not mistaken, David did warn that this kind of situation may occur the last time around :) IIRC, David was seeing a refactor a year or two off, if ever. I'm concerned that the refactor will go on and on and on, not least because I haven't seen any plan or discussion as to what the precise goals of the refactor are, much less a plan for how to get there. It's hard to have a sprint when no one knows what they are trying to achieve. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. You are correct that there is not much time.However, our timeline is middle of July and we do have dedicated resources. I was also hoping to have discussions at SciPy to accelerate the process. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 4:49 PM, David Goldsmith wrote: Travis: do you already have a place on the NumPy Development Wiki where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. This is a great idea of course and we will definitely post progess there. So far, the code has been reviewed and several functions identified for re-factoring. This is taking place in a github branch of numpy called numpy refactor. -Travis DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 5:06 PM, David Cournapeau wrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is an interesting idea and also workable. I am really worried about breaking things once now and once in a few months (or even a year). I am too. That's why this discussion.We will have the NumPy refactor done by end of July at the latest. Numpy 2.0 should be able to come out in August. -Travis David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion --- Travis Oliphant Enthought, Inc. oliph...@enthought.com 1-512-536-1057 http://www.enthought.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On 25 May 2010 21:22, Travis Oliphant oliph...@enthought.com wrote: This is a great idea of course and we will definitely post progess there. So far, the code has been reviewed and several functions identified for re-factoring. This is taking place in a github branch of numpy called numpy refactor. Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! Regards Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
