Re: [Numpy-discussion] Extending documentation to c code
Excerpts from David Cournapeau's message of 2010-05-25 05:06:09 +0200: On Tue, May 25, 2010 at 3:01 AM, Charles R Harris [snip] Maybe as a first step, something that could extract function signature would be enough, and writing this should not take too much time (Sebastien B wrote something which could be a start, to autogenerate cython code from header:http://bitbucket.org/binet/cylon). note that llvm/clang is versatile enough to easily provide indices into the source code, which of course includes the comments... I am actually working on improving the python bindings to clang (which are already quite useful for this thread's topic as they are used for code completing C/C++ code - but are not yet complete enough for providing complete function signatures) cheers, sebastien. -- # # Dr. Sebastien Binet # Laboratoire de l'Accelerateur Lineaire # Universite Paris-Sud XI # Batiment 200 # 91898 Orsay # ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
On Tue, May 25, 2010 at 2:09 PM, David Goldsmith d.l.goldsm...@gmail.com wrote: This does sound promising/a good first step. But it doesn't really answer Charles' question about a standard (which would be useful to have to help guide doc editor design). it does - I looked into synopsis because we could use rest, and I don't think anyone wants to go the doxygen route. Just putting rest comments into sources is not useful (since just *extracting* them is non trivial for C/C++). I think the documentation project taught us that being able to build a decent looking document is required for people to actually document things. Also, I may have not been clear, but when I said I thought about it, I meant I have tried it and it did not work after one hour of two of tinkering (then I realized that you need to parse C to do anything useful). David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
Hi,
ma, 2010-05-24 kello 12:01 -0600, Charles R Harris kirjoitti:
I'm wondering if we could extend the current documentation format to
the c source code. The string blocks would be implemented something
like
[clip]
I'd perhaps stick closer to C conventions and use something like
/**
* Spam foo out of the parrots.
*
* Parameters
* --
* a
* Amount of spam
* b
* Number of parrots
*/
int foo(int a, int b)
{
}
Using some JavaDoc-type syntax might also be possible, although I
personally find it rather ugly. Also, parsing C might not be too
difficult, as projects aimed doing that in Python exist, for instance
http://code.google.com/p/pycparser/
--
Pauli Virtanen
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
On Tue, May 25, 2010 at 4:04 PM, Sebastien Binet seb.bi...@gmail.com wrote: Excerpts from David Cournapeau's message of 2010-05-25 05:06:09 +0200: On Tue, May 25, 2010 at 3:01 AM, Charles R Harris [snip] Maybe as a first step, something that could extract function signature would be enough, and writing this should not take too much time (Sebastien B wrote something which could be a start, to autogenerate cython code from header:http://bitbucket.org/binet/cylon). note that llvm/clang is versatile enough to easily provide indices into the source code, which of course includes the comments... I am actually working on improving the python bindings to clang Ah, nice - where do you put your work ? It seems that llvm-py does not have recent commits, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
Excerpts from David Cournapeau's message of 2010-05-25 09:51:36 +0200: On Tue, May 25, 2010 at 4:04 PM, Sebastien Binet seb.bi...@gmail.com wrote: Excerpts from David Cournapeau's message of 2010-05-25 05:06:09 +0200: On Tue, May 25, 2010 at 3:01 AM, Charles R Harris [snip] Maybe as a first step, something that could extract function signature would be enough, and writing this should not take too much time (Sebastien B wrote something which could be a start, to autogenerate cython code from header:http://bitbucket.org/binet/cylon). note that llvm/clang is versatile enough to easily provide indices into the source code, which of course includes the comments... I am actually working on improving the python bindings to clang Ah, nice - where do you put your work ? It seems that llvm-py does not have recent commits, I am talking about the clang python bindings, not the llvm ones :) I am pushing stuff over there: http://llvm.org/viewvc/llvm-project/cfe/trunk/bindings/python/ FYI, my cylon project is waiting for this last patch to be applied: http://lists.cs.uiuc.edu/pipermail/cfe-dev/2010-May/009091.html then world domination... cheers, sebastien. -- # # Dr. Sebastien Binet # Laboratoire de l'Accelerateur Lineaire # Universite Paris-Sud XI # Batiment 200 # 91898 Orsay # ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
On 25/05/10 09:04, Sebastien Binet wrote: note that llvm/clang is versatile enough to easily provide indices into the source code, which of course includes the comments... I am actually working on improving the python bindings to clang (which are already quite useful for this thread's topic as they are used for code completing C/C++ code - but are not yet complete enough for providing complete function signatures) Have you seen http://code.google.com/p/pycparser/ ? It is a pure python implementation with small or no external dependencies. I was thinking about using it for writing a simple cython interfaces generator. I do not know if it supports extracting comments, but I think it would be simple to extend it this way. Cheers, -- Daniele ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
Excerpts from Daniele Nicolodi's message of 2010-05-25 11:37:50 +0200: On 25/05/10 09:04, Sebastien Binet wrote: note that llvm/clang is versatile enough to easily provide indices into the source code, which of course includes the comments... I am actually working on improving the python bindings to clang (which are already quite useful for this thread's topic as they are used for code completing C/C++ code - but are not yet complete enough for providing complete function signatures) Have you seen http://code.google.com/p/pycparser/ ? yes, and I have been using it to teach myself PyParsing. the problem is that I need to be able to parse C++, not just C. Parsing (correctly) C++ is a full blown project and I prefer to rely on the work of others :) cheers, sebastien. -- # # Dr. Sebastien Binet # Laboratoire de l'Accelerateur Lineaire # Universite Paris-Sud XI # Batiment 200 # 91898 Orsay # ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] calling C function from Python via f2py
Thanks for your time and assistance, Nadav. I will look into the SWIG list and or cython. On Mon, May 24, 2010 at 2:37 PM, Nadav Horesh nad...@visionsense.com wrote: Sorry, can not figure it out, if you don't gen an answer on this list maybe you should address it on swig list. Personally I use cython for this purpose. Nadav -Original Message- From: numpy-discussion-boun...@scipy.org on behalf of Matt Fearon Sent: Mon 24-May-10 15:43 To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] calling C function from Python via f2py Nadav, Thank you. I believe it is working now, as the pos(2) error is gone. However, though the error is gone, my return variable from the C function is not being updated as if the C code is not executing? Syntax to the call the C function from Python is the following: FFMCcalc.FFMCcalc(T,H,W,ro,Fo) Should this execute the C code? thanks, Matt On Sun, May 23, 2010 at 1:44 AM, Nadav Horesh nad...@visionsense.com wrote: in test.py change to print FFMCcalc.FFMCcalc(T,H,W,ro,Fo) As implied from the line print FFMCcalc.FFMCcalc.__doc__ Nadav -Original Message- From: numpy-discussion-boun...@scipy.org on behalf of Matt Fearon Sent: Fri 21-May-10 21:55 To: numpy-discussion@scipy.org Subject: [Numpy-discussion] calling C function from Python via f2py Hello, I am trying to use f2py to generate a wrapped C function that I can call from Python (passing arguments to and from). I have this almost working, but I receive trouble with exp and pow related to C and some pos (2) error with one of my passed variables. My f2py syntax is: f2py -c -lm FFMCcalc.pyf FFMCcalc.c Also, my 3 scripts are short and attached. 1. FFMCcalc.c, C function 2. FFMCcalc.pyf, wrapper file 3. test.py, short python code that calls C function Any advice would greatly appreciated to get this working. thanks, Matt ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending documentation to c code
it does - I looked into synopsis because we could use rest, and I don't think anyone wants to go the doxygen route. I am curious as to why doxygen isn't a viable option. While I don't have experience with the other suggestions, I have used doxygen in a few of my personall projects and have been quite happy with it serving as internal documentation. doxygen can still produce bare-bones documentation, call-graphs and cross-references from C/C++ code even if there are no special comments for it to parse. While doxygen may not be perfect, I think it does well enough to produce good documentation for developers to use. Just my 2 cents. Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] __eq__ with str and object
I don't understand this: a1 = np.array(['a', 'b'], dtype=object) a2 = np.array(['a', 'b']) a1 == a2 array([ True, True], dtype=bool) # Looks good a2 == a1 False # Should I have expected this? This works like I expected: a1 = np.array([1, 2], dtype=object) a2 = np.array([1, 2]) a1 == a2 array([ True, True], dtype=bool) a2 == a1 array([ True, True], dtype=bool) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ensuring docstrings in new code
Over on [Numpy-discussion] Extending documentation to c code, David G. gave voice to a frustration he and I share about the status of documentation in the new-code development process. I don't want to paint with a broad brush, yet in recent months there have been a number of checkins, unanimously passed off by the entire core development group/Steering Committee, without a single mention that this code entered SVN with deficient or even completely absent docstrings. I'm not pointing out the offenses simply because I don't think public embarrassment is a good policy, but David has a list that we can share with developers privately. It is really hard to do much without docs on anything you didn't write yourself. When I learned how to program, it was beaten into me to document as I went along, that good projects always did, and that major problems up to and including a complete implosion and abandonment of the software could and sometimes did result if docs were not kept current. The Doc Project is not the writing arm of the developers. So far, we have only worked on the existing software, not anything new, and most of us don't even have SVN access. Rather, the Doc Project is cleaning up what others, in their haste, did not do, and it frustrates us to see others still not doing it. What change of procedures would ensure that no code, possibly including bug fixes, enters SVN without a ready-for-review docstring? Perhaps new code could be reviewed according to a checklist that includes has a ready-for-review docstring. Or perhaps announcements to the lists of pending code checkins should include [CHECKIN] in the subject, to attract the attention of the Doc Project's Editor (i.e., David), so that he can act as an outside cop. Or maybe the SVN can somehow be made to check for docstrings, or to ask for them on checkin. Or maybe something in the build could make such a list and compare it to the last list and complain if there are new routines on it. We've called for a rule of no new code without docs many times. I'm not sure of the best solution, but we need something or things won't change. It's not like these are hard to write. The lack of a ready-for-review docstring is a major bug in itself. --jh-- ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython. There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. -Travis In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliph...@enthought.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
Travis: do you already have a place on the NumPy Development Wikihttp://wiki.numpy.org/where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.comwrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.comwrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Mathematician: noun, someone
Re: [Numpy-discussion] Extending documentation to c code
On Tue, May 25, 2010 at 11:18 PM, Benjamin Root ben.r...@ou.edu wrote: it does - I looked into synopsis because we could use rest, and I don't think anyone wants to go the doxygen route. I am curious as to why doxygen isn't a viable option. While I don't have experience with the other suggestions, I have used doxygen in a few of my personall projects and have been quite happy with it serving as internal documentation. It is yet another format to use inside C sources (I don't think doxygen supports rest), and I would rather have something that is similar, ideally integrated into sphinx. It also generates rather ugly doc by default, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Wed, May 26, 2010 at 4:37 AM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it. My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). This sounds great. As for how this is to be done, what would be the numpy aspects to be worked on first ? The obvious candidates are broadcasting, indexing, and ufunc ? Concerning the goal of making numpy available to other languages, is the strategy already decided ? For example, will the core C API be reference-counted though python, or will this be abstracted ? It may be not feasable in the timeframe that Enthought has in mind, but I have been wondering for some time if we could do something like LUA does. LUA has a stronger story than python in terms of embedding within other languages, and the stack may be one solution for abstracting the reference counting (only the API around the stack and its implementation would need to be aware of a particular implementation of memory management). David While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 4:06 PM, David Cournapeau courn...@gmail.comwrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. If I had to choose between those, I would pick making a 1.5 release, that is, branch the current trunk and then excise datetime and all the related changes. Let me propose a schedule: - Branch 1.5 in late June. The time until then to be spent closing tickets. - Release 1.5 towards the end of July. That should be doable now that the release folks have had some practice. - Release 2.0 next spring. I don't think 3 man months is enough time to redesign/refactor numpy, get it tested, and document the changes. If we hide stuff away it will be even longer before folks who have written extensions can make the needed changes. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, 25 May 2010 15:54:26 -0500, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython. There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. -Travis In any case, I think the primary goal for 2.0 should remain the python3k port. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliph...@enthought.com Chuck wants to make 2.0 be the py3k release. I expressed a desire some time ago to hold 2.0 until we had reviewed docs, on the basis that this should have been done for 1.0, and 2.0 is our real 1.0 for this and other reasons. Now Travis wants to refactor the C code in a way that breaks extensions (another real 1.0 item). But Chuck wants to get his project out the door soon, and not without justification. NumPy is not a small project anymore and it would make sense not only to have an open community process (which we sort-of do, except when we don't), but to define that process and to conduct more of it in plain view, on the web site, rather than hiding it here in the list archives where only subscribers are likely to see it. There are many interested parties (users) who are not on the lists. It's fine (even good) not to commit to specific dates until we know them for sure, but the process and rough timeline should be laid out publicly. I hope we can agree on one
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On Tue, May 25, 2010 at 4:31 PM, David Cournapeau courn...@gmail.comwrote: On Wed, May 26, 2010 at 4:37 AM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). This sounds great. As for how this is to be done, what would be the numpy aspects to be worked on first ? The obvious candidates are broadcasting, indexing, and ufunc ? If it was ufuncs alone it could be broken out into a separate project where ufuncs operated on objects that exposed the buffer interface. This would keep it separate from from numpy and at some point we could drop it into numpy trunk. If things went that way it would also make sense to drop support for versions of python that don't support the new buffer protocol. Speaking of which, can the datetime types be supported with the buffer interface? If not, there is some design work to be done there as to how to use the buffer interface for new types. snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On 25 May 2010 15:06, David Cournapeau courn...@gmail.com wrote: Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is a good suggestion. Release 1.5 without ABI breakage and then leave enough time to discuss an optimal API, refactor the C code and include datetime functionality for 2.0. We don't stand anything to gain by rushing. If I'm not mistaken, David did warn that this kind of situation may occur the last time around :) Cheers Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Bug in nanmin called with unsigned integers
I got bit again by this bug with unsigned integers. (My original changes got overwritten when I updated from svn and, unfortunately, merged conflicts without actually looking over the changes.) In any case, I thought it'd be a good time to bump the issue (with patch). Cheers, -Tony PS: Just for context, this issue comes up when displaying images with Chaco (which converts images to unsigned integer arrays and calls nanmin). ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Geometric, negative binomial and poisson fail for extreme arguments
Josef, This is ticket #896 from two years ago. IIRC, there was some more recent discussion on the list of some of these. Do you know what the current state of these distributions is? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
2010/5/25 Stéfan van der Walt ste...@sun.ac.za On 25 May 2010 15:06, David Cournapeau courn...@gmail.com wrote: Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is a good suggestion. Release 1.5 without ABI breakage and then leave enough time to discuss an optimal API, refactor the C code and include datetime functionality for 2.0. We don't stand anything to gain by rushing. If I'm not mistaken, David did warn that this kind of situation may occur the last time around :) IIRC, David was seeing a refactor a year or two off, if ever. I'm concerned that the refactor will go on and on and on, not least because I haven't seen any plan or discussion as to what the precise goals of the refactor are, much less a plan for how to get there. It's hard to have a sprint when no one knows what they are trying to achieve. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Bug in nanmin called with unsigned integers
On Tue, May 25, 2010 at 8:21 PM, Tony S Yu tsy...@gmail.com wrote: I got bit again by this bug with unsigned integershttp://projects.scipy.org/numpy/ticket/1300. (My original changes got overwritten when I updated from svn and, unfortunately, merged conflicts without actually looking over the changes.) In any case, I thought it'd be a good time to bump the issue (with patchhttp://projects.scipy.org/numpy/ticket/1300#comment:2 ). Cheers, -Tony PS: Just for context, this issue comes up when displaying images with Chaco (which converts images to unsigned integer arrays and calls nanmin). Fixed in r8445. Please add some tests. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Geometric, negative binomial and poisson fail for extreme arguments
On Tue, May 25, 2010 at 10:34 PM, Charles R Harris charlesr.har...@gmail.com wrote: Josef, This is ticket #896 from two years ago. IIRC, there was some more recent discussion on the list of some of these. Do you know what the current state of these distributions is? I don't have any information on these and I don't remember any discussion (and a quick search didn't find anything). I never looked at the integer overflow problem, besides reading the ticket. All 3 distributions are used in scipy.stats and tested for some regular values. (my not very strong opinion: for consistency with the other distributions, I would go with Robert's approach of rejecting overflow samples. I don't know any application where the truncation would have a significant effect. In scipy.stats I switched to returning floats instead of integers for ppf, because we need inf and nans.) BTW: If you are fixing things in np.random, then depreciating and renaming pareto as we discussed recently on the list would help reduce some confusion. I don't think we filed a ticket. Josef Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] FW: Numpy python build
Hi,
Can I just install numpy and scipy without ATLAS? And what does this means
gnu: no Fortran 90 compiler found?
Im installing on RHEL
Thanks in advance!
[r...@giswk002 numpy-1.3.0]# python setup.py build
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in /usr/local/Python-2.6.2/lib
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/Python-2.6.2/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in /usr/local/Python-2.6.2/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
NOT AVAILABLE
/usr/local/numpy-1.3.0/numpy/distutils/system_info.py:1383: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
libraries blas not found in /usr/local/Python-2.6.2/lib
libraries blas not found in /usr/local/lib
FOUND:
libraries = ['blas']
library_dirs = ['/usr/lib']
language = f77
FOUND:
libraries = ['blas']
library_dirs = ['/usr/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in /usr/local/Python-2.6.2/lib
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/Python-2.6.2/lib
libraries lapack_atlas not found in /usr/local/Python-2.6.2/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info:
libraries f77blas,cblas,atlas not found in /usr/local/Python-2.6.2/lib
libraries lapack_atlas not found in /usr/local/Python-2.6.2/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
NOT AVAILABLE
/usr/local/numpy-1.3.0/numpy/distutils/system_info.py:1290: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
libraries lapack not found in /usr/local/Python-2.6.2/lib
libraries lapack not found in /usr/local/lib
FOUND:
libraries = ['lapack']
library_dirs = ['/usr/lib']
language = f77
FOUND:
libraries = ['lapack', 'blas']
library_dirs = ['/usr/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler
options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler
options
running build_src
building py_modules sources
building library npymath sources
building extension numpy.core._sort sources
adding 'build/src.linux-i686-2.6/numpy/core/include/numpy/config.h' to
sources.
adding 'build/src.linux-i686-2.6/numpy/core/include/numpy/numpyconfig.h'
to sources.
numpy/core/code_generators/genapi.py:9: DeprecationWarning: the md5 module
is deprecated; use hashlib instead
import md5
executing numpy/core/code_generators/generate_numpy_api.py
adding
'build/src.linux-i686-2.6/numpy/core/include/numpy/__multiarray_api.h' to
sources.
numpy.core - nothing done with h_files =
['build/src.linux-i686-2.6/numpy/core/include/numpy/config.h',
'build/src.linux-i686-2.6/numpy/core/include/numpy/numpyconfig.h',
'build/src.linux-i686-2.6/numpy/core/include/numpy/__multiarray_api.h']
building extension numpy.core.multiarray sources
adding
Re: [Numpy-discussion] Geometric, negative binomial and poisson fail for extreme arguments
On Tue, May 25, 2010 at 9:20 PM, josef.p...@gmail.com wrote: On Tue, May 25, 2010 at 10:34 PM, Charles R Harris charlesr.har...@gmail.com wrote: Josef, This is ticket #896 from two years ago. IIRC, there was some more recent discussion on the list of some of these. Do you know what the current state of these distributions is? I don't have any information on these and I don't remember any discussion (and a quick search didn't find anything). I never looked at the integer overflow problem, besides reading the ticket. All 3 distributions are used in scipy.stats and tested for some regular values. (my not very strong opinion: for consistency with the other distributions, I would go with Robert's approach of rejecting overflow samples. I don't know any application where the truncation would have a significant effect. In scipy.stats I switched to returning floats instead of integers for ppf, because we need inf and nans.) BTW: If you are fixing things in np.random, then depreciating and renaming pareto as we discussed recently on the list would help reduce some confusion. I don't think we filed a ticket. OK, but it would help if you did file a ticket. And if you think truncation is the way to go on the #896 could you post a note there also? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This refactoring isn't going to be as straight forward as the python3k port because a lot of design decisions need to be made along the way. You are correct that there is not much time.However, our timeline is middle of July and we do have dedicated resources. I was also hoping to have discussions at SciPy to accelerate the process. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 4:49 PM, David Goldsmith wrote: Travis: do you already have a place on the NumPy Development Wiki where you're (b)logging your design decisions? Seems like a good way for concerned parties to monitor your choices in more or less real time and thus provide comment in a timely fashion. This is a great idea of course and we will definitely post progess there. So far, the code has been reviewed and several functions identified for re-factoring. This is taking place in a github branch of numpy called numpy refactor. -Travis DG On Tue, May 25, 2010 at 2:19 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant oliph...@enthought.com wrote: On May 25, 2010, at 2:50 PM, Charles R Harris wrote: On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant oliph...@enthought.com wrote: Hi everyone, There has been some talk about re-factoring NumPy to separate out the Python C-API layer and make NumPy closer to a C-library. I know there are a few different ideas about what this means, and also that people are very busy. I also know there is a NumPy 2.0 release that is in the works. I'm excited to let everyone know that we (at Enthought) have been able to find resources (about 3 man months) to work on this re-factoring project and Scott and Jason (both very experienced C and Python programmers) are actively pursuing it.My hope is that NumPy 2.0 will contain this re-factoring (which should be finished just after SciPy 2010 --- where I'm going to organize a Sprint on NumPy which will include at least date-time improvements and re-factoring work). While we have specific goals for the re-factoring, we want this activity to be fully integrated with the NumPy community and Scott and Jason want to interact with the community as much as feasible as they suggest re-factoring changes (though they both have more experience with phone-conversations to resolve concerns than email chains and so some patience from everybody will be appreciated). Because Jason and Scott are new to this mailing list (but not new to NumPy), I wanted to introduce them so they would feel more comfortable posting questions and people would have some context as to what they were trying to do. Scott and Jason are both very proficient and skilled programmers and I have full confidence in their abilities. That said, we very much want the input of as many people as possible as we pursue the goal of grouping together more tightly the Python C-API interface layer to NumPy. I will be involved in some of the discussions, but am currently on a different project which has tight schedules and so I will only be able to provide limited mailing-list visibility. I think 2.0 would be a bit early for this. Is there any reason it couldn't be done in 2.1? What is the planned policy with regards to the visible interface for extensions? It would also be nice to have a rough idea of how the resulting code would be layered, i.e., what is the design for this re-factoring. Simply having a design would be a major step forward. The problem with doing it in 2.1 is that this re-factoring will require extensions to be re-built. The visible interface to extensions will not change, but there will likely be ABI incompatibility.It seems prudent to do this in NumPy 2.0. Perhaps we can also put in place the ABI-protecting indirection approaches that David C. was suggesting earlier. Some aspects of the design are still being fleshed out, but the basic idea is to separate out a core library that is as independent of the Python C-API as possible.There will likely be at least some dependency on the Python C-API (reference counting and error handling and possibly others) which any interface would have to provide in a very simple Python.h -- equivalent, for example. Our purpose is to allow NumPy to be integrated with other languages or other frameworks systems without explicitly relying on CPython.There are a lot of questions as to how this will work, and so much of that is being worked out. Part of the reason for this mail is to help ensure that as much of this discussion as possible takes place in public. Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Admittedly that makes keeping a refactored branch current with fixes going into the trunk a hassle, but perhaps that can be worked around somewhat by clearly labeling what files will be touched in the refactoring and possibly rearranging the content of the existing files. This requires a game plan and a clear idea of the goal. Put simply, I think the proposed schedule is too ambitious and needs to be fleshed out. This
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On May 25, 2010, at 5:06 PM, David Cournapeau wrote: On Wed, May 26, 2010 at 6:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: Sounds good, but what if it doesn't get finished in a few months? I think we should get 2.0.0 out pronto, ideally it would already have been released. I think a major refactoring like this proposal should get the 3.0.0 label. Naming it 3.0 or 2.1 does not matter much - I think we should avoid breaking things twice. I can see a few solutions: - postpone 2.0 indefinitely, until this new work is done - backport py3k changes to 1.5 (which would be API and ABI compatible with 1.4.1), and 2.0 would contain all the breaking changes. This is an interesting idea and also workable. I am really worried about breaking things once now and once in a few months (or even a year). I am too. That's why this discussion.We will have the NumPy refactor done by end of July at the latest. Numpy 2.0 should be able to come out in August. -Travis David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion --- Travis Oliphant Enthought, Inc. oliph...@enthought.com 1-512-536-1057 http://www.enthought.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought
On 25 May 2010 21:22, Travis Oliphant oliph...@enthought.com wrote: This is a great idea of course and we will definitely post progess there. So far, the code has been reviewed and several functions identified for re-factoring. This is taking place in a github branch of numpy called numpy refactor. Awesome! Since github now supports SVN interaction, and all the core devs use Git, now might be a good time to move the entire numpy source tree? It will certainly make it easier to merge the refactor changes! Regards Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
