[sage-devel] Re: Renaming the debugging function trace() to trace_execution()
Hi Jeroen! On 14 Nov., 00:02, Jeroen Demeyer wrote: > At #9704, there is a positively reviewed proposal to rename the > debugging function trace() to trace_execution() and to use trace() in > the mathematical sense. > > I have nothing to do with that ticket, but I would like to point it out > in case some people really think that it's a bad idea. My 0.02 Euros: Since Sage's user interface is based on Python, I find it generally a bad idea to have a *function* for computing mathematical data: It should rather be by calling a method! So, the computation of the trace of a matrix M, it should be M.trace(), not trace(M). I am "-1" to introducing a mathematical trace function. On the other hand, due to the lack of grammar in the English language, it is not clear whether "trace" is used as a noun or not. If you want "the trace" of a matrix, it is a noun. If it is "to trace" a computation, it is a verb. So, in order to avoid confusion between the two notions of trace, one might indeed consider to rename the non- mathematical trace. According to "LEO" (an online dictionary), the awkward German word "Ablaufverfolgung" is "trace" in the non-mathematical sense, and may also be translated as "tracing" or "backtrace". So, why not have a function "tracing" or "backtrace" in Sage for tracing a computation and getting a backtrace? Cheers, Simon -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: self.sum() method on vector made of int is not defined
Hi,
This thread is about sum.
1. Dima said:
> I don't think sum() method is needed. It's certainly a code bloat.
This is a very, very misinformed comment. I want to address it,
especially the statement "It's certainly a code bloat." It is
absolutely *NOT* code bloat to define a sum method for vectors over a
specific base instead of relying on the generic sum method built into
Python. Yes, the two methods accomplish the same thing. However,
there is a *massive* difference in efficiency. The builtin Python
one is over 500 times *slower* than the specialized Sage one:
sage: v = vector(RDF,range(10^5))
sage: timeit('v.sum()')
625 loops, best of 3: 116 µs per loop
sage: import __builtin__
sage: timeit('__builtin__.sum(v)')
5 loops, best of 3: 59.6 ms per loop
sage: 59.6/.116
513.793103448276
The reason is because the builtin sum has to turn every single entry
of v into a Python object, which involves creation of massive amounts
of Python objects. The vector v itself is just a malloc'd array of
10^5 floats, at the C level. If you do the sum using the Python
builtin method, about 59.5 milliseconds is spent creating Python
objects, and the other 0.1 millisecond is spent doing actual
summation.
Other comments:
Jason Grout:
> If we're now making a convention for the global sum function (i.e., try
> calling the .sum() method first), I'm curious how many other objects
> have a .sum() method? Will this break things?
If you type "sum??" and read you'll find out that the sum method
already calls ".sum()" first:
if hasattr(expression, 'sum'):
return expression.sum(*args, **kwds)
I had to use __builtin__ above to get the genuine builtin Python sum
function, which is dog slow. The sum in Sage's global namespace
overrides the builtin python one and calls sum on the input object.
This has little overhead if the actual summing dominates:
sage: v = vector(RDF,range(10^5))
sage: timeit('v.sum()', number=10^3)
1000 loops, best of 3: 116 µs per loop
sage: timeit('sum(v)', number=10^3)
1000 loops, best of 3: 122 µs per loop
If you try vectors with a smaller number of entries, e.g., a more
realistic maybe 20, then the time of doing the sum
in all cases is dominated by stupid overhead. In the case of Sage's
RDF vector, the time is still pretty bad, since in fact the sum is
done by converting to numpy, and the numpy sum is itself very slow
(which may be surprising to people, but is true):
sage: v = vector(RDF,range(20))
sage: timeit('v.sum()',number=10^4)
1 loops, best of 3: 6.54 µs per loop
sage: n = v.numpy()
sage: timeit('n.sum()',number=10^4)
1 loops, best of 3: 3.34 µs per loop
To see what the timings *should* be like, try stats.TimeSeries, which
is code I wrote from scratch using only Cython and C level data
structures. It's very straightforward code.
sage: t = stats.TimeSeries(range(20))
sage: t.sum()
190.0
sage: timeit('t.sum()',number=10^4)
1 loops, best of 3: 136 ns per loop
This is not fast because of caching. If you do "t.sum??" you'll see the code:
sage: t.sum??
cpdef double sum(self):
...
cdef double s = 0
cdef Py_ssize_t i
for i from 0 <= i < self._length:
s += self._values[i]
return s
The speed difference for a vector with many entries is of course much less:
sage: t = stats.TimeSeries(range(10^5))
sage: timeit('t.sum()')
625 loops, best of 3: 107 µs per loop
sage: v = vector(RDF,range(10^5))
sage: timeit('v.sum()')
625 loops, best of 3: 120 µs per loop
For some reason, beginning the middle of this summer, more and more
I'm starting to care a *lot* about the speed of code I write. In
many cases the difference in speed between carefully crafted
specialized code and generic code is a factor of hundreds or
thousands. This is the sort of thing that doesn't matter at all for
a lot of undergrad teaching or playing around with toy examples. It
matters a huge amount for research mathematics. I've been thinking
lately about ways to make it easier to write really fast code in the
context of Python, which isn't really hard to read and ugly (e.g.,
code that involves a lot of arithmetic operations but written without
operator overloading is terrible). I think the new awesome code in
Cython for wrapping C++ code may be very helpful...
-- William
On Sat, Nov 13, 2010 at 10:55 AM, Jason Grout
wrote:
> On 11/13/10 12:38 PM, Tom Boothby wrote:
>>
>> I agree, John. It would be better to define a .sum() method on the
>> generic vector class, which calls the global sum(). Then, it'll be
>> overridden by various classes which can do it faster.
>
>
> If we're now making a convention for the global sum function (i.e., try
> calling the .sum() method first), I'm curious how many other objects have a
> .sum() method? Will this break things? This question not only holds for
> Sage stuff, but for any library out there. Since we are talking about
> changing the behavior of a standard python function (sum), I can see a lo
[sage-devel] Re: Memory hits the roof when I divide a sage vector.
This is now track #10262 (memory leak in scalar*vector multiplication) Dima On Nov 13, 5:10 pm, Jason Grout wrote: > On 11/12/10 10:30 PM, Robert Bradshaw wrote: > > > On Fri, Nov 12, 2010 at 7:10 PM, Jason Grout > > It only has to construct an element if it can't figure out what to do > > after consulting the Parents themselves. > > Ah, okay. > > > > > > > > >> And then there's the matter you talk about; why is an element so big? > > > The example above is quite strange. No idea why it should be so big. > > (Note that these are are arbitrary precision integers, so the relative > > overhead for small ones is still quite large). > > > As for the original example, > > > sage: type(vector(RR, range(100))) > > > > > Which means that each element is a full Python RealNumber, quite a bit > > more than a double. Still doesn't explain the memory usage. For almost > > any kind of linear algebra, you're better off using RDF, or even numpy > > directly. > > RDF also seems to have a big problem: > > sage: v=vector(RDF, range(1)) > sage: get_memory_usage() > 206.84765625 > sage: w=v.parent().an_element() > sage: get_memory_usage() > 983.60546875 > > This is very strange, as RDF vectors are literally light wrappers around > numpy arrays. I wonder how much Sage library code has to be loaded to > do this conversion? > > Note that an_element() is cached in the parent. Still, that seems odd > that it goes up by ~780MB. > > Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: "Why Sage?" Website Section
On 11/14/10 02:54 AM, Eviatar wrote: Anyway, I made an updated version (linking seems to work now): http://oi55.tinypic.com/rclh5l.jpg I think that's better. At least the maths part is first. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: "Why Sage?" Website Section
On Nov 12, 5:16 pm, rjf wrote: > On Nov 12, 10:46 am, kcrisman wrote: > > > > My view is that there has been excessive boosterism for Python, > > Perhaps. > > > So having packages in Python doesn't matter, and therefore it is not > > > really a selling point and > > > maybe should not be mentioned prominently because users don't care?? > > > At least pedagogical users care. > > Really? The parts of Sage that are written in python appear to be > entirely conventional kinds of computer science projects ... e.g. > write a read-eval-print loop that has does math and calls programs. > > Judging from the kinds of bug reports I see here, it has no special > pedagogical import, and might as well have been written in any > language du jour. > > Some people here at Berkeley are enthusiastic about > Ruby on Rails. Tcl/TK passed by some time ago. > > Python/web blah blah is so retro. If it is so great, and Google > loves it, why can't I download python apps to my Google android > phone? :) > Like Sage. Oh, I forgot, Sage isn't really written in Python, > anyway. I mean pedagogically from my perspective - as you well know, I've never claimed to be anything remotely like a computer scientist. But I do teach students taking math (not necessarily always mathematics students!), and often ones who are quite afraid of anything where they have to type accurately (they love to text, of course) and create loops. And Python (even with a few Sage-specific changes, nearly 100% of which are to make invalid Python syntax which has semi-mathematical meaning into something useful, so not changing valid Python syntax) happens to be a language that is much less scary for them to use. There may be other such ones; Sage happens to be written in one such language, essentially. That is both all I mean, as well as what I mean, if you take what I mean. - kcrisman -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: "Why Sage?" Website Section
On Nov 13, 6:32 am, "Johan S. R. Nielsen" wrote: > First of all, I think that such a page put at a prominent place (as > the entry-page for what is now the Tour, perhaps?) will be a good > selling point. You need to hook people before they have read pages of > text, because otherwise most of them will already have continued on. > > > David Kirkby wrote > > There are two separate issues > > > 1) The interface language > > > 2) What the source code is written in. > > This is very true and a big issue! Maybe enough so to deserve having > two info boxes on this suggested "Why Sage"-page. > > I don't think that Python is the perfect language to write mathematics > software with; I would definitely vote on a much more functional > language here, e.g. OCaml or maybe even Haskell. However, this would > cut out so many potential developers, and from this viewpoint, it is a > bad choice for a large-scale open source project like Sage. Python has > many merits like it is easy to learn and has little syntax. Just > trying to imagine writing mathematics algorithms in Java makes my > stomach turn; soo many type declarations and soo many interfaces, all > having to be explicitly named and imported, instead of just "going > with the flow" in the duck-typing of Python (of course, preferably > using the type inference of functional languages, but oh well). The > bad side is then no static validation of anything (like type-checking > and only invoking declared methods and such), which makes it so much > more important to reread, test, double-test, auto-test and re-test all > code -- all the time. But as long as the developers succeed in doing > this well, the users won't see it too much (it makes me worried for > whether there might be a sort of upper limit on how big Sage can grow > while still being stable, though). > > But I digress; my point is that Python IS a selling point _both_ as > the underlying interpreter language (which is so many leagues ahead of > anything in the Ma*-software, simply because of a well-thought, > coherent syntax and standard library) and as the (main?) development > language. The first draws in users who care about ease of programming > (advanced users, teachers and potential developers. I always hated > Maple for its unsystematic syntax and Matlab for its happy-go-lucky > interpreter), and the second thing makes it easier for a user to > transition into being a developer. I personally don't know Cython yet, > but if a feature or a bug I was interested in came up, I would spend a > weekend learning it so I could develop with it on Sage; however, I > might not have cared about learning Cython when I was "only" a Sage > user in the first place. > > Oh yeah, and Eviatar, I just can't get over the fully committed > geekyness of posting the link as ASCII binary X-D That's just plain > cool. > > Johan :D thanks. I take pride in it. Anyway, I made an updated version (linking seems to work now): http://oi55.tinypic.com/rclh5l.jpg Anyways, I think most of us agree Python is a selling point. In fact, on the front page it says, "It combines the power of many existing open-source packages into a common Python-based interface." I think that makes discussion on whether Sage should be a selling point irrelevant for this thread; maybe more suited to sage-flame (in fact, a thread has already been started there). I made this thread to propose an accessible introductory page, not redefine the goals of the Sage project. Let's please keep that discussion out. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: simplify_rational and Pynac
On Nov 13, 1:39 pm, Ben Goodrich wrote: > Hi, > > The simplify_rational method has three choices for Maxima functions, > but I wanted to try GiNaC's normal method described here > > http://www.ginac.de/tutorial/Rational-expressions.html#Rational-expre... > > to see if it was faster. Has someone already tried this and concluded > Maxima was better? > Maxima has a very large number of very specific types of simplifications/expansions. It allows for quite a bit of control. That said, the main reason our Maxima stuff is slow is because we use 'pexpect' to communicate with it. Nils Bruin has a patch allowing us to "turn on" a library interface (which is now already built inside Sage) to Maxima instead; it would be great if someone who knew enough about Lisp and whatever else needed would give it a positive review, so that we could check such things at the library level. That's a little orthogonal to your main question, which I should know the answer to, but have forgotten off hand. Might this be in sage/ libs/ginac/ ? - kcrisman -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: Website Feature Request: Public Examples Webpage / Workbook
On 11/13/10 08:00 PM, rjf wrote: How many legs does a sheep have, if you count its tail as a leg? It has 4 legs. Counting its tail as a leg does not make it a leg. I'll answer your question - a sheep has 4 legs, unless it is unlucky. Could I request you answer my two questions I posted earlier 1) What would have been your choice for a an interfance langauge for Sage - I gave you about 10 choice that came to my mind. 2) What would be the best "glue" language. I'd be interested in your opinions. I tend to agree that Python is overrated by some Sage developers, but I'm yet to be totally convinced there was a better choice. As I many people have alluded to, having a language that's well known is useful. I do believe using Lisp for example would have significantly reduced the number of developers. Knowledge of Python is more likely to be of use to someone on their CV than knowledge of Lisp. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: "Why Sage?" Website Section
On Sat, Nov 13, 2010 at 7:13 AM, rjf wrote: > > > On Nov 13, 6:32 am, "Johan S. R. Nielsen" > wrote: > >> two info boxes on this suggested "Why Sage"-page. >> >> I don't think that Python is the perfect language to write mathematics >> software with; I would definitely vote on a much more functional >> language here, e.g. OCaml or maybe even Haskell. However, this would >> cut out so many potential developers, > ... yada yada... > > excessive boosterism. > > Consider that symbolic software systems like Maxima/Macsyma, Reduce, > Axiom, Jacal ... > were written in Lisp, > > and that Mathematica and Maple were written in C dialects... > > and even YOU would prefer a different, more functional language. > > And then you say Python is still better. > > Certainly not for writing math software. Maybe for writing web > applications? > > Because it has a coherent syntax Compared to Lisp or Haskell or > OCaml? > Because people who know little mathematics and little about > programming > can write/alter/debug applications for SAGE??? About which they > presumably know > nothing? And this is because Python is such a winner? > > And of course so much of SAGE is not even in Python, but C, Fortran, C+ > +, Lisp, whatever, > that even that is nonsense. > > excessive boosterism. > > At best, you might say, some features of Sage can be augmented by > writing in Python, and > the user interface looks like Python (actually it is not, but has to > be pre-processed). > > RJF FUD http://en.wikipedia.org/wiki/Fear,_uncertainty_and_doubt -- William Stein Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Renaming the debugging function trace() to trace_execution()
On 11/13/10 11:02 PM, Jeroen Demeyer wrote:
At #9704, there is a positively reviewed proposal to rename the
debugging function trace() to trace_execution() and to use trace() in
the mathematical sense.
I have nothing to do with that ticket, but I would like to point it out
in case some people really think that it's a bad idea.
Jeroen.
FWIW,
Mathematica has a "Trace" command, which does the non-Mathematical thing.
Mathematica 7.0 for Sun Solaris x86 (64-bit)
Copyright 1988-2009 Wolfram Research, Inc.
In[1]:= ?Trace
Trace[expr] generates a list of all expressions used in the evaluation of expr
. Trace[expr, form] includes only those expressions which match form
. Trace[expr, s] includes all evaluations which use transformation rules
associated with the symbol s.
In[2]:= ? *Trace*
AlgebraicNumberTrace TraceForward TraceScan
TraceTraceInternal$TraceOff
TraceAbove TraceLevel $TraceOn
TraceAction TraceOff $TracePattern
TraceBackwardTraceOn $TracePostAction
TraceDepth TraceOriginal$TracePreAction
TraceDialog TracePrint
In[5]:= Trace[Simplify[Sin[x]^2+Cos[x]^2]]
2 22 2 2 2
Out[5]= {{Sin[x] + Cos[x] , Cos[x] + Sin[x] }, Simplify[Cos[x] + Sin[x] ],
>1}
In[6]:=
Dave
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
[sage-devel] Renaming the debugging function trace() to trace_execution()
At #9704, there is a positively reviewed proposal to rename the debugging function trace() to trace_execution() and to use trace() in the mathematical sense. I have nothing to do with that ticket, but I would like to point it out in case some people really think that it's a bad idea. Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 11/13/10 08:48 PM, Robert Bradshaw wrote: On Sat, Nov 13, 2010 at 11:09 AM, Jeroen Demeyer wrote: On 2010-11-13 19:28, Dr. David Kirkby wrote: Actually, I realised I had in fact made a trac ticket for it. http://trac.sagemath.org/sage_trac/ticket/9418 Here's the comment from William that he is happy for this package to be added, despite he personally voted -1 for it https://groups.google.com/group/sage-devel/msg/973161fbcc09b7e6?hl=en Thank for the link, I just read the whole thread. I saw the various suggestions and simply using "patch" gets a big +1 from me. It is very easy to implement and allows for: * patching files depending on the system * patching multiple files with one diff * patching the same file with multiple diffs * easily update a spkg to a new upstream version * easily add a new patch to a spkg * not being forced to update any spkgs, we can keep the old system if we want (if there is a reason to do that for a particular spkg). (As far as I can tell, none of the other proposals have all these properties.) Maybe using "patch" is not the optimal solution, but it would certainly be a strict improvement to what we have now. The reason William was against "patch" is the following: We should never, ever add any new packages to sage without being very, very careful first. Every package added to sage increases later porting work, maintenance work, etc. forever. This is particularly a concern to *me*, since some of you come and go, but I'll be dealing with sage pretty much forever. But I agree with David Kirkby that "patch" is such a trivial spkg that this should not be a big concern. I think so too. Is it so trivial that someone is willing to commit to maintaining this spkg for 1-2 years? - Robert I said I would, when I proposed it before. https://groups.google.com/group/sage-devel/msg/6fff0383842b9fdb?hl=en%F3%AD%A5%A9fbcc09b7e6 I think it would be pretty low-maintenance package anyway. There have been no new releases in 2010. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On Sat, Nov 13, 2010 at 11:09 AM, Jeroen Demeyer wrote: > On 2010-11-13 19:28, Dr. David Kirkby wrote: >> Actually, I realised I had in fact made a trac ticket for it. >> >> http://trac.sagemath.org/sage_trac/ticket/9418 >> >> Here's the comment from William that he is happy for this package to be >> added, despite he personally voted -1 for it >> >> https://groups.google.com/group/sage-devel/msg/973161fbcc09b7e6?hl=en > > Thank for the link, I just read the whole thread. > > I saw the various suggestions and simply using "patch" gets a big +1 > from me. It is very easy to implement and allows for: > * patching files depending on the system > * patching multiple files with one diff > * patching the same file with multiple diffs > * easily update a spkg to a new upstream version > * easily add a new patch to a spkg > * not being forced to update any spkgs, we can keep the old system if we > want (if there is a reason to do that for a particular spkg). > (As far as I can tell, none of the other proposals have all these > properties.) > > Maybe using "patch" is not the optimal solution, but it would certainly > be a strict improvement to what we have now. > > The reason William was against "patch" is the following: >> We should never, ever add any new packages to sage without being >> very, very careful first. Every package added to sage increases later >> porting work, maintenance work, etc. forever. This is particularly a >> concern to *me*, since some of you come and go, but I'll be dealing >> with sage pretty much forever. > > But I agree with David Kirkby that "patch" is such a trivial spkg that > this should not be a big concern. I think so too. Is it so trivial that someone is willing to commit to maintaining this spkg for 1-2 years? - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Website Feature Request: Public Examples Webpage / Workbook
On Nov 12, 11:30 pm, David Kirkby wrote: ... > > If one looks at the Wolfram Demonstrations Project > > http://demonstrations.wolfram.com/ > > there are 6582 demos covering a wide range of topics, from games like hangman > > http://demonstrations.wolfram.com/PlayHangman/ > > using the "times table" > > http://demonstrations.wolfram.com/MultiplicationTable/ > > to quantum mechanics > > http://demonstrations.wolfram.com/topic.html?topic=Quantum+Mechanics&;... > > These are reviewed by Wolfram Reasearch staff (WRI even claim they > count as academic publications!!) How many legs does a sheep have, if you count its tail as a leg? It has 4 legs. Counting its tail as a leg does not make it a leg. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: How to profile I/O?
2010/11/13 Simon King : > Hi Dave! > > On 12 Nov., 16:22, "Dr. David Kirkby" wrote: >> On 11/12/10 03:10 PM, Dr. David Kirkby wrote: >> >> I meant to say Sage is NOT the place to >> >> look. vmstat, sar (possibly top) are worth looking at. > > top would not help. It simply says that my process is sometimes only > taking 30% CPU time, but it gives no indication *why*. Note that it is > only a guess that I/O are to blame. > > Moreover, a system tool would not tell me in what part of my programs > the problem is located. "prun" might give some hint. However, since my > programs use external programs as well as compiled components and > interfaces, I was hoping for a tool that tracks all I/O processes > occuring during my computation. > > Perhaps there is a system tool for that purpose. I think it would help > me if some tool would produce a list of all file names to which I/O > happened, and record the I/O time for each file. Probably the best tool on linux should be perf, e.g: $ perf record -f /bin/sh -c "complex command line" $ perf report -f to overwrite previous recording and "/bin/sh -c" if needs pipes, etc better to do it on an idle system as it reports time kernel and in other processes. Example in my computer (while doing a system update) $ perf record -f sage -- | Sage Version 4.6, Release Date: 2010-10-30 | | Type notebook() for the GUI, and license() for information.| -- sage: Exiting Sage (CPU time 0m0.08s, Wall time 0m7.88s). [ perf record: Woken up 2 times to write data ] [ perf record: Captured and wrote 0.451 MB perf.data (~19711 samples) ] $ perf report|less # Samples: 14408076653 # # Overhead Command Shared Object Symbol # .. .. # 13.46%python libpython2.7.so.1.0 [.] PyParser_AddToken 11.32%python libpython2.7.so.1.0 [.] 0x027ca8 7.84%python libpython2.7.so.1.0 [.] 0x0643bd 5.46%python libpython2.7.so.1.0 [.] 0x0ca601 3.79%python libpython2.7.so.1.0 [.] 0x0725fb 2.63%python libpython2.7.so.1.0 [.] PyEval_EvalFrameEx 2.26%python libpython2.7.so.1.0 [.] PyTokenizer_Get 2.25%python libpython2.7.so.1.0 [.] 0x0dc72c 2.16%python libpython2.7.so.1.0 [.] PyObject_Malloc 2.15%python libpython2.7.so.1.0 [.] PyObject_Free 2.08%python libpython2.7.so.1.0 [.] PyNode_AddChild [...] > Best regards, > Simon Paulo -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 2010-11-13 19:28, Dr. David Kirkby wrote: > Actually, I realised I had in fact made a trac ticket for it. > > http://trac.sagemath.org/sage_trac/ticket/9418 > > Here's the comment from William that he is happy for this package to be > added, despite he personally voted -1 for it > > https://groups.google.com/group/sage-devel/msg/973161fbcc09b7e6?hl=en Thank for the link, I just read the whole thread. I saw the various suggestions and simply using "patch" gets a big +1 from me. It is very easy to implement and allows for: * patching files depending on the system * patching multiple files with one diff * patching the same file with multiple diffs * easily update a spkg to a new upstream version * easily add a new patch to a spkg * not being forced to update any spkgs, we can keep the old system if we want (if there is a reason to do that for a particular spkg). (As far as I can tell, none of the other proposals have all these properties.) Maybe using "patch" is not the optimal solution, but it would certainly be a strict improvement to what we have now. The reason William was against "patch" is the following: > We should never, ever add any new packages to sage without being > very, very careful first. Every package added to sage increases later > porting work, maintenance work, etc. forever. This is particularly a > concern to *me*, since some of you come and go, but I'll be dealing > with sage pretty much forever. But I agree with David Kirkby that "patch" is such a trivial spkg that this should not be a big concern. Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: self.sum() method on vector made of int is not defined
On 11/13/10 12:38 PM, Tom Boothby wrote: I agree, John. It would be better to define a .sum() method on the generic vector class, which calls the global sum(). Then, it'll be overridden by various classes which can do it faster. If we're now making a convention for the global sum function (i.e., try calling the .sum() method first), I'm curious how many other objects have a .sum() method? Will this break things? This question not only holds for Sage stuff, but for any library out there. Since we are talking about changing the behavior of a standard python function (sum), I can see a lot of confusion if sum(object) starts behaving differently because object happens to have an object.sum() method. For that reason, I think the magic method that sum() tries should be something more obscure, like ._sum_sage() or something like that. Then it would be much more unlikely to mess up objects out there that happened to have .sum() methods. Thanks, Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] simplify_rational and Pynac
Hi, The simplify_rational method has three choices for Maxima functions, but I wanted to try GiNaC's normal method described here http://www.ginac.de/tutorial/Rational-expressions.html#Rational-expressions to see if it was faster. Has someone already tried this and concluded Maxima was better? I see that there is an expand_rational method in sage that uses Pynac whose main part is just _sig_on cdef GEx x = self_.gobj.expand(0) _sig_off return new_Expression_from_GEx(self._parent, x) So I guess I would need to copy those lines and change .expand(0) to .normal(). But how do I add the .normal() method to Pynac's table of eligible GiNaC functions? Thanks, Ben -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: self.sum() method on vector made of int is not defined
I agree, John. It would be better to define a .sum() method on the generic vector class, which calls the global sum(). Then, it'll be overridden by various classes which can do it faster. On Sat, Nov 13, 2010 at 6:15 AM, John Cremona wrote: > I'm not sure I agree with this. I would expect the global sum() to > try to use a sum method first, and only do the default if that did not > exist, since some classes might have better ways of summing themselves > than the default. But then the only reason for having sum methods > would be if there was such a more efficient way of summing. The > present situation is still bad, though, as it is inconsistent. > > I am (of course) open to persuasion! > > John > > On Sat, Nov 13, 2010 at 5:18 AM, Dima Pasechnik wrote: >> I don't think sum() method is needed. It's certainly a code bloat. >> Could you check that sum() in vector_double_dense can be removed? >> (remove it there, do sage -b, run testsuite, see if there were any >> errors caused by it) >> >> On Nov 13, 12:59 pm, Maxim wrote: >>> If I try to find the sum of a vector of floats that way, it works as >>> expected: >>> sage: vector([1,float(2),3]).sum() >>> -> 6.0 >>> >>> However, applying the same logic on a vector made of integers: >>> sage: vector([1,2,3]).sum() >>> -> Traceback (click to the left of this block for traceback) >>> ... >>> AttributeError: >>> 'sage.modules.vector_integer_dense.Vector_integer_dense' >>> object has no attribute 'sum' >>> >>> Workaround (thanks to Dr. Drake) is to call the sum function directly >>> like this: >>> sage: sum(vector([1,2,3])) >>> -> 6 >>> >>> Browsing through the code, one can find the .sum() method being >>> defined for floats at the end of the file vector_double_dense.pyx, >>> while this method is absent in the vector_integer_dense.pyx. >>> >>> Did I luckily stumbled on a rarity, or should this self.sum() method >>> be part of every types' definition? (where applicable). in any case, >>> uniformity would help (method present in every type definition / >>> completely removed) to prevent confusion. >> >> -- >> To post to this group, send an email to sage-devel@googlegroups.com >> To unsubscribe from this group, send an email to >> sage-devel+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/sage-devel >> URL: http://www.sagemath.org >> > > -- > To post to this group, send an email to sage-devel@googlegroups.com > To unsubscribe from this group, send an email to > sage-devel+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sage-devel > URL: http://www.sagemath.org > -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: How to profile I/O?
On 11/13/10 12:29 PM, Simon King wrote: Hi Dave! On 12 Nov., 16:22, "Dr. David Kirkby" wrote: On 11/12/10 03:10 PM, Dr. David Kirkby wrote: I meant to say Sage is NOT the place to look. vmstat, sar (possibly top) are worth looking at. top would not help. It simply says that my process is sometimes only taking 30% CPU time, but it gives no indication *why*. Note that it is only a guess that I/O are to blame. Moreover, a system tool would not tell me in what part of my programs the problem is located. "prun" might give some hint. However, since my programs use external programs as well as compiled components and interfaces, I was hoping for a tool that tracks all I/O processes occuring during my computation. Perhaps there is a system tool for that purpose. I think it would help me if some tool would produce a list of all file names to which I/O happened, and record the I/O time for each file. Best regards, Simon lsof will certainly tell you what files are open, but I don't think it can tell you the CPU usage of the processes having those files open. On Solaris, dtrace could undoubtedly do what you want. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 11/13/10 05:38 PM, Dr. David Kirkby wrote: On 11/13/10 10:33 AM, Jeroen Demeyer wrote: On 2010-11-13 05:32, Robert Bradshaw wrote: I think the solution everyone liked was either making patch a dependency or spkgs, and using something like quilt. The current situation of copying files and/or manually maintaining patches is a pain and has several drawbacks. Is there a reason why we don't include GNU patch as a spkg, or it is just a matter of "nobody ever did it"? For the pari spkg (remember #9343?), I actually automated the process of generating patched files from the patches in a file spkg-make. This was really needed, because there were like 10 patches or so and I was making a new spkg at least once per day... Jeroen. I did it a few months ago http://boxen.math.washington.edu/home/kirkby/patches/patch-2.6.1.spkg William was against including it, though there was a concensus it should be included, so William accepted it. But there were still many arguments about it, and I've never put the package up for review. But I did create it some months back. Actually, I realised I had in fact made a trac ticket for it. http://trac.sagemath.org/sage_trac/ticket/9418 Here's the comment from William that he is happy for this package to be added, despite he personally voted -1 for it https://groups.google.com/group/sage-devel/msg/973161fbcc09b7e6?hl=en I don't know the best way to handle this in the deps file. It's clearly something that needs to be built very early on, since you could not create a patch for any package until this is built. Perhaps this should be added to 'BASE', so put in spkg/base rather than spkg/standard? Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 11/13/10 10:33 AM, Jeroen Demeyer wrote: On 2010-11-13 05:32, Robert Bradshaw wrote: I think the solution everyone liked was either making patch a dependency or spkgs, and using something like quilt. The current situation of copying files and/or manually maintaining patches is a pain and has several drawbacks. Is there a reason why we don't include GNU patch as a spkg, or it is just a matter of "nobody ever did it"? For the pari spkg (remember #9343?), I actually automated the process of generating patched files from the patches in a file spkg-make. This was really needed, because there were like 10 patches or so and I was making a new spkg at least once per day... Jeroen. I did it a few months ago http://boxen.math.washington.edu/home/kirkby/patches/patch-2.6.1.spkg William was against including it, though there was a concensus it should be included, so William accepted it. But there were still many arguments about it, and I've never put the package up for review. But I did create it some months back. -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] mercurial question
This is more a Mercurial question than a Sage one. I have two branches of an hg repository: two separate clones, each with different sets of edits, in both case all committed. In branch1 there are several files added and deleted (forgotten). In branch2 there are no changes relevant to those files at all. Now in branch1 I pull in all the changes from branch2 using hg pull, and (as prompted) follow that by hg merge and hg commit. In the current directory, the files which had been edited in branch1 are still edited, but the files which were added have disappeared completely. I could get them back (using hg serve and cut-and-paste) but that's stupid. I have read the mercurial documentation, and I don't think this is what is supposed to happen. Is there anything obvious (or otherwise) which I did wrong? This has happened to me twice now, and it is getting rather tedious. John -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: "Why Sage?" Website Section
On Nov 13, 6:32 am, "Johan S. R. Nielsen" wrote: > two info boxes on this suggested "Why Sage"-page. > > I don't think that Python is the perfect language to write mathematics > software with; I would definitely vote on a much more functional > language here, e.g. OCaml or maybe even Haskell. However, this would > cut out so many potential developers, ... yada yada... excessive boosterism. Consider that symbolic software systems like Maxima/Macsyma, Reduce, Axiom, Jacal ... were written in Lisp, and that Mathematica and Maple were written in C dialects... and even YOU would prefer a different, more functional language. And then you say Python is still better. Certainly not for writing math software.Maybe for writing web applications? Because it has a coherent syntax Compared to Lisp or Haskell or OCaml? Because people who know little mathematics and little about programming can write/alter/debug applications for SAGE??? About which they presumably know nothing? And this is because Python is such a winner? And of course so much of SAGE is not even in Python, but C, Fortran, C+ +, Lisp, whatever, that even that is nonsense. excessive boosterism. At best, you might say, some features of Sage can be augmented by writing in Python, and the user interface looks like Python (actually it is not, but has to be pre-processed). RJF -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: "Why Sage?" Website Section
First of all, I think that such a page put at a prominent place (as the entry-page for what is now the Tour, perhaps?) will be a good selling point. You need to hook people before they have read pages of text, because otherwise most of them will already have continued on. > David Kirkby wrote > There are two separate issues > > 1) The interface language > > 2) What the source code is written in. This is very true and a big issue! Maybe enough so to deserve having two info boxes on this suggested "Why Sage"-page. I don't think that Python is the perfect language to write mathematics software with; I would definitely vote on a much more functional language here, e.g. OCaml or maybe even Haskell. However, this would cut out so many potential developers, and from this viewpoint, it is a bad choice for a large-scale open source project like Sage. Python has many merits like it is easy to learn and has little syntax. Just trying to imagine writing mathematics algorithms in Java makes my stomach turn; soo many type declarations and soo many interfaces, all having to be explicitly named and imported, instead of just "going with the flow" in the duck-typing of Python (of course, preferably using the type inference of functional languages, but oh well). The bad side is then no static validation of anything (like type-checking and only invoking declared methods and such), which makes it so much more important to reread, test, double-test, auto-test and re-test all code -- all the time. But as long as the developers succeed in doing this well, the users won't see it too much (it makes me worried for whether there might be a sort of upper limit on how big Sage can grow while still being stable, though). But I digress; my point is that Python IS a selling point _both_ as the underlying interpreter language (which is so many leagues ahead of anything in the Ma*-software, simply because of a well-thought, coherent syntax and standard library) and as the (main?) development language. The first draws in users who care about ease of programming (advanced users, teachers and potential developers. I always hated Maple for its unsystematic syntax and Matlab for its happy-go-lucky interpreter), and the second thing makes it easier for a user to transition into being a developer. I personally don't know Cython yet, but if a feature or a bug I was interested in came up, I would spend a weekend learning it so I could develop with it on Sage; however, I might not have cared about learning Cython when I was "only" a Sage user in the first place. Oh yeah, and Eviatar, I just can't get over the fully committed geekyness of posting the link as ASCII binary X-D That's just plain cool. Johan On Nov 13, 11:48 am, David Kirkby wrote: > On 12 November 2010 18:18, rjf wrote: > > > > > On Nov 12, 8:46 am, "Dr. David Kirkby" > >> I think you have a big bias against python and towards lisp. > > > My view is that there has been excessive boosterism for Python, > > I agree with you there. > > > asserting that > > it is the solution to some important issues in building a system that > > is supposed > > to displace Mathematica, Maple, Magma, (Maxima?). > > I don't think anyone believes it will displace those languages. > > > While Python may > > have > > some merit in some situations, the case being made for it for Sage is > > weak, > > Given it's your opinion the case for Python is week, what would have > been best as a user interface language for users? > > I can think of a few possibilities worth consideration myself. > > 1) Create an entirely new language. > 2) Python > 3) Maxima > 4) A Mathematica like interface > 5) A MATLAB like interface > 6) A Maple-like interface > 7) A Magma-like interface > 8) Lisp > 9) Q - > seehttp://en.wikipedia.org/wiki/Q_%28equational_programming_language%29 > 10) Purehttp://code.google.com/p/pure-lang/is another possibility, > but that did not exist until 2008, but is based on Q, which existed > prior to Sage > > None seem perfect to me. > > Given the basic rationale of Sage is to "glue" various bits of code > together, what in your opinion would have been the best language to > glue them together? > > > which is why I find some bizarre enjoyment in tweaking people who make > > these claims. > > You certainly get some enjoyment out of winding people up. I wish you > would spend a bit more time contributing practical information though. > You have helped me, and you have helped others too. You almost > certainly know more about computer algebra systems than any Sage > developer. > > A bit less time "tweaking people" and a bit more time "tweaking" the > Sage source code would be nice! > > I'd also have a bit more respect for your opinions if you actually > tried Sage. It would give you a chance to point out even more > shortcomings, which I think you would enjoy. > > > > > Like so many people know it. (Why not use Java? or PHP?) > > Like it is slow but that's OK, you can use Cython (or some
Re: [sage-devel] Re: self.sum() method on vector made of int is not defined
I'm not sure I agree with this. I would expect the global sum() to try to use a sum method first, and only do the default if that did not exist, since some classes might have better ways of summing themselves than the default. But then the only reason for having sum methods would be if there was such a more efficient way of summing. The present situation is still bad, though, as it is inconsistent. I am (of course) open to persuasion! John On Sat, Nov 13, 2010 at 5:18 AM, Dima Pasechnik wrote: > I don't think sum() method is needed. It's certainly a code bloat. > Could you check that sum() in vector_double_dense can be removed? > (remove it there, do sage -b, run testsuite, see if there were any > errors caused by it) > > On Nov 13, 12:59 pm, Maxim wrote: >> If I try to find the sum of a vector of floats that way, it works as >> expected: >> sage: vector([1,float(2),3]).sum() >> -> 6.0 >> >> However, applying the same logic on a vector made of integers: >> sage: vector([1,2,3]).sum() >> -> Traceback (click to the left of this block for traceback) >> ... >> AttributeError: >> 'sage.modules.vector_integer_dense.Vector_integer_dense' >> object has no attribute 'sum' >> >> Workaround (thanks to Dr. Drake) is to call the sum function directly >> like this: >> sage: sum(vector([1,2,3])) >> -> 6 >> >> Browsing through the code, one can find the .sum() method being >> defined for floats at the end of the file vector_double_dense.pyx, >> while this method is absent in the vector_integer_dense.pyx. >> >> Did I luckily stumbled on a rarity, or should this self.sum() method >> be part of every types' definition? (where applicable). in any case, >> uniformity would help (method present in every type definition / >> completely removed) to prevent confusion. > > -- > To post to this group, send an email to sage-devel@googlegroups.com > To unsubscribe from this group, send an email to > sage-devel+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sage-devel > URL: http://www.sagemath.org > -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: self.sum() method on vector made of int is not defined
There could be code in other parts of devel/sage/sage that depends upon .sum(), althouh not too likely. You should run "make testlong" to see if anything breaks (well, it takes a while of course ) And then, you know, open a trac ticket with your patch. On Nov 13, 9:54 pm, Maxim wrote: > On 13 nov, 00:18, Dima Pasechnik wrote: > > > I don't think sum() method is needed. It's certainly a code bloat. > > Could you check that sum() in vector_double_dense can be removed? > > (remove it there, do sage -b, run testsuite, see if there were any > > errors caused by it) > > I just tried that, it worked. The sage -b completed without issue, > then I ran this to test the vector_double_dense object integrity: > sage: v = vector([1,float(2),3)] > sage: type(v) > -> 'sage.modules.vector_real_double_dense.Vector_real_double_dense'> > sage: TestSuite(v).run(verbose = True) > running ._test_category() . . . pass > running ._test_eq() . . . pass > running ._test_not_implemented_methods() . . . pass > running ._test_pickling() . . . pass > > and now doing a: > sage: v.sum() > Traceback (click to the left of this block for traceback) > ... > AttributeError: > 'sage.modules.vector_real_double_dense.Vector_real_double_dense' > object > has no attribute 'sum' > > Which is the same consistent behaviour as for a vector_integer_dense > object, forcing one to use the universal sum(v). -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: self.sum() method on vector made of int is not defined
On 13 nov, 00:18, Dima Pasechnik wrote: > I don't think sum() method is needed. It's certainly a code bloat. > Could you check that sum() in vector_double_dense can be removed? > (remove it there, do sage -b, run testsuite, see if there were any > errors caused by it) I just tried that, it worked. The sage -b completed without issue, then I ran this to test the vector_double_dense object integrity: sage: v = vector([1,float(2),3)] sage: type(v) -> sage: TestSuite(v).run(verbose = True) running ._test_category() . . . pass running ._test_eq() . . . pass running ._test_not_implemented_methods() . . . pass running ._test_pickling() . . . pass and now doing a: sage: v.sum() Traceback (click to the left of this block for traceback) ... AttributeError: 'sage.modules.vector_real_double_dense.Vector_real_double_dense' object has no attribute 'sum' Which is the same consistent behaviour as for a vector_integer_dense object, forcing one to use the universal sum(v). -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: How to profile I/O?
On 2010-11-13 13:29, Simon King wrote: > Perhaps there is a system tool for that purpose. I think it would help > me if some tool would produce a list of all file names to which I/O > happened, and record the I/O time for each file. You might have a look at strace. It is probably not exactly what you want, but it might help. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: How to profile I/O?
Hi Dave! On 12 Nov., 16:22, "Dr. David Kirkby" wrote: > On 11/12/10 03:10 PM, Dr. David Kirkby wrote: > > I meant to say Sage is NOT the place to > > look. vmstat, sar (possibly top) are worth looking at. top would not help. It simply says that my process is sometimes only taking 30% CPU time, but it gives no indication *why*. Note that it is only a guess that I/O are to blame. Moreover, a system tool would not tell me in what part of my programs the problem is located. "prun" might give some hint. However, since my programs use external programs as well as compiled components and interfaces, I was hoping for a tool that tracks all I/O processes occuring during my computation. Perhaps there is a system tool for that purpose. I think it would help me if some tool would produce a list of all file names to which I/O happened, and record the I/O time for each file. Best regards, Simon -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: How to profile I/O?
Hi Jeroen, On 12 Nov., 16:09, Jeroen Demeyer wrote: > Forgive me for stating the obvious, but it might be caused by > 1) swapping out memory to disk > 2) other processes taking up CPU time No, according to top it was the only thing taking CPU time, and the memory consumption was marginal. Cheers, Simon -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: spkg policy: which files to track in "patches" directory?
On Nov 13, 6:29 pm, Jeroen Demeyer wrote: > On 2010-11-13 00:49, John H Palmieri wrote: > > > This question came up on a trac ticket recently: when working on an > > spkg, in the patches directory there might be files like > > > file.py > > file.py.patch > > > Then the spkg-install file copies file.py to the appropriate place in > > src, while file.py.patch documents the changes. Certainly > > file.py.patch should be tracked in the Mercurial repo for the spkg. > > > * Should file.py also be tracked, or should it be in the .hgignore > > file? > > One argument for NOT tracking patched files is to make reviewing easier: > I really dislike looking at spkg patches which consist mostly of > *upstream* patches. I don't need to see how an upstream file changed > between version X and version Y, I need to see the actual changes which > were made for Sage. For that, the patched files are usually sufficient. > > To put it a bit exaggerated: whatever reason we have for not putting the > whole source repository src/ under revision control, the same reason > should apply for not putting patched files under revision control. I'd rather only do the revision control of the src/ files that need to be changed for installation. (the only inconvenience is that "hg status" would report (lots of) ?- marked files.) In this way at least the patches for spkgs are very easy to make and use (and can be automated, too); sage -f/-i can take a directory (rather than an spkg=file) as the argument. Actually, hg can be used instead of patch, so no need to create yet another spkg. Any spkg that is installed after mercurial can just do "sage -hg import/push". Dima > > Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: "Why Sage?" Website Section
On 12 November 2010 18:18, rjf wrote: > > > On Nov 12, 8:46 am, "Dr. David Kirkby" >> I think you have a big bias against python and towards lisp. > > My view is that there has been excessive boosterism for Python, I agree with you there. > asserting that > it is the solution to some important issues in building a system that > is supposed > to displace Mathematica, Maple, Magma, (Maxima?). I don't think anyone believes it will displace those languages. > While Python may > have > some merit in some situations, the case being made for it for Sage is > weak, Given it's your opinion the case for Python is week, what would have been best as a user interface language for users? I can think of a few possibilities worth consideration myself. 1) Create an entirely new language. 2) Python 3) Maxima 4) A Mathematica like interface 5) A MATLAB like interface 6) A Maple-like interface 7) A Magma-like interface 8) Lisp 9) Q - see http://en.wikipedia.org/wiki/Q_%28equational_programming_language%29 10) Pure http://code.google.com/p/pure-lang/ is another possibility, but that did not exist until 2008, but is based on Q, which existed prior to Sage None seem perfect to me. Given the basic rationale of Sage is to "glue" various bits of code together, what in your opinion would have been the best language to glue them together? > which is why I find some bizarre enjoyment in tweaking people who make > these claims. You certainly get some enjoyment out of winding people up. I wish you would spend a bit more time contributing practical information though. You have helped me, and you have helped others too. You almost certainly know more about computer algebra systems than any Sage developer. A bit less time "tweaking people" and a bit more time "tweaking" the Sage source code would be nice! I'd also have a bit more respect for your opinions if you actually tried Sage. It would give you a chance to point out even more shortcomings, which I think you would enjoy. > Like so many people know it. (Why not use Java? or PHP?) > Like it is slow but that's OK, you can use Cython (or something else > that is not Python). > Like it runs everywhere (but if Guido does X, Y, or Z, we can't use > the new version) > Like it is my favorite language (of the one or two I know). > Like it has a natural math syntax (contradicted by almost all > examples). > > >>This comes out in a >> lot of what you say. I would think users of Mathematica consider they write >> in >> Mathematica, though the underlying code is probably C, C++, perhaps even >> Lisp in >> many cases. > > Everyone writes in binary, in underlying code. Yes, but using a hex editor and creating binary directly is not a very practical way of developing software. > More to the point, I think the big selling point of Mathematica > initially and maybe even now, > is presentation of graphics, plotting, etc. For which there are > actually much fancier > programs. But Wolfram (or whoever designed the graphics) made a nice > cut between > complexity and simplicity. Steven Wolfram is no fool. He is a winner of the MacArthur Fellowship (nicknamed the Genius Award) which is currently valued at $500,000 He is a very bright guy, and loves to let people know it. As you say, there are better plotting programs than Mathematica. I forget the one I used to use, but it was closed-source and very good. But Mathematica does a pretty decent job of it too. What Wolfram did well with Mathematica was to combine a wide range of maths tools into one product and to integrate them well. But unlike MATLAB, I'm not convinced Mathematica has wide use outside universities. Sure some companies use it, particularly in the financial sector, but I don't think there's much use of it in industry. I base that on my experience as a late 40's engineer/scientist who has worked in a number of commercial companies, along with results from job searches. > RJF Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 2010-11-13 05:32, Robert Bradshaw wrote: > I think the solution everyone liked was either making patch a > dependency or spkgs, and using something like quilt. The current > situation of copying files and/or manually maintaining patches is a > pain and has several drawbacks. Is there a reason why we don't include GNU patch as a spkg, or it is just a matter of "nobody ever did it"? For the pari spkg (remember #9343?), I actually automated the process of generating patched files from the patches in a file spkg-make. This was really needed, because there were like 10 patches or so and I was making a new spkg at least once per day... Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 2010-11-13 00:49, John H Palmieri wrote: > This question came up on a trac ticket recently: when working on an > spkg, in the patches directory there might be files like > >file.py >file.py.patch > > Then the spkg-install file copies file.py to the appropriate place in > src, while file.py.patch documents the changes. Certainly > file.py.patch should be tracked in the Mercurial repo for the spkg. > > * Should file.py also be tracked, or should it be in the .hgignore > file? One argument for NOT tracking patched files is to make reviewing easier: I really dislike looking at spkg patches which consist mostly of *upstream* patches. I don't need to see how an upstream file changed between version X and version Y, I need to see the actual changes which were made for Sage. For that, the patched files are usually sufficient. To put it a bit exaggerated: whatever reason we have for not putting the whole source repository src/ under revision control, the same reason should apply for not putting patched files under revision control. Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 2010-11-13 09:26, David Kirkby wrote: > 2) A somewhat stronger argument for tracking the patched files too, is > that without doing that, you would be unable to look at an older > version of the file without downloading the source code for that older > version. So if I'm updating version N of a program, and want to know > how we patched version N-2, I need to download the source for N-2 in > order to see how that was patched. Would it not be more useful to see the patch file for that version instead of the patched file for that version? -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: "Why Sage?" Website Section
On 12 November 2010 14:04, Johan S. R. Nielsen wrote: > A little C-procrastination and relocation to my own site, and Google > is ok: > > http://www.student.dtu.dk/~jsrn/whysage.jpg I don't like that. It suffers from promoting Sage as being free first. It is only when you get to number 4 or 5 do we find out it's maths software. You show "Free" as being the number one point, as but I provided in a link, and Richard Fatemen has pointed out, being free is not the most important thing about Sage, or open-source software in general. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: "Why Sage?" Website Section
On 12 November 2010 17:49, Bill Hart wrote: > > > On Nov 12, 4:46 pm, "Dr. David Kirkby" >> When people buy cars, the comfort, economy, look, performance, how nice it >> is to >> drive, are likely to be important to them. I don't suppose 99% care if the >> engine block is made from steel aluminum. People are usually more >> interstellar >> in the interface they are presented with. "interstellar" was meant to be "interested". >> I've yet to see a single complaint from a user that package X is written in C >> and not in Python. >> > > Sage doesn't run on Windows, so I beg to differ. > > Bill. That's a different issue. If Sage was written entirely in assembly code, but run fine on Windows, then Windows uses would be happy. There are two separate issues 1) The interface language 2) What the source code is written in. In the case of Sage, the interface is python. The source code is written in at least all of these: * Assembly code * Bash shell scripts * C * C++ * Fortran * GNU variant of C * GNU variant of C++ * Lisp * Maxima * Python etc. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Memory hits the roof when I divide a sage vector.
On 11/12/10 10:30 PM, Robert Bradshaw wrote: On Fri, Nov 12, 2010 at 7:10 PM, Jason Grout It only has to construct an element if it can't figure out what to do after consulting the Parents themselves. Ah, okay. And then there's the matter you talk about; why is an element so big? The example above is quite strange. No idea why it should be so big. (Note that these are are arbitrary precision integers, so the relative overhead for small ones is still quite large). As for the original example, sage: type(vector(RR, range(100))) Which means that each element is a full Python RealNumber, quite a bit more than a double. Still doesn't explain the memory usage. For almost any kind of linear algebra, you're better off using RDF, or even numpy directly. RDF also seems to have a big problem: sage: v=vector(RDF, range(1)) sage: get_memory_usage() 206.84765625 sage: w=v.parent().an_element() sage: get_memory_usage() 983.60546875 This is very strange, as RDF vectors are literally light wrappers around numpy arrays. I wonder how much Sage library code has to be loaded to do this conversion? Note that an_element() is cached in the parent. Still, that seems odd that it goes up by ~780MB. Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 13 November 2010 02:14, Minh Nguyen wrote: > Hi John, > > On Sat, Nov 13, 2010 at 10:49 AM, John H Palmieri > wrote: >> (Personally, I think it should be tracked, but this is not a strongly >> held opinion. One argument the other way: if it's in .hgignore, then >> the spkg will be smaller, and besides, the relevant information is >> already available in the patch file.) > > I really don't have a strong opinion about the subject of this thread. > Regardless of what policy is adopted, I urge spkg maintainers to > carefully document each patch in the file SPKG.txt. What I'm looking > for is documentation on each patch, and as verbose and as much > documentation as possible (but you don't need to write an essay for > each patch :-). If we adopt a policy, this needs to be documented in > the Developer's Guide. > > -- > Regards > Minh Van Nguyen What would be worth adding to that documentation is the checksum of the files they are patching. "Overwriting src/foobar.c (cksum = 3468988494) with patches/foobar.c (cksum=2395938660), having created patches/foobar.c.diff (cksum=4269132816) " That would allow one to more easily detect cases where unchanged patched files having been copied over upstream source code which has changed. To me, that the biggest problem we have - people update packages, copying over the old patch files, ignoring the fact the upstream source has changed. A more foolproof automatic way of handing this would be nice, but in the absence of that (and to date there;s never been any agreement about what is best approach), keeping a record in SPKG.txt of all checksums would be useful. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] spkg policy: which files to track in "patches" directory?
On 12 November 2010 23:49, John H Palmieri wrote: > This question came up on a trac ticket recently: when working on an > spkg, in the patches directory there might be files like > > file.py > file.py.patch > > Then the spkg-install file copies file.py to the appropriate place in > src, while file.py.patch documents the changes. Certainly > file.py.patch should be tracked in the Mercurial repo for the spkg. > > * Should file.py also be tracked, or should it be in the .hgignore > file? > > (Personally, I think it should be tracked, but this is not a strongly > held opinion. One argument the other way: if it's in .hgignore, then > the spkg will be smaller, and besides, the relevant information is > already available in the patch file.) > > -- > John I don't have a strong opinion on the matter either, though my slight preference is to track them, 1) I don't however believe the disk space argument for not tracking individual files is very strong, as normally the patches are such a small fraction of the actual package, it's not going to make much practical difference. 2) A somewhat stronger argument for tracking the patched files too, is that without doing that, you would be unable to look at an older version of the file without downloading the source code for that older version. So if I'm updating version N of a program, and want to know how we patched version N-2, I need to download the source for N-2 in order to see how that was patched. I have on occasions wanted to see how previous versions were patched, but this is only because I'm sometimes suspicious someone has just overwritten a file when the upstream source has changed. The current system is so error prone, that debugging patches can take some time and so having all the information available can be useful. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
