Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
On May 4, 2015, at 7:56 AM, Tamas Papp tkp...@gmail.com wrote: On Mon, May 04 2015, Scott Jones scott.paul.jo...@gmail.com wrote: On May 4, 2015, at 3:21 AM, Tamas Papp tkp...@gmail.com wrote: I think you misunderstand: IOBuffer is suggested not for mutable string operations in general, but only for efficient concatenation of many strings. Best, Tamas I don’t think that I misunderstood - it’s that using IOBuffer is the only solution that has been given here… and it doesn’t handle what I need to do efficiently... If you have a better solution, please let me know… 1. Can you share the benchmarks (and simplified, self-contained code) for your problem using IOBuffer? I have always found it very fast, but maybe what you are working on is different. It is very fast, for building up things in a buffer… the problem isn’t the speed of IOBuffer, it’s that you can’t do string operations on it (AFAIK), without going back and forth converting it to a immutable string… The other issue is not computer efficiency, but programmer efficiency… The syntax is clumsy, compared to doing something like: `MyBuff ..= “.ext”` I’m a firm believer that most of the time, programmer efficiency is more important than computer efficiency… (For the most part, I think Julia is incredibly good in that aspect, with the powerful metaprogramming and the way with parameterization it can generate a lot of special case code for me, saving me a lot of time, while producing code that is as fast as my hand specialized and optimized C code) 2. Do you have a specific algorithm in mind that would be more efficient? No, just many years of experience trying to speed up the compiler / interpreter of a language used heavily for string / database processing… I never wrote application code, just had customer requests for certain types of operations to be made faster… Best, Tamas Thanks for all the responses… It helps a Julia beginner like me a lot! Scott
RE: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
I would suggest that this topic is moved over to a github issue by those that are interested/participating. I have observed a highly useful pattern of communication over the last year on this list: someone brings up a topic that relates to a design question/improvement of julia, there is a bit of discussion on the mailing list, but once the discussion becomes extensive, someone will open up a github issue, post the link to the issue on this mailinglist, and then the discussion continues on github. This is effective because it keeps long discussion about specific julia design areas that are of no interest to the larger julia-users crowd off this list here (which specifically is NOT about the development/design of julia, but about its use). At the same time those people that are interested in the topic can hammer out a great design on github. I feel this topic checks all the boxes: it is obviously super important and I’m glad there are people looking into how string handling can be improved in julia, but for the vast majority of readers of this list this is really off-topic, and it has taken up a very large share of traffic over the last week or so. So, a github issue seems ideal. Thanks, David From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Scott Jones Sent: Monday, May 4, 2015 5:30 AM To: julia-users@googlegroups.com; Tamas Papp Subject: Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language? On May 4, 2015, at 7:56 AM, Tamas Papp tkp...@gmail.com mailto:tkp...@gmail.com wrote: On Mon, May 04 2015, Scott Jones scott.paul.jo...@gmail.com mailto:scott.paul.jo...@gmail.com wrote: On May 4, 2015, at 3:21 AM, Tamas Papp tkp...@gmail.com mailto:tkp...@gmail.com wrote: I think you misunderstand: IOBuffer is suggested not for mutable string operations in general, but only for efficient concatenation of many strings. Best, Tamas I don’t think that I misunderstood - it’s that using IOBuffer is the only solution that has been given here… and it doesn’t handle what I need to do efficiently... If you have a better solution, please let me know… 1. Can you share the benchmarks (and simplified, self-contained code) for your problem using IOBuffer? I have always found it very fast, but maybe what you are working on is different. It is very fast, for building up things in a buffer… the problem isn’t the speed of IOBuffer, it’s that you can’t do string operations on it (AFAIK), without going back and forth converting it to a immutable string… The other issue is not computer efficiency, but programmer efficiency… The syntax is clumsy, compared to doing something like: `MyBuff ..= “.ext”` I’m a firm believer that most of the time, programmer efficiency is more important than computer efficiency… (For the most part, I think Julia is incredibly good in that aspect, with the powerful metaprogramming and the way with parameterization it can generate a lot of special case code for me, saving me a lot of time, while producing code that is as fast as my hand specialized and optimized C code) 2. Do you have a specific algorithm in mind that would be more efficient? No, just many years of experience trying to speed up the compiler / interpreter of a language used heavily for string / database processing… I never wrote application code, just had customer requests for certain types of operations to be made faster… Best, Tamas Thanks for all the responses… It helps a Julia beginner like me a lot! Scott
Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
David: good call, but on string concatenation specifically an issue was already opened: https://github.com/JuliaLang/julia/issues/11030 On Monday, May 4, 2015 at 12:00:00 PM UTC-7, David Anthoff wrote: I would suggest that this topic is moved over to a github issue by those that are interested/participating. I have observed a highly useful pattern of communication over the last year on this list: someone brings up a topic that relates to a design question/improvement of julia, there is a bit of discussion on the mailing list, but once the discussion becomes extensive, someone will open up a github issue, post the link to the issue on this mailinglist, and then the discussion continues on github. This is effective because it keeps long discussion about specific julia design areas that are of no interest to the larger julia-users crowd off this list here (which specifically is NOT about the development/design of julia, but about its use). At the same time those people that are interested in the topic can hammer out a great design on github. I feel this topic checks all the boxes: it is obviously super important and I’m glad there are people looking into how string handling can be improved in julia, but for the vast majority of readers of this list this is really off-topic, and it has taken up a very large share of traffic over the last week or so. So, a github issue seems ideal. Thanks, David *From:* julia...@googlegroups.com javascript: [mailto: julia...@googlegroups.com javascript:] *On Behalf Of *Scott Jones *Sent:* Monday, May 4, 2015 5:30 AM *To:* julia...@googlegroups.com javascript:; Tamas Papp *Subject:* Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language? On May 4, 2015, at 7:56 AM, Tamas Papp tkp...@gmail.com javascript: wrote: On Mon, May 04 2015, Scott Jones scott.pa...@gmail.com javascript: wrote: On May 4, 2015, at 3:21 AM, Tamas Papp tkp...@gmail.com javascript: wrote: I think you misunderstand: IOBuffer is suggested not for mutable string operations in general, but only for efficient concatenation of many strings. Best, Tamas I don’t think that I misunderstood - it’s that using IOBuffer is the only solution that has been given here… and it doesn’t handle what I need to do efficiently... If you have a better solution, please let me know… 1. Can you share the benchmarks (and simplified, self-contained code) for your problem using IOBuffer? I have always found it very fast, but maybe what you are working on is different. It is very fast, for building up things in a buffer… the problem isn’t the speed of IOBuffer, it’s that you can’t do string operations on it (AFAIK), without going back and forth converting it to a immutable string… The other issue is not computer efficiency, but programmer efficiency… The syntax is clumsy, compared to doing something like: `MyBuff ..= “.ext”` I’m a firm believer that most of the time, programmer efficiency is more important than computer efficiency… (For the most part, I think Julia is incredibly good in that aspect, with the powerful metaprogramming and the way with parameterization it can generate a lot of special case code for me, saving me a lot of time, while producing code that is as fast as my hand specialized and optimized C code) 2. Do you have a specific algorithm in mind that would be more efficient? No, just many years of experience trying to speed up the compiler / interpreter of a language used heavily for string / database processing… I never wrote application code, just had customer requests for certain types of operations to be made faster… Best, Tamas Thanks for all the responses… It helps a Julia beginner like me a lot! Scott
Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
Scott, You shouldn't take my reply personal. It wasn't really about the specific string case you mentioned but more in general about Python julia performance comparisons. On Friday, May 1, 2015 at 3:10:14 PM UTC+2, Scott Jones wrote: On May 1, 2015, at 8:23 AM, Steven Sagaert steven@gmail.com javascript: wrote: On Friday, May 1, 2015 at 12:26:54 PM UTC+2, Scott Jones wrote: On Friday, May 1, 2015 at 4:25:50 AM UTC-4, Steven Sagaert wrote: I think the performance comparisons between Julia Python are flawed. They seem to be between standard Python Julia but since Julia is all about scientific programming it really should be between SciPi Julia. Since SciPi uses much of the same underlying libs in Fortran/C the performance gap will be much smaller and to be really fair it should be between numba compiled SciPi code julia. I suspect the performance will be very close then (and close to C performance). Why should Julia be limited to scientific programming? I think it can be a great language for general programming, I agree but for now the short time future I think the core domain of julia is scientific computing/data science and so to have fair comparisons one should not just compare julia to vanilla Python but especially scipi numba. I stated that my comparisons were of string processing… what’s unfair about that? I have no expertise to compare Julia to any scientific computing system, I’ll leave that to the people here that do (and there are many, very highly qualified). Also, even in technical computing, the performance issues I raise may be of some importance, for example, issues about performance connection to a database… I assume that sometimes you need to read scientific data from a database, and store results to one? Scott
Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
No, I don't take hardly anything personally [my problem is that expect others to do the same]. I just want to understand Julia as best as possible, and improve her if I can... and I think reasoned debates about the technical issues (as opposed to... I just like this better, I think that looks ugly, or I'd rather do things how I'm already familiar with) is great, and I learn a lot. On Friday, May 1, 2015 at 12:52:04 PM UTC-4, Steven Sagaert wrote: Scott, You shouldn't take my reply personal. It wasn't really about the specific string case you mentioned but more in general about Python julia performance comparisons. On Friday, May 1, 2015 at 3:10:14 PM UTC+2, Scott Jones wrote: On May 1, 2015, at 8:23 AM, Steven Sagaert steven@gmail.com wrote: On Friday, May 1, 2015 at 12:26:54 PM UTC+2, Scott Jones wrote: On Friday, May 1, 2015 at 4:25:50 AM UTC-4, Steven Sagaert wrote: I think the performance comparisons between Julia Python are flawed. They seem to be between standard Python Julia but since Julia is all about scientific programming it really should be between SciPi Julia. Since SciPi uses much of the same underlying libs in Fortran/C the performance gap will be much smaller and to be really fair it should be between numba compiled SciPi code julia. I suspect the performance will be very close then (and close to C performance). Why should Julia be limited to scientific programming? I think it can be a great language for general programming, I agree but for now the short time future I think the core domain of julia is scientific computing/data science and so to have fair comparisons one should not just compare julia to vanilla Python but especially scipi numba. I stated that my comparisons were of string processing… what’s unfair about that? I have no expertise to compare Julia to any scientific computing system, I’ll leave that to the people here that do (and there are many, very highly qualified). Also, even in technical computing, the performance issues I raise may be of some importance, for example, issues about performance connection to a database… I assume that sometimes you need to read scientific data from a database, and store results to one? Scott
Re: [julia-users] Performance variability - can we expect Julia to be the fastest (best) language?
On May 1, 2015, at 8:23 AM, Steven Sagaert steven.saga...@gmail.com wrote: On Friday, May 1, 2015 at 12:26:54 PM UTC+2, Scott Jones wrote: On Friday, May 1, 2015 at 4:25:50 AM UTC-4, Steven Sagaert wrote: I think the performance comparisons between Julia Python are flawed. They seem to be between standard Python Julia but since Julia is all about scientific programming it really should be between SciPi Julia. Since SciPi uses much of the same underlying libs in Fortran/C the performance gap will be much smaller and to be really fair it should be between numba compiled SciPi code julia. I suspect the performance will be very close then (and close to C performance). Why should Julia be limited to scientific programming? I think it can be a great language for general programming, I agree but for now the short time future I think the core domain of julia is scientific computing/data science and so to have fair comparisons one should not just compare julia to vanilla Python but especially scipi numba. I stated that my comparisons were of string processing… what’s unfair about that? I have no expertise to compare Julia to any scientific computing system, I’ll leave that to the people here that do (and there are many, very highly qualified). Also, even in technical computing, the performance issues I raise may be of some importance, for example, issues about performance connection to a database… I assume that sometimes you need to read scientific data from a database, and store results to one? Scott
