Re: [julia-users] Weird error about nonexistence of a method
Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles (http://julia.readthedocs.org/en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com javascript: wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
Re: [julia-users] Roadmap
Sorry for the stupid question, but how is TCO relevant for Julia? Not even all Lisps have TCO in the standard (eg Common Lisp doesn't). Is Little Schemer-like heavily recursive code advocated anywhere in the Julia community? I thought the paradigm Julia favors is loops and maybe some functional code. Best, Tamas On Thu, Dec 11 2014, Mike Innes mike.j.in...@gmail.com wrote: https://github.com/JuliaLang/julia/issues/4964 On 11 December 2014 at 11:55, Uwe Fechner uwe.fechner@gmail.com wrote: What do you mean with TCO? On Thursday, December 11, 2014 10:50:19 AM UTC+1, Mike Innes wrote: It seems to me that a lot of FAQs could be answered by a simple list of the communities'/core developers' priorities. For example: We care about module load times and static compilation, so that's going to happen eventually. We care about package documentation, which is basically done. We don't care as much about deterministic memory management or TCO, so neither of those things are happening any time soon. It doesn't have to be a commitment to releases or dates, or even be particularly detailed, to give a good sense of where Julia is headed from a user perspective. Indeed, it's only the same things you end up posting on HN every time someone complains that Gadfly is slow. On 11 December 2014 at 03:01, Tim Holy tim@gmail.com wrote: Really nice summaries, John and Tony. On Thursday, December 11, 2014 02:08:54 AM Boylan, Ross wrote: BTW, is 0.4 still in a you don't want to go there state for users of julia? In short, yes---for most users I'd personally recommend sticking with 0.3. Unless you simply _must_ have some of its lovely new features. But be prepared to update your code basically every week or so to deal with changes. --Tim
[julia-users] [ANN] TermWin 0.0.26 new features on DataFrames
Fellow julia users,
I'd like to share a new feature in the TermWin
https://github.com/tonyhffong/TermWin.jl package for navigating
dataframes.
Basic usage:
using TermWin
using DataFrames
df = ... # read/create your dataframe
tshow( df )
With a bit of customization, you can generate very elaborate pivots to help
get a feel of the data.
Here's an example using RDatasets's Ecdat/caschool data (the code is in
test/dataframe.jl):
https://lh5.googleusercontent.com/-4hvC7oq4dZE/VIrVN6BfcII/AXY/8XT9eJLTKOU/s1600/caschool.png
Here the top5district pivot (after County) is a calculated pivot. It is
not a static column but a dynamic one that is generated based on the path
of the tree nodes, so
as you move this pivot further up or down the pivot chain, it would be
adjusted to the context/subdataframe correctly. Another example of
calculated pivot is discretization on aggregated values, which can be found
on the view selector ('v' keyboard shortcut) in the same example. You can
also change the pivots directly using the 'p' shortcut.
The framework borrows ideas heavily from DataFramesMeta.jl (and relies on
some of its code, too) to structure a lazy tree while keeping aggregation
and pivoting quite performant by lifting the specifications into compiled
functions. Numbers formatting leverages the rather performant code from
Formatting.jl
Give it a spin and let me know how it goes.
Tony
[julia-users] Re: THANKS to Julia core developers!
You mean it's better than even the R guys? Maintaining culture in the long run will be hard.
Re: [julia-users] Weird error about nonexistence of a method
I would love to detect this kind of confusing situation and indicate what's happening better because obviously it's pretty confusing when you first encounter it. On Fri, Dec 12, 2014 at 4:37 AM, Test This curiousle...@gmail.com wrote: Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles ( http://julia.readthedocs.org/en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
Re: [julia-users] How do I add a column to a subdataframe?
Tony, it's not possible to manipulate a subdataframe like that with the current code. You can make something that works close, though. In Julia v0.4, you can create a DataFrame with columns that are views into the original columns using `sub`. Here is an example: df = DataFrame(a = rand(1:3, 10), x = rand(10), y = rand(10)) idx = find(df[:a] .== 1) sd = DataFrame(Any[sub(df[i], idx) for i in 1:size(df, 2)], names(df)) Try `dump(sd)` to view the structure of the result. Now, you can use `sd` like a SubDataFrame, but you can do any normal DataFrame manipulation. If the starting point is a SubDataFrame, you can convert it as follows: sdf = sub(df, df[:a] .== 1) sd2 = DataFrame(Any[sub(sdf.parent[i], idx) for i in 1:size(sdf, 2)], names(sdf)) One huge improvement of this is approach is that `sd[:colA]` doesn't allocate anything. With the existing SubDataFrames implementation, a column indexing operation like that does a row indexing operation into the parent DataFrame, so it will allocate memory. That differs from a DataFrame where column indexing never allocates anything. I've been thinking that this might be a good change in general for `sub`. If Base Arrays switch to views by default, this might match better with that. In fact, we could get rid of `sub` and do this for all row indexing operations. It could use some more thought. This approach doesn't work in Julia v0.3 because `sub` did not support arbitrary indexes in v0.3. On Fri, Dec 12, 2014 at 1:58 AM, Tony Fong tony.hf.f...@gmail.com wrote: In a vanilla dataframe, I can do this df[ :a ] = mydarray It doesn't seem to allow me to do the same for a subdataframe returned by a groupby. I'm trying to compute a new column based on the subdataframe, attach this column to it and then do further groupby. Is there a way to do that? Tony
Re: [julia-users] [ANN] TermWin 0.0.26 new features on DataFrames
This looks amazing! I'm really excited to try this...
On Fri, Dec 12, 2014 at 7:10 AM, Tony Fong tony.hf.f...@gmail.com wrote:
Fellow julia users,
I'd like to share a new feature in the TermWin
https://github.com/tonyhffong/TermWin.jl package for navigating
dataframes.
Basic usage:
using TermWin
using DataFrames
df = ... # read/create your dataframe
tshow( df )
With a bit of customization, you can generate very elaborate pivots to
help get a feel of the data.
Here's an example using RDatasets's Ecdat/caschool data (the code is in
test/dataframe.jl):
https://lh5.googleusercontent.com/-4hvC7oq4dZE/VIrVN6BfcII/AXY/8XT9eJLTKOU/s1600/caschool.png
Here the top5district pivot (after County) is a calculated pivot. It
is not a static column but a dynamic one that is generated based on the
path of the tree nodes, so
as you move this pivot further up or down the pivot chain, it would be
adjusted to the context/subdataframe correctly. Another example of
calculated pivot is discretization on aggregated values, which can be found
on the view selector ('v' keyboard shortcut) in the same example. You can
also change the pivots directly using the 'p' shortcut.
The framework borrows ideas heavily from DataFramesMeta.jl (and relies on
some of its code, too) to structure a lazy tree while keeping aggregation
and pivoting quite performant by lifting the specifications into compiled
functions. Numbers formatting leverages the rather performant code from
Formatting.jl
Give it a spin and let me know how it goes.
Tony
Re: [julia-users] Roadmap
Personally I'm using it in Lazy https://github.com/one-more-minute/Lazy.jl/blob/b5927a01f7ab8f95565d5bbe36a175b64825eda6/src/liblazy.jl#L102-L103, just because it's the natural way to express that kind of problem. I'm only using tail self-calls though, which are easy enough to optimise away via a macro. On 12 December 2014 at 10:39, Tamas Papp tkp...@gmail.com wrote: Sorry for the stupid question, but how is TCO relevant for Julia? Not even all Lisps have TCO in the standard (eg Common Lisp doesn't). Is Little Schemer-like heavily recursive code advocated anywhere in the Julia community? I thought the paradigm Julia favors is loops and maybe some functional code. Best, Tamas On Thu, Dec 11 2014, Mike Innes mike.j.in...@gmail.com wrote: https://github.com/JuliaLang/julia/issues/4964 On 11 December 2014 at 11:55, Uwe Fechner uwe.fechner@gmail.com wrote: What do you mean with TCO? On Thursday, December 11, 2014 10:50:19 AM UTC+1, Mike Innes wrote: It seems to me that a lot of FAQs could be answered by a simple list of the communities'/core developers' priorities. For example: We care about module load times and static compilation, so that's going to happen eventually. We care about package documentation, which is basically done. We don't care as much about deterministic memory management or TCO, so neither of those things are happening any time soon. It doesn't have to be a commitment to releases or dates, or even be particularly detailed, to give a good sense of where Julia is headed from a user perspective. Indeed, it's only the same things you end up posting on HN every time someone complains that Gadfly is slow. On 11 December 2014 at 03:01, Tim Holy tim@gmail.com wrote: Really nice summaries, John and Tony. On Thursday, December 11, 2014 02:08:54 AM Boylan, Ross wrote: BTW, is 0.4 still in a you don't want to go there state for users of julia? In short, yes---for most users I'd personally recommend sticking with 0.3. Unless you simply _must_ have some of its lovely new features. But be prepared to update your code basically every week or so to deal with changes. --Tim
Re: [julia-users] Re: Aren't loops supposed to be faster?
On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
Re: [julia-users] Re: Aren't loops supposed to be faster?
From the original version of the code I see this timing after code gen: julia @time Ke1(N); elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc time) julia @time Ke2(N); elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc time) After making all the globals const, I see this: julia @time Ke1(N); elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc time) julia @time Ke2(N); elapsed time: 0.026985097 seconds (384 bytes allocated) Type annotations everywhere are neither necessary nor recommended. It is recommended not to use lots of non-constant globals. On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim.h...@gmail.com wrote: On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
Re: [julia-users] Re: Aren't loops supposed to be faster?
Stefan, please consider the updated version https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923 which has no global variables. 2014-12-12 10:07 GMT-05:00 Stefan Karpinski ste...@karpinski.org: From the original version of the code I see this timing after code gen: julia @time Ke1(N); elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc time) julia @time Ke2(N); elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc time) After making all the globals const, I see this: julia @time Ke1(N); elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc time) julia @time Ke2(N); elapsed time: 0.026985097 seconds (384 bytes allocated) Type annotations everywhere are neither necessary nor recommended. It is recommended not to use lots of non-constant globals. On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim.h...@gmail.com wrote: On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
Re: [julia-users] Roadmap
I think that tail call optimization is mostly relevant in the sense that the question is asked from time to time if/when Julia will support it/why it's not supported already.
Re: [julia-users] Re: Aren't loops supposed to be faster?
Yikes, that's much harder to read than the original. Making the globals const is a simple change and the fact that this improves the performance so much shows that all those type annotations are not necessary. The only issue with the original code was non-constant globals. We can also change the globals to function arguments and the problem goes away: julia @time Ke1(N); elapsed time: 0.237183958 seconds (131200224 bytes allocated, 34.34% gc time) julia @time Ke2(N); elapsed time: 0.021313495 seconds (400 bytes allocated) The gist has both the const and function-arg versions: https://gist.github.com/StefanKarpinski/e1d7d8804e373cc1de07 On Fri, Dec 12, 2014 at 10:12 AM, Andreas Noack andreasnoackjen...@gmail.com wrote: Stefan, please consider the updated version https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923 which has no global variables. 2014-12-12 10:07 GMT-05:00 Stefan Karpinski ste...@karpinski.org: From the original version of the code I see this timing after code gen: julia @time Ke1(N); elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc time) julia @time Ke2(N); elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc time) After making all the globals const, I see this: julia @time Ke1(N); elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc time) julia @time Ke2(N); elapsed time: 0.026985097 seconds (384 bytes allocated) Type annotations everywhere are neither necessary nor recommended. It is recommended not to use lots of non-constant globals. On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim.h...@gmail.com wrote: On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
Re: [julia-users] Weird error about nonexistence of a method
Hi Stefan, Please see below for what may be relevant code structure. I am happy to email the actual code off the list if you think that is necessary. The code which resulted in the problem had three different files. 1. dataTypes.jl (this file defines the Params composite type). 2. paramcombos.jl (there is a baseparams() function in this file, which creates an instance of Params type). 3. runsim.jl (the simulate function which gave the error was in this file) I did not declare any module statements in either dataTypes.jl or runsim.jl, but did declare module paramcombos in paramcombos.jl. The relevant structure of the files: - *## File dataTypes.jl* type Params . . . end - *## File paramcombos.jl* module paramcombos import Distributions include(dataTypes.jl) function baseparams() ... # body of this function return Params(...) end end - *## File runsim.jl* include(dataTypes.jl) require(paramcombos.jl) function simulate(params::Params, rseed::Int) ... # body of this function end function main() basep = paramcombos.baseparams() simulate(basep, 1) end main() Thanks for looking into this. On Friday, December 12, 2014 8:16:40 AM UTC-5, Stefan Karpinski wrote: I would love to detect this kind of confusing situation and indicate what's happening better because obviously it's pretty confusing when you first encounter it. On Fri, Dec 12, 2014 at 4:37 AM, Test This curiou...@gmail.com javascript: wrote: Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles ( http://julia.readthedocs.org/en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
Re: [julia-users] Weird error about nonexistence of a method
Yeah, that will do it. You should only include dataTypes.jl in one place. On Fri, Dec 12, 2014 at 10:28 AM, Test This curiousle...@gmail.com wrote: Hi Stefan, Please see below for what may be relevant code structure. I am happy to email the actual code off the list if you think that is necessary. The code which resulted in the problem had three different files. 1. dataTypes.jl (this file defines the Params composite type). 2. paramcombos.jl (there is a baseparams() function in this file, which creates an instance of Params type). 3. runsim.jl (the simulate function which gave the error was in this file) I did not declare any module statements in either dataTypes.jl or runsim.jl, but did declare module paramcombos in paramcombos.jl. The relevant structure of the files: - *## File dataTypes.jl* type Params . . . end - *## File paramcombos.jl* module paramcombos import Distributions include(dataTypes.jl) function baseparams() ... # body of this function return Params(...) end end - *## File runsim.jl* include(dataTypes.jl) require(paramcombos.jl) function simulate(params::Params, rseed::Int) ... # body of this function end function main() basep = paramcombos.baseparams() simulate(basep, 1) end main() Thanks for looking into this. On Friday, December 12, 2014 8:16:40 AM UTC-5, Stefan Karpinski wrote: I would love to detect this kind of confusing situation and indicate what's happening better because obviously it's pretty confusing when you first encounter it. On Fri, Dec 12, 2014 at 4:37 AM, Test This curiou...@gmail.com wrote: Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles (http://julia.readthedocs.org/ en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
Re: [julia-users] Re: Aren't loops supposed to be faster?
The problem in the updated version I linked to seems to be extracting the type of the properties of nested composite types so wondered if you had any general comment to that, i.e. using complicated types as inputs. 2014-12-12 10:25 GMT-05:00 Stefan Karpinski ste...@karpinski.org: Yikes, that's much harder to read than the original. Making the globals const is a simple change and the fact that this improves the performance so much shows that all those type annotations are not necessary. The only issue with the original code was non-constant globals. We can also change the globals to function arguments and the problem goes away: julia @time Ke1(N); elapsed time: 0.237183958 seconds (131200224 bytes allocated, 34.34% gc time) julia @time Ke2(N); elapsed time: 0.021313495 seconds (400 bytes allocated) The gist has both the const and function-arg versions: https://gist.github.com/StefanKarpinski/e1d7d8804e373cc1de07 On Fri, Dec 12, 2014 at 10:12 AM, Andreas Noack andreasnoackjen...@gmail.com wrote: Stefan, please consider the updated version https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923 which has no global variables. 2014-12-12 10:07 GMT-05:00 Stefan Karpinski ste...@karpinski.org: From the original version of the code I see this timing after code gen: julia @time Ke1(N); elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc time) julia @time Ke2(N); elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc time) After making all the globals const, I see this: julia @time Ke1(N); elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc time) julia @time Ke2(N); elapsed time: 0.026985097 seconds (384 bytes allocated) Type annotations everywhere are neither necessary nor recommended. It is recommended not to use lots of non-constant globals. On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim.h...@gmail.com wrote: On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
[julia-users] Good exemplar Julia programs for code style/organization?
Hi, all. I'm slowly moving over to Julia from Python. There is just enough of a difference of Julia from other languages that I'd like to find a well-written, small to medium-sized Julia program to study its coding style and code organization. It doesn't have to be huge; just a well-organized collection of modules and classes/types. Any nominations for exemplary Julia? (I've gone through various library code files, but libraries aren't organized the same as programs. I've skimmed a few of the programs in Julia.jl but it's hard to determine what's considered well-written.) Thanks, -Tom
[julia-users] Good exemplar Julia programs for code style/organization?
Hi, all. I'm slowly moving over to Julia from Python. There is just enough of a difference of Julia from other languages that I'd like to find a well-written, small to medium-sized Julia program to study its coding style and code organization. It doesn't have to be huge; just a well-organized collection of modules and classes/types. Any nominations for exemplary Julia? (I've gone through various library code files, but libraries aren't organized the same as programs. I've skimmed a few of the programs in Julia.jl but it's hard to determine what's considered well-written.) Thanks, -Tom
Re: [julia-users] Re: Aren't loops supposed to be faster?
Stefan, If you mean by the original code Ke1() and Ke2(), those were contrived examples that unfortunately did not even represent the problems in my actual code (conductivity(), https://gist.github.com/PetrKryslUCSD/ae4a0f218fe50abe370f https://www.google.com/url?q=https%3A%2F%2Fgist.github.com%2FPetrKryslUCSD%2Fae4a0f218fe50abe370fsa=Dsntz=1usg=AFQjCNHJQsa8ICKiXobesCkCSCKvhdW9dQ). That code had no globals whatsoever. The solution (https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923) was to declare anything and everything. Now I've had more time to investigate,, and I've turned off the declarations selectively to see which ones were actually needed. The result: I actually need four declarations. Some of it may have to do with my inexperience with the language – I may not have made it possible for the compiler to discover the types. Some of it was addressed by John above (dynamically called function, with stable type, but not enough information for the compiler). The code (note the comments): https://gist.github.com/PetrKryslUCSD/4f57d4d62758ecc5bf20 Petr PS: Concerning my attempt to formulate a moral for myself: I stand corrected, I'm beginning to get the idea that Julia does not need everything declared. On Friday, December 12, 2014 7:26:28 AM UTC-8, Stefan Karpinski wrote: Yikes, that's much harder to read than the original. Making the globals const is a simple change and the fact that this improves the performance so much shows that all those type annotations are not necessary. The only issue with the original code was non-constant globals. We can also change the globals to function arguments and the problem goes away: julia @time Ke1(N); elapsed time: 0.237183958 seconds (131200224 bytes allocated, 34.34% gc time) julia @time Ke2(N); elapsed time: 0.021313495 seconds (400 bytes allocated) The gist has both the const and function-arg versions: https://gist.github.com/StefanKarpinski/e1d7d8804e373cc1de07 On Fri, Dec 12, 2014 at 10:12 AM, Andreas Noack andreasno...@gmail.com javascript: wrote: Stefan, please consider the updated version https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923 which has no global variables. 2014-12-12 10:07 GMT-05:00 Stefan Karpinski ste...@karpinski.org javascript:: From the original version of the code I see this timing after code gen: julia @time Ke1(N); elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc time) julia @time Ke2(N); elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc time) After making all the globals const, I see this: julia @time Ke1(N); elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc time) julia @time Ke2(N); elapsed time: 0.026985097 seconds (384 bytes allocated) Type annotations everywhere are neither necessary nor recommended. It is recommended not to use lots of non-constant globals. On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim@gmail.com javascript: wrote: On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote: The moral of this story is: If you can't or won't declare every single variable, don't do loops. They are likely to be a losing proposition. Just to follow up further, this is not at all the right moral to absorb from this. A better set of morals is: - use code_typed or TypeCheck or Lint to diagnose problems - remember that julia optimizes functions as a unit. Therefore, in a loop with a type issue, one brute force solution is to create a separate function just for running that loop. The types will be known when that function gets compiled (even if you don't annotate anything with their types), and so your type problem should evaporate. --Tim
Re: [julia-users] Weird error about nonexistence of a method
Does that restriction (including file in only one place) apply to import or using? I think not, but just confirming. Sorry if the docs clearly address that, but I don't remember. Thank you. On Friday, December 12, 2014 10:40:56 AM UTC-5, Stefan Karpinski wrote: Yeah, that will do it. You should only include dataTypes.jl in one place. On Fri, Dec 12, 2014 at 10:28 AM, Test This curiou...@gmail.com javascript: wrote: Hi Stefan, Please see below for what may be relevant code structure. I am happy to email the actual code off the list if you think that is necessary. The code which resulted in the problem had three different files. 1. dataTypes.jl (this file defines the Params composite type). 2. paramcombos.jl (there is a baseparams() function in this file, which creates an instance of Params type). 3. runsim.jl (the simulate function which gave the error was in this file) I did not declare any module statements in either dataTypes.jl or runsim.jl, but did declare module paramcombos in paramcombos.jl. The relevant structure of the files: - *## File dataTypes.jl* type Params . . . end - *## File paramcombos.jl* module paramcombos import Distributions include(dataTypes.jl) function baseparams() ... # body of this function return Params(...) end end - *## File runsim.jl* include(dataTypes.jl) require(paramcombos.jl) function simulate(params::Params, rseed::Int) ... # body of this function end function main() basep = paramcombos.baseparams() simulate(basep, 1) end main() Thanks for looking into this. On Friday, December 12, 2014 8:16:40 AM UTC-5, Stefan Karpinski wrote: I would love to detect this kind of confusing situation and indicate what's happening better because obviously it's pretty confusing when you first encounter it. On Fri, Dec 12, 2014 at 4:37 AM, Test This curiou...@gmail.com wrote: Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles (http://julia.readthedocs.org/ en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
Re: [julia-users] Weird error about nonexistence of a method
include works as if you had pasted the contents of the file at the point where it's called – if you call it in multiple places, it's just as if you'd cut-and-pasted the code. import, using and require load things once and import and using are expected to produce a module by the given name. On Fri, Dec 12, 2014 at 12:59 PM, Test This curiousle...@gmail.com wrote: Does that restriction (including file in only one place) apply to import or using? I think not, but just confirming. Sorry if the docs clearly address that, but I don't remember. Thank you. On Friday, December 12, 2014 10:40:56 AM UTC-5, Stefan Karpinski wrote: Yeah, that will do it. You should only include dataTypes.jl in one place. On Fri, Dec 12, 2014 at 10:28 AM, Test This curiou...@gmail.com wrote: Hi Stefan, Please see below for what may be relevant code structure. I am happy to email the actual code off the list if you think that is necessary. The code which resulted in the problem had three different files. 1. dataTypes.jl (this file defines the Params composite type). 2. paramcombos.jl (there is a baseparams() function in this file, which creates an instance of Params type). 3. runsim.jl (the simulate function which gave the error was in this file) I did not declare any module statements in either dataTypes.jl or runsim.jl, but did declare module paramcombos in paramcombos.jl. The relevant structure of the files: - *## File dataTypes.jl* type Params . . . end - *## File paramcombos.jl* module paramcombos import Distributions include(dataTypes.jl) function baseparams() ... # body of this function return Params(...) end end - *## File runsim.jl* include(dataTypes.jl) require(paramcombos.jl) function simulate(params::Params, rseed::Int) ... # body of this function end function main() basep = paramcombos.baseparams() simulate(basep, 1) end main() Thanks for looking into this. On Friday, December 12, 2014 8:16:40 AM UTC-5, Stefan Karpinski wrote: I would love to detect this kind of confusing situation and indicate what's happening better because obviously it's pretty confusing when you first encounter it. On Fri, Dec 12, 2014 at 4:37 AM, Test This curiou...@gmail.com wrote: Thanks Stefan. Yes, it looks like this is related to my lack of clear understanding about include vs. require. I am reading the docs John Myles (http://julia.readthedocs.org/ en/release-0.3/manual/modules/) directed me to and will report back on whether that helps me fix the issue. On Thursday, December 11, 2014 11:54:55 PM UTC-5, Stefan Karpinski wrote: It is possible that you have managed to get into a state where there are two different types by the name Params. On Dec 11, 2014, at 9:10 PM, Test This curiou...@gmail.com wrote: I am running into what appears to be weird error. I have this function simulate that takes two arguments. When I try to run the file containing this function I get the following error. I have added println( methods(simulate) ) to the code so that you can see its methods. *# 1 method for generic function simulate:* *simulate(params::Params,rseed::Int64) at /Users/code/simulationcode.jl:340* *ERROR: `simulate` has no method matching simulate(::Params, ::Int64)* Are the 2nd and 3rd lines not contradictory? Thanks in advance for your help.
[julia-users] Announcing RobotOS.jl
After about 6 weeks of initial part-time development, I'm announcing the first release of the RobotOS.jl package, which enables essentially seamless integration of Julia code with ROS http://wiki.ros.org (Robot Operating System). At the core, it is mostly a wrapper for the rospy python library (many thanks to Steve Johnson for PyCall), but on top of that I added an automatic Julia type generation system so the back-end python details are completely hidden from the end-user. https://github.com/phobon/RobotOS.jl I believe the robotics community is especially well suited to adopt Julia, as speed and natural mathematical expressiveness are both highly desirable features in whichever programming language is used. For about six months now, I've been doing all my research in Julia and it has been vastly more pleasurable than using either Matlab or Python. I am quite eager to continue development on the package with whatever community feedback I can get but hopefully it can already prove useful to anyone out there who has already thought of using the two systems together. Thanks, Josh Langsfeld Graduate Research Assistant Maryland Robotics Center - UMD
Re: [julia-users] Announcing RobotOS.jl
Very cool. Glad you've been enjoying Julia so much, Josh. -- John On Dec 12, 2014, at 2:06 PM, Josh Langsfeld jdla...@gmail.com wrote: After about 6 weeks of initial part-time development, I'm announcing the first release of the RobotOS.jl package, which enables essentially seamless integration of Julia code with ROS (Robot Operating System). At the core, it is mostly a wrapper for the rospy python library (many thanks to Steve Johnson for PyCall), but on top of that I added an automatic Julia type generation system so the back-end python details are completely hidden from the end-user. https://github.com/phobon/RobotOS.jl I believe the robotics community is especially well suited to adopt Julia, as speed and natural mathematical expressiveness are both highly desirable features in whichever programming language is used. For about six months now, I've been doing all my research in Julia and it has been vastly more pleasurable than using either Matlab or Python. I am quite eager to continue development on the package with whatever community feedback I can get but hopefully it can already prove useful to anyone out there who has already thought of using the two systems together. Thanks, Josh Langsfeld Graduate Research Assistant Maryland Robotics Center - UMD
Re: [julia-users] Re: Aren't loops supposed to be faster?
I have now been able to remove the need for all declarations except one,
Rm still needs to be declared as returning an array of floats.
Just before the loop I look at the type of the function that retrieves Rm
and I get
{:($(Expr(:lambda, {:XYZ,:tangents,:fe_label},
{{},{{:XYZ,Array{Float64,2},0},{:tangents,Array{Float64,2},0},{:fe_label,Int64,0}},{{:eR
m,Array{Float64,2},19}}}, :(begin #
C:\Users\pkrysl\Documents\Research\Software folder\FEA software
folder\jfineale\FEMMBaseModule.jl,
line 82:
return eRm::Array{Float64,2}
end::Array{Float64,2}}
This seems to be saying that the matrix returns the correct type. Am I
wrong?
Thanks,
Petr
On Friday, December 12, 2014 9:17:41 AM UTC-8, Petr Krysl wrote:
Stefan,
If you mean by the original code Ke1() and Ke2(), those were contrived
examples that unfortunately did not even represent the problems in my
actual code (conductivity(),
https://gist.github.com/PetrKryslUCSD/ae4a0f218fe50abe370f
https://www.google.com/url?q=https%3A%2F%2Fgist.github.com%2FPetrKryslUCSD%2Fae4a0f218fe50abe370fsa=Dsntz=1usg=AFQjCNHJQsa8ICKiXobesCkCSCKvhdW9dQ).
That code had no globals whatsoever.
The solution (https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923)
was to declare anything and everything. Now I've had more time to
investigate,, and I've turned off the declarations selectively to see
which ones were actually needed. The result: I actually need four
declarations. Some of it may have to do with my inexperience with the
language – I may not have made it possible for the compiler to discover
the types. Some of it was addressed by John above (dynamically called
function, with stable type, but not enough information for the compiler).
The code (note the comments):
https://gist.github.com/PetrKryslUCSD/4f57d4d62758ecc5bf20
Petr
PS: Concerning my attempt to formulate a moral for myself: I stand
corrected, I'm beginning to get the idea that Julia does not need
everything declared.
On Friday, December 12, 2014 7:26:28 AM UTC-8, Stefan Karpinski wrote:
Yikes, that's much harder to read than the original. Making the globals
const is a simple change and the fact that this improves the performance so
much shows that all those type annotations are not necessary. The only
issue with the original code was non-constant globals. We can also change
the globals to function arguments and the problem goes away:
julia @time Ke1(N);
elapsed time: 0.237183958 seconds (131200224 bytes allocated, 34.34% gc
time)
julia @time Ke2(N);
elapsed time: 0.021313495 seconds (400 bytes allocated)
The gist has both the const and function-arg versions:
https://gist.github.com/StefanKarpinski/e1d7d8804e373cc1de07
On Fri, Dec 12, 2014 at 10:12 AM, Andreas Noack andreasno...@gmail.com
wrote:
Stefan, please consider the updated version
https://gist.github.com/PetrKryslUCSD/7bd14515e1a853275923
which has no global variables.
2014-12-12 10:07 GMT-05:00 Stefan Karpinski ste...@karpinski.org:
From the original version of the code I see this timing after code gen:
julia @time Ke1(N);
elapsed time: 0.251365495 seconds (134400208 bytes allocated, 30.13% gc
time)
julia @time Ke2(N);
elapsed time: 3.923532621 seconds (996800384 bytes allocated, 13.97% gc
time)
After making all the globals const, I see this:
julia @time Ke1(N);
elapsed time: 0.273683599 seconds (131200208 bytes allocated, 28.52% gc
time)
julia @time Ke2(N);
elapsed time: 0.026985097 seconds (384 bytes allocated)
Type annotations everywhere are neither necessary nor recommended. It
is recommended not to use lots of non-constant globals.
On Fri, Dec 12, 2014 at 9:36 AM, Tim Holy tim@gmail.com wrote:
On Thursday, December 11, 2014 08:10:38 PM Petr Krysl wrote:
The moral of this story is: If you can't or won't declare every
single
variable, don't do loops. They are likely to be a losing proposition.
Just to follow up further, this is not at all the right moral to
absorb from
this. A better set of morals is:
- use code_typed or TypeCheck or Lint to diagnose problems
- remember that julia optimizes functions as a unit. Therefore, in a
loop with
a type issue, one brute force solution is to create a separate
function just
for running that loop. The types will be known when that function gets
compiled (even if you don't annotate anything with their types), and
so your
type problem should evaporate.
--Tim
Re: [julia-users] Re: home page content
On Wednesday, December 10, 2014 7:30:42 PM UTC-5, Tony Kelman wrote: This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. Yes, indeed it was. Conferences, end-of-semester, and my own version of Jeff's issue #8839 https://github.com/JuliaLang/julia/issues/8839 all meant that this slid off my desk. But, there's good news! Some of the folks in the /r/Julia http://www.reddit.com/r/Julia/ subreddit are picking up my slack! Check out their first (stickied) post: This Fortnight In Julia: 12 Dec - 26 Dec http://www.reddit.com/r/Julia/comments/2p36av/this_fortnight_in_julia_12_dec_26_dec/ http://www.reddit.com/r/Julia/comments/2p36av/this_fortnight_in_julia_12_dec_26_dec/
[julia-users] Re: MatrixDepot.jl: A Test Matrix Collection
great references, Weijian ... it turns out that the UF Sparse collection has a filed issue at: https://github.com/JuliaSparse/MatrixMarket.jl/issues/1 i understand that the UF Sparse collection is primarily in Rutherford-Boeing format and that a read-in tool for this may be in the works ... best, cdm On Friday, December 12, 2014 12:09:41 PM UTC-8, Weijian Zhang wrote: Hi cdm, Thanks a lot for the comments. Just for reference: If you want to read matrices in the Matrix Market format, you can use MatrixMarket.jl (https://github.com/JuliaSparse/MatrixMarket.jl). If your matrices are in Harwell Boeing Format, there is HarwellBoeing.jl ( https://github.com/dpo/HarwellBoeing.jl) Best wishes, Weijian
[julia-users] Running on multiple machines
I have parallelized my algorithm using pmap. The performance improvement on one machine union -p option is is great. Now I would like to run on multiple machines. I used --machinefile option on julia start. It works but it launches only one process on remote machine. I would like to have multiple processes running on each machine. Option -p is enabling multiple processes only on local machine. Is there a way to specify number of processes on remote machines. Thanks Dejan
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
[julia-users] Re: Running on multiple machines
Thanks!!! On Friday, December 12, 2014 2:18:27 PM UTC-8, ele...@gmail.com wrote: Simply repeat the entries in the machine file the appropriate number of times. On Saturday, December 13, 2014 8:09:04 AM UTC+10, Dejan Miljkovic wrote: I have parallelized my algorithm using pmap. The performance improvement on one machine union -p option is is great. Now I would like to run on multiple machines. I used --machinefile option on julia start. It works but it launches only one process on remote machine. I would like to have multiple processes running on each machine. Option -p is enabling multiple processes only on local machine. Is there a way to specify number of processes on remote machines. Thanks Dejan
[julia-users] Are some matrices marked as special?
Are perhaps matrices initialized as all zeros or identity matrices marked as special behind the scenes, so that no actual processing is done when there used for instance in multiplication? Thanks, Petr
Re: [julia-users] Re: Community Support for Julia on Travis CI
Can it be possible to add some kind of support for code coverage ? I use this package (https://github.com/IainNZ/Coverage.jl) which seems to be a standard one in Julia. For now I use a custom script in my .tramis.yml file, but the only modification in the build script is |julia -e 'Pkg.test(MyPkg, coverage=true)'| vs the official |julia -e 'Pkg.test(MyPkg)'| so why not make this coverage=true the default ? To enable the coverall.io output, one would then only have to add after_success: -julia -e 'cd(Pkg.dir(MyPkg)); Pkg.add(Coverage); using Coverage; Coveralls.submit(Coveralls.process_folder())' In their .tramis.yml file. Or there can be a special syntax for that if needed. Guillaume Pontus Stenetorp a écrit : On 12 December 2014 at 00:28, Stefan Karpinskiste...@karpinski.org wrote: Btw, can I be added to the JuliaCI org? No idea who, but someone has taken care of it. If someone else wants to join, just give us a poke. On 12 December 2014 at 00:49, Nils Gudatnils.gu...@gmail.com wrote: Excuse the ignorant question, but what exactly does this mean? I haven't seen Travis CI before and clicking on the home page is slightly confusing... Ignorance is fine, at least in academia you need some of it in order to accomplish anything. I agree that the homepage is a mess, essentially they have a bunch of virtual servers that check if you have a pending pull request or made a push to your repository. Once this happens they will pull the code from GitHub and build/run tests on their machines to check that everything is all right. If something is wrong, you will get an e-mail/notification on GitHub. This enables you to test against multiple versions of Julia and saves you the time to pull code from other people to test locally on your machine before accepting pull requests. It is worth pointing out that I am by no means a testing fanatic, but I still really enjoy the service that Travis provide. Pontus
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Alright. This looks like it's because there's no RPATH in the libraries that are generated by the BinDeps source build. Here's my debugging info showing how we can dlopen libcasa_table.so just fine, but when we try to dlopen libcasacorewrapper we get an error that libcasa_table.so.1 can't be found, and ldd can't find it either. So I think we need to tack some linker flags onto the end of that. However, this build process takes 30 minutes, and isn't very nice to shove onto Travis every time we make a Julia change. Is there a debian package or something that contains libcasacore? -E On Fri, Dec 12, 2014 at 2:18 PM, Elliot Saba staticfl...@gmail.com wrote: Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Link to debugging info: https://travis-ci.org/staticfloat/CasaCore.jl/builds/43891665 -E On Fri, Dec 12, 2014 at 4:00 PM, Elliot Saba staticfl...@gmail.com wrote: Alright. This looks like it's because there's no RPATH in the libraries that are generated by the BinDeps source build. Here's my debugging info showing how we can dlopen libcasa_table.so just fine, but when we try to dlopen libcasacorewrapper we get an error that libcasa_table.so.1 can't be found, and ldd can't find it either. So I think we need to tack some linker flags onto the end of that. However, this build process takes 30 minutes, and isn't very nice to shove onto Travis every time we make a Julia change. Is there a debian package or something that contains libcasacore? -E On Fri, Dec 12, 2014 at 2:18 PM, Elliot Saba staticfl...@gmail.com wrote: Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
[julia-users] Re: MatrixDepot.jl: A Test Matrix Collection
could you discuss the properties? Can we add more properties?What do the 'inverse' and 'eigen' properties mean? On Friday, December 12, 2014 1:53:09 PM UTC-8, cdm wrote: great references, Weijian ... it turns out that the UF Sparse collection has a filed issue at: https://github.com/JuliaSparse/MatrixMarket.jl/issues/1 i understand that the UF Sparse collection is primarily in Rutherford-Boeing format and that a read-in tool for this may be in the works ... best, cdm On Friday, December 12, 2014 12:09:41 PM UTC-8, Weijian Zhang wrote: Hi cdm, Thanks a lot for the comments. Just for reference: If you want to read matrices in the Matrix Market format, you can use MatrixMarket.jl (https://github.com/JuliaSparse/MatrixMarket.jl). If your matrices are in Harwell Boeing Format, there is HarwellBoeing.jl ( https://github.com/dpo/HarwellBoeing.jl) Best wishes, Weijian
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Wow, I really appreciate all the work you've put into this so far. I hadn't been aware of any Debian packages for CasaCore, but it looks like the SKA has packaged CasaCore (https://launchpad.net/~ska-sa/+archive/ubuntu/main). It's not immediately obvious to me which package is needed from that repository, but I'll look into it. On Fri, Dec 12, 2014 at 4:01 PM, Elliot Saba staticfl...@gmail.com wrote: Link to debugging info: https://travis-ci.org/staticfloat/CasaCore.jl/builds/43891665 -E On Fri, Dec 12, 2014 at 4:00 PM, Elliot Saba staticfl...@gmail.com wrote: Alright. This looks like it's because there's no RPATH in the libraries that are generated by the BinDeps source build. Here's my debugging info showing how we can dlopen libcasa_table.so just fine, but when we try to dlopen libcasacorewrapper we get an error that libcasa_table.so.1 can't be found, and ldd can't find it either. So I think we need to tack some linker flags onto the end of that. However, this build process takes 30 minutes, and isn't very nice to shove onto Travis every time we make a Julia change. Is there a debian package or something that contains libcasacore? -E On Fri, Dec 12, 2014 at 2:18 PM, Elliot Saba staticfl...@gmail.com wrote: Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
[julia-users] Re: Are some matrices marked as special?
If the Array is anything other than normal, it should have a non-standard
type. So the answer for your two examples is no, as you can see by
creating them and looking at the type, which is simply Array{Float64,2}.
There are however some special Array like types, such as I, which acts as
an matrix with diagonal ones, but is implemented in a more efficient manner.
*julia **ones(2,2)*
*2x2 Array{Float64,2}:*
* 1.0 1.0*
* 1.0 1.0*
*julia **zeros(2,2)*
*2x2 Array{Float64,2}:*
* 0.0 0.0*
* 0.0 0.0*
*julia **ones(2,2)+I*
*2x2 Array{Float64,2}:*
* 2.0 1.0*
* 1.0 2.0*
*julia **I*
*UniformScaling{Int64}*
*1*I*
On Friday, December 12, 2014 3:45:16 PM UTC-7, Petr Krysl wrote:
Are perhaps matrices initialized as all zeros or identity matrices marked
as special behind the scenes, so that no actual processing is done when
there used for instance in multiplication?
Thanks,
Petr
[julia-users] I am losing caller scope but don't understand why.
Using julia: Version 0.3.3 (2014-10-21 20:18 UTC) This is the simplest example I could come up with to explain the issue. In this case, the method something is not in scope. module Person export say_something function say_something() println(something()) end end using Person function something() default something end Person.say_something() The output: $ julia test2.jl ERROR: something not defined in say_something at /tmp/test2.jl:4 in include at ./boot.jl:245 in include_from_node1 at loading.jl:128 in process_options at ./client.jl:285 in _start at ./client.jl:354 in _start_3B_1720 at /usr/bin/../lib/x86_64-linux-gnu/julia/sys.so while loading /tmp/test2.jl, in expression starting on line 14 More along the lines of my use case is this example, where my goal is to call to a generic method that knows to look back to a method the caller defines, in this case, it's something again: module Person export say_something function say_something() println(something()) end end module Boy using Person function something() I'm a boy. end end module Girl using Person function something() I'm a girl. end end Boy.say_something() Girl.say_something() Same error: $ julia test.jl ERROR: something not defined in say_something at /tmp/test.jl:4 in include at ./boot.jl:245 in include_from_node1 at loading.jl:128 in process_options at ./client.jl:285 in _start at ./client.jl:354 in _start_3B_1720 at /usr/bin/../lib/x86_64-linux-gnu/julia/sys.so while loading /tmp/test.jl, in expression starting on line 22 In ruby, this example works, albeit with classes: module Person def say_something p something end end class Boy include Person def something I'm a boy. end end class Girl include Person def something I'm a girl. end end Boy.new.say_something Girl.new.say_something Outputting: $ ruby test.rb I'm a boy. I'm a girl. Is there anyway to accomplish this in julia? Thank you, - Charlie
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
It turns out that after installing CasaCore from the repository, all the problems go away. CasaCore.jl is now building successfully and passing its tests on Travis (and I'm no longer hammering Travis with a 30-40 minute build every time I change a line). Again, thanks for your help. -- Michael On Fri, Dec 12, 2014 at 4:15 PM, Michael Eastwood michael.w.eastw...@gmail.com wrote: Wow, I really appreciate all the work you've put into this so far. I hadn't been aware of any Debian packages for CasaCore, but it looks like the SKA has packaged CasaCore ( https://launchpad.net/~ska-sa/+archive/ubuntu/main). It's not immediately obvious to me which package is needed from that repository, but I'll look into it. On Fri, Dec 12, 2014 at 4:01 PM, Elliot Saba staticfl...@gmail.com wrote: Link to debugging info: https://travis-ci.org/staticfloat/CasaCore.jl/builds/43891665 -E On Fri, Dec 12, 2014 at 4:00 PM, Elliot Saba staticfl...@gmail.com wrote: Alright. This looks like it's because there's no RPATH in the libraries that are generated by the BinDeps source build. Here's my debugging info showing how we can dlopen libcasa_table.so just fine, but when we try to dlopen libcasacorewrapper we get an error that libcasa_table.so.1 can't be found, and ldd can't find it either. So I think we need to tack some linker flags onto the end of that. However, this build process takes 30 minutes, and isn't very nice to shove onto Travis every time we make a Julia change. Is there a debian package or something that contains libcasacore? -E On Fri, Dec 12, 2014 at 2:18 PM, Elliot Saba staticfl...@gmail.com wrote: Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
[julia-users] Re: MatrixDepot.jl: A Test Matrix Collection
Hi Christian, could you discuss the properties? What do the 'inverse' and 'eigen' properties mean? That's a very good question. Here is the answer: * inverse : the inverse of the matrix is known explicitly. * ill-cond : the matrix is ill-conditioned for some parameter values (NOT all). * symmetric : as above. * pos -def: as above. * eigen: something is known about the eigensystem (eigenvalues, eigenvectors, bounds of eigenvalues etc.) I will add more details in the next version. Can we add more properties? Yes, of course. Currently, you can add your properties to the matrixclass inside src/higham.jl then modify the function matrixdepot() inside src/MatrixDepot.jl a bit then everything will work. But this is not nice, right? I will add some features so that you can add your properties easily in the next version. Thanks for your questions. Weijian On Saturday, 13 December 2014 00:12:13 UTC, Christian Peel wrote: could you discuss the properties? Can we add more properties?What do the 'inverse' and 'eigen' properties mean? On Friday, December 12, 2014 1:53:09 PM UTC-8, cdm wrote: great references, Weijian ... it turns out that the UF Sparse collection has a filed issue at: https://github.com/JuliaSparse/MatrixMarket.jl/issues/1 i understand that the UF Sparse collection is primarily in Rutherford-Boeing format and that a read-in tool for this may be in the works ... best, cdm On Friday, December 12, 2014 12:09:41 PM UTC-8, Weijian Zhang wrote: Hi cdm, Thanks a lot for the comments. Just for reference: If you want to read matrices in the Matrix Market format, you can use MatrixMarket.jl (https://github.com/JuliaSparse/MatrixMarket.jl). If your matrices are in Harwell Boeing Format, there is HarwellBoeing.jl (https://github.com/dpo/HarwellBoeing.jl) Best wishes, Weijian
[julia-users] Re: Are some matrices marked as special?
Well, I can see no difference either. But here is the tale of four matrices. They are each fed in turn to the same operation as input: everything else is precisely the same. The timings of that operation are as follows: Rm1=eye(2,2) # 25 seconds Rm2=[-0.668742918086684 -0.743493718540316; -0.743493718540316 0.668742918086684]; # 46 seconds Rm3=[1.0 0.0; 0.0 1.0] # 25 seconds Rm4=[0.0 1.0; -1.0 0.0] # 39 seconds So are some of these matrices more matrixy than others? Petr
[julia-users] Re: Announcing RobotOS.jl
Josh, This is very cool, thanks for releasing this! Some of my colleagues are starting to use ROS and I've been gently nudging them towards Julia rather than legacy C or Matlab solutions. The timing on this package is a convenient coincidence. How much performance do you think this might be leaving on the table by having to go through PyCall and rospy? Are most of the underlying API's exposed through a C interface that could be wrapped directly via ccall? -Tony On Friday, December 12, 2014 11:06:37 AM UTC-8, Josh Langsfeld wrote: After about 6 weeks of initial part-time development, I'm announcing the first release of the RobotOS.jl package, which enables essentially seamless integration of Julia code with ROS http://wiki.ros.org (Robot Operating System). At the core, it is mostly a wrapper for the rospy python library (many thanks to Steve Johnson for PyCall), but on top of that I added an automatic Julia type generation system so the back-end python details are completely hidden from the end-user. https://github.com/phobon/RobotOS.jl I believe the robotics community is especially well suited to adopt Julia, as speed and natural mathematical expressiveness are both highly desirable features in whichever programming language is used. For about six months now, I've been doing all my research in Julia and it has been vastly more pleasurable than using either Matlab or Python. I am quite eager to continue development on the package with whatever community feedback I can get but hopefully it can already prove useful to anyone out there who has already thought of using the two systems together. Thanks, Josh Langsfeld Graduate Research Assistant Maryland Robotics Center - UMD
[julia-users] Re: Are some matrices marked as special?
If some of the matrix values lead to floating point exceptions (NaN or Inf) in your benchmark code, that would slow it down (because of how the CPU works, not because of Julia). Or maybe you are doing some other computation whose computation steps depend on the values. What is your benchmark?
[julia-users] Re: I am losing caller scope but don't understand why.
Hi Charlie, I'm not the best person to answer this because I'm not familiar with Ruby, so I'm not sure exactly what you're trying to do, but I'll give it a shot. I'm not sure how Ruby modules relate to Julia modules, but this is a way to accomplish what I think you're trying to do: module Persons export Person, say_something abstract Person say_something(person::Person) = println(something(person)) something(::Person) = error(not implemented) end using Persons #we have to import Persons.something in order to extend it #even if it was exported import Persons.something type Boy : Person end type Girl : Person end something(::Boy) = I'm a boy. something(::Girl) = I'm a girl. say_something(Boy()) say_something(Girl()) I'm sure others will be able to respond with more details. Sam On Friday, December 12, 2014 4:33:56 PM UTC-8, Charles Snider wrote: Using julia: Version 0.3.3 (2014-10-21 20:18 UTC) This is the simplest example I could come up with to explain the issue. In this case, the method something is not in scope. module Person export say_something function say_something() println(something()) end end using Person function something() default something end Person.say_something() The output: $ julia test2.jl ERROR: something not defined in say_something at /tmp/test2.jl:4 in include at ./boot.jl:245 in include_from_node1 at loading.jl:128 in process_options at ./client.jl:285 in _start at ./client.jl:354 in _start_3B_1720 at /usr/bin/../lib/x86_64-linux-gnu/julia/sys.so while loading /tmp/test2.jl, in expression starting on line 14 More along the lines of my use case is this example, where my goal is to call to a generic method that knows to look back to a method the caller defines, in this case, it's something again: module Person export say_something function say_something() println(something()) end end module Boy using Person function something() I'm a boy. end end module Girl using Person function something() I'm a girl. end end Boy.say_something() Girl.say_something() Same error: $ julia test.jl ERROR: something not defined in say_something at /tmp/test.jl:4 in include at ./boot.jl:245 in include_from_node1 at loading.jl:128 in process_options at ./client.jl:285 in _start at ./client.jl:354 in _start_3B_1720 at /usr/bin/../lib/x86_64-linux-gnu/julia/sys.so while loading /tmp/test.jl, in expression starting on line 22 In ruby, this example works, albeit with classes: module Person def say_something p something end end class Boy include Person def something I'm a boy. end end class Girl include Person def something I'm a girl. end end Boy.new.say_something Girl.new.say_something Outputting: $ ruby test.rb I'm a boy. I'm a girl. Is there anyway to accomplish this in julia? Thank you, - Charlie
[julia-users] Gadfly in userimg.jl
Hello all, Is there anyone else who has attempted to precompile Gadfly using userimg.jl? It keeps claiming that the variable STD_ variables (STDIN,STDOUT,STDERR) are not defined, and this seems to be a pretty common/general issue (it also comes up when trying to precompile HDF5, etc.). I've pulled the load time on Gadfly down to about 15 seconds by precompiling DataFrames, but it'd be really nice if I could just put using My,Most,Used,Packages unto my juliarc.jl file and not have a solid 30 second startup time for a new instance of Julia. I'm wondering if a) is this an issue only I have and b) if not, is there some way to make it so the STD_ variables are all defined prior to running/compiling userimg.jl?
[julia-users] Re: Are some matrices marked as special?
Steven, I also thought of floating-point exceptions. However, the results are perfectly fine. Not identical, only within numerical precision, but pretty good nonetheless. The matrix passed in is the material of rotation matrix. Since the material conductivity matrix is an identity, the rotation matrix makes absolutely no difference as to the results. In fact, its transpose should multiply through with the matrix itself to identity. Petr On Friday, December 12, 2014 6:23:45 PM UTC-8, Steven G. Johnson wrote: If some of the matrix values lead to floating point exceptions (NaN or Inf) in your benchmark code, that would slow it down (because of how the CPU works, not because of Julia). Or maybe you are doing some other computation whose computation steps depend on the values. What is your benchmark?
[julia-users] Re: Gadfly in userimg.jl
I believe the trouble you're running into is that not all packages are precompile-friendly. While Gadfly by itself is, it dynamically tries to load other packages if they exist. I've run into this trouble when I have some JuMP packages installed. In my case, I fixed it by changing the following in Compose.jl: diff --git a/src/table.jl b/src/table.jl index d66d88b..bbbfa31 100644 --- a/src/table.jl +++ b/src/table.jl @@ -369,7 +369,7 @@ function realize_brute_force(tbl::Table, drawctx::ParentDrawContext) end -if Pkg.installed(JuMP) != nothing +if false Pkg.installed(JuMP) != nothing (Pkg.installed(GLPKMathProgInterface) != nothing || Pkg.installed(Cbc) != nothing) include(table-jump.jl) On Friday, December 12, 2014 9:47:56 PM UTC-5, Joshua Job wrote: Hello all, Is there anyone else who has attempted to precompile Gadfly using userimg.jl? It keeps claiming that the variable STD_ variables (STDIN,STDOUT,STDERR) are not defined, and this seems to be a pretty common/general issue (it also comes up when trying to precompile HDF5, etc.). I've pulled the load time on Gadfly down to about 15 seconds by precompiling DataFrames, but it'd be really nice if I could just put using My,Most,Used,Packages unto my juliarc.jl file and not have a solid 30 second startup time for a new instance of Julia. I'm wondering if a) is this an issue only I have and b) if not, is there some way to make it so the STD_ variables are all defined prior to running/compiling userimg.jl?
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Yep, no problem. So as to help make your CasaCore.jl maximally useful to others, I'd suggest trying to get the AptGet provider able to install `libcasacore-dev` automatically; I'm not sure if we have the ability to request AptGet to install from a different PPA, but if we don't that's something we can deal with in a issue on the BinDeps.jl tracker. Once we have that in place, users should be able to just run Pkg.add(CasaCore), and BinDeps will take care of the necessary apt-get magic. As it stands, BinDeps will attempt to build from source, and because the source build doesn't set RPATHs properly, the resultant libraries are unusable because the linker doesn't know to look in the proper directories for the other libraries. We should also pass the proper linker flags to the build scripts to get the from source builds to work properly. Something along the lines of setting LDFLAGS to -Wl,rpath,$(joinpath(depsdir,usr,lib))should do the trick, I believe. -E On Fri, Dec 12, 2014 at 4:40 PM, Michael Eastwood michael.w.eastw...@gmail.com wrote: It turns out that after installing CasaCore from the repository, all the problems go away. CasaCore.jl is now building successfully and passing its tests on Travis (and I'm no longer hammering Travis with a 30-40 minute build every time I change a line). Again, thanks for your help. -- Michael On Fri, Dec 12, 2014 at 4:15 PM, Michael Eastwood michael.w.eastw...@gmail.com wrote: Wow, I really appreciate all the work you've put into this so far. I hadn't been aware of any Debian packages for CasaCore, but it looks like the SKA has packaged CasaCore ( https://launchpad.net/~ska-sa/+archive/ubuntu/main). It's not immediately obvious to me which package is needed from that repository, but I'll look into it. On Fri, Dec 12, 2014 at 4:01 PM, Elliot Saba staticfl...@gmail.com wrote: Link to debugging info: https://travis-ci.org/staticfloat/CasaCore.jl/builds/43891665 -E On Fri, Dec 12, 2014 at 4:00 PM, Elliot Saba staticfl...@gmail.com wrote: Alright. This looks like it's because there's no RPATH in the libraries that are generated by the BinDeps source build. Here's my debugging info showing how we can dlopen libcasa_table.so just fine, but when we try to dlopen libcasacorewrapper we get an error that libcasa_table.so.1 can't be found, and ldd can't find it either. So I think we need to tack some linker flags onto the end of that. However, this build process takes 30 minutes, and isn't very nice to shove onto Travis every time we make a Julia change. Is there a debian package or something that contains libcasacore? -E On Fri, Dec 12, 2014 at 2:18 PM, Elliot Saba staticfl...@gmail.com wrote: Just wanted to say that I'm taking a look at this on my own, over here https://travis-ci.org/staticfloat/CasaCore.jl. So far, it looks like libcasacorewrapper is having a hard time finding libcasa_table.so.1, although I can't quite say why yet. -E
[julia-users] Re: I am losing caller scope but don't understand why.
Very cool Sam! Julia is new to me, but I really like ruby and was trying to do things I would do in ruby in julia, just didn't know how. This sheds all new light on the language! I really appreciate you taking your time to look at this, really very helpful. Thank you, - Charlie
Re: [julia-users] Gadfly in userimg.jl
HDF5 _should_ work---I just put in a bunch of work to fix some problems that were preventing precompilation. But, you must be using HDF5 0.4.7 (or master). --Tim On Friday, December 12, 2014 06:47:55 PM Joshua Job wrote: Hello all, Is there anyone else who has attempted to precompile Gadfly using userimg.jl? It keeps claiming that the variable STD_ variables (STDIN,STDOUT,STDERR) are not defined, and this seems to be a pretty common/general issue (it also comes up when trying to precompile HDF5, etc.). I've pulled the load time on Gadfly down to about 15 seconds by precompiling DataFrames, but it'd be really nice if I could just put using My,Most,Used,Packages unto my juliarc.jl file and not have a solid 30 second startup time for a new instance of Julia. I'm wondering if a) is this an issue only I have and b) if not, is there some way to make it so the STD_ variables are all defined prior to running/compiling userimg.jl?
[julia-users] scale! is amusingly slow for a sparse matrix
I stumbled on this behavior when profiling a small finite element code.
julia N = 10_000;
julia A = sprand(N, N, 0.01);
julia @time scale!(A, 2.0);
elapsed time: 7.333984625 seconds (2275708 bytes allocated)
julia @time scale!(A, 0.5);
elapsed time: 7.258595034 seconds (80 bytes allocated)
julia @time scale!(A.nzval, 2.0);
elapsed time: 0.022604954 seconds (269764 bytes allocated)
julia @time scale!(A.nzval, 0.5);
elapsed time: 0.001367558 seconds (80 bytes allocated)
julia 7.258595034/0.001367558
5307.7054384530675
The problem is that scale! treats A as an AbstractArray.
julia methods(scale!, (typeof(A),Float64))
1-element Array{Any,1}:
scale!(X::AbstractArray{T,N},s::Number) at linalg/generic.jl:24
If called with A.nzval, however, scale! calls BLAS.scal!
[julia-users] Re: scale! is amusingly slow for a sparse matrix
Hah, thats quite funny. This would be easy to fix - you could submit a pull
request to Julia?
On Saturday, December 13, 2014 12:39:59 AM UTC-5, Bill McLean wrote:
I stumbled on this behavior when profiling a small finite element code.
julia N = 10_000;
julia A = sprand(N, N, 0.01);
julia @time scale!(A, 2.0);
elapsed time: 7.333984625 seconds (2275708 bytes allocated)
julia @time scale!(A, 0.5);
elapsed time: 7.258595034 seconds (80 bytes allocated)
julia @time scale!(A.nzval, 2.0);
elapsed time: 0.022604954 seconds (269764 bytes allocated)
julia @time scale!(A.nzval, 0.5);
elapsed time: 0.001367558 seconds (80 bytes allocated)
julia 7.258595034/0.001367558
5307.7054384530675
The problem is that scale! treats A as an AbstractArray.
julia methods(scale!, (typeof(A),Float64))
1-element Array{Any,1}:
scale!(X::AbstractArray{T,N},s::Number) at linalg/generic.jl:24
If called with A.nzval, however, scale! calls BLAS.scal!
Re: [julia-users] Re: Community Support for Julia on Travis CI
Good point! (I'm a bit biased :D) On Friday, December 12, 2014 6:13:55 PM UTC-5, Luthaf wrote: Can it be possible to add some kind of support for code coverage ? I use this package (https://github.com/IainNZ/Coverage.jl) which seems to be a standard one in Julia. For now I use a custom script in my .tramis.yml file, but the only modification in the build script is julia -e 'Pkg.test(MyPkg, coverage=true)' vs the official julia -e 'Pkg.test(MyPkg)' so why not make this coverage=true the default ? To enable the coverall.io output, one would then only have to add after_success:- julia -e 'cd(Pkg.dir(MyPkg)); Pkg.add(Coverage); using Coverage; Coveralls.submit(Coveralls.process_folder())' In their .tramis.yml file. Or there can be a special syntax for that if needed. Guillaume Pontus Stenetorp a écrit : On 12 December 2014 at 00:28, Stefan Karpinski st...@karpinski.org javascript: wrote: Btw, can I be added to the JuliaCI org? No idea who, but someone has taken care of it. If someone else wants to join, just give us a poke. On 12 December 2014 at 00:49, Nils Gudat nils...@gmail.com javascript: wrote: Excuse the ignorant question, but what exactly does this mean? I haven't seen Travis CI before and clicking on the home page is slightly confusing... Ignorance is fine, at least in academia you need some of it in order to accomplish anything. I agree that the homepage is a mess, essentially they have a bunch of virtual servers that check if you have a pending pull request or made a push to your repository. Once this happens they will pull the code from GitHub and build/run tests on their machines to check that everything is all right. If something is wrong, you will get an e-mail/notification on GitHub. This enables you to test against multiple versions of Julia and saves you the time to pull code from other people to test locally on your machine before accepting pull requests. It is worth pointing out that I am by no means a testing fanatic, but I still really enjoy the service that Travis provide. Pontus
Re: [julia-users] Good exemplar Julia programs for code style/organization?
I like reading library code from the Julia core team for learning purposes, especially libraries for 0.4. Given how fast Julia is moving, I would not expect anything super-polished at this stage outside libraries, especially since in scientific computing the programs themselves are used only a few times (compared to other kinds of software). Best, Tamas On Fri, Dec 12 2014, Tom Fawcett tom.fawc...@gmail.com wrote: Hi, all. I'm slowly moving over to Julia from Python. There is just enough of a difference of Julia from other languages that I'd like to find a well-written, small to medium-sized Julia program to study its coding style and code organization. It doesn't have to be huge; just a well-organized collection of modules and classes/types. Any nominations for exemplary Julia? (I've gone through various library code files, but libraries aren't organized the same as programs. I've skimmed a few of the programs in Julia.jl but it's hard to determine what's considered well-written.) Thanks, -Tom
Re: [julia-users] Good exemplar Julia programs for code style/organization?
Tamas makes a good point, it depends on if your interest is in Julia as a language or Julia as a way of getting stuff done. Both communities are active, but I doubt they have the same definition of well written :) Cheers Lex On Saturday, December 13, 2014 5:23:41 PM UTC+10, Tamas Papp wrote: I like reading library code from the Julia core team for learning purposes, especially libraries for 0.4. Given how fast Julia is moving, I would not expect anything super-polished at this stage outside libraries, especially since in scientific computing the programs themselves are used only a few times (compared to other kinds of software). Best, Tamas On Fri, Dec 12 2014, Tom Fawcett tom.f...@gmail.com javascript: wrote: Hi, all. I'm slowly moving over to Julia from Python. There is just enough of a difference of Julia from other languages that I'd like to find a well-written, small to medium-sized Julia program to study its coding style and code organization. It doesn't have to be huge; just a well-organized collection of modules and classes/types. Any nominations for exemplary Julia? (I've gone through various library code files, but libraries aren't organized the same as programs. I've skimmed a few of the programs in Julia.jl but it's hard to determine what's considered well-written.) Thanks, -Tom
