Re: [julia-users] THANKS to Julia core developers!
Le mercredi 17 décembre 2014 à 10:43 -0800, Douglas Bates a écrit : On Tuesday, December 16, 2014 5:50:12 PM UTC-6, ivo welch wrote: my note was partly a joke, partly a warning. the R community started out very nice, too. many still are. but some of the tone has shifted towards the obnoxious. I''m glad to hear you say that. I was wondering whether your comment was serious or not. the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. I am quite optimistic that the Julia community will not evolve the way that R did. Two reasons for this are: 1. Stefan, Jeff and Viral are surprisingly well adjusted for such technically accomplished people. :-) 2. I really do think that github and the other tools used for developing Julia encourage social coding. I believe that over 300 people have contributed to github.com/JuliaLang/julia in the few years that the project has been on github. That is incredible. Indeed, apart from the harsh tone on the R mailing lists (I couldn't have described it better than Ivo did), the factor that I think drives developers away from R is the idea that it's documented, so it's not a bug and doesn't need to change. For some time you try to contribute improvements to the project, but at some point you get tired of trying to convince core developers just to be able to help somewhere. Result: SVN tells me about 30 people have committed to the R code base in all those years -- 10 times less than Julia, and 100 times less than the number of R packages... Indeed one of the greatest achievements of Julia so far is social, in part thanks to git and Github, but mostly (I suspect) thanks to the spirit of the core team. I see no reason why it would change. Regards this reminds me: for those of us who cannot contribute, is there a julia foundation membership? it would not be a bad idea to have us using users get used to contributing, too.
Re: [julia-users] THANKS to Julia core developers!
On Thu, Dec 18 2014, Milan Bouchet-Valat nalimi...@club.fr wrote: the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. Indeed, apart from the harsh tone on the R mailing lists (I couldn't have described it better than Ivo did), the factor that I think drives developers away from R is the idea that it's documented, so it's not a bug and doesn't need to change. For some time you try to contribute improvements to the project, but at some point you get tired of trying to convince core developers just to be able to help somewhere. Result: While I agree with the above description of the R community, we should take the context into account: R itself comes from S-plus, and there is quite a bit of legacy code that they would not want to break with incompatible changes. Hopefully Julia will never get ossified like this. Best, Tamas
[julia-users] odd behavior for eval scope in macros
Hi, I'm trying to return an expression that contains symbols from a macro and I'm seeing odd behavior when the macro is declared in a module vs. when a macro is declared outside of a module. An exported macro declared from within a module can't resolve symbols from the returned expression when it tries to eval. I'm running julia on a mac compiled from the release-0.3 branch (Version 0.3.4-pre+74 (2014-12-16 03:44 UTC)). Here's a minimized example of what I'm seeing: julia macro xxx(str) symbol(str) end julia module Test export @yyy; macro yyy(str) symbol(str) end end julia using Test julia global x = 10 10 julia @xxx(x) 10 julia @yyy(x) ERROR: x not defined I guess that I must be misunderstanding something about hygiene. Any help would be appreciated. thanks, wade
[julia-users] Re: odd behavior for eval scope in macros
I think I figured it out. It looks like I need to esc() the returned symbol to avoid hygiene issue in this case. thanks, wade On Thursday, December 18, 2014 7:31:38 AM UTC-5, S Wade wrote: Hi, I'm trying to return an expression that contains symbols from a macro and I'm seeing odd behavior when the macro is declared in a module vs. when a macro is declared outside of a module. An exported macro declared from within a module can't resolve symbols from the returned expression when it tries to eval. I'm running julia on a mac compiled from the release-0.3 branch (Version 0.3.4-pre+74 (2014-12-16 03:44 UTC)). Here's a minimized example of what I'm seeing: julia macro xxx(str) symbol(str) end julia module Test export @yyy; macro yyy(str) symbol(str) end end julia using Test julia global x = 10 10 julia @xxx(x) 10 julia @yyy(x) ERROR: x not defined I guess that I must be misunderstanding something about hygiene. Any help would be appreciated. thanks, wade
Re: [julia-users] THANKS to Julia core developers!
Compatibility is tough and we've really just barely started to deal with it, but I think so far it's going pretty well. Multiple dispatch is uncannily good at deprecating things, and having a dynamic language with nice metaprogramming makes it possible to do clever things like Compat.jl, which makes it much easier to the language. Changes will slow down once we hit 1.0, but they shouldn't stop. If we can make transitions between major versions relatively painless, then we'll be able to continue to innovate without getting stuck between versions the way Perl and Python have (for quite opposite reasons). The Ruby transition from 1.8 to 2.0 is a good model – they broke some things, but not so much that the language became unfamiliar, and the pain of switching was more than fairly compensated for by better performance and features that make the language more pleasant to use. People should be excited about upgrading, not dread it. On Thu, Dec 18, 2014 at 3:51 AM, Tamas Papp tkp...@gmail.com wrote: On Thu, Dec 18 2014, Milan Bouchet-Valat nalimi...@club.fr wrote: the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. Indeed, apart from the harsh tone on the R mailing lists (I couldn't have described it better than Ivo did), the factor that I think drives developers away from R is the idea that it's documented, so it's not a bug and doesn't need to change. For some time you try to contribute improvements to the project, but at some point you get tired of trying to convince core developers just to be able to help somewhere. Result: While I agree with the above description of the R community, we should take the context into account: R itself comes from S-plus, and there is quite a bit of legacy code that they would not want to break with incompatible changes. Hopefully Julia will never get ossified like this. Best, Tamas
Re: [julia-users] Re: Can Julia be the next web scripting language?
I suppose Julia would be the successor to JavaScript if it was converted by Emscripten and run by asm.js. But for now there seems to be an Enscripten bug which invalidates Julia IR code.
Re: [julia-users] Rust and Julia - Together? Advice
For this purpose, I would deploy a web service in Julia, using e.g. http://juliawebstack.org/#HttpServer
[julia-users] Efficient data sharing between Julia and R in production
This is a fairly general question about structuring an application that relies on both Julia and R code. The Julia part takes time series input and performs extensive parallel computations (thank you core developers for the efficient pmap), producing the output that is not that demanding in terms of data storage. The R part collects some data from the web every minute via REST APIs forming the time series for Julia input, and also uses Julia output for further computations (not as demanding as the Julia functions so that the R-side performance is not limiting the overall process). Now as I am trying to integrate everything for a production environment, the question is about data sharing between Julia and R. Saving chunks of time series as HDFS files works in development but is inefficient. Is there a suitable non-relational database with both Julia and R bindings available? Should I look into setting up Julia web-server and have R talk to it via REST API? Any thoughts/advice on structuring my application (planning to use Google cloud) would be helpful. Thanks for reading.
[julia-users] Re: Community Support for Julia on Travis CI
Perhaps related: Warning: error initializing module GMP: ErrorException(The dynamically loaded GMP library (version 5.1.3 with __gmp_bits_per_limb == 64) does not correspond to the compile time version (version 6.0.0 with __gmp_bits_per_limb == 64). Please rebuild Julia.) I have not investigated this a whole bunch yet, but I might have installed gmp locally for another reason/project and there is a conflict now. Other installs (Travis) might have had the same? 6.0.0 has newishness about it.
Re: [julia-users] THANKS to Julia core developers!
Will be exciting to see how fast Julia 0.3 dies down when we start recommending 0.4 by default. kl. 15:27:45 UTC+1 torsdag 18. desember 2014 skrev Stefan Karpinski følgende: Compatibility is tough and we've really just barely started to deal with it, but I think so far it's going pretty well. Multiple dispatch is uncannily good at deprecating things, and having a dynamic language with nice metaprogramming makes it possible to do clever things like Compat.jl, which makes it much easier to the language. Changes will slow down once we hit 1.0, but they shouldn't stop. If we can make transitions between major versions relatively painless, then we'll be able to continue to innovate without getting stuck between versions the way Perl and Python have (for quite opposite reasons). The Ruby transition from 1.8 to 2.0 is a good model – they broke some things, but not so much that the language became unfamiliar, and the pain of switching was more than fairly compensated for by better performance and features that make the language more pleasant to use. People should be excited about upgrading, not dread it. On Thu, Dec 18, 2014 at 3:51 AM, Tamas Papp tkp...@gmail.com javascript: wrote: On Thu, Dec 18 2014, Milan Bouchet-Valat nali...@club.fr javascript: wrote: the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. Indeed, apart from the harsh tone on the R mailing lists (I couldn't have described it better than Ivo did), the factor that I think drives developers away from R is the idea that it's documented, so it's not a bug and doesn't need to change. For some time you try to contribute improvements to the project, but at some point you get tired of trying to convince core developers just to be able to help somewhere. Result: While I agree with the above description of the R community, we should take the context into account: R itself comes from S-plus, and there is quite a bit of legacy code that they would not want to break with incompatible changes. Hopefully Julia will never get ossified like this. Best, Tamas
Re: [julia-users] Rust and Julia - Together? Advice
My experience has been that using a full web stack is overkill for this kind of internal communication between processes. ZMQ let's you set up point-to-point, broadcast, anycast, pubsub, etc. messaging patterns really easily and Julia's built-in serialization and deserialization lets you send nearly arbitrary objects over the wire with no fuss. On Thu, Dec 18, 2014 at 5:26 AM, Jerzy Głowacki jerzyglowa...@gmail.com wrote: For this purpose, I would deploy a web service in Julia, using e.g. http://juliawebstack.org/#HttpServer
Re: [julia-users] Efficient data sharing between Julia and R in production
Redis, perhaps? There are some Julia bindings: https://github.com/msainz/Redis.jl https://github.com/markmo/HiRedis.jl I'm not sure which is better to use – the Redis.jl package is a pure Julia implementation of a Redis client while HiRedis.jl uses the hiredis library to interact with Redis. It would be nice to eventually have one true way to interact with Redis, but for now there's a choice. You could also try using Rif.jl https://github.com/lgautier/Rif.jl to call R directly from Julia. Yet another option would be to push chunks of data from Julia to R using ZeroMQ messaging. Here's a thread about that approach: https://groups.google.com/forum/#!topic/julia-users/umHiBwVLQ4g On Thu, Dec 18, 2014 at 4:26 AM, Pavel pavel.paramo...@gmail.com wrote: This is a fairly general question about structuring an application that relies on both Julia and R code. The Julia part takes time series input and performs extensive parallel computations (thank you core developers for the efficient pmap), producing the output that is not that demanding in terms of data storage. The R part collects some data from the web every minute via REST APIs forming the time series for Julia input, and also uses Julia output for further computations (not as demanding as the Julia functions so that the R-side performance is not limiting the overall process). Now as I am trying to integrate everything for a production environment, the question is about data sharing between Julia and R. Saving chunks of time series as HDFS files works in development but is inefficient. Is there a suitable non-relational database with both Julia and R bindings available? Should I look into setting up Julia web-server and have R talk to it via REST API? Any thoughts/advice on structuring my application (planning to use Google cloud) would be helpful. Thanks for reading.
Re: [julia-users] Rust and Julia - Together? Advice
Update:
I miraculously figured out how to use ZMQ in Julia, i.e. you type using
ZMQ :)
More seriously, I managed to get jeromq working in Eclipse and started
trying to get it to talk to Julia's ZMQ. I was trying to follow the
examples in the zguide (which is awesome btw):
- http://zguide.zeromq.org/page:all
But had some difficulties. For example, it is not totally obvious how to
implement the Hello World server in Julia. Here is the C version:
#include zmq.h
#include stdio.h
#include unistd.h
#include string.h
#include assert.h
int main (void)
{
*// Socket to talk to clients*
void *context = zmq_ctx_new ();
void *responder = zmq_socket (context, ZMQ_REP);
int rc = zmq_bind (responder, tcp://*:);
assert (rc == 0);
*while* (1) {
char buffer [10];
zmq_recv (responder, buffer, 10, 0);
printf (Received Hello*\n*);
sleep (1); *// Do some 'work'*
zmq_send (responder, World, 5, 0);
}
*return* 0;
}
I got this far:
*julia **using ZMQ*
*julia **ctx = Context(1)*
*Context(Ptr{Void} @0x7fc8a25b8d60,[])*
*julia **s = Socket(ctx, REP)*
*Socket(Ptr{Void} @0x7fc8a7ce2600)*
*julia **ZMQ.bind(s, tcp://*:)*
This all seems to work. But I don't know how to implement the while loop
properly. Any hints?
Best regards,
Eric
Re: [julia-users] Rust and Julia - Together? Advice
I have some examples in my fork of ZMQ.jl here
https://github.com/jakebolewski/ZMQ.jl. I don't know if they will work
on the latest ZMQ master.
On Thursday, December 18, 2014 10:58:59 AM UTC-5, Eric Forgy wrote:
Update:
I miraculously figured out how to use ZMQ in Julia, i.e. you type using
ZMQ :)
More seriously, I managed to get jeromq working in Eclipse and started
trying to get it to talk to Julia's ZMQ. I was trying to follow the
examples in the zguide (which is awesome btw):
- http://zguide.zeromq.org/page:all
But had some difficulties. For example, it is not totally obvious how to
implement the Hello World server in Julia. Here is the C version:
#include zmq.h
#include stdio.h
#include unistd.h
#include string.h
#include assert.h
int main (void)
{
*// Socket to talk to clients*
void *context = zmq_ctx_new ();
void *responder = zmq_socket (context, ZMQ_REP);
int rc = zmq_bind (responder, tcp://*:);
assert (rc == 0);
*while* (1) {
char buffer [10];
zmq_recv (responder, buffer, 10, 0);
printf (Received Hello*\n*);
sleep (1); *// Do some 'work'*
zmq_send (responder, World, 5, 0);
}
*return* 0;
}
I got this far:
*julia **using ZMQ*
*julia **ctx = Context(1)*
*Context(Ptr{Void} @0x7fc8a25b8d60,[])*
*julia **s = Socket(ctx, REP)*
*Socket(Ptr{Void} @0x7fc8a7ce2600)*
*julia **ZMQ.bind(s, tcp://*:)*
This all seems to work. But I don't know how to implement the while loop
properly. Any hints?
Best regards,
Eric
Re: [julia-users] Gadfly in userimg.jl
HDF5 works fine if I don't try to precompile it. And now Gadfly works great! Which is really fantastic as that was, by a longshot, the biggest problem slowdown in getting to work with a new Julia instance. Thanks so much! I'm not too worried about HDF5, as it only takes a couple seconds to load. I'll try playing around with it myself a bit and if I can't figure it out I'll file an issue. On Tuesday, December 16, 2014 11:08:05 PM UTC-5, Shashi Gowda wrote: @Joshua, if you do Pkg.update() and get the latest Gadfly, you should be able to precompile it. Homebrew.jl had a new release 5 hours ago as well, did you try updating? We were getting the UndefVarError(var=:STDERR) with Compose, this happens when the code you are loading wants to print an error message (I think). Does `using HDF5` work for you if you build without precompiling? You should not have any load time errors for this to work. I was able to fix Compose precompilation problem by accounting for the error case https://github.com/JuliaLang/JuliaBox/issues/123#issuecomment-67187273 On Wed, Dec 17, 2014 at 7:58 AM, Tim Holy tim@gmail.com javascript: wrote: If you'd like some help fixing this, please open an issue over at HDF5.jl. If you do, please be explicit about the commands you're executing that lead to such error messages. --Tim On Tuesday, December 16, 2014 11:12:58 AM Joshua Job wrote: I'm on Mac OS X, but I get an error from HDF5 calling/pulling in the Homebrew package with error during bootstrap: LoadError(at sysimg.jl line 286: LoadError(at /Users/joshuajob/julia-0.3/base/userimg.jl line 1: LoadError(at /Users/joshuajob/.julia/v0.3/HDF5/src/HDF5.jl line 1: LoadError(at /Users/joshuajob/.julia/v0.3/HDF5/src/plain.jl line 14: LoadError(at /Users/joshuajob/.julia/v0.3/Homebrew/src/Homebrew.jl line 345: UndefVarError(var=:STDERR)) It probably would work fine if I was on Linux. On Friday, December 12, 2014 11:21:39 PM UTC-5, Tim Holy wrote: HDF5 _should_ work---I just put in a bunch of work to fix some problems that were preventing precompilation. But, you must be using HDF5 0.4.7 (or master). --Tim On Friday, December 12, 2014 06:47:55 PM Joshua Job wrote: Hello all, Is there anyone else who has attempted to precompile Gadfly using userimg.jl? It keeps claiming that the variable STD_ variables (STDIN,STDOUT,STDERR) are not defined, and this seems to be a pretty common/general issue (it also comes up when trying to precompile HDF5, etc.). I've pulled the load time on Gadfly down to about 15 seconds by precompiling DataFrames, but it'd be really nice if I could just put using My,Most,Used,Packages unto my juliarc.jl file and not have a solid 30 second startup time for a new instance of Julia. I'm wondering if a) is this an issue only I have and b) if not, is there some way to make it so the STD_ variables are all defined prior to running/compiling userimg.jl?
[julia-users] Column names in DataFrame object to lowercase
vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
[julia-users] Recompiling ast and document about uncompressed ast
Hi, I'm wondering if there's a way to recompile the lowered AST back into a function and what additional information (other than the module it should be evaluate in) is necessary to do that what I've tried and would like to do is demonstrated with the code attached. The idea is that I would like to modify the code of a function/method before calling it (in this case, I am replacing it with the code from another function with the same signature). (In another word, the question is how to construct a function from code1 with ast2 and run it.) It will also be nice if there's more documentation on this lowered AST since it seems to be very different form the one you get in a function definition. (e.g. what is the array in `code_lowered(+, (Float64, Float64))[1].args[2]` mean) A brief description of what I want to do. I'm writing a control system with dynamic logic. However, since the code might not be running on a single host, it is hard to use an existing interpreter to do what I want. Trying to avoid inventing yet another language, I hope to use a subset of an existing language and somehow transform it into a representation that can be run somewhere else and it seems that the inspection and meta programming feature in julia is very useful for this. It might be possible to just apply a macro to every function / method definition but it will be much harder to write. I would rather try to trace the excution and use the AST acquired at runtime. For that I would like to know more about the lowered AST and possibly how to execute a modified version in order to trace recursive / nested function calls. This might not be the best way to do it and I'm still in planning stage so any suggestions on the design is also welcome. Cheers, Yichao Yu # function get_code(func, args) return which(func, Base.typesof(args...)).func.code end function recompile(func1, func2, args) code1 = get_code(func1, args) code2 = get_code(func2, args) @assert code1.module == code2.module eval_code1 = code1.module.eval(code1) eval_res1 = eval_code1(args...) @assert eval_res1 == func1(args...) println(eval_code: $(eval_res1)) eval_code2 = code2.module.eval(code2) eval_res2 = eval_code2(args...) @assert eval_res2 == func2(args...) println(eval_code: $(eval_res2)) ast1 = ccall(:jl_uncompress_ast, Any, (Any, Any), code1, code1.ast) println(ast1: $ast1) ast2 = ccall(:jl_uncompress_ast, Any, (Any, Any), code2, code2.ast) println(ast2: $ast2) code1.ast = ccall(:jl_compress_ast, Any, (Any, Any), code1, ast2) println(new_code: $code1) eval_new_code = code1.module.eval(code1) eval_new_res = eval_new_code(args...) @assert eval_new_res == func2(args...) println(eval_new_code: $(eval_new_res)) end recompile(+, -, (1.2, 3.4)) signature.asc Description: This is a digitally signed message part.
Re: [julia-users] Column names in DataFrame object to lowercase
The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
Re: [julia-users] Column names in DataFrame object to lowercase
My mistake. It should be println($x). v was a typo. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com javascript: wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
Re: [julia-users] Column names in DataFrame object to lowercase
Trying again. Here's the code that generated an error: vars = names(df) for (x, v) in enumerate(vars) println($x, $v, lowercase(v)) end The error comes from the lowercase() function, saying that it does not know how to handle Symbol type. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com javascript: wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
Re: [julia-users] Column names in DataFrame object to lowercase
Try lowercase(string(v)). On Thu, Dec 18, 2014 at 1:47 PM, Min-Woong Sohn mws...@gmail.com wrote: Trying again. Here's the code that generated an error: vars = names(df) for (x, v) in enumerate(vars) println($x, $v, lowercase(v)) end The error comes from the lowercase() function, saying that it does not know how to handle Symbol type. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
[julia-users] types, arrays, and syntax () {} []
Hi,
I see that I can define my own type, and create instances of it like so:
~~~
type MyType{T}
x::T
end
a = MyType{Int64}(3)
~~~
I tried to do similarly to create an array, but it fails... my first guess
was that this might work:
b = Array{Int64}(10, 11, 12) # trying to get [10, 11, 12]
but maybe I need to specify the dimensions... `help(Array)` says I need to
specify type and dims, so tried:
b = Array{Int64, 3, 1}(10, 11, 12) # (make a 3x1 array) but that's not
right either.
I saw in the learnxinyminutes for Julia that you can do:
b = Int64[10, 11, 12]
which seems to be a typed version of just `[10, 11, 12]`. Is that a
shorthand syntax for some more general `Array{...}(...)` syntax?
Re: [julia-users] Column names in DataFrame object to lowercase
Thank you so much. It works beautifully. On Thursday, December 18, 2014 1:51:21 PM UTC-5, Stefan Karpinski wrote: Try lowercase(string(v)). On Thu, Dec 18, 2014 at 1:47 PM, Min-Woong Sohn mws...@gmail.com javascript: wrote: Trying again. Here's the code that generated an error: vars = names(df) for (x, v) in enumerate(vars) println($x, $v, lowercase(v)) end The error comes from the lowercase() function, saying that it does not know how to handle Symbol type. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
Re: [julia-users] Column names in DataFrame object to lowercase
No problem. On Thu, Dec 18, 2014 at 2:11 PM, Min-Woong Sohn mws...@gmail.com wrote: Thank you so much. It works beautifully. On Thursday, December 18, 2014 1:51:21 PM UTC-5, Stefan Karpinski wrote: Try lowercase(string(v)). On Thu, Dec 18, 2014 at 1:47 PM, Min-Woong Sohn mws...@gmail.com wrote: Trying again. Here's the code that generated an error: vars = names(df) for (x, v) in enumerate(vars) println($x, $v, lowercase(v)) end The error comes from the lowercase() function, saying that it does not know how to handle Symbol type. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
Re: [julia-users] Column names in DataFrame object to lowercase
Here's a function that changes all variable names (column names) in a DataFrame to lower case. function renvars_lower(df) vars = names(df) for (i,v) in enumerate(vars) vars[i] = lowercase(string(v)) end names!(df,vars) end Any suggestions to improve this is welcome. On Thursday, December 18, 2014 2:11:09 PM UTC-5, Min-Woong Sohn wrote: Thank you so much. It works beautifully. On Thursday, December 18, 2014 1:51:21 PM UTC-5, Stefan Karpinski wrote: Try lowercase(string(v)). On Thu, Dec 18, 2014 at 1:47 PM, Min-Woong Sohn mws...@gmail.com wrote: Trying again. Here's the code that generated an error: vars = names(df) for (x, v) in enumerate(vars) println($x, $v, lowercase(v)) end The error comes from the lowercase() function, saying that it does not know how to handle Symbol type. On Thursday, December 18, 2014 1:15:49 PM UTC-5, Stefan Karpinski wrote: The variable seems to be named x not v? On Thu, Dec 18, 2014 at 12:55 PM, Min-Woong Sohn mws...@gmail.com wrote: vars = names(df) where df is a DataFrame object and vars is an array of Symbols. How do I change the elements in vars to lower case? When I did for x in vars println($v) end it gives me an error. Thanks a bunch.
[julia-users] Re: types, arrays, and syntax () {} []
I see that I can do:
b = Array{Int64, 1}
#= Array{Int64,1}
but why can't I push anything to that?:
push!(b, 10)
# ERROR: `push!` has no method matching push!(::Type{Array{Int64,1}},
::Int64)
Hm. Can't push anything to this either though:
~~~
c = []
#= 0-element Array{None,1}
push!(c, 10)
ERROR: [] cannot grow. Instead, initialize the array with T[], where T is
the desired element type.
in push! at ./array.jl:457
~~~
Re: [julia-users] Re: types, arrays, and syntax () {} []
Array{Int64,1} is a type – it's the type of an array with 1 dimension (a
vector) whose elements are 64-bit signed integers. You can't push anything
onto a type because types aren't containers. If you want to construct an
uninitialized three value Int array, you can do this:
v = Array(Int, 3)
This will just have junk values gotten via malloc. If you want to
initialize an Int array with the values 10, 11 and 12 you can do this:
v = [10, 11, 12]
It will have element type Int because all the values you gave it are Ints.
On Thu, Dec 18, 2014 at 2:21 PM, jgabriele...@gmail.com wrote:
I see that I can do:
b = Array{Int64, 1}
#= Array{Int64,1}
but why can't I push anything to that?:
push!(b, 10)
# ERROR: `push!` has no method matching push!(::Type{Array{Int64,1}},
::Int64)
Hm. Can't push anything to this either though:
~~~
c = []
#= 0-element Array{None,1}
push!(c, 10)
ERROR: [] cannot grow. Instead, initialize the array with T[], where T
is the desired element type.
in push! at ./array.jl:457
~~~
Re: [julia-users] Re: Can Julia be the next web scripting language?
On Thursday, December 18, 2014 10:59:53 AM UTC, Jerzy Głowacki wrote: I suppose Julia would be the successor to JavaScript if it was converted by Emscripten and run by asm.js. But for now there seems to be an Enscripten bug which invalidates Julia IR code. Hi, Curious, what bug is that? You have a bug number or know what it relates to (on Emscripten side)? Invalidates Julia IR code? Intermediate Representation? Does the bug only affect Julia? Have anything to do with JavaScript ints (that it their non-existence, JS uses doubles as int)? Did you really mean to say Julia would be the successor or Julia could be a successor? Is there any obvious successor or fairly much used alternative currently? I've heard about Coffeescript, Typescript and Dart etc. languages made to compile to JavaScript. And Scala.js. Scala was meant to run in JVM and Scala.js has slightly different semantics because if JS's ints. I wandered if Julia could do the same. I assume you are talking about compile Julia the language/environment to JavaScript and run the the REPL in a browser? Should it be easier to compile Julia code to C (possible now) and that C code via Emscripten to JavaScript? Emscripten says they handle portable C code. Doesn't Intel's Julia2C claim portable C? I don't really see how Emscripten handles ints. There is no requirement that C's ints are less than 64 bit? Do not even have to be two's compliment, right? I fail to see how they densely pack ints (that are 64-bit doubles) into 32-bit in arrays.. PS. I dreamt my first dream in Julia code tonight. Might be a sign it's the language for me.. :) Reminds me: Dreaming in code is a good book on (Python and) Chandler (and e.g. Kapor/Mozilla).. PS2. Is there an alternative gateway to the forums. I just hate using the web(browser it's slow). Google Groups used to be for Usenet, but not this forum..
[julia-users] Re: Best way to minimize memory allocation in iterative method?
Thanks! Doing the updates in place instead of copying tripled speed, and using a:b instead of [a:b] got another 50% speed increase. It's amazing how writing longer code can sometimes make things go faster.
Re: [julia-users] THANKS to Julia core developers!
As a long-time R user, I second that. Brian Ripley is a core contributor to base R, does reply on R-help to many requests, but is very acerbic. In general, there is quite a big divide between R core contributors and R users. This is totally absent in Julia, with the founders of the language very involved with the early adopters. It's a great asset to have for a new language. Just in recent times, I would say that the early adopters of Clojure were instrumental in the adoption of the language. -gappy On Tuesday, December 16, 2014 6:50:12 PM UTC-5, ivo welch wrote: my note was partly a joke, partly a warning. the R community started out very nice, too. many still are. but some of the tone has shifted towards the obnoxious. the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. this reminds me: for those of us who cannot contribute, is there a julia foundation membership? it would not be a bad idea to have us using users get used to contributing, too.
Re: [julia-users] THANKS to Julia core developers!
Please don't do that. Unnecessary comments directed at specific people are exactly the kind of thing that start to change the character of a mailing list and lead to burnout. Such comments are especially unwelcome when directed against people who are not active on the list. We should set a higher standard, especially in a thread about how nice the mailing list is. On Thu, Dec 18, 2014 at 4:13 PM, Giuseppe Paleologo paleol...@gmail.com wrote: As a long-time R user, I second that. Brian Ripley is a core contributor to base R, does reply on R-help to many requests, but is very acerbic. In general, there is quite a big divide between R core contributors and R users. This is totally absent in Julia, with the founders of the language very involved with the early adopters. It's a great asset to have for a new language. Just in recent times, I would say that the early adopters of Clojure were instrumental in the adoption of the language. -gappy On Tuesday, December 16, 2014 6:50:12 PM UTC-5, ivo welch wrote: my note was partly a joke, partly a warning. the R community started out very nice, too. many still are. but some of the tone has shifted towards the obnoxious. the weirdest part is that there are some people who seem to enjoy *really* helping users, all the while being somewhat insulting. it is my (incomplete) understanding that internal strife in the core development team has become negative, too. it will be up to the julia core team to set the community standard and watch themselves and the community to keep it alive. this reminds me: for those of us who cannot contribute, is there a julia foundation membership? it would not be a bad idea to have us using users get used to contributing, too.
Re: [julia-users] Re: Best way to minimize memory allocation in iterative method?
Just to be clear (otherwise someone is going to link to this thread and say that Julia insists that you should write long, devectorized code): while writing slightly longer, more explicit code is – for now – sometimes the easiest way to make your code really fast, this is not a Julia mantra nor a philosophical underpinning of the language. We would like, through improved compiler technology, to make the shortest, most convenient way to write something also the fastest. But we won't cheat to accomplish this by doing something hacky that doesn't generalize – and that *is* a philosophical underpinning of the language. On Thu, Dec 18, 2014 at 3:37 PM, DumpsterDoofus peter.richter@gmail.com wrote: Thanks! Doing the updates in place instead of copying tripled speed, and using a:b instead of [a:b] got another 50% speed increase. It's amazing how writing longer code can sometimes make things go faster.
Re: [julia-users] Re: Can Julia be the next web scripting language?
It's getting closer. First, the --compile=all and --dump-bitcode=yes options to Julia are a big help in dumping LLVM bitcodes. The contrib/build_sysimg.jl code is useful for this after modifying it to include these Julia command-line options. There are a few issues with using Emscripten with Julia code: * 32-bit code -- Emscripten expects 32-bit bitcode. On a 64-bit platform, Julia generates 64-bit bitcode. I tried using the build_sysimg.jl script with a target of i686, but Emscripten still gave me errors. * struct error -- See this bug: https://code.google.com/p/nativeclient/issues/detail?id=3932 * Symbol naming -- Code generated by Julia has a lot of symbols like #sym that Emscripten doesn't like. This can be handled by either Emscripten or on the Julia side. Lastly, libjulia bitcode needs to be generated. I've gotten about 80% of it to compile after stripping out the libuv stuff. There are also libraries like BLAS, but these can be added as needed. On Thu, Dec 18, 2014 at 2:48 PM, Páll Haraldsson pall.haralds...@gmail.com wrote: On Thursday, December 18, 2014 10:59:53 AM UTC, Jerzy Głowacki wrote: I suppose Julia would be the successor to JavaScript if it was converted by Emscripten and run by asm.js. But for now there seems to be an Enscripten bug which invalidates Julia IR code. Hi, Curious, what bug is that? You have a bug number or know what it relates to (on Emscripten side)? Invalidates Julia IR code? Intermediate Representation? Does the bug only affect Julia? Have anything to do with JavaScript ints (that it their non-existence, JS uses doubles as int)? Did you really mean to say Julia would be the successor or Julia could be a successor? Is there any obvious successor or fairly much used alternative currently? I've heard about Coffeescript, Typescript and Dart etc. languages made to compile to JavaScript. And Scala.js. Scala was meant to run in JVM and Scala.js has slightly different semantics because if JS's ints. I wandered if Julia could do the same. I assume you are talking about compile Julia the language/environment to JavaScript and run the the REPL in a browser? Should it be easier to compile Julia code to C (possible now) and that C code via Emscripten to JavaScript? Emscripten says they handle portable C code. Doesn't Intel's Julia2C claim portable C? I don't really see how Emscripten handles ints. There is no requirement that C's ints are less than 64 bit? Do not even have to be two's compliment, right? I fail to see how they densely pack ints (that are 64-bit doubles) into 32-bit in arrays.. PS. I dreamt my first dream in Julia code tonight. Might be a sign it's the language for me.. :) Reminds me: Dreaming in code is a good book on (Python and) Chandler (and e.g. Kapor/Mozilla).. PS2. Is there an alternative gateway to the forums. I just hate using the web(browser it's slow). Google Groups used to be for Usenet, but not this forum..
Re: [julia-users] Re: Best way to minimize memory allocation in iterative method?
The OP said that it was the three lines involving arr1, arr2, and arr3 that took all the time and allocated lots of memory. But I believe that the lines arr3 = arr2 arr2 = arr1 should not allocate memory nor take any significant time to execute because they do not involve any copying, but merely arrange for the variable names to be associated with already declared arrays. Correct? --Peter On Thursday, December 18, 2014 1:59:26 PM UTC-8, Stefan Karpinski wrote: Just to be clear (otherwise someone is going to link to this thread and say that Julia insists that you should write long, devectorized code): while writing slightly longer, more explicit code is – for now – sometimes the easiest way to make your code really fast, this is not a Julia mantra nor a philosophical underpinning of the language. We would like, through improved compiler technology, to make the shortest, most convenient way to write something also the fastest. But we won't cheat to accomplish this by doing something hacky that doesn't generalize – and that *is* a philosophical underpinning of the language. On Thu, Dec 18, 2014 at 3:37 PM, DumpsterDoofus peter.ri...@gmail.com javascript: wrote: Thanks! Doing the updates in place instead of copying tripled speed, and using a:b instead of [a:b] got another 50% speed increase. It's amazing how writing longer code can sometimes make things go faster.
[julia-users] Re: THANKS to Julia core developers!
Adding my sentiments here too. I went from C#, Matlab - Java (University and project requirements) - Python - Julia Julia has been the only language where I have felt free to do as I please! There was a bit of swearing until I figured out how to properly make use of templates and modules. Now I am just happily working and getting results. Python I enjoyed because it was easy and I managed to get a lot done. Though speedwise it left too much lacking and I had to do too much for minor gains. I moved the code over to Julia and it was faster than the c++ version I had done. I could have probably improved the C++ version but that took days whereas Julia took hours! Other than me being a complete idiot regarding Real and Float (Thank you Stefan for helping with that btw!) it was easy to create fast enough things. It was then easy to take my code and extend it further. Multimethods really are amazing! Then reading more the community, while not being active in it, really was something to see. Where discussions take place that actually have meaning more than just being fluff. When I am bored I go and read the github issues and pull requests and see if there is anything interesting. This happens quite regularly! So just another thank you to all involved!
Re: [julia-users] Re: Best way to minimize memory allocation in iterative method?
Yeah, actually that was another mistake I made: I totally forgot that arrays are passed by reference, which means that modifications made to arr1 will affect arr2, which caused a load of nonsense. Not sure why it seemed to be allocating memory and taking time, though.
Re: [julia-users] Rust and Julia - Together? Advice
Hi Jake, That is awesome. Thank you for sharing. I'll give it a try next time I'm in my (second) office. Assuming they work, would you consider adding them to the ZGuide? All examples in the ZGuide are given in many different languages, but there are no Julia examples. It would be a great exercise and a great contribution to implement all examples and add them there. By the way, the Julia ZMQ and Java JeroMQ I have are based on different versioms of libzmq, i.e. 3.2.4 vs 3.2.2. I guess I'll find out next time I'm in front of my dev laptop, but should I be optimistic the two can still talk to each other? Best regards, Eric
[julia-users] Re: Global variables
I realise my original question should have been more specific and not digress about implementing fast globals. Please bear with me as I'm relatively new to programming let alone Julia (from my little experience I quite like Julia) Any guidelines on best practice would be appreciated, but I'll also ask some specific questions below. I guess conceptually I have a lot of global parameters that usually don't change much. I also have tried to define short, one-purpose functions (which I believe is encouraged?) These functions are called in many places and most have specific arguments but also access the global parameters My current implementation uses Julia global variables these parameters, which has 2 benefits for me: - Easy to setup and leads to simpler code. - Global parameters are accessible from REPL (or in my case Juno LT), which allows interactive work. However using global variables probably leads to poor performance. And probably poor program design? I'm quite happy to not use globals, but I'm not sure about the best way to go about this. In any case, my next incarnation was to define a composite structure [ GlobalParameters] and instantiate as a global variable [g = GlobalParameters() # constructors initialises all fields]. Within functions, I access global parameters from this global composite type variable [g.field] - g is not passed as argument to function, but accessed as global This improved performance. However, I wondered whether there was any real type stability improvement, since variable is still global. Then I declared the global variable as const. [const g = GlobalParameters()] This appeared to lead to a further speedup. Q1. Are globals slow solely because type cannot be determined or guaranted? or is it also because globals are slower for other reasons as well? Q2. Can you verify that declaring global composite type as const should theoretically lead to better performance? Q3. Would there be further benefit from passing composite type as argument to all the functions? If anyone wants to provide general tips or best practice (general programming or Julia specific) I would welcome that as well. Thanks! Greg On Tuesday, December 16, 2014 8:40:50 AM UTC+11, Greg Plowman wrote: Hi, I understand using global variables can lead to poor performance. This is because types cannot be guaranteed? Globals could be reassigned with different types? For my purpose, I set up a lot of parameters as global variables. Then define lots of functions that have specific arguments but also use some subset of the globals as well. I guess the response is going to be: do not to use globals. I get that. However, to me it seems sometimes a natural and easy way to think about program a solution. What are the alternatives: 1. Live with poor performance 2. Create composite type containing the global variables, and pass around a reference to single global variable of this composite type - Would this boost performance? - Would it boost performance if I didn't pass the composite-type variable as argument, but instead access it inside functions as a global variable? 3. What else? To compare performance, in some functions I assigned global variables to local variables annotated with type. Then used local variable in function. This produced a considerable speed-up, between 3-4x faster. function foo(arg1::arg1type, ...) l_var1::var1type = g_var1 l_var2::var1type = g_var2 ... x = l_var1 * ... end Can I do something like the following with equivalent speedup? function foo(arg1::arg1type, ...) g_var1::var1type g_var2::var2type ... x = g_var1 * ... end This would be sort of like declaring global variables as arguments to function but with types. Compiler could optimise. Runtime error if type not correct. As an aside, it occurred to me that there might be 2 cases for global variables. 1. globals used in interactive REPL environment, where global can be reassigned 2. globals used for bad or lazy or whatever programming Why can't we have static *type* globals? Not const but const type. So effectively, we have 3 levels of const/variable-ness global x = 5 x = 6 # OK x = 9.1 # OK static x = 5 x = 6 # OK x = 9.1 # ERROR, must release/clear/reset x first const x = 5 x = 6 # ERROR x = 9.1 # ERROR Cheers, Greg
Re: [julia-users] Re: Global variables
This should get you started: http://c2.com/cgi/wiki?GlobalVariablesAreBad -- John On Dec 18, 2014, at 7:36 PM, Greg Plowman greg.plow...@gmail.com wrote: I realise my original question should have been more specific and not digress about implementing fast globals. Please bear with me as I'm relatively new to programming let alone Julia (from my little experience I quite like Julia) Any guidelines on best practice would be appreciated, but I'll also ask some specific questions below. I guess conceptually I have a lot of global parameters that usually don't change much. I also have tried to define short, one-purpose functions (which I believe is encouraged?) These functions are called in many places and most have specific arguments but also access the global parameters My current implementation uses Julia global variables these parameters, which has 2 benefits for me: Easy to setup and leads to simpler code. Global parameters are accessible from REPL (or in my case Juno LT), which allows interactive work. However using global variables probably leads to poor performance. And probably poor program design? I'm quite happy to not use globals, but I'm not sure about the best way to go about this. In any case, my next incarnation was to define a composite structure [GlobalParameters] and instantiate as a global variable [g = GlobalParameters() # constructors initialises all fields]. Within functions, I access global parameters from this global composite type variable [g.field] - g is not passed as argument to function, but accessed as global This improved performance. However, I wondered whether there was any real type stability improvement, since variable is still global. Then I declared the global variable as const. [const g = GlobalParameters()] This appeared to lead to a further speedup. Q1. Are globals slow solely because type cannot be determined or guaranted? or is it also because globals are slower for other reasons as well? Q2. Can you verify that declaring global composite type as const should theoretically lead to better performance? Q3. Would there be further benefit from passing composite type as argument to all the functions? If anyone wants to provide general tips or best practice (general programming or Julia specific) I would welcome that as well. Thanks! Greg On Tuesday, December 16, 2014 8:40:50 AM UTC+11, Greg Plowman wrote: Hi, I understand using global variables can lead to poor performance. This is because types cannot be guaranteed? Globals could be reassigned with different types? For my purpose, I set up a lot of parameters as global variables. Then define lots of functions that have specific arguments but also use some subset of the globals as well. I guess the response is going to be: do not to use globals. I get that. However, to me it seems sometimes a natural and easy way to think about program a solution. What are the alternatives: Live with poor performance Create composite type containing the global variables, and pass around a reference to single global variable of this composite type Would this boost performance? Would it boost performance if I didn't pass the composite-type variable as argument, but instead access it inside functions as a global variable? What else? To compare performance, in some functions I assigned global variables to local variables annotated with type. Then used local variable in function. This produced a considerable speed-up, between 3-4x faster. function foo(arg1::arg1type, ...) l_var1::var1type = g_var1 l_var2::var1type = g_var2 ... x = l_var1 * ... end Can I do something like the following with equivalent speedup? function foo(arg1::arg1type, ...)g_var1::var1typeg_var2::var2type ... x = g_var1 * ... end This would be sort of like declaring global variables as arguments to function but with types. Compiler could optimise. Runtime error if type not correct. As an aside, it occurred to me that there might be 2 cases for global variables. globals used in interactive REPL environment, where global can be reassigned globals used for bad or lazy or whatever programming Why can't we have static type globals? Not const but const type. So effectively, we have 3 levels of const/variable-ness global x = 5x = 6 # OKx = 9.1 # OK static x = 5x = 6 # OKx = 9.1 # ERROR, must release/clear/reset x first const x = 5x = 6 # ERRORx = 9.1 # ERROR Cheers, Greg
[julia-users] pod-like and use documentation for julia?
I am still a lurker, not a user---although I hope to adopt julia in a year or so for my masters student at a B-school (wish me luck!). teaching it in a class will force me to switch to julia. I just looked at some random .jl files in the julia source. they did not have a lot of documentation in them---maybe I looked at the wrong ones. one aspect of a language that I think is important is standard doc integration. [I always wished for some small knuth-like intelligently documented code, but none of my past languages had much. (knuth's had too much.)] perl's pod is reasonably good, although a bit clumsy (no easy one-liner pod docs). some more standards would be good, too. every function without a one-liner preceding it could give a warning, for example. every file without an author could give a warning. the julia module name should be automatically considered to be the package name, unless overridden (to avoid double-typing and out-of-sync). and so on. maybe this is all already there and I just did not see it. the julia docs on the website did not have a section on comments and user documentation. one of the aspects that I liked about R was that the question mark gives longer documentation than julia does. the best R doc parts are *always* the plenty of example uses towards the end. I almost always go there first. the see also pointers are also very good. right now, I am looking at julia's ?linreg, and I wonder how I would get standard errors, predictions, etc. one of the aspects that I did not like about R was that this was not easy to imitate with an end-user R program. I would have liked to integrate similar ?myfun for my end-user R function. a second suggestion, also important to me (well, to me and to help my students): ?R.lm should tell me what the equivalent julia function for lm() is, or at least where I should start looking. ?ols could take an intelligent guess at what it is that the user wants to learn, rather than tell me it is not defined. the first would be even more useful if we had a list of the most-used R functions. regards, /iaw
[julia-users] Designing a macro to do something before and after an expression
I want to write a macro that performs certain actions before and after an expression, returning the value of the expression. Here is my first try: macro wrap(expr) :(global a=1; r = $(esc(expr)); global a=0; r) end Obviously, setting the global variable `a` to 1 and 0 are just placeholder actions. This works for expressions, but doesn't work when `return` statements are involved. julia @wrap 3 3 julia @wrap (if true return 3 else 0 end) ERROR: syntax: misplaced return statement This simpler case demonstrates the problem: julia @wrap (return 3) = ERROR: syntax: misplaced return statement The problem goes away if I don't try to conserve the result into a variable `r`. However, I really need to do that. How? -erik PS: The macro `@inbounds` in Base has the same issue -- I assume this is the reason why it doesn't return a value. -- Erik Schnetter schnet...@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
[julia-users] ANN: microcoverage.jl
I wrote a package that offers code-coverage for Julia source code, i.e., it reports how many times each line of code was executed during a test-run. It produces a report that is finer-grained than the built-in code coverage capability, namely, it also reports how many times each branch was followed in expressions with branching operators: ||, , ?:. My code is a bit of a hack, so comments on how to improve it are welcome. It is available as follows: Pkg.clone(git://github.com/StephenVavasis/microcoverage) -- Steve Vavasis
Re: [julia-users] Global variables
It's important to realize that Julia's global variables are slow because they are generally not a good design decision, not the other way around. They could be optimized, but it's preferable to encourage taking a different approach. Julia makes other similar tradeoffs, including prohibiting eval() from interacting with the local scope. It's not that this couldn't be implemented, it's just that there are much better ways of expressing the problem that don't require this feature. You've mentioned reasons that you use global variables, so I'll mention a few reasons that I generally try to avoid them. In short, if I use globals heavily, I find my code becomes unmaintainable immediately after writing it, and therefore the program rapidly becomes awful, bug-ridden, and impossible to add features to. Having globals simultaneously increases the code complexity and decreases readability. This is actually true of mutation in general, but mutation becomes more difficult to track when the variables that are changing are global. With local variables, you can see the explicit patterns of data-flow. With global variables, you cannot safely assume that the variable will not change during the execution of that function. Even if it is true now, someone (or even you) might later want to add some cool feature, but since they can't trace where that variable gets used, it becomes incredibly difficult to ensure that the change won't impact other parts of the code unintentionally. A second part of avoiding globals is that it makes writing good tests much harder. Ideally, a function does exactly one thing: it computes a result. In some cases, that computation is to store a value somewhere. When a function tries to do multiple things, it bugs become harder to trace, since you can't separately test the cause and the effect. Of course, I don't actually avoid them. Types, functions, STDIO, memory allocation/gc are all global because the tradeoff is not worthwhile. I'm also going to make a distinction between constants and mutatable globals. It's perfectly reasonable to have constant data accessible from the global scope. Note: some people avoid passing variables around because they assume this is a lot of work for the computer. It isn't. Like humans, it's a lot easier for the compiler to track dependencies between local variables and function parameters than to analyze a global variable. oh, and I see John beat me to finishing this reply, but I will second his recommendation of the c2.com wiki. On Mon Dec 15 2014 at 4:40:53 PM Greg Plowman greg.plow...@gmail.com wrote: Hi, I understand using global variables can lead to poor performance. This is because types cannot be guaranteed? Globals could be reassigned with different types? For my purpose, I set up a lot of parameters as global variables. Then define lots of functions that have specific arguments but also use some subset of the globals as well. I guess the response is going to be: do not to use globals. I get that. However, to me it seems sometimes a natural and easy way to think about program a solution. What are the alternatives: 1. Live with poor performance 2. Create composite type containing the global variables, and pass around a reference to single global variable of this composite type - Would this boost performance? - Would it boost performance if I didn't pass the composite-type variable as argument, but instead access it inside functions as a global variable? 3. What else? To compare performance, in some functions I assigned global variables to local variables annotated with type. Then used local variable in function. This produced a considerable speed-up, between 3-4x faster. function foo(arg1::arg1type, ...) l_var1::var1type = g_var1 l_var2::var1type = g_var2 ... x = l_var1 * ... end Can I do something like the following with equivalent speedup? function foo(arg1::arg1type, ...) g_var1::var1type g_var2::var2type ... x = g_var1 * ... end This would be sort of like declaring global variables as arguments to function but with types. Compiler could optimise. Runtime error if type not correct. As an aside, it occurred to me that there might be 2 cases for global variables. 1. globals used in interactive REPL environment, where global can be reassigned 2. globals used for bad or lazy or whatever programming Why can't we have static *type* globals? Not const but const type. So effectively, we have 3 levels of const/variable-ness global x = 5 x = 6 # OK x = 9.1 # OK static x = 5 x = 6 # OK x = 9.1 # ERROR, must release/clear/reset x first const x = 5 x = 6 # ERROR x = 9.1 # ERROR Cheers, Greg
Re: [julia-users] Designing a macro to do something before and after an expression
try/finally does this. for an example, see the implementation of `cd(::Function)` in Base. (`@inbounds` doesn't return a value to force you to put it outside an expression, not because it couldn't) On Thu Dec 18 2014 at 10:41:38 PM Erik Schnetter schnet...@gmail.com wrote: I want to write a macro that performs certain actions before and after an expression, returning the value of the expression. Here is my first try: macro wrap(expr) :(global a=1; r = $(esc(expr)); global a=0; r) end Obviously, setting the global variable `a` to 1 and 0 are just placeholder actions. This works for expressions, but doesn't work when `return` statements are involved. julia @wrap 3 3 julia @wrap (if true return 3 else 0 end) ERROR: syntax: misplaced return statement This simpler case demonstrates the problem: julia @wrap (return 3) = ERROR: syntax: misplaced return statement The problem goes away if I don't try to conserve the result into a variable `r`. However, I really need to do that. How? -erik PS: The macro `@inbounds` in Base has the same issue -- I assume this is the reason why it doesn't return a value. -- Erik Schnetter schnet...@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Recompiling ast and document about uncompressed ast
The lowered AST is what you get when you call expand on any other AST. It is a simplified format that is easier to analyze, type-infer, and execute. However, it is still a regular AST object, so you can modify it and recreate another function with it just like the pre-expanded version (in fact, this is what inference.jl does). The first few arrays in the lambda provide information to the compiler. The first is the argument list. The next is a list of (local variables list, variable info, captured variable list) The problem with your code is that you are trying to mutate internal compiler state, and it has no idea that you are trying to do so. It has already optimized the call for that function, and no longer cares about the code.ast field for that lambda. What you really want is a new function entirely: julia function recompile(func1, func2, args) code1 = get_code(func1, args) code2 = get_code(func2, args) @assert code1.module == code2.module eval_code1 = code1.module.eval(code1) eval_res1 = eval_code1(args...) @assert eval_res1 == func1(args...) println(eval_code: $(eval_res1)) eval_code2 = code2.module.eval(code2) eval_res2 = eval_code2(args...) @assert eval_res2 == func2(args...) println(eval_code: $(eval_res2)) ast1 = ccall(:jl_uncompress_ast, Any, (Any, Any), code1, code1.ast) println(ast1: $(ast1.args[3])) ast2 = ccall(:jl_uncompress_ast, Any, (Any, Any), code2, code2.ast) code3 = Expr(:function, Expr(:tuple, ast2.args[1]...), Expr(:block, ast2.args[3].args...)) # what did we construct? how does it compare to `:( function(a,b) a+b end )|dump`? # dump(code3); Meta.show_sexpr(code3); println() eval_new_code = code1.module.eval(code3) eval_new_res = eval_new_code(args...) @assert eval_new_res == func2(args...) println(eval_new_code: $(eval_new_res)) end recompile (generic function with 1 method) julia recompile(+, -, (1.2, 3.4)) eval_code: 4.6 eval_code: -2.2 ast1: begin # float.jl, line 192: return box(Float64,sub_float(unbox(Float64,x),unbox(Float64,y))) end Expr head: Symbol function args: Array(Any,(2,)) 1: Expr head: Symbol tuple args: Array(Any,(2,)) 1: Symbol x 2: Symbol y typ: Any 2: Expr head: Symbol block args: Array(Any,(2,)) 1: Expr head: Symbol line args: Array(Any,(2,)) typ: Any 2: Expr head: Symbol return args: Array(Any,(1,)) typ: Any typ: Any typ: Any (:function, (:tuple, :x, :y), (:block, (:line, 192, symbol(float.jl)), (:return, (:call, :box, :Float64, (:call, :sub_float, (:call, :unbox, :Float64, :x), (:call, :unbox, :Float64, :y )) eval_new_code: -2.2 On Thu Dec 18 2014 at 1:14:32 PM Yichao Yu yyc1...@gmail.com wrote: Hi, I'm wondering if there's a way to recompile the lowered AST back into a function and what additional information (other than the module it should be evaluate in) is necessary to do that what I've tried and would like to do is demonstrated with the code attached. The idea is that I would like to modify the code of a function/method before calling it (in this case, I am replacing it with the code from another function with the same signature). (In another word, the question is how to construct a function from code1 with ast2 and run it.) It will also be nice if there's more documentation on this lowered AST since it seems to be very different form the one you get in a function definition. (e.g. what is the array in `code_lowered(+, (Float64, Float64))[1].args[2]` mean) A brief description of what I want to do. I'm writing a control system with dynamic logic. However, since the code might not be running on a single host, it is hard to use an existing interpreter to do what I want. Trying to avoid inventing yet another language, I hope to use a subset of an existing language and somehow transform it into a representation that can be run somewhere else and it seems that the inspection and meta programming feature in julia is very useful for this. It might be possible to just apply a macro to every function / method definition but it will be much harder to write. I would rather try to trace the excution and use the AST acquired at runtime. For that I would like to know more about the lowered AST and possibly how to execute a modified version in order to trace recursive / nested function calls. This might not be the best way to do it and I'm still in planning stage so any suggestions on the design is also welcome. Cheers, Yichao Yu
Re: [julia-users] pod-like and use documentation for julia?
Yes, inline doc capabilities are in-progress on master. See for example: https://github.com/JuliaLang/julia/blob/master/base/basedocs.jl Some people are trying this out in packages but there are some issues not completely decided such as the syntax: https://groups.google.com/d/msg/julia-users/wBZGlzFo7eI/pB0qdVGVACEJ Regarding inline docs for base functions, as far as I know it is still kind of an open question whether everything will be migrated. Regarding the suggestion, it is interesting. The one comment I have is that it may be impractical to have this in base because much of the functionality will live (and does already) in packages. Something like this might work better as a web-based service (possibly queryable from the REPL?). I'm thinking specifically of something like Haskell's Hoogle system. On Thu, Dec 18, 2014 at 10:37 PM, ivo welch ivo...@gmail.com wrote: I am still a lurker, not a user---although I hope to adopt julia in a year or so for my masters student at a B-school (wish me luck!). teaching it in a class will force me to switch to julia. I just looked at some random .jl files in the julia source. they did not have a lot of documentation in them---maybe I looked at the wrong ones. one aspect of a language that I think is important is standard doc integration. [I always wished for some small knuth-like intelligently documented code, but none of my past languages had much. (knuth's had too much.)] perl's pod is reasonably good, although a bit clumsy (no easy one-liner pod docs). some more standards would be good, too. every function without a one-liner preceding it could give a warning, for example. every file without an author could give a warning. the julia module name should be automatically considered to be the package name, unless overridden (to avoid double-typing and out-of-sync). and so on. maybe this is all already there and I just did not see it. the julia docs on the website did not have a section on comments and user documentation. one of the aspects that I liked about R was that the question mark gives longer documentation than julia does. the best R doc parts are *always* the plenty of example uses towards the end. I almost always go there first. the see also pointers are also very good. right now, I am looking at julia's ?linreg, and I wonder how I would get standard errors, predictions, etc. one of the aspects that I did not like about R was that this was not easy to imitate with an end-user R program. I would have liked to integrate similar ?myfun for my end-user R function. a second suggestion, also important to me (well, to me and to help my students): ?R.lm should tell me what the equivalent julia function for lm() is, or at least where I should start looking. ?ols could take an intelligent guess at what it is that the user wants to learn, rather than tell me it is not defined. the first would be even more useful if we had a list of the most-used R functions. regards, /iaw
Re: [julia-users] Re: types, arrays, and syntax () {} []
On Thursday, December 18, 2014 5:33:11 PM UTC-5, Steve Cordwell wrote:
Hi John,
`help(Array)` is telling you to use Array(SomeType, some_num_of_dims). If
it was telling you about the type it would look more like this:
help(Int)
DataType :
Int64
supertype: Signed
The `Base.Array(type, dims)` line at the top of the help is giving you the
calling signature. Since Array is both the type name and a method to
construct arrays, it can be used with both {} and ().
Ah, both! I see. Also, I see that I can call `help()` on both types *and*
methods, but since Array is both, `help()` has to pick one, and I get the
Array *method* help text.
Still though, the example Stefan posted above doesn't look like it makes
sense to me. He wrote:
If you want to construct an uninitialized three value Int array, you
can do this:
v = Array(Int, 3)
but `help(Array)` says I'm supposed to pass in dims; I'd assumed this
means the same thing as what `ndims` returns --- i.e., 1D (Vector), 2D
(Matrix), etc. But calling `Array(Int, 3)` returns a *1D* array, rather
than 3D. Am I misreading the docs for Base.Array()?
Try this:
typeof(Array{Int, 3})
DataType
typeof(Array(Int, 3))
Array{Int64,1}
typeof([1, 2, 3])
Array{Int64,1}
Ok. The object I get back `Array(Int64, 3)` is of exactly the same type as
the one I get back from `[1, 2, 3]` (though of course the latter comes with
garbage values in it as Stefan noted.) I can `push!()` to both of them.
I also see that `Array` is an abstract (non-leaf) type, whereas `Array{Int,
3}` is a concrete type:
~~~
julia isleaftype(Array)
false
julia isleaftype(Array{Int, 3})
true
~~~
Thanks!
-- John
Re: [julia-users] Re: types, arrays, and syntax () {} []
To get a 3 dimensional array using the syntax Stefan described, you need to
tell it the number of indices along each dimesion. Eg `Array(Int, 3, 4, 5)`
produces a 3x4x5 array
On Friday, December 19, 2014 7:29:50 PM UTC+13, jgabri...@gmail.com wrote:
On Thursday, December 18, 2014 5:33:11 PM UTC-5, Steve Cordwell wrote:
Hi John,
`help(Array)` is telling you to use Array(SomeType, some_num_of_dims). If
it was telling you about the type it would look more like this:
help(Int)
DataType :
Int64
supertype: Signed
The `Base.Array(type, dims)` line at the top of the help is giving you
the calling signature. Since Array is both the type name and a method to
construct arrays, it can be used with both {} and ().
Ah, both! I see. Also, I see that I can call `help()` on both types *and*
methods, but since Array is both, `help()` has to pick one, and I get the
Array *method* help text.
Still though, the example Stefan posted above doesn't look like it makes
sense to me. He wrote:
If you want to construct an uninitialized three value Int array, you
can do this:
v = Array(Int, 3)
but `help(Array)` says I'm supposed to pass in dims; I'd assumed this
means the same thing as what `ndims` returns --- i.e., 1D (Vector), 2D
(Matrix), etc. But calling `Array(Int, 3)` returns a *1D* array, rather
than 3D. Am I misreading the docs for Base.Array()?
Try this:
typeof(Array{Int, 3})
DataType
typeof(Array(Int, 3))
Array{Int64,1}
typeof([1, 2, 3])
Array{Int64,1}
Ok. The object I get back `Array(Int64, 3)` is of exactly the same type as
the one I get back from `[1, 2, 3]` (though of course the latter comes with
garbage values in it as Stefan noted.) I can `push!()` to both of them.
I also see that `Array` is an abstract (non-leaf) type, whereas
`Array{Int, 3}` is a concrete type:
~~~
julia isleaftype(Array)
false
julia isleaftype(Array{Int, 3})
true
~~~
Thanks!
-- John
Re: [julia-users] Re: types, arrays, and syntax () {} []
zeros, ones, rand etc all work in this way also
On Friday, December 19, 2014 7:45:43 PM UTC+13, Steve Cordwell wrote:
To get a 3 dimensional array using the syntax Stefan described, you need
to tell it the number of indices along each dimesion. Eg `Array(Int, 3, 4,
5)` produces a 3x4x5 array
On Friday, December 19, 2014 7:29:50 PM UTC+13, jgabri...@gmail.com wrote:
On Thursday, December 18, 2014 5:33:11 PM UTC-5, Steve Cordwell wrote:
Hi John,
`help(Array)` is telling you to use Array(SomeType, some_num_of_dims).
If it was telling you about the type it would look more like this:
help(Int)
DataType :
Int64
supertype: Signed
The `Base.Array(type, dims)` line at the top of the help is giving you
the calling signature. Since Array is both the type name and a method to
construct arrays, it can be used with both {} and ().
Ah, both! I see. Also, I see that I can call `help()` on both types *and*
methods, but since Array is both, `help()` has to pick one, and I get the
Array *method* help text.
Still though, the example Stefan posted above doesn't look like it makes
sense to me. He wrote:
If you want to construct an uninitialized three value Int array, you
can do this:
v = Array(Int, 3)
but `help(Array)` says I'm supposed to pass in dims; I'd assumed this
means the same thing as what `ndims` returns --- i.e., 1D (Vector), 2D
(Matrix), etc. But calling `Array(Int, 3)` returns a *1D* array, rather
than 3D. Am I misreading the docs for Base.Array()?
Try this:
typeof(Array{Int, 3})
DataType
typeof(Array(Int, 3))
Array{Int64,1}
typeof([1, 2, 3])
Array{Int64,1}
Ok. The object I get back `Array(Int64, 3)` is of exactly the same type
as the one I get back from `[1, 2, 3]` (though of course the latter comes
with garbage values in it as Stefan noted.) I can `push!()` to both of them.
I also see that `Array` is an abstract (non-leaf) type, whereas
`Array{Int, 3}` is a concrete type:
~~~
julia isleaftype(Array)
false
julia isleaftype(Array{Int, 3})
true
~~~
Thanks!
-- John
