Re: [julia-users] randperm run time is slow

> Let's capture this as a Julia performance issue on github, > if we can't figure out an easy way to speed this up right away. I think I remember having identified a potentially sub-optimal implementation of this function few weeks back (perhaps no more than what Tim suggested) and had planned to

[julia-users] Scheduling external programs in slurm with Julia and ClusterManagers

This code @everywhere function test_run_job() return readall(`echo test`) end should go after addprocs otherwise it is only defined in existing workers at the time it is defined. Moving it there works locally.

[julia-users] Scheduling external programs in slurm with Julia and ClusterManagers

This code @everywhere function test_run_job() return readall(`echo test`) end should go after addprocs otherwise it is only defined in existing workers at the time it is defined. Moving it there works locally.

Re: [julia-users] randperm run time is slow

If you see the code as @timholy suggested, you will see the knuth shuffle implementation. Let's capture this as a Julia performance issue on github, if we can't figure out an easy way to speed this up right away. -viral On Saturday, January 23, 2016 at 9:47:21 AM UTC+5:30, Viral Shah wrote: > >

Re: [julia-users] randperm run time is slow

Generating one by one or together is a small part of the execution time: julia> @time rand(3*10^8); 1.281946 seconds (10 allocations: 2.235 GB, 0.23% gc time) julia> r(n) = for i=1:n; rand(); end; julia> @time r(3*10^8) 0.632389 seconds (6 allocations: 192 bytes) The one by one is perhaps fa

[julia-users] Running multiple scripts in parallel Julia sessions

Let's say I have 10 julia scripts, scripts = ["script1.jl", "script2.jl", , "script10.jl"] and I would like to run them in parallel in separate Julia sessions, but 4 at a time (since I only have 4 cores on my machine). Is there any way to do this programmatically? I tried doing this: addp

Re: [julia-users] immutability, value types and reference types?

I'm trying to accomplish understanding what Julia does. On Friday, January 22, 2016 at 7:13:58 AM UTC-8, Stefan Karpinski wrote: > > What are you trying to accomplish? > > On Thu, Jan 21, 2016 at 7:00 PM, > wrote: > >> >> >> On Thursday, January 21, 2016 at 8:48:19 AM UTC-8, Stefan Karpinski wrot

[julia-users] Scheduling external programs in slurm with Julia and ClusterManagers

Hi, I am new to Julia and this is my first post in this group. I am writing a script in Julia that schedules external programs with Slurm on a Rocks Cluster. When it came to actually scheduling the jobs I ran into difficulties getting external programs to run inside a function. The code below i

[julia-users] newbie

I've been using IDL, matlab, and octave for a while (mostly IDL) but Juila looks really interesting. I have been interested in wavelets (one of the reasons for matlab) and think that if Julia wavelet package got non-separable wavelets (e.g., quincunx based) it would really stand out in that de

Re: [julia-users] Tuples vs. vectors for indexing into an array

I'll give it a try. It's missing scalar addition, though. CartesianIndex((4,5)) + 1 # error Thanks! On Friday, January 22, 2016 at 8:26:53 PM UTC-5, Tim Holy wrote: > > Check out CartesianIndex. It's a little annoying to type long names, but > it > supports all those operations, and julia al

Re: [julia-users] Tuples vs. vectors for indexing into an array

Check out CartesianIndex. It's a little annoying to type long names, but it supports all those operations, and julia already has all the indexing operations defined for it. --Tim On Friday, January 22, 2016 05:20:59 PM Cedric St-Jean wrote: > I manipulate a lot of images, and I have to deal wit

Re: [julia-users] randperm run time is slow

Try @edit randperm(10) and see for yourself. My bet is that you could speed it up by generating all the random numbers you'll need in one go, rather than generating them one-by-one. Want to give it a shot? --Tim On Friday, January 22, 2016 02:54:51 PM Brian Lucena wrote: > I am running a s

[julia-users] Tuples vs. vectors for indexing into an array

I manipulate a lot of images, and I have to deal with image coordinates (aka indexes) all the time. I've been storing them as vectors so far, but the inefficiency of creating a full-blown array object to store 2 integers is gross (and it might add up, though it's hard to profile). OTOH, tuples

[julia-users] Re: randperm run time is slow

is asd typed differently earlier in your code ... ? see: https://groups.google.com/forum/#!searchin/julia-users/randperm|sort:date/julia-users/16EO_-jkz8Y/DyPE-rG76GYJ

[julia-users] optim stopping without reaching convergence

Look at the source https://github.com/JuliaOpt/Optim.jl/blob/master/src/nelder_mead.jl The number of iterations is not necessarily the same as the number of objective function evaluations

[julia-users] optim stopping without reaching convergence

Look at the source https://github.com/JuliaOpt/Optim.jl/blob/master/src/nelder_mead.jl The number of iterations is not necessarily the same as the number of other justice function evaluations.

[julia-users] Cannot pull with rebase when I try to use Pkg.update()

I am trying to use Julia on Windows for the first time, and when I try to run Pkg.update(), I got this error: error: Cannot pull with rebase: You have unstaged changes. When I run "git status" in the METADATA folder, I got: On branch metadata-v2 Your branch is behind 'origin/metadata-v2' by 5

[julia-users] randperm run time is slow

I am running a simulation that requires me to generate random permutations of size 300 million. I would ideally like to run this generation in a loop 10K or 100K times. I am surprised that randperm is not faster in Julia than it is. It seems to be considerably slower than the equivalent in R (

[julia-users] optim stopping without reaching convergence

There is really something weird happening with optim. really puzzled but what could go wrong. So when I run: optr = optimize(objfun,sps0,method = :nelder_mead,iterations=2000) The algorithm stops after precisely 10 function evaluations. Output: Results of Optimization Algorithm * Algor

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

No no, It's perfectly fine, it was my fault. What I haven't realized is that if I start the server async then my script will finish immediately, which also terminated the server. It was my responsibility to keep the whole app alive now. It works like a charm! sâmbătă, 23 ianuarie 2016, 00:0

[julia-users] Re: Help with some code where we would normally use inheritance

I think I almost got it - just need to figure out how to inject onCompletedInterval with the proper vars in macro. macro gen_intervalize(onCompletedInterval) return quote if(isNextInterval(i.intervalState, time)) onCompletedInterval(getCurrentIntervalTime(i.intervalState), i. cu

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

The shell works with processes, Julia has tasks where are not the same thing... On Fri, Jan 22, 2016 at 5:49 PM, Adrian Salceanu wrote: > The problem seems to that HttpServer can not run @async - it exits > immediately. > > === > > using HttpServer > > http = HttpHandler() do req::Request, res::

Re: [julia-users] Macros, functions and module

On Fri, Jan 22, 2016 at 4:34 PM, wrote: > Ok this could seem to make no sense, and this actually does not... It will not work since @eval eval's in the module that calls the macro, you need to use the eval function and supply with the right module. `current_module()` might work but it really dep

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

Oh, @async has worked actually! It correctly run the command, but the startup script itself was finishing and exiting immediately after. Thank you very much Stefan and Erik! vineri, 22 ianuarie 2016, 23:26:23 UTC+1, Adrian Salceanu a scris: > > Thanks! > > @async works perfectly with your

[julia-users] Help with some code where we would normally use inheritance

Hopefully you can help me out with this. I left out function implementations that we should't need to see. This part of the code I think seems ok: abstract Intervalizer{Time <: Integer, Value <: Union{AbstractFloat, Integer }} type StreamingIntervalizer{Time, Value} <: Intervalizer{Time, Val

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

The problem seems to that HttpServer can not run @async - it exits immediately. === using HttpServer http = HttpHandler() do req::Request, res::Response Response( ismatch(r"^/hello/", req.resource) ? exit(2) : 404 ) end server = Server( http ) run( server, 8001 ) # <--- this works but bl

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

Thanks Per my previous comment, unfortunately @async / @spawn cause the app / server to exit immediately. Let me give @persist a try. Cheers! vineri, 22 ianuarie 2016, 22:45:10 UTC+1, Erik Schnetter a scris: > > If you want to be able to terminate your local Julia process, and have > the s

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

Thanks! @async works perfectly with your example. And also works great with running the "ping" command. The problem is web app / Mux / HttpServer exit immediately if run @async. Same with @spawn, the app exits immediately. vineri, 22 ianuarie 2016, 22:40:56 UTC+1, Stefan Karpinski a scris: >

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

If you want to be able to terminate your local Julia process, and have the server continue to run in the background, then you might want to check out . This does the equivalent of run/detach, but in such a way that the detached process runs as daemon. Otherw

Re: [julia-users] Re: How to run a detached command and return execution to the parent script?

@spawn runs a command on a (random) worker process. If you want to do "background" work in the current process, you can use @async: julia> t = @async (sleep(5); rand()) Task (runnable) @0x000112d746a0 julia> wait(t) 0.14543742643271207 On Fri, Jan 22, 2016 at 4:33 PM, Adrian Salceanu wrote

Re: [julia-users] Macros, functions and module

Ok this could seem to make no sense, and this actually does not... Of course it solved the error in this precise case but the point of generating functions through macros inside the module was precisely because I have to define a type outside the module that should be used inside one of the func

[julia-users] Re: How to run a detached command and return execution to the parent script?

Oh! The ruby analogy made me think about actually spawning the detached command! Which produced the desired effect! julia> @spawn run(detach(`ping www.google.com`)) vineri, 22 ianuarie 2016, 22:29:27 UTC+1, Adrian Salceanu a scris: > > I guess what I'm looking for is the equivalent of Ruby's

[julia-users] Re: How to run a detached command and return execution to the parent script?

I guess what I'm looking for is the equivalent of Ruby's Process#spawn In REPL: >> pid = Process.spawn("ping www.google.com", :out => '/dev/null') 83210 >> <-- the process is running in the background and control has been returned to the REPL vineri, 22 ianuarie 2016,

Re: [julia-users] Macros, functions and module

Thanks to both of you, I actually solved my problem in the mean time by calling @eval, as Yiacho suggested, working code for the record: module Mod export g, h macro generate_f() return esc(quote macro f(name) return quote print($name) end

[julia-users] Re: Macros, functions and module

I don't know what your exact use case is, but you should't be using macros like that. Does @inline g() suit your purposes? If not, I recommend making your example a bit more pertinent to what you want accomplished. On Friday, January 22, 2016 at 3:43:10 PM UTC-5, ami...@gmail.com wrote: > > H

[julia-users] How to run a detached command and return execution to the parent script?

Hi, I'm hammering at a web app and I'm trying to setup functionality to monitor the file system for changes and restart/reload the server automatically so the changes are picked up (I'm using Mux which uses HttpServer). The approach I have in mind is: 1. have a startup script which is run f

Re: [julia-users] Macros, functions and module

On Fri, Jan 22, 2016 at 3:43 PM, wrote: > Hi, > > I have this quite weird situation: > > > module Mod > > export @generate_macro_f, @generate_function_g, h > > macro generate_f() > return esc(quote > macro f(name) > return quote > print($name) >

[julia-users] Macros, functions and module

Hi, I have this quite weird situation: module Mod export @generate_macro_f, @generate_function_g, h macro generate_f() return esc(quote macro f(name) return quote print($name) end end end) end macro generate_g() return esc(qu

Re: [julia-users] Re: Optimizing Function Injection in Julia

On Friday, January 22, 2016 12:24:30 PM Cedric St-Jean wrote: > What about creating a parametric type, with one parameter/closed-over > variable? Currently there's no caching of old FA types. Pull requests to FA are welcome ;-), though I wonder if it's worth it given Jeff's work. All FA does is

Re: [julia-users] Re: Optimizing Function Injection in Julia

What about creating a parametric type, with one parameter/closed-over variable? On Friday, January 22, 2016 at 3:20:48 PM UTC-5, Tim Holy wrote: > > On Friday, January 22, 2016 12:03:02 PM Cedric St-Jean wrote: > > It looks like my understanding of FastAnonymous was flawed. Why doesn't > it >

Re: [julia-users] Re: Optimizing Function Injection in Julia

On Friday, January 22, 2016 12:03:02 PM Cedric St-Jean wrote: > It looks like my understanding of FastAnonymous was flawed. Why doesn't it > create the type at macro time, and just instantiate it at runtime, yielding > 1 type / @anon ? Is there any complication that prevents that? Yes: for z in (

Re: [julia-users] Re: Optimizing Function Injection in Julia

On Friday, January 22, 2016 11:28:49 AM Bryan Rivera wrote: > I tried to mitigate that by not timing the code where @anon is called. Is > that still the case given the two snippets? Actually, it looks like you didn't call function1 in both cases. You might want to re-test. You're also using a l

[julia-users] Re: Optimizing Function Injection in Julia

Good to know, this is quite a core issue. I see performant lambda's as a must. Next would be to have C++ like options for how local vars are handled. We should be able to negate some options with intelligent compilation. Those Python vs. Julia benchmarks are about to get a whole lot faster

Re: [julia-users] Re: Optimizing Function Injection in Julia

Tim, Every time you call @anon, it creates a brand-new type (and an instance of > that type) that julia has never seen before. It looks like my understanding of FastAnonymous was flawed. Why doesn't it create the type at macro time, and just instantiate it at runtime, yielding 1 type / @anon

Re: [julia-users] Re: Optimizing Function Injection in Julia

I tried to mitigate that by not timing the code where @anon is called. Is that still the case given the two snippets?

Re: [julia-users] Re: Optimizing Function Injection in Julia

I tried to mitigate that by not timing the code where @anon is called. Is that still the case given the two snippets? On Friday, January 22, 2016 at 2:15:20 PM UTC-5, Tim Holy wrote: > > On Friday, January 22, 2016 10:21:31 AM Bryan Rivera wrote: > > For 1000 elements: > > > > 0.00019s vs 0.0

Re: [julia-users] Re: Optimizing Function Injection in Julia

On Friday, January 22, 2016 10:21:31 AM Bryan Rivera wrote: > For 1000 elements: > > 0.00019s vs 0.035s respectively > > Thanks! Glad it helped. > Is the reason the faster code has more allocations bc it is > inserting vars into the single function? (Opposed to the slower > code already having

Re: [julia-users] Re: Optimizing Function Injection in Julia

It doesn't look like I can edit my posts anymore. Is that normal? I have to cleanup the copied OP.

Re: [julia-users] Re: Optimizing Function Injection in Julia

Yea those were averages of 5 runs each, minus the first for JIT. On Friday, January 22, 2016 at 1:24:00 PM UTC-5, Stefan Karpinski wrote: > > Make sure you time it twice – the faster version may generate more code. > > On Fri, Jan 22, 2016 at 1:21 PM, Bryan Rivera > wrote: > >> dude.. >> >> dictZ

Re: [julia-users] Re: Optimizing Function Injection in Julia

Make sure you time it twice – the faster version may generate more code. On Fri, Jan 22, 2016 at 1:21 PM, Bryan Rivera wrote: > dude.. > > dictZ = Dict{Int, Int}() > > for z = 1:1000 > dictZ[z] = z > end > > function testNoFunCopy() > for z = 1:1000 > function2.z = dictZ[z] > #

Re: [julia-users] Re: Optimizing Function Injection in Julia

dude.. dictZ = Dict{Int, Int}() for z = 1:1000 dictZ[z] = z end function testNoFunCopy() for z = 1:1000 function2.z = dictZ[z] # do whatever with function2, including making a copy end end @code_llvm testNoFunCopy() @code_native testNoFunCopy() @time testNoFunCopy() # Test

[julia-users] Re: Interrupting script gives constant error

Afterall changing readbytes in ~IJulia/src/stdio.jl to read is just worked. Before this, IJulia prompted many warnings about depreciation of readbytes for one cell evaluation and browser was freezing while showing those warnings.

Re: [julia-users] Re: Optimizing Function Injection in Julia

Just use z = 1 function2 = @anon c -> c + z for z = 1:100 function2.z = z # do whatever with function2, including making a copy end --Tim On Friday, January 22, 2016 08:55:25 AM Cedric St-Jean wrote: > (non-mutating) Closures and FastAnonymous work essentially the same way. > They store

[julia-users] Re: Interrupting script gives constant error

Since I am on arm machine I use latest master branch for Arm ABI. Recent update to read* and write* methods has not updated in IJulia. I will just update depreciated methods in IJulia by myself I think.

[julia-users] Re: Optimizing Function Injection in Julia

(non-mutating) Closures and FastAnonymous work essentially the same way. They store the data that is closed over (more or less) and a function pointer. The thing is that there's only one data structure in Julia for all regular anonymous functions, whereas FastAnonymous creates one per @anon sit

Re: [julia-users] Issue with return values from subtypes

On Friday, January 22, 2016 at 1:03:05 AM UTC-5, Mauro wrote: > > you mean why the second errors: > > julia> Irrational > Irrational{sym} > > julia> Irrational{sym} > ERROR: UndefVarError: sym not defined > > ? Irrational is all that is needed. Would this help: > I guess my point is, why

[julia-users] Re: Optimizing Function Injection in Julia

I have to do some investigating here. I thought we could do something like that but wasn't quite sure how it would look. Check this out: This code using FastAnonymous optimizes to the very same code below it where functions have been manually injected: using FastAnonymous function function1

Re: [julia-users] Optimizing Function Injection in Julia

Thanks for the responses guys. @Mauro Hopefully I'm approved soon :) @Viral Makes sense, looks like it works. @Kevin This is the only email I've used to post here. It might be another Bryan.

[julia-users] Re: Interrupting script gives constant error

You are on the potentially unstable dev branch 0.5. Try 0.4 instead. You should probably file this as an issue on Julia's github. On Friday, January 22, 2016 at 7:32:55 AM UTC-5, Lutfullah Tomak wrote: > > I have a julia script that runs IJulia > as > > #!/bin/sh > julia -e "using IJulia; note

Re: [julia-users] Interrupting script gives constant error

Thanks for your answer. I later googled jl_uv_writecb. Basically, it happens because of depreciation warning. And, it also in repl too if a depreciated method is interrupted. Depreciation warning for readall would make IJulia unusable even if I use jupyter.

Re: [julia-users] Interrupting script gives constant error

I tend to just run Jupyter by itself from a command line, not from Julia. Interrupting Julia's notebook() doesn't work reliably. -- mb On Fri, Jan 22, 2016 at 7:32 AM, Lutfullah Tomak wrote: > I have a julia script that runs IJulia > as > > #!/bin/sh > julia -e "using IJulia; notebook()" > > In

Re: [julia-users] immutability, value types and reference types?

What are you trying to accomplish? On Thu, Jan 21, 2016 at 7:00 PM, wrote: > > > On Thursday, January 21, 2016 at 8:48:19 AM UTC-8, Stefan Karpinski wrote: >> >> Semantically all objects are reference types in Julia. It just happens >> that if they're immutable the system is free to copy them or

Re: [julia-users] Ambiguous methods warnings for DataFrames and Images

I'm pretty sure I convinced Jeff that we should change ambiguities, at least between modules, into runtime errors instead of definition-time warnings. So you can expect this to change in the future. Of course, that doesn't help right now. On Thu, Jan 21, 2016 at 6:06 PM, Tim Holy wrote: > There

Re: [julia-users] Julia Dict key in keys() but not in sort(collect(keys()))

The only way this could legitimately happen is if badnum is NaN – otherwise something fishy is going on. If you can provide a reproducing example, that would be helpful. This could also happen if you've changed hashing or equality for numbers. On Fri, Jan 22, 2016 at 2:42 AM, Mauro wrote: > I ca

[julia-users] Re: help with using Julia + Cilk

Thanks for the quick response. Missing python lib does not seem to be problem, because (as I said) PyPlot etc have all been working just fine except when a c module using cilk (such as the hello code above) is called *before* the "using PyPlot". Indeed, if I do using PyPlot include("hello.jl"

[julia-users] Interrupting script gives constant error

I have a julia script that runs IJulia as #!/bin/sh julia -e "using IJulia; notebook()" Interrupting this script gives recurring error reading jl_uv_writecb() ERROR: bad file descripter EBADF and cannot be stopped. I needed to cancel because IJulia freezes giving warning lots of depreciation w

[julia-users] help with using Julia + Cilk

It is unrelated to Julia because you are missing a python library that is needed by PyPlot. From error, I think you're missing numpy.

[julia-users] Re: Using MNE with Julia using PyCall

Thanks, this solved my problem. On Tuesday, 19 January 2016 00:59:33 UTC+1, Steven G. Johnson wrote: > > > > On Monday, January 18, 2016 at 6:03:15 PM UTC-5, Rob wrote: >> >> I am trying to use the python MNE library in Julia. >> >> When I call the python function it returns a `Dict{Any,Any}` inst

Re: [julia-users] Optimizing Function Injection in Julia

I have noticed that I've had to approve more than one post by Bryan. Bryan, have you been using the same email address when you post here? Kevin On Friday, January 22, 2016, Viral Shah wrote: > Only the first post is moderated, to prevent spammers from getting > through. > > -viral > > On Frid

Re: [julia-users] how to i get number of arguments of a function?

> Is it also possible to get a list of names of the variables used in a > function? > > e.g. for > > function f(x,y) > k=0.1 > return x*y+k > end > > I'd like to get a list ["k","x","y"] > > My first thought was to make a method f() that returns this list, but if > its possible to do this

[julia-users] Re: Optimizing Function Injection in Julia

Only the first post is moderated, to prevent spammers from getting through. -viral On Friday, January 22, 2016 at 7:32:50 AM UTC+5:30, Bryan Rivera wrote: > > Guys, it's killing me having to wait hours until my posts are approved. > > (Ability to edit would be nice as well.. Although it looks li

Re: [julia-users] how to i get number of arguments of a function?

Is it also possible to get a list of names of the variables used in a function? e.g. for function f(x,y) k=0.1 return x*y+k end I'd like to get a list ["k","x","y"] My first thought was to make a method f() that returns this list, but if its possible to do this otherwise and more g

[julia-users] help with using Julia + Cilk

Hi all, I have been working on a project that involves a Julia front-end calling a C back-end (it's a neural network simulator, in case you're wondering). Since the C side is essentially a bunch of nested for-loops, I've been experimenting with using Cilk+ (via GCC5) to parallelize things. Lo