[julia-users] Getting variable names of function though the AST?

[Jon Norberg]

I should have added that growthV was a function

Function growthV(s,f)
Return s+f+q
End

An i am looking to get a list containing s, f and q

[julia-users] Getting variable names of function though the AST?

[Jon Norberg]

Hi, I asked in a thread some year ago how to get the parameters and variables 
used in a function. I got some amazing help from the always very helpful 
community (Thanks Mauro and more, 
https://groups.google.com/forum/m/#!search/Jon$20norberg/julia-users/bV4VZxbzZyk).
 However, as already hinted at in that thread, this trick would cease to work 
in 0.5. So my question now is: How can I do this in 0.5?

I have come this far
julia
ast=Base.uncompressed_ast(methods(growthV).mt.defs.func.lambda_template)
```

which gives me the code of the function. previously one could also get at all 
parameters used in the function (se link to above htread) but I am at a loss 
how to get there now in julia 0.5.

Re: [julia-users] Re: Introducing Knet8: beginning deep learning with 100 lines of Julia!

[Jon Norberg]

Ah yes of course, sorry :-) and thanks

Re: [julia-users] Re: Introducing Knet8: beginning deep learning with 100 lines of Julia!

[Jon Norberg]

I get "LoadError: unknown package Knet" when using Pkg.add("Knet"). I am on 0.5.

Very interested in this julia native ML library, thanks for sharing

[julia-users] TensorFlow.jl help

[Jon Norberg]

I have very little experience with tensor flow but am hoping to make a simple 
version of the [Karpathy 
game](http://cs.stanford.edu/people/karpathy/reinforcejs/waterworld.html) 
eventually. However, Already at first attempt I get stuck with the kernel dying 
on me at the last line (julia 0.5 rc 3)

```julia
using TensorFlow
using Distributions
situationDim=10
actionDim=10
n_hidden_1=100
n_hidden_2=100


function weight_variable(shape)
initial = map(Float64, rand(Normal(0, .001), shape...))
return Variable(initial)
end

function bias_variable(shape)
initial = fill(Float64(.1), shape...)
return Variable(initial)
end

session = Session(Graph())

x = placeholder(Float64, shape=[-1, situationDim])
y = placeholder(Float64, shape=[-1, actionDim])
layer1=nn.tanh(x*weight_variable([situationDim, 
n_hidden_1])+bias_variable(n_hidden_1))
layer2=nn.tanh(layer1*weight_variable([n_hidden_1, 
n_hidden_2])+bias_variable(n_hidden_2))
y_out=nn.softmax(layer2*weight_variable([n_hidden_2, 
actionDim])+bias_variable(actionDim))
cross_entropy = -reduce_sum(y.*log(y_out))
train_step = train.minimize(train.AdamOptimizer(1e-4), cross_entropy)
```

The error is:

```
train_step = train.minimize(train.AdamOptimizer(1e-4), cross_entropy)
F ./tensorflow/core/lib/gtl/inlined_vector.h:155] Check failed: i < size() (0 
vs. 0)

signal (6): Abort trap: 6
while loading no file, in expression starting on line 0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 10484772 (Pool: 10481619; Big: 3153); GC: 20
Abort trap: 6
```

Any suggestions would be much appreciated. If anyone has a simple MLP 
reinforcement example implemented and sharable I'd appreciate learning from it.

[julia-users] help with integrating websockets and protobuf

[Jon Norberg]

Anyone have any experiences with protobuf/web Sockets?

[julia-users] help with integrating websockets and protobuf

[Jon Norberg]

just to check that there is no problem in the formats, the following does send 
a ljulia-protobuf formatted message to the html-client but somewhat 
indirectly

msg = read(client)
iob = PipeBuffer();
write(iob,msg)
test = readproto(iob,com())
println("got MESSAGE: $test.id")  #i.e. works!
writeproto(iob,com(id=[1],val=[0.2]))
msg=read(iob)
write(client,msg)#the new value 
turns up at the client, so no issues with the formatting, just to get the web 
socket to talk via readproto and writeproto

[julia-users] help with integrating websockets and protobuf

[Jon Norberg]

Dear julia community. 

I can't figure out whats wrong here. I have a web socket implementation that 
works just fine without using protobuf on the julsa/websocket side (in fact, 
the html-client does send a protobuf array which just gets sent back without 
unpacking and packing on the julia web socket side, i.e. echo server.

the relevant lines in the web socket server are:

msg = read(client) # This receives messages
write(client,msg)   # This sends it back

I can then unpack it on the html-client side again just fine.

I have defined a protobuf type as 

type com
id::Array{Int32,1}
val::Array{Float32,1}
com(; kwargs...) = (o=new(); fillunset(o); isempty(kwargs) || 
ProtoBuf._protobuild(o, kwargs); o)
end #type com
hash(v::com) = ProtoBuf.protohash(v)
isequal(v1::com, v2::com) = ProtoBuf.protoisequal(v1, v2)
==(v1::com, v2::com) = ProtoBuf.protoeq(v1, v2)

and the following works as it should:

iob = PipeBuffer();
writeproto(iob,com(id=[1],val=[0.2]))
msg = readproto(iob,com())
println(msg)

But when I change the relevant lines in the web socket code on the server side 
to:

msg = readproto(client.socket,com())
writeproto(client.socket,com(id=[1],val=[0.2]))

it does not work.

 I did figure out that client is a composite type and that client.socket is of 
type TCPSocket which I assume is the right variable to pass to the protobuf 
functions.

Can anyone spot what I am missing? 

the protofile is:

message com {
  repeated int32 id = 1;
  repeated float val =2;
}

[julia-users] Help on channels

[Jon Norberg]

A=RemoteRef(()->Channel{Int64}(10), 1)

Works but 

Put!(A,1,1) gives

LoadError: MethodError: `put!` has no method matching put!(::Channel{Int64}, 
::Int64, ::Int64)
Closest candidates are:
  put!(::Channel{T}, ::Any)
  put!(!Matched::Base.RemoteValue, ::Any...)
  put!(!Matched::RemoteRef{T<:AbstractChannel}, ::Any...)
while loading In[15], in expression starting on line 1

 in put! at multi.jl:799
 in put_ref at multi.jl:800
 in call_on_owner at multi.jl:777
 in put! at multi.jl:801

I would also like to get the following to work also but get same errors:

type test
a::Float64
end

B=RemoteRef(()->Channel{test}(10), 1)
Put!(B,test(0.1),1)

And

B=RemoteRef(()->Channel{Array{Float64,1}}(10), 1)
put!(B,rand(10),1)

Any advice appreciated

[julia-users] dictchannel.jl example

[Jon Norberg]

Strangely though, if I modify the put! function as:

function put!(D::DictChannel, k, v)
D.d[k] = v
notify(D.cond_take)
Println(keys(D.d))
D
end

And run this I get 

Any[2,3,1]

(I added keps in that order), i.e. This works

So it seemed the function definition of keys does not work in the same way as 
for put! For some reason and I can't figure out why

[julia-users] dictchannel.jl example

[Jon Norberg]

Strangely makning a function

function keys(D::RemoteRef{DictChannel})
keys(D.d)
end

Gives also error:

LoadError: type RemoteRef has no field d

Also, for the function put! That does work the function looks like:

function put!(D::DictChannel, k, v)
D.d[k] = v
notify(D.cond_take)
D
end

It just makes no sense to me way keys(D.d) Should not work in the same way. 

[julia-users] dictchannel.jl example

[Jon Norberg]

the dictchannel is in the julia package as an example of Channels

https://github.com/JuliaLang/julia/blob/master/examples/dictchannel.jl

[julia-users] dictchannel.jl example

[Jon Norberg]

I have great use of the dictchannel.jl example. I wanted to add the method keys 
to it so I thought that I simply add the function:

function keys(D::DictChannel)
keys(D.d)
end

as well s add the function name in the import (I tried also without adding the 
"keys") :

import Base: put!, wait, isready, take!, fetch, keys 

to the file, I then include("dictchannel.jl").

All functions e.g. put!() and fetch() work as should but for my new function I 
get:

LoadError: MethodError: `keys` has no method matching 
keys(::RemoteRef{DictChannel})

Not sure what I am doing wrong so any help appreciated

Re: [julia-users] async web socket read & write?

[Jon Norberg]

Thanks, that works perfectly

[julia-users] async web socket read & write?

[Jon Norberg]

I am trying to use a websocket to provide online input to a simulation that 
runs on another process and update back with a given frequency. The examples I 
have found all wait for a read event and then immediately write back. I would 
like to have the ability to have separate read and write loops (or any other 
method to achieve the same thing), essentially running these loops or 
equivalent simultaneously (preferably even possibly on separate processes):

while true
  msg=read(client)
  put!(msg into channel)
end

while true
  take!(msg from channel)
  write(client, msg)
  sleep(0.1)
end

 I want the web socket to communicate with the simulation via a channel but I 
am not sure if this is the best way. The simulation runs in real time and will 
only be affected by user input if any info comes through the web socket but the 
user can see the simulation in near realtime with or without input.

Any help much appreciated!

[julia-users] tensor flow question

[Jon Norberg]

I am following the examples in tensor flow.jl I get all the examples to work 
nicely. however, when I want to change an activation function to tan or sigmoid 
I get an error. I suspect I am doing something wrong rather than a problem with 
the package so I ask here...

If I do this:

using TensorFlow
using TensorFlow.Train
using TensorFlow.InputData
import TensorFlow: DT_FLOAT32
import TensorFlow.API: relu, tanh, sigmoid,softmax_cross_entropy_with_logits, 
AdamOptimizer, arg_max, equal, cast
methods(relu)

I get:

2 methods for generic function relu:
relu(features::Union{TensorFlow.CoreTypes.AbstractTensor,Void}) at 
/Users/Raukhur/.julia/v0.4/TensorFlow/src/API/TfNn.jl:1006
relu(features::Union{TensorFlow.CoreTypes.AbstractTensor,Void}, 
name::Union{AbstractString,Void}) at 
/Users/Raukhur/.julia/v0.4/TensorFlow/src/API/TfNn.jl:1006

but if I do this

methods(tanh)

I get 13 methods for tanh but none for tensor flow related types and for 

methods(sigmoid) I get

LoadError: UndefVarError: sigmoid not defined
while loading In[42], in expression starting on line 6

So it seems relu gets imported but not tanh or sigmoid. also 

methods(TensorFlow.API.tanh) nor
methods(TensorFlow.tanh) works

Its probably something simple I have overlooked

Re: [julia-users] Julia text editor on iPad?

[Jon Norberg]

I mentioned a few issues on the Jupyter group regarding iPad usability, such as 
kinetic scrolling and some issues mentioned above. It's working ok, but a few 
things are still awkward for ipad

[julia-users] Julia text editor on iPad?

[Jon Norberg]

I've found that the best solution for me was to install Jupyter server on my 
desktop machine and just run Julia as well as write code by opening a 
webbrowser on my iPad Connection to it. Jupyter has a text-only editor that 
supports Julia syntax in addition to the notebooks. I do miss atom though...

[julia-users] Remove Gadfly gridlines?

[Jon Norberg]

I have searched and tried a few things but cannot remove the background grids 
in Gadfly. Its probably simple and I am missing something obvious...Any 
suggestions would be appreciated.

layer(x=E,y=wetness(E,10.0), Geom.line,Theme(default_color=a[1], 
line_width=2pt, grid_color=colorant"white")

also tried grid_line_width=0pt

but they still show up ( I save it as SVG, but also output in jupiter shows 
them)

[julia-users] recommended graphics packages for multiple curves on a single canvas

[Jon Norberg]

using Gadfly

L=Layer[]
push!(L,layer(x=1:10,y=rand(10),Geom.line)[])
push!(L,layer(x=1:10,y=rand(10),Geom.line)[])
push!(L,layer(x=1:10,y=rand(10),Geom.line)[])
plot(L)

Only awkward thing is the empty square bracket for some reason is needed

Styling colours using Themes (see gadfly documentation)

cheers

Re: [julia-users] how to i get number of arguments of a function?

[Jon Norberg]

Wow, thanks a lot, That one I would never had a chance to figure out. 

Re: [julia-users] how to i get number of arguments of a function?

[Jon Norberg]

Is it also possible to get a list of names of the variables used in a 
function? 

e.g.  for 

function f(x,y)
k=0.1
return x*y+k
end

I'd like to get a list ["k","x","y"]

My first thought was to make a method f() that returns this list, but if 
its possible to do this otherwise and more generally that would be very 
useful 

On Thursday, January 21, 2016 at 6:47:38 PM UTC+1, ami...@gmail.com wrote:
>
> Great, thanks a lot.
> In fact, I also need to evaluate the number of arguments of an anonymous 
> function:
>
> julia> function factory(y)
>  return x -> x + y
>end
> factory (generic function with 1 method)
>
> julia> type Foo
>  f::Function
>end
>
> julia> foo = Foo(factory(2))
> Foo((anonymous function))
>
> julia> methods(foo.f)
> ERROR: ArgumentError: argument is not a generic function
>  in methods at reflection.jl:180
>
>
> Any way to do that..?
>
>

Re: [julia-users] Re: Linking values in composite type and array?

[Jon Norberg]

Many thanks all, I learned a lot again from this great community.

Jon

[julia-users] Linking values in composite type and array?

[Jon Norberg]

I would like to create a composite type and then also create an array that 
has values from this type by reference. The behaviour I am looking for is 
like this:

type c
  a::Float64
  b::Float64
end

x=c(0.1,0.2)
y=c(0.3,0.4)

z=[x.a,x.b,y.a,y.b]

show(z)
[0.1,0.2,0.3,0.4]

x.a=0.5

show(z)
[0.5,0.2,0.3,0.4]

z[4]=0.6

show(y.b)
0.6

Is this possible?

Thanks, Jon

[julia-users] Re: make composite type from array of strings?

[Jon Norberg]

I really like the compactness of composite types, I think I remember 
testing speed when having parameters in composite type vs dictionary and 
composite type was much faster, but maybe I did it wrong. 

Anyhow, I made this function and it seems to work, I pass an array of 
symbols in s thats why i use string(s).

function makeComposite(s)
  q="type param "
  for i=1:length(s)
  q=q*string(s[i])*"; "
  end
  q=q*"end"
  eval(parse(q))
end

Is there some problem with creating a composite type like this dynamically 
since you suggest using a dictionary instead?

On Thursday, December 17, 2015 at 4:28:32 AM UTC+1, Steven G. Johnson wrote:
>
> On Wednesday, December 16, 2015 at 10:11:20 AM UTC-5, Jon Norberg wrote:
>>
>> Is it possible to dynamically create a composite type if I have a list of 
>> strings for the fieldnames?
>
>
> While this is possible, you should probably consider using a dictionary 
> type rather than creating a composite type on the fly. 
>

[julia-users] Re: make composite type from array of strings?

[Jon Norberg]

I really like the compactness of composite types, I think I remember 
testing speed when having parameters in composite type vs dictionary and 
composite type was much faster, but maybe I did it wrong. 

Anyhow, I made this function and it seems to work.

function makeComposite(s)
  q="type param "
  for i=1:length(s)
  q=q*string(s[i])*"; "
  end
  q=q*"end"
  eval(parse(q))
end

Is there some problem with creating a composite type like this dynamically 
since you suggest using a dictionary instead?


On Thursday, December 17, 2015 at 4:28:32 AM UTC+1, Steven G. Johnson wrote:
>
> On Wednesday, December 16, 2015 at 10:11:20 AM UTC-5, Jon Norberg wrote:
>>
>> Is it possible to dynamically create a composite type if I have a list of 
>> strings for the fieldnames?
>
>
> While this is possible, you should probably consider using a dictionary 
> type rather than creating a composite type on the fly. 
>

Re: [julia-users] make composite type from array of strings?

[Jon Norberg]

Thanks for that pointer! Not a computer scientist nor even a very good coder, 
but I just learned a bit more how julia works internally.

I did the xdump and saw the AST (learned what that is too)

xdump(:(type T X; Y; Z; end)) gives 

Expr 
  head: Symbol type
  args: Array(Any,(3,))
1: Bool true
2: Symbol T
3: Expr 
  head: Symbol block
  args: Array(Any,(6,))
1: LineNumberNode 
  file: Symbol In[7]
  line: Int64 1
2: Symbol X
3: LineNumberNode 
  file: Symbol In[7]
  line: Int64 1
4: Symbol Y
5: LineNumberNode 
  file: Symbol In[7]
  line: Int64 1
6: Symbol Z
  typ: Any::DataType  <: Any
  typ: Any::DataType  <: Any

for fun I tried eval(:(type T X; Y; Z; end)) which results in T becoming 
available

typeof(T) -> DataType

and 

xdump(T) gives:

T::DataType  <: Any
  X::Any::DataType  <: Any
  Y::Any::DataType  <: Any
  Z::Any::DataType  <: Any



so something happened but not what I need to accomplish. Any more pointers 
would be welcome, but I'll dig into the documentation too

Jon

[julia-users] make composite type from array of strings?

[Jon Norberg]

Is it possible to dynamically create a composite type if I have a list of 
strings for the fieldnames?

Is it possible to do this in a function?

A=[]
push!(A,"X")
push!(A,"Y")
push!(A,"Z")

function defineCompositeType(A)
   magic?
end

regards, Jon

[julia-users] jupiter server issue on OSX

[Jon Norberg]

Maybe not strictly a julia question but maybe someone can help...

I just got brand new iMac. I set up everything for jupiter, julia and 
ijulia. can launch notebook just fine. then I change the jupiter config to 
serve the notebook via http. On the computer itself i can open using the ip 
address x.x.x.x:portnr and it opens fine asking for password and julia 
notebook works. But I cannot open it from any other computer. I tried using 
my iPhone as a internet gate in case my university blocked a port, but same 
issue. The port doesn't seem to be open for outside. On a note, I did the 
same on my old iMac and had the same problem. I thought I wait for a new 
computer in case I had screwed anything up on the old one, but alas, same 
problem. Anyone had similar issue and solved it?

set up: 

Xcode (App Store)

Anaconda (graphical installer)

Julia (downloaded 0.4.1 and symlink)

IJulia (Pkg.add)

Jupyter (conda install)


generated jupiter config file


added:


c=get_config()

c.NotebookApp.port=

c.NotebookApp.open_browser=False

c.NotebookApp.password = u'sha1:xyz'

c.NotebookApp.ip = '*'


I have also tried with the ssh certfile and keyhole setup, but no change.

[julia-users] Re: indexing with non Integer Reals is deprecated

[Jon Norberg]

Thanks Eric!

On Wednesday, November 18, 2015 at 10:08:08 AM UTC+1, Eric Forgy wrote:
>
> Hi Jon,
>
> Have a look at this comment above: 
> https://groups.google.com/d/msg/julia-users/sEUvnjvtryI/g_vbGQiDBgAJ
>
> You can try:
>
> D[i+1-div(d,2):j-3+q*5]
>
> or the unicode equivalent:
>
> D[i+1-d÷2:j-3+q*5]
>
>
>
> On Wednesday, November 18, 2015 at 4:53:10 PM UTC+8, Jon Norberg wrote:
>>
>> On a very practical note:
>>
>> I need to to do inline calculations of indexes such as
>>
>> D[i+1-d/2:j-3+q*5]
>>
>> where i,d,j and q are Int64 and d is a multiple of 2 i.e. d/2 should 
>> always be integer
>>
>> What is the most efficient way to use an expression as above but to avoid 
>> the Warning: indexing with non Integer... ?
>>
>> Thanks, Jon
>>
>

[julia-users] Re: indexing with non Integer Reals is deprecated

[Jon Norberg]

On a very practical note:

I need to to do inline calculations of indexes such as

D[i+1-d/2:j-3+q*5]

where i,d,j and q are Int64 and d is a multiple of 2 i.e. d/2 should always 
be integer

What is the most efficient way to use an expression as above but to avoid 
the Warning: indexing with non Integer... ?

Thanks, Jon

[julia-users] Path to module in Atom IDE

[Jon Norberg]

I didn't get the above to work, including trying to set path with 
cd("/Users/raukhur/Documents/Github/Landscapes")

However, for some reason the following did work:

push!(LOAD_PATH, "/Users/raukhur/Documents/Github/Landscapes")

I am fine using this, but if anyone knows what could be the cause for the 
suggestions given above not to work it would be useful to know and help others.

Thanks!

Jon

[julia-users] Path to module in Atom IDE

[Jon Norberg]

Hm, command palette just says "no match found" and even if I type it nothing 
changes.

Also, 1) if I have a file outside the module folder, can I load it somehow by 
providing a path? and 2) I quite like hydrogen, does it work the same way to 
load a module with Julia Client:  etc?

[julia-users] Path to module in Atom IDE

[Jon Norberg]

Dear Julia users,

I just switched to atom IDE from Juno. I am puzzled by one thing. 

As in Juno, I first cd(myDir) to move to the directory where I have the files 
for the module I am developing. In Juno I could then just type using myModule 
to load it but in Atom I now only get


LoadError: ArgumentError: myModule not found in path
while loading In[5], in expression starting on line 1

in require at 
/Applications/Julia-0.4.app/Contents/Resources/julia/lib/julia/sys.dylibLoadError
LoadError: ArgumentError: myModule not found in path
while loading In[5], in expression starting on line 1

typing pwd() I get the right path and typing reader() I get a list of files 
that includes the files I am expecting.

Anyone have any idea what I am doing wrong?

Thanks, Jon

Re: [julia-users] A grateful scientist

[Jon Norberg]

Utterly seconding that. Amazing community and beautiful language. 

Thanks all!

Jon Norberg 

[julia-users] Re: poisson distribution?

[Jon Norberg]

Sorry, thought I could delete threadso deleted first postmy bad

[julia-users] Re: poisson distribution?

[Jon Norberg]

I think I had a problem in how I used the resulting data that slowed it 
down. 

Solved, Thanks

[julia-users] poisson distribution?

[Jon Norberg]

I need to get random variables from poisson distributions with different 
lambda

this is the only way I got it to work at the moment BUT its very slow!

using Distributions
lambda=linspace(0.1,2,100)
out=zeros(Int64,100)
for i=1:length(lambda)
  P=Poisson(lambda[i])
  out[i]=rand(P)
end

Anyone have suggestions for speed improvements?

Thanks

Re: [julia-users] type of single values AND array?

[Jon Norberg]

Many thanks! works

Re: [julia-users] type of single values AND array?

[Jon Norberg]

type x
  value::Union{Array{Float64},Float64}
end

gives me error:

type: instantiate_type: expected TypeConstructor, got function

Re: [julia-users] type of single values AND array?

[Jon Norberg]

not sure what leaf type is?

I actually need more fields in the type so I do need it as some field

type x
value::Array{Float64}
   otherfield...
end

Re: [julia-users] type of single values AND array?

[Jon Norberg]

leaftype?

On Monday, September 21, 2015 at 2:41:02 PM UTC+2, Yichao Yu wrote:
>
> On Mon, Sep 21, 2015 at 8:37 AM, Jon Norberg  > wrote: 
> > quick question: How do I define a type most effective so that I can give 
> it both single values, vectors and Arrays as variables? 
> > 
> > I now have: 
> > 
> > type x 
> > value::Array{Float64} 
> > end 
> > 
> > I can do x(rand(3)) as well as x(rand(3,3)) but not x(0.1) 
>
> Depends on what you need. If you don't need the field to be a leaftype 
> (as in your code above), Union should work fine. 
>
> > 
> > the answer is probably in the forums but I couldn't construct a good 
> search to find it :-/ 
> > 
> > Thanks 
>

[julia-users] type of single values AND array?

[Jon Norberg]

quick question: How do I define a type most effective so that I can give it 
both single values, vectors and Arrays as variables?

I now have:

type x
value::Array{Float64}
end

I can do x(rand(3)) as well as x(rand(3,3)) but not x(0.1)

the answer is probably in the forums but I couldn't construct a good search to 
find it :-/

Thanks

[julia-users] Re: ANN: Immerse package, and more videos on interactive plotting

[Jon Norberg]

What does the ANN: in the title mean?

[julia-users] Re: beginning deep learning with 500 lines of Julia

[Jon Norberg]

Remarkably good idea, thanks for sharing. I like the clean readable code 
but efficient if needed idea. It would be amazing if the recent 
developments could be added as you hint at for people to learn.

many thanks

On Saturday, February 28, 2015 at 4:19:18 PM UTC+1, Deniz Yuret wrote:
>
> KUnet.jl  (beginning deep 
> learning with 500 lines of Julia 
> 
> ) is out with its alpha release.  Only the basic functionality is in 
> place (i.e. backprop with relu, softmax, sgd, momentum, nesterov, adagrad, 
> dropout, l1-l2 regularization etc.) but the GPU functionality is in, its 
> speed is competitive with Caffe , and I 
> think convolutional and recurrent nets can be added without too much 
> effort.  I wrote a blog post 
> 
>  about 
> the code structure and there is some basic documentation 
> .  You can 
> send me suggestions for improvement (both in coding style and new 
> functionality) using comments 
>  
> to 
> the blog post 
> ,
>  
> or using issues  or pull 
> requests  on GitHub 
> .
>
> I tried to make the code (cpu/gpu) generic and high level.  Getting the 
> same code working on the GPU and the CPU in Julia proved to be a bit 
> challenging and showed that both a more standard treatment of CPU and GPU 
> arrays, and a standard syntax for in-place operations would be welcome 
> additions to the language.  I'd like to thank Tim Holy (CUDArt), Nick 
> Henderson (CUBLAS), and Simon Byrne (InplaceOps) for their generous help.
>
> best,
> deniz
>

[julia-users] Re: parallel loop with mutable types

[Jon Norberg]

also this works but does not change values in b

@parallel for i=1:20 b[i]=k[i].r*k[i].K end

I tried making b=DArray{Float64,1} or b=dones(20,1) but still values in b 
are not updated

do I need to use spawn/fetch or pmap or something like this?

Sorry, not fluent in parallel programming yet, but I am trying to make my 
simple code scalable from start

//J

[julia-users] Re: parallel loop with mutable types

[Jon Norberg]

Great, solve first problem, thanks. 

using DArray though gives

julia> k=DArray(parms,20)
exception on 2: ERROR: no method parms((UnitRange{Int64},))
 in anonymous at multi.jl:840
 in run_work_thunk at multi.jl:613
 in run_work_thunk at multi.jl:622
 in anonymous at task.jl:6
ERROR: assertion failed
 in DArray at darray.jl:18
 in DArray at darray.jl:39
 in DArray at darray.jl:48

[julia-users] Re: An appreciation of two contributors among many

[Jon Norberg]

I'll second that, great community and some very very helpful people that 
put a lot of effort into this.

Thanks

[julia-users] parallel loop with mutable types

[Jon Norberg]

I have: 

type parms
  r::Float64
  K::Float64
end

k=Array(parms,20)

for i =1:20 k[i]=parms(1.1,2.2) end

addprocs(1)

nprocs() -> 2

@parallel for i=1:20 k[i].r=2.0 end

gives error:

julia> @parallel for i=1:20 k[i].r=2.0 end
fatal error on 2:
julia> ERROR: parms not defined
 in deserialize at serialize.jl:470
 in handle_deserialize at serialize.jl:327
 in deserialize at serialize.jl:398
 in handle_deserialize at serialize.jl:327
 in deserialize at serialize.jl:310
 in anonymous at serialize.jl:330
 in ntuple at tuple.jl:30
 in deserialize_tuple at serialize.jl:330
 in handle_deserialize at serialize.jl:322
 in deserialize at serialize.jl:368
 in handle_deserialize at serialize.jl:327
 in deserialize at serialize.jl:310
 in anonymous at serialize.jl:330
 in ntuple at tuple.jl:30
 in deserialize_tuple at serialize.jl:330
 in handle_deserialize at serialize.jl:322
 in deserialize at serialize.jl:368
 in handle_deserialize at serialize.jl:327
 in anonymous at task.jl:835
Worker 2 terminated.

Do I need to share the k array or somehow let the parallel code know how to 
handle an array of type parms?

Thanks for any help here!

Jon

Re: [julia-users] Help in Array of mutable types

[Jon Norberg]

Thanks!

On Monday, June 16, 2014 8:26:11 AM UTC+2, John Myles White wrote:
>
> This line
>
> p=Vector{parms}
>
> seems like it’s not what you want. You want to create an object of that 
> type, not the type, I imagine.
>
> To do that, try
>
> parms[]
>
> or
>
> Array(parms, 0)
>
>  — Jon
>
> On Jun 15, 2014, at 11:24 PM, Jon Norberg  > wrote:
>
> p=Vector{parms}
>
>
>

[julia-users] Help in Array of mutable types

[Jon Norberg]

Any hint on how to do this?:

type parms 
 r::Vector{Float64}
K::Vector{Float64}
end

r=rand(N)*(0.05-0.001)+0.001
K=rand(N)*100+1

p=Vector{parms}

p[1]=parms(r,K)
p[2]=parms(r,K)
etc

I get error:


no method setindex!(Type{Array{parms,1}}, parms, Int64)
while loading In[36], in expression starting on line 1


Many thanks

[julia-users] Re: packaging parameters for functions revisited

[Jon Norberg]

I added the keyword version (not sure it is as intended though)

function with_keyword(x::Float64=1.1; 
a::Float64=1.0,b::Float64=2.0,c::Float64=3.0,d::Float64=4.0,e::Float64=5.0)
dx= a*b*c*d*e::Float64
end

Results now are: (slower due to being on my macbook and couldn't change in 
the ijulia book yet):

@time for i in 1:N with_keyword(1.1) end
@time for i in 1:N Brute() end
@time for i in 1:N assignInF() end
@time for i in 1:N assignFromArray(f) end
@time for i in 1:N assignFromArray(ff) end
@time for i in 1:N packUnpack(p) end
@time for i in 1:N sepParams(a,b,c,d,e) end
@time for i in 1:N globadAssign() end
@time for i in 1:N indexedArray(f) end
@time for i in 1:N immutableType(s) end
@time for i in 1:N indexedArray(ff) end

elapsed time: 1.859415076 seconds (639983688 bytes allocated)
elapsed time: 1.824780742 seconds (639983688 bytes allocated)
elapsed time: 1.815331646 seconds (639983688 bytes allocated)
elapsed time: 2.260640645 seconds (640031792 bytes allocated)
elapsed time: 2.491979295 seconds (800031648 bytes allocated)
elapsed time: 2.498344644 seconds (800030640 bytes allocated)
elapsed time: 2.478566736 seconds (81228 bytes allocated)
elapsed time: 2.605157314 seconds (799983688 bytes allocated)
elapsed time: 2.365273473 seconds (640021904 bytes allocated)
elapsed time: 2.58554929 seconds (84984 bytes allocated)
elapsed time: 2.520120133 seconds (800021760 bytes allocated)


Thus, keyword goes close to top, but not sure how to pass a,b,c,d,e to the 
function using keywords

Re: [julia-users] packaging parameters for functions revisited

[Jon Norberg]

Do you mean using a dict to pack/unpack them?

function f(x; args...)
###
end

where args is a dict?

[julia-users] packaging parameters for functions revisited

[Jon Norberg]

There have been several posts about this, so I tried to compile what I 
could find to compare speed and pretty coding:

http://nbviewer.ipython.org/urls/dl.dropboxusercontent.com/u/38371278/Function%20parms%20passing%20speed%20test.ipynb

Best speed is assigning values or variables inside the function
Second best is reassigning parameters from array for each parameter. The 
@pack/@unpack macro also is pretty fast and much prettier
Third best is separate parameters and global assignment (longer function 
call code if many parameters or/and just no no to use global?)
Slowest is using indexed array as parameters in function (but this is ugly 
to read) and immutable types

So I am wondering, did I miss any that improve speed/prettiness?

Best, Jon

[julia-users] function parameter packaging revisited

[Jon Norberg]

Hi, I am trying to find out the best (fast & pretty) way to pass a bunch of 
parameters to a function. I checked the group posts and compiled some I 
found

const pnames = [:a, :b, :c, :d, :e]
macro unpack(ex)
  Expr(:block, [:($(pnames[i]) = $ex[$i]) for i = 1:length(pnames) ]...) |> 
esc
end

macro pack(ex)
N = length(pnames)
Expr(:block, [
:($ex = zeros($N,1)),
 [:( $ex[$i] = $(pnames[i]) ) for i = 1:length(pnames) ]
]...) |> esc
end

function test1(a,b,c,d,e)
dx= a*b*c*d*e
return dx
end;

function test2(f)
a=f[1]
b=f[2]
c=f[3]
d=f[4]
e=f[5]
dx= a*b*c*d*e
return dx
end;

function test3(f)
dx= f[1]*f[2]*f[3]*f[4]*f[5]
return dx
end;

function test4()
a=1
b=2
c=3
d=4
e=5
dx= a*b*c*d*e
return dx
end;

function test5()
dx= 1*2*3*4*5
return dx
end;

function test6(p)
@unpack p
 dx= a*b*c*d*e
return dx
end;

function test7()
 dx= aa*bb*cc*dd*ee
return dx
end;

N=1000
a=1; b=2; c=3; d=4; e=5;
f=[]
f=[1,2,3,4,5];
ff=[a,b,c,d,e];
@pack p
immutable Var
a::Float64
b::Float64
c::Float64
d::Float64
e::Float64
end
s=Var(1.0,2.0,3.0,4.0,5.0)
global aa, bb, cc, dd, ee;
aa=1; bb=2; cc=3; dd=4; ee=5

@time for i in 1:N test1(a,b,c,d,e) end
@time for i in 1:N test2(f) end
@time for i in 1:N test3(f) end
@time for i in 1:N test2(ff) end
@time for i in 1:N test3(ff) end
@time for i in 1:N test4() end
@time for i in 1:N test5() end
@time for i in 1:N test6(p) end
@time for i in 1:N test7() end

Resulting in:

elapsed time: 1.885115751 seconds (639983704 bytes allocated)
elapsed time: 1.707763955 seconds (639983704 bytes allocated)
elapsed time: 2.829910887 seconds (639983704 bytes allocated)
elapsed time: 1.717995529 seconds (639983704 bytes allocated)
elapsed time: 2.803335021 seconds (639983688 bytes allocated)
elapsed time: 1.483327602 seconds (639983688 bytes allocated)
elapsed time: 1.468555675 seconds (639983688 bytes allocated)
elapsed time: 1.860042772 seconds (799983688 bytes allocated)
elapsed time: 1.904847681 seconds (639983704 bytes allocated)


Defining the value in the function seems fastest test5() and test4(), followed 
by reassigning within the function test2(). Next is passing all values 
test1(a,b,c,d,e) followed by using @pack macro aand global declaration (Do I 
understand correctly that @pack/@unpack rewrites the code before compiling?). 
Slowest is using indexing in an array (ugly too).


So my question is: any other ways that improve readability/speed?


Thanks

[julia-users] Re: suggestion of OSX julia/ijulia setup

[Jon Norberg]

Hi, and many thanks, very useful info here. How would I link this to julia 
studio? seems julia studio has hardcoded to look for bin/julia-basic. Any 
one been able to still point it to the right file successfully?

On Wednesday, May 21, 2014 10:41:51 PM UTC+2, Adam Smith wrote:
>
> I'm on a 2012 Macbook Air and I run julia v0.3 nightly builds. Building 
> Julia myself on this machine takes forever, so I just download the nightly 
> build from http://status.julialang.org/download/osx10.7+ about once a 
> week.
>
> I then drag it into /Applications and rename it from its commit hash 
> version to "Julia-0.3.0-prerelease" (and move/delete the prior build). With 
> this line in my ~/.profile, the new binary is available in my path so I can 
> just run "julia" in any terminal:
> export PATH=/Applications/Julia-0.3.0-prerelease.app/Contents/Resources/
> julia/bin:$PATH
>
> This is a pretty quick process that does not require crippling my laptop 
> for 10+ minutes while it builds.
>
> On Friday, May 16, 2014 8:32:38 AM UTC-4, Jon Norberg wrote:
>>
>> Hi all,
>>
>> I have been using julia and ijulia for a while and everything worked 
>> fine. over time I get more and more issues, trying to upgrade/reinstall etc 
>> and now I can't get it to work at all anymore. As I intend to reinstall osx 
>> anyway, I was wondering if you good people have any good setup for 
>> python/ijulia/julia/juliastudio that is simple to maintain over time. I 
>> want to use one version of julia so somehow let julia-studio use that 
>> (annoying that they hardcode the julia path to julia-basic...)
>>
>> I was ambitious to try to keep a --HEAD version of julia but maybe I will 
>> have to settle for a latest stable pre-release version to avoid trouble.
>>
>> But I also find that sometimes there are issues with the dependencies and 
>> libraries, how do you keep those right?
>>
>> So how do you people keep julia smoothly updating and working. Are you 
>> using brew, anaconda, enthought, for the python part, are you using xcode 
>> compiler or others, are you building julia yourself...
>>
>> I'd appreciate any help in setting this up for the longer run than I have 
>> been able to keep it working so far.
>>
>> Many thanks
>>
>>
>>

Re: [julia-users] suggestion of OSX julia/ijulia setup

[Jon Norberg]

Also, I sometimes seem to get issues with what libraries are being used. I 
am not very good at this but I understand there are different compiler 
libraries and native osx ones. How do you guys handle this? or is it not an 
issue using brew?

On Sunday, May 18, 2014 6:39:38 PM UTC+2, Jon Norberg wrote:
>
> Many thanks Cameron, I'll try that setup. Did I understand that you use 
> brew to compile julia?
>
> On Friday, May 16, 2014 4:21:19 PM UTC+2, Ethan Anderes wrote:
>>
>> +1 for Cameron. I use the same workflow. 
>
>

Re: [julia-users] suggestion of OSX julia/ijulia setup

[Jon Norberg]

Many thanks Cameron, I'll try that setup. Did I understand that you use 
brew to compile julia?

On Friday, May 16, 2014 4:21:19 PM UTC+2, Ethan Anderes wrote:
>
> +1 for Cameron. I use the same workflow. 

[julia-users] suggestion of OSX julia/ijulia setup

[Jon Norberg]

Hi all,

I have been using julia and ijulia for a while and everything worked fine. 
over time I get more and more issues, trying to upgrade/reinstall etc and 
now I can't get it to work at all anymore. As I intend to reinstall osx 
anyway, I was wondering if you good people have any good setup for 
python/ijulia/julia/juliastudio that is simple to maintain over time. I 
want to use one version of julia so somehow let julia-studio use that 
(annoying that they hardcode the julia path to julia-basic...)

I was ambitious to try to keep a --HEAD version of julia but maybe I will 
have to settle for a latest stable pre-release version to avoid trouble.

But I also find that sometimes there are issues with the dependencies and 
libraries, how do you keep those right?

So how do you people keep julia smoothly updating and working. Are you 
using brew, anaconda, enthought, for the python part, are you using xcode 
compiler or others, are you building julia yourself...

I'd appreciate any help in setting this up for the longer run than I have 
been able to keep it working so far.

Many thanks

[julia-users] Re: inserting javascript driven graphs in ijulia

[Jon Norberg]

I Will check That out, thanks

[julia-users] inserting javascript driven graphs in ijulia

[Jon Norberg]

What is the best way to display graphics driven by javascript in IJulia?

For example these two:

http://rectangleworld.com/demos/ColorBoids/ColorBoids01.html

http://bl.ocks.org/mbostock/raw/4636377/

I tried from what I could find googling i.e. 

readchomp html 

and displaying and trying with 

type HTML
   s::String
end
import Base.writemime
writemime(io::IO, ::@MIME("text/html"), x::HTML) = print(io, x.s)

the but nothing worked. 

Any suggestions are appreciated

[julia-users] ijulia problem

[Jon Norberg]

Hi, anyone know how to help me with getting ijulia to run again (using 
mavericks osx). I have clean install latest julia. I added a soft link so 

captiveportal-49-129:~ jon$ which julia
/usr/local/bin/julia

and typing julia starts julia as expected from anywhere

Starting python wight he julia profile works and opens a webpage with the 
files. 

captiveportal-49-129:~ jon$ ipython notebook --profile julia
2014-02-28 17:15:39.482 [NotebookApp] Using existing profile dir: 
u'/Users/jon/.ipython/profile_julia'
2014-02-28 17:15:39.488 [NotebookApp] Using MathJax from CDN: 
http://cdn.mathjax.org/mathjax/latest/MathJax.js
2014-02-28 17:15:39.498 [NotebookApp] Serving notebooks from local 
directory: /Users/jon
2014-02-28 17:15:39.498 [NotebookApp] The IPython Notebook is running at: 
http://127.0.0.1:8998/
2014-02-28 17:15:39.498 [NotebookApp] Use Control-C to stop this server and 
shut down all kernels (twice to skip confirmation).

But opening one of the ijulia files opened the page but the terminal gives:

2014-02-28 17:15:48.342 [tornado.application] ERROR | Uncaught exception 
POST /kernels?notebook=74403c31-b655-4406-b36c-6c5f19fc03ed (127.0.0.1)
HTTPRequest(protocol='http', host='127.0.0.1:8998', method='POST', 
uri='/kernels?notebook=74403c31-b655-4406-b36c-6c5f19fc03ed', 
version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Origin': 
'http://127.0.0.1:8998', 'Content-Length': '0', 'Accept-Language': 'en-us', 
'Accept-Encoding': 'gzip, deflate', 'Host': '127.0.0.1:8998', 'Accept': 
'application/json, text/javascript, */*; q=0.01', 'User-Agent': 
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.73.11 
(KHTML, like Gecko) Version/7.0.1 Safari/537.73.11', 'Connection': 
'keep-alive', 'Referer': 
'http://127.0.0.1:8998/74403c31-b655-4406-b36c-6c5f19fc03ed', 
'X-Requested-With': 'XMLHttpRequest'})
Traceback (most recent call last):
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/tornado/web.py",
 
line 1141, in _when_complete
callback()
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/tornado/web.py",
 
line 1162, in _execute_method
self._when_complete(method(*self.path_args, **self.path_kwargs),
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/tornado/web.py",
 
line 2297, in wrapper
return method(self, *args, **kwargs)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/html/services/kernels/handlers.py",
 
line 46, in post
kernel_id = km.start_kernel(notebook_id, cwd=nbm.notebook_dir)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/html/services/kernels/kernelmanager.py",
 
line 86, in start_kernel
kernel_id = super(MappingKernelManager, self).start_kernel(**kwargs)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/kernel/multikernelmanager.py",
 
line 115, in start_kernel
km.start_kernel(**kwargs)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/kernel/manager.py",
 
line 205, in start_kernel
**kw)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/kernel/manager.py",
 
line 161, in _launch_kernel
return launch_kernel(kernel_cmd, **kw)
  File 
"/Users/jon/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/IPython/kernel/launcher.py",
 
line 251, in launch_kernel
stdin=_stdin, stdout=_stdout, stderr=_stderr, cwd=cwd, env=os.environ)
  File 
"/Applications/Canopy.app/appdata/canopy-1.3.0.1715.macosx-x86_64/Canopy.app/Contents/lib/python2.7/subprocess.py",
 
line 709, in __init__
errread, errwrite)
  File 
"/Applications/Canopy.app/appdata/canopy-1.3.0.1715.macosx-x86_64/Canopy.app/Contents/lib/python2.7/subprocess.py",
 
line 1326, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
2014-02-28 17:15:48.344 [tornado.access] ERROR | 500 POST 
/kernels?notebook=74403c31-b655-4406-b36c-6c5f19fc03ed (127.0.0.1) 14.90ms

Anyone know why this is? Can't it find julia despite me setting it up that 
way?

Thanks for any help

[julia-users] trouble running julia

[Jon Norberg]

I have been running julia for a while lately with ijulia and everything 
worked nicely. then I started building julia with brew --HEAD to keep up to 
date with the latest changes, worked well enough. But now something 
happened and nothing works, test run won't work even though julia installs 
without errorseven when I download the prerelease the process just 
shuts down

exec 
'/Applications/Julia-0.3.0-prerelease-a673e4c4de.app/Contents/Resources/julia/bin/julia'


[Process completed]

So I have a couple of questions:

1) whats likely wrong here? is it some dependencies? how do I make a REALLY 
clean install
2) is it worth brewing --HEAD julia? How often does/should it fail?
3) when sticking to a stable release I notice that the number of warnings 
increases as time goes towards the next release. How to avoid? 
4) Whats the best strategy to have a healthy and reasonably updated julia

Many thanks (again)

Re: [julia-users] Re: Parallel sparse matrix vector multiplication

[Jon Norberg]

Thank you 

On Friday, February 14, 2014 10:19:22 AM UTC+1, Amit Murthy wrote:
>
> The `using ParallelSparseMatMul` must be after any `addprocs` statements. 
>
>
> On Fri, Feb 14, 2014 at 2:21 PM, Jon Norberg 
> 
> > wrote:
>
>> Amazing, just what I was looking for. 
>>
>> However :-/ I did exactly s your read me, installed, and using 
>> exactly your example I get:
>>
>> julia> y = S*x
>>
>> fatal error on 2: ERROR: ParallelSparseMatMul not defined
>>
>> Worker 2 terminated.
>>
>> ProcessExitedException()
>>
>>
>>
>> is it enough to just write 
>>
>> using ParallelSparseMatMul
>>
>>
>> to have access to it? Seems julia can't find some function...
>>
>>
>> Any help?
>>
>>
>> Many thanks
>>
>>
>> On Friday, February 14, 2014 2:31:08 AM UTC+1, Madeleine Udell wrote:
>>
>>> Thanks, Jiahao! It looks like you've already made a great dent in the 
>>> iterative solvers wishlist. I'm planning on using the 
>>> ParallelSparseMatMul library along with some iterative solvers 
>>> (possibly just LSQR) to implement iterative solvers for nonnegative 
>>> least squares, lasso, elastic net, etc using ADMM. It would be nice to 
>>> ensure that eg shared arrays stay shared in IterativeSolvers to ensure 
>>> it works well with parallel matrix multiplication. 
>>>
>>> On Thu, Feb 13, 2014 at 5:05 PM, Jiahao Chen  wrote: 
>>> > Fantastic work! 
>>> > 
>>> > I've been meaning to get back to work on IterativeSolvers... 
>>> > 
>>> > Thanks, 
>>> > 
>>> > Jiahao Chen, PhD 
>>> > Staff Research Scientist 
>>> > MIT Computer Science and Artificial Intelligence Laboratory 
>>>
>>>
>>>
>>> -- 
>>> Madeleine Udell 
>>> PhD Candidate in Computational and Mathematical Engineering 
>>> Stanford University 
>>> www.stanford.edu/~udell 
>>>
>>
>

Re: [julia-users] Re: Parallel sparse matrix vector multiplication

[Jon Norberg]

Amazing, just what I was looking for. 

However :-/ I did exactly s your read me, installed, and using exactly 
your example I get:

julia> y = S*x

fatal error on 2: ERROR: ParallelSparseMatMul not defined

Worker 2 terminated.

ProcessExitedException()



is it enough to just write 

using ParallelSparseMatMul


to have access to it? Seems julia can't find some function...


Any help?


Many thanks


On Friday, February 14, 2014 2:31:08 AM UTC+1, Madeleine Udell wrote:
>
> Thanks, Jiahao! It looks like you've already made a great dent in the 
> iterative solvers wishlist. I'm planning on using the 
> ParallelSparseMatMul library along with some iterative solvers 
> (possibly just LSQR) to implement iterative solvers for nonnegative 
> least squares, lasso, elastic net, etc using ADMM. It would be nice to 
> ensure that eg shared arrays stay shared in IterativeSolvers to ensure 
> it works well with parallel matrix multiplication. 
>
> On Thu, Feb 13, 2014 at 5:05 PM, Jiahao Chen > 
> wrote: 
> > Fantastic work! 
> > 
> > I've been meaning to get back to work on IterativeSolvers... 
> > 
> > Thanks, 
> > 
> > Jiahao Chen, PhD 
> > Staff Research Scientist 
> > MIT Computer Science and Artificial Intelligence Laboratory 
>
>
>
> -- 
> Madeleine Udell 
> PhD Candidate in Computational and Mathematical Engineering 
> Stanford University 
> www.stanford.edu/~udell 
>

[julia-users] Re: distance.jl

[Jon Norberg]

And that smart way is of course kNN algorithm...

[julia-users] Re: distance.jl

[Jon Norberg]

yes, you are right, just saw the bug with si should be ii and jj

will use sub as soon as I get 0.3.

Also, I did this little script to sort the xy array before hand. just need 
ot figure out a smart way to constrain the loops to avoid calculating pairs 
of block that can't have any distance below threshold.

n=5000

a=rand(3,n).*1000

b=zeros(Int64,4,n)

b[4,:]=1:n

b[1:3,:]=round(a/100)

bb=deepcopy(sortcols(b))

aa=deepcopy(a)

for i=1:n

 a[:,i]=aa[:,bb[4,i]]

end


s=edist(a,10,100)


s[1000,:]

[julia-users] Re: distance.jl

[Jon Norberg]

Thank you very much Douglas! I learned a lot there. I didn't get it to work 
straight away (running julia studio so its using julia 0.2). But these 
changes made it work:

function subinds(k, N)

step = iceil(N/k)

rng = 1:step

res = [rng + j * step for j in 0:k-2]

push!(res, ((k - 1) * step + 1):N)

res

end


function edist(a::AbstractMatrix, nblocks::Int, threshold)

N = size(a,2)

si = subinds(nblocks,N)

i = Int[]

j = Int[]

x = eltype(a)[]

for ii in si, jj in si

#r = pairwise(Euclidean(), view(a,:,ii), view(a,:,jj))

r = pairwise(Euclidean(), a[:,si[2]], a[:,si[2]])

ioffset = ii[1] - 1

joffset = jj[1] - 1

for ir in 1:size(r,1), jr in 1:size(r,2)

if (ie = ioffset + ir) != (je = joffset + jr) && (rij = r[ir,jr]) < 
threshold

push!(i,ie)

push!(j,je)

push!(x,rij)

end

end

end

sparse(i,j,x)

end

[julia-users] Re: distance.jl

[Jon Norberg]

Well, I solved it for now with subsampling: 

using Distance


n=5

a=rand(3,n)

#@time r=pairwise(Euclidean(),a,a)

subsample=10

m=integer(n/subsample)

s=spzeros(n,n)

r=zeros(m,m)

threshold=0.2

for i=1:subsample-1

ii=(i-1)*m+1

for j=1:subsample-1

jj=(j-1)*m+1

r=pairwise(Euclidean(),a[:,ii:ii+m-1],a[:,jj:jj+m-1])

r[r.>threshold]=0

s[ii:ii+m-1,jj:jj+m-1]=sparse(r)

end

end


If anyone know any performance improving tricks I'd be grateful.

[julia-users] Re: distance.jl

[Jon Norberg]

I also noted that pairwise has a hard limit just above 1*1 array 
size output, at least it crashes julia for me. Is there a way to increase 
this? 

[julia-users] distance.jl

[Jon Norberg]

I am working on some sparse representation models.

I was wondering if its difficult to implement a version of distance.jl so 
that one could write:

r=pairwise(spzeros(n,n),Euclidean(),A,B,threshold)

and produce a sparse output with distances only where 
Euclidean(A,B)

[julia-users] Re: quick sparse related question

[Jon Norberg]

Ok, I was tired when I wrote the above :-/

This works:

rand(10,30).*rand(10,30)

rand(10,1).*rand(10,30)

as one expects,


and this one works too

sprand(10,30,0.1).*sprand(10,30,0.1)


but this one don't:

sprand(10,1,0.1).*sprand(10,30,0.1)

ErrorException("Incompatible sizes")


Is this just not implemented yet or am I doing it wrong?

[julia-users] quick sparse related question

[Jon Norberg]

rand(10).*rand(10,30)

but

sv=sparsevec([3,5,7],[0.1,0.0,3.2],4)

sv.*sprand(4,20,0.1)


sprandbool(10, 1, 0.1).*sprand(10,30,0.1)


rand(10).*sprand(10,30,0.1)


all give 

Incompatible sizes


Is my syntax wrong?

Re: [julia-users] nested type arrays?

[Jon Norberg]

Performance WILL be an issue later. So dict is a beter choice for that? Can i 
create them similarly?

[julia-users] Re: nested type arrays?

[Jon Norberg]

Ok, thanks, I am all set now :-)

[julia-users] Re: nested type arrays?

[Jon Norberg]

I mean like: a = [ Bother(zeros(Int64,10), ArrayOf{Bother}) for x in 1:2 ]

On Sunday, January 26, 2014 1:28:49 PM UTC+1, Jon Norberg wrote:
>
> Actually, one more question, can I in the creation of  a = [ Bother(zeros(
> Int64,10), zeros(20)) for x in 1:2 ] also put in a vector of custom types 
> e.g. a type of "Bother"?
>

[julia-users] Re: nested type arrays?

[Jon Norberg]

Actually, one more question, can I in the creation of  a = [ Bother(zeros(
Int64,10), zeros(20)) for x in 1:2 ] also put in a vector of custom types 
e.g. a type of "Bother"?

[julia-users] Re: nested type arrays?

[Jon Norberg]

many thanks, thats perfect

Jon

On Sunday, January 26, 2014 1:03:38 PM UTC+1, Johan Sigfrids wrote:
>
> If you create a type  with fields b and other and put it in a vector a it 
> should work:
>
> # Create the type
> type Bother
> b::Vector
> other::Vector
> end
> # Initialize the array
> a = [ Bother(zeros(Int64,10), zeros(20)) for x in 1:2 ]
> # Then your assignments work (after fixing the parenthesis)
> a[1].b[1:10] = 10
> a[1].other[1:20] = 9.2
> a[2].b[1:10] = 5
> a[2].other[1:20] = 1.22
>
>
>
> On Sunday, January 26, 2014 12:47:15 PM UTC+2, Jon Norberg wrote:
>>
>> Hi
>>
>> I am trying to create nested structs like in matlab. I understand types 
>> are the equivalent. I want to be able to create something like 
>>
>> a[1].b[1:10]=10
>> a[1].other[1:20)=9.2
>> a[2].b[1:10]=5
>> a[2].other[1:20)=1.22
>>
>> Also, how would one list all fields like
>>
>> fields(a)
>>
>> "b"
>> "other"
>>
>> I have been trying to get this to work with types but if anyone can help 
>> me I'd be much grateful.
>>
>>

[julia-users] nested type arrays?

[Jon Norberg]

Hi

I am trying to create nested structs like in matlab. I understand types are 
the equivalent. I want to be able to create something like 

a[1].b[1:10]=10
a[1].other[1:20)=9.2
a[2].b[1:10]=5
a[2].other[1:20)=1.22

Also, how would one list all fields like

fields(a)

"b"
"other"

I have been trying to get this to work with types but if anyone can help me 
I'd be much grateful.

Re: [julia-users] New Year's resolutions for DataArrays, DataFrames and other packages

[Jon Norberg]

is this why I get this on latest julia studio on mac with recently updated 
packages:

julia> using DataFrames

julia> using RDatasets

julia> iris = data("datasets", "iris")

data not defined


??

Re: [julia-users] Natural language processing in Julia

[Jon Norberg]

Thats great, now I am starting to be able to do what I want. Any idea how 
one can list all properties and methods of a pyobject

Re: [julia-users] Natural language processing in Julia

[Jon Norberg]

Great. And how dose one get the text from pyobject to a julia string?

Thanks very much

Re: [julia-users] Natural language processing in Julia

[Jon Norberg]

Hi Jonathan

would you by any chance have some example code to share  how you work with 
NLTK using pycall. I wish there were more julia examples scripts available 
for browning and learning.

Thanks,

[julia-users] Re: nltk with pycall help

[Jon Norberg]

outch, a very clear case of RTFM

wn[:synsets]("car")


just as it is described if you read the pycall CAREFULLY


please make fun of me. 


Love Julia :-D

[julia-users] Re: nltk with pycall help

[Jon Norberg]

I did mean 

wn.synsets("car")

or 

wn.synsets('car')

won't work. should be simple solution but can't figure it out from the 
pycall manual :-/

On Wednesday, January 8, 2014 11:35:53 AM UTC, Jon Norberg wrote:
>
> Ok, so far so good, but it doesn't accept 
>
> wn.symsets("car")
>
> is there a different way to use the functions?
>
> Thanks
>
>
> On Tuesday, January 7, 2014 10:34:37 PM UTC, Steven G. Johnson wrote:
>>
>> @pyimport nltk
>> wn = nltk.corpus["wordnet"]
>>
>

[julia-users] Re: nltk with pycall help

[Jon Norberg]

Ok, so far so good, but it doesn't accept 

wn.symsets("car")

is there a different way to use the functions?

Thanks


On Tuesday, January 7, 2014 10:34:37 PM UTC, Steven G. Johnson wrote:
>
> @pyimport nltk
> wn = nltk.corpus["wordnet"]
>

[julia-users] Re: Natural language processing in Julia

[Jon Norberg]

Autocorrect does not work well.I meant: Forgot to say everything works in 
Python 

[julia-users] Re: Natural language processing in Julia

[Jon Norberg]

Yes, fordon to say it workshops perfectly in Python

[julia-users] nltk with pycall help

[Jon Norberg]

Everything works well in Python btw

[julia-users] nltk with pycall help

[Jon Norberg]

Its probably simple but I can't get it to work.

I simply want to use pythons nltk module through julia. 

http://nltk.org/howto/wordnet.html

I tried

@pyimport wordnet as wn


and

@pyimport nltk as wn


but can't seem to interface to use eg. out=wn.synsets("car")


do I need to use 

pyeval()


instead?


Thanks

[julia-users] Re: Natural language processing in Julia

[Jon Norberg]

Could anyone give me a little help in the right direction reg using nltk in 
julia.

I do 

@pyimport nltk.wordnet as wn


but I can's seem to figure out how to interact with e.g wn.synsets("car")


thanks for help