[julia-users] Re: Too many packages?

[yuuki]

On Sunday, 12 July 2015 22:47:42 UTC+2, Tony Kelman wrote:

> > I think there's a big differences between developing core features in 
> packages and shipping them with the default version and having optional 
> third party packages implementing core features.
>
> Like what, exactly? If the complaint is about ease of installation of 
> packages, then that's a known and acknowledged bug (set of bugs) that 
> people are thinking about how to do a better job of. We could always use 
> more help making things better.
>

If there's a bunch of official packages that are shipped with default 
version it's like having no packages, it's just a way for the devs to 
organize their work internally that doesn't concern the user too much.

On the other hand for third party packages the user has to find them, 
install them, debug them, worry about long term maintenance, etc. In 
reality it's a bit more fuzzy than that, so maybe my distinction isn't so 
relevant.


For plotting I think it would be better to have any plotting than none, 
even though not everybody will agree on the best choice for the one. The 
least dependencies seems the most important criteria to me, as long as you 
can draw lines, points and surfaces with decent performances. The high 
level interface doesn't matter that much in my opinion. 

[julia-users] Re: Too many packages?

[yuuki]

I think there's a big differences between developing core features in 
packages and shipping them with the default version and having optional 
third party packages implementing core features.

Personally I also find the huge amount of packages to be slightly annoying, 
but it's also clear the Julia team cannot do everything like Matlab would. 
The only thing I really would like to have 
included by default it plotting tools, because they are so essential for a 
lot of things. 

[julia-users] Re: Why Julia is not faster than Matlab ?

[yuuki]

Without a specific example it's hard to say anything. Matlab can be faster 
for some cases, but Julia should be faster in many cases, if you write your 
code correctly.

Re: [julia-users] Re: 4D interpolation from scattered points

[Yuuki Soho]

Something simple you can do is to compute the distance from your new point 
to all the other points, and do a weighted average of their values 
depending on the distance,
using for example Gaussian weights.

Re: [julia-users] Re: Teaching Julia to an 8 year old (and a 12 year old)

[yuuki]

Personally I always recommend Processing as a starting environment, it 
allows to easily do cool stuff. 
The bottleneck when starting is often motivation, so having positive 
feedback is important.

http://www.openprocessing.org/sketch/192912

Re: [julia-users] Re: Newbie help... First implementation of 3D heat equation solver VERY slow in Julia

[Yuuki Soho]

The code allocate only 432 bytes on my computer once I removed all global 
variables, and it's pretty fast.

Multiplying by the inverse of dx2 ... instead of dividing also make quite a 
difference, 2-3x.

http://pastebin.com/PSZyLXJX

Re: [julia-users] Yet Another String Concatenation Thread (was: Re: Naming convention)

[Yuuki Soho]

I think one should go over all the names in Base and see if there's some 
rules that can be applied sanely to come up with better naming scheme.

If you introduce factorize(MyType,...) and want to be consistent about this 
kind of things you might end up changing a lot of the functions in base. 
E.g.

sqrtm -> sqrt(Matrix,...)
hist2d -> hist(Dimension{2},...)
...

Re: [julia-users] Code starting to be slow after 30 min of execution

[Yuuki Soho]

The README.md is just the default page shown on github,

https://github.com/EconForge/NLsolve.jl/blob/master/README.md

but there's no example of anonymous function use there. I think you need to 
do something of the sort:

(x, fvec) -> begin
   fvec[1] = (x[1]+3)*(x[2]^3-7)+18
   fvec[2] = sin(x[2]*exp(x[1])-1)
end

Otherwise it's *@gensym *that is generating a new function name at each 
iteration, I'm not sure you need it.

Re: [julia-users] Code starting to be slow after 30 min of execution

[Yuuki Soho]

Do you really need to use a different name for your function each time ? 
you could just use the same name it seems. I'm not sure it would solve the 
problem though.

Re: [julia-users] Re: Naming convention

[Yuuki Soho]

On Monday, April 27, 2015 at 4:21:11 PM UTC+2, Kevin Squire wrote:
>
> Just a note, Matlab-style [a b] concatenation has been deprecated in 
> Julia v0.4 
> .
>   
> See the linked issues for details.  The main issue is that this 
> functionality makes it challenging to create arrays of arrays (or arrays of 
> ranges), which are less useful in Matlab, but quite useful for general 
> programming.
>

As I understand it, only [a, b] is deprecated, with [a b] and [a; b] still 
being valid horizontal and vertical concatenation.

I find a bit inconsistent that [a b] means something different when a and b 
are string as opposed to arrays, but maybe that rest upon the wrong idea 
that string are array-like objects.

Re: [julia-users] Re: Naming convention

[Yuuki Soho]

There's one argument for * over +. That string concatenation is not 
commutative, and that + main property is to be commutative.

Personally I don't mind * for string concatenation. If anything I would 
prefer to have matlab style concatenation, using []; it would make sense to 
use concatenation syntax for concatenation.

[julia-users] Re: Need help/direction on how to optimize some Julia code

[Yuuki Soho]

I don't know too much about the subject but it seems for your problem 
description that the limiting factor will be network bandwidth. 
If I understand correctly now you are doing only 5 requests at the same 
time? It seems to me you could do much more. 

It does depend on the server though, some will block you if you are doing 
too many requests.

At some point I was doing some web crawling, using a single process and a 
recursive function:

function exploreSite(...)

for i=1:length(links)
refs = [refs; remotecall(1, exploreSite, depth-1, links[i]) ]
end

for r in refs
wait(r)
end

This would fire hundreds of queries asynchronously, and saturate either my 
network or the server.

[julia-users] Re: Product Function

[Yuuki Soho]

I think your are looking for prod()

Base.prod(itr)

   Returns the product of all elements of a collection.

Base.prod(A, dims)

   Multiply elements of an array over the given dimensions.


julia> prod(1:10)
3628800

[julia-users] Re: About Speed (what am I doing wrong?)

[Yuuki Soho]

You can loop on i in the inner loop, it helps quite a bit:

@inbounds for j = 1:size(b,2)

for k = 1:size(b,1)

@simd for i =1:size(a,1)

c[i,j] += a[i,k]*b[k,j] 

end

end

end 

Re: [julia-users] How to introduce scope inside macro

[Yuuki Soho]

Wrapping it in a function seems to work fine, although it will be if you 
don't declare c as const:


   1. macro map_(expr, args...)
   2. quote
   3. @assert all([map(length, ($(args...),))...] .== length($(args[
   1])))
   4. out = Array(typeof($(indexify(expr, 1, args))), size($(args[1]
   )))
   5. for i in 1:length(out)
   6. @inbounds out[i] = $(indexify(expr, :i, args))
   7. end
   8. out
   9. end
   10. end
   11.  
   12. macro map(expr, args...)
   13. quote
   14. function dummy_fun($(args...))
   15. @map_($expr,$(args...))
   16. end
   17. dummy_fun($(args...))
   18. end
   19. end
   

[julia-users] Re: Why is Forward Reference possible?

[Yuuki Soho]

I think Julia implicitly assume that i is a global variable, i.e. your 
function is equivalent to

f = () -> begin
global i
return i
end

So it compiles but throws an error at run-time if i is not defined in the 
global scope.

Re: [julia-users] Re: Functions in degrees

[Yuuki Soho]

One advantage of having a Radian type would be to have circular statistics 
built-in, mainly mean, var, std or cor. Other languages usually have 
special functions or packages for that (circmean).

[julia-users] Re: beginning deep learning with 500 lines of Julia

[Yuuki Soho]

Nice article, it reminded me of my machine learning courses :)

The only comment I have is maybe use a bit more explicit names, like forw 
-> forward, xtst -> xtest

[julia-users] Re: api - web scraping

[yuuki]

It's not much but here a script that reads a page (using HTTPClient) and 
get some HTML elements from it (using Gumbo).
There's probably a better way of selecting specific html elements, but it 
was working at the time I did it:

http://pastebin.com/vG67Udkj

Re: [julia-users] Re: How to avoid temporary arrays being created in a function.

[Yuuki Soho]

Sometimes allocations do not seem to make such a difference but for some 
functions if you do repeated calls, the first ones seem fine, but then the 
third of fourth one trigger gc and gets a 2-3 times slowdown, so the cost 
of allocation is not always immediately visible.

Otherwise I think you need to pre-allocated array and pass them as 
arguments. With multiple dispatch it's quite easy, you can write your core 
algorithm in a function that modify it's input:

foo!(out,A,B)

and then just write another one that allocate the output:

function foo(A,B)

out = zeros ...
foo!(out,A,B)

end

So you have both a convenient version and a faster one if you really need 
it.

Re: [julia-users] How to manipulate dimensions ?

[Yuuki Soho]

Thanks, perfect. 

Looking at my console, I tried something like tuple(sA,sA...), but I didn't 
realized you can "splice" several arguments. 
The learning curve for tuples might be a bit bumpy. 

[julia-users] Re: No help ? ? Re: Binomial , how compute discret (or / and cumulated) value ?

[Yuuki Soho]

This should give you the answer:

http://juliastats.github.io/Distributions.jl/univariate.html

[julia-users] How to manipulate dimensions ?

[Yuuki Soho]

I'm trying to do something very simple but I'm having no luck.

I have to arrays A and B and I want to initialize an array C with dimension 
that depend on the ones of A and B, in matlab I simply would do:

sA = size(A); 
sB = size(B); 
C = zeros([sA(1:end-1) sB(2:end)]); 

How does one do that in Julia ? I guess I could write a function and use 
ntuple, but it feels really awkward.

Re: [julia-users] Re: compute quantity along contour

[Yuuki Soho]

It seems you want to unwrap the phase (plus pi) along your path:

https://gist.github.com/ssfrr/7995008

But as you data are quite discrete, I'm not sure it will work. Maybe if you 
interpolate.

[julia-users] Re: help with performance

[Yuuki Soho]

The reason why test_1 is slower than test_2 is that you forgot to pass σ as 
a parameter in test_1 definition, so Julia looks for a global variable 
instead. 

In general don't hesitate to add type annotation to your functions, it 
often helps to prevent types problems (e.g. function 
test_1(μ::Float64,Δσ::Float64,σ::Float64) 
).

[julia-users] Re: How to make vector 'c' such that they are the values of the vector "a" but not in the vector "b"?

[Yuuki Soho]

setdiff(a,b)

You can find all this kind of functions here, they are quite useful:

http://docs.julialang.org/en/release-0.3/stdlib/collections/#set-like-collections

[julia-users] Re: Memory usage and speed in nested loop.

[Yuuki Soho]

I think you can just add a loop over the dimensions to do the copy and 
avoid the allocations:

function slow(points::Array{Float64,2})
n_dim = size(points,1)
n_points::Int = size(points,2)
point_2 = zeros(n_dim)
cum = 0.0
for i in 1:n_points
for j in (i+1):n_points
for k=1:n_dim
point_2[k] = points[k, j]
end
cum += myDist(point_2)
end
end
return cum
end

[julia-users] Re: Memory usage and speed in nested loop.

[Yuuki Soho]

If you access directly the elements of your arrays it will not allocate 
memory:

cum += points[1, j] 

It's often worth adding a loop or reshaping your inputs so you can write 
your inner code like that 
and avoid memory allocation.

Re: [julia-users] Re: Strange performance behavior

[Yuuki Soho]

Well, my matlab code wasn't very good, I did a fully vectorized version, 
and it's about the same speed than Julia.


I'm playing now with a problem that is easier to vectorize and there the 
matlab version is 300 times faster than Julia, which is a bit crazy.

Maybe I messed something up again, that's really a big difference. I've 
check that the output is identical, so at least I'm computing the same 
thing.


Julia:

   - 
  1. A = rand(50,40,30);
  2. B = rand(50,50);
  3. C = rand(40,40);
  4. D = rand(30,30);
  5. E = rand(50,40,30);
  6. alpha = zeros(size(A))
  7.  
  8. function testSum!(A::Array{Float64,3},B::Array{Float64,2},C::Array{
  Float64,2},
  9.   D::Array{Float64,2},E::Array{Float64,3},alpha::
  Array{Float64,3})
  10.
  11. tmp_1 = zero(Float64)
  12. tmp_2 = zero(Float64)
  13. tmp_3 = zero(Float64)
  14.
  15. @inbounds begin
  16. for x_3p = 1:30
  17. for x_2p = 1:40
  18. for x_1p = 1:50
  19. tmp_3 = zero(tmp_3)
  20. for x_3 = 1:30
  21. tmp_2 = zero(tmp_2)
  22. for x_2 = 1:40
  23. tmp_1 = zero(tmp_1)
  24. @simd for x_1 = 1:50
  25.  tmp_1 += A[x_1,x_2,x_3] * B[x_1,x_1p]   
   
  26. end
  27. tmp_2 += tmp_1 * C[x_2,x_2p]
  28. end
  29. tmp_3 += tmp_2 * D[x_3,x_3p]
  30. end
  31. alpha[x_1p,x_2p,x_3p] =  E[x_1p,x_2p,x_3p] * tmp_3
  32. end
  33. end
  34. end
  35. end
  36. end
  
Matlab:


function alpha = test2(A,B,C,D,E)

 

alpha = multiprod(multiprod(multiprod(B', A,[1 2],[1 2]),C,[1 2],[1 
2]),D,[2 3],[1 2]).*E;

end

Re: [julia-users] Re: Strange performance behavior

[Yuuki Soho]

Omg, I forgot to remove it! I should proof-read my code more carefully 
before posting... Julia is now 5x faster than matlab.

Re: [julia-users] Re: Strange performance behavior

[Yuuki Soho]

   
   1. That's my matlab code, I was hoping to be able to write it with loops 
   in Julia, because sometimes these vectorize operations can become 
   a real headache, but it's maybe hopeless to try to beat these highly 
   optimized libraries.

   
   function alpha = testSum(A,B,C,D,E,idx)
   alpha = zeros(100,1);
   for t=1:100 
  tmp = squeeze( sum( bsxfun(@times,A,B(:,t)), 1) );
  tmp = sum( bsxfun(@times,tmp,C(:,t)), 1);
  tmp = tmp * D(:,t);

  alpha(t) = tmp * E(t,idx(t));
   end
end
   
   

Re: [julia-users] Re: Strange performance behavior

[Yuuki Soho]

Coming back to my original problem, I did a simplified version of it,

which is about 10x slower than a vectorized matlab version. Have I missed 
anything here ?


A = ones(50,40,40);

B = ones(50,100)/2;

C = ones(40,100)/3;

D = ones(40,100)/4;

E = ones(100,100)/5;

idx = int([100:-1:1]);

 

function 
testSum(A::Array{Float64,3},B::Array{Float64,2},C::Array{Float64,2},

 
D::Array{Float64,2},E::Array{Float64,2},idx::Array{Int64,1})

 

alpha = zeros(100)

tmp_1 = zero(Float64)

tmp_2 = zero(Float64)

tmp_3 = zero(Float64)



@inbounds for t = 1:100

for thp = 1:50 



tmp_3 = zero(tmp_3) 

for x_3 = 1:40 

 

tmp_2 = zero(tmp_2) 

for x_2 = 1:40 

 

tmp_1 = zero(tmp_1) 

@simd for x_1 = 1:50 

 tmp_1 += A[x_1,x_2,x_3] * B[x_1,t] 
   

end

tmp_2 += tmp_1 * C[x_2,t]

end

tmp_3 += tmp_2 * D[x_3,t]

end



alpha[t] =  E[t,idx[t]] * tmp_3

end

end



return alpha



end

Re: [julia-users] Re: Strange performance behavior

[yuuki]

This seems right, I can't really tell what is going on from the machine 
code, but if I use @inbounds in f2 and do out of bounds reads it doesn't 
crash, so it seems the code is never executed. Thanks for the answers.

Re: [julia-users] Re: Strange performance behavior

[yuuki]

This replicates my "problem", f2 is much faster than f1, but like I said 
isn't that because the loops are optimized away during the compilation?
The weird thing I just noticed is that that it still depend on the size of 
A.

function f1(A::Array{Float64,2})

  tmp = 0.0

  for i=1:size(A,1)
for j=1:size(A,2)
 tmp += A[i,j]
end
  end

  out = tmp
end

function f2(A::Array{Float64,2})

  tmp = 0.0

  for i=1:size(A,1)
for j=1:size(A,2)
 tmp += A[i,j]
end
  end

  out = 1.0
end

A = randn(10^4,10^4);

f1(A);
f2(A);
@time f1(A)
@time f2(A)

[julia-users] Re: Strange performance behavior

[yuuki]

Hum, I think the compiler is just removing the inner loop when I don't use 
tmp_3 since it's never used. So I guess 3.8s is the correct time, which 
still feel a bit slow.

[julia-users] Strange performance behavior

[Yuuki Soho]

Hi, I've got a performance issue that I really can't seem to understand.


This if the inside of my main loop, I'm trying to fill up my alpha variables 
(there's some outer loops over thp,Ap,Bp,tp1)


tmp_1 = zero(Float64)

tmp_2 = zero(Float64)

tmp_3 = zero(Float64)

tmp_4 = zero(Float64)



for thp = 1:50 # line 858:

tmp_3 = 0.0 # line 557:

tmp_4 = rand() # line 557:




for B = 1:40 # line 558:

 

tmp_2 = 0.0 # line 557:

for A = 1:40 # line 558:

 

tmp_1 = 0.0 # line 550:

@simd for th = 1:50 # line 
551:

 

@inbounds tmp_1 += 
alpha_t[th,A,B] * tr_1[th,thp] 
   

end

 

@inbounds tmp_2 += tmp_1 * 
tr_2[A,Ap]

end



@inbounds tmp_3 += tmp_2 * 
tr_3[B,Bp]

end

 

@inbounds alpha[thp,Ap,Bp,tp1] =  
em_1[thp,Ap,Bp,obs[tp1]] * tmp_3 

end


This is pretty slow, it take 3.8s on my computer. The profiler tells me 
that the last line takes of lot of time:



The weird thing is that I replace tmp_3 by tmp_4 or rand() time gets down 
to 0.004s. 

I really don't get what's going on. All my arrays are annotated as Float64. 
Any idea ? I'm on version 0.3.2.

Thanks!

Re: [julia-users] Re: Plotting-package-independent API

[Yuuki Soho]

It seems there's two levels where things could be standardized, for me a 
good plotting API primarily need to provide good primitives: dot, line, 
rectangle, ellipse, patch, text, ...

I don't know if those could be standardized so that plotting methods could 
be written in a rendering back-end independent manner. 

Then there's the higher level methods like plot, show, piechart...

[julia-users] Re: 0.4 Documentation System

[Yuuki Soho]

I had a problem with my packages (it was even crashing on update...), I 
reinstalled the packages from scratch and the error is gone. I should have 
checked that first, sorry.

That said I'm not sure it's working properly, that's what I get (I renamed 
the test function above test1):

help?> test1
INFO: Loading help data...
test1 (generic function with 1 method)

[julia-users] Re: 0.4 Documentation System

[Yuuki Soho]

My file was huge, but I did a minimal one:

if VERSION < v"0.4-"

using Docile

macro doc_mstr(text)

Base.triplequoted(text)

end

macro doc_str(text)

text

end

end

 

@doc doc"Tells you if there are too foo items in the array." ->

function test(x)

 

return 1

end

julia> reload("testDocile.jl")

ERROR: No metadata defined in module Main.

 in error at error.jl:21

 in getdoc at /Users/laupin/.julia/Docile/src/types.jl:110

 in Entry at /Users/laupin/.julia/Docile/src/types.jl:35

 in Entry at /Users/laupin/.julia/Docile/src/types.jl:34

 in setmeta! at /Users/laupin/.julia/Docile/src/types.jl:101

 in include at 
/Applications/Julia-0.3.2.app/Contents/Resources/julia/lib/julia/sys.dylib

 in include_from_node1 at 
/Applications/Julia-0.3.2.app/Contents/Resources/julia/lib/julia/sys.dylib

 in reload_path at loading.jl:152

 in reload at loading.jl:85

while loading /Users/laupin/Desktop/julia/testDocile.jl, in expression 
starting on line 11

[julia-users] Re: 0.4 Documentation System

[Yuuki Soho]

I just used the workaround but now I get another error:

ERROR: No metadata defined in module Main.


I just copy-pasted one of the example from the doc before one of my 
functions. v"0.3.2"

On Tuesday, January 20, 2015 at 8:19:50 AM UTC+1, Michael Hatherly wrote:
>
> If anyone happens to run into a
>
> ERROR: @doc_mstr not defined
>
> message when using Docile and the 0.4 doc system please have a look at 
> this  issue for 
> the workaround.
>
> — Mike
> ​
> On Tuesday, 20 January 2015 00:26:36 UTC+2, Mike Innes wrote:
>>
>> Hey Julians,
>>
>> Just wanted to drop a note to say that the new documentation system now 
>> has some of its own documentation in the manual 
>> . This will 
>> be relevant even if you're not using 0.4, since you can use the excellent 
>> Docile.jl package to create and access documentation in 0.3.
>>
>> It'd be great to see people kicking the tires on this, and with any luck 
>> the state of package docs will look much better by the time 0.4 is released.
>>
>> – Mike
>>
>

[julia-users] Re: Juno command line - total noob question

[yuuki]

Usually I have a second file open that I use like a console, it's a bit 
awkward but it works.

[julia-users] Re: terminology: vector, array, matrix

[yuuki]

Both Nx1 and 1xN arrays are matrices (ndims=2), but N-vectors (ndims=1) are 
almost functionally equivalent to Nx1 matrices, i.e. you can use them in 
matrix multiplication and such. This is a bit weird at first but it doesn't 
cause much problems in practice, even though it can be a bit funny 
sometimes:

julia> x == x''
false

Re: [julia-users] julia in Sublime without IJulia functionality

[Yuuki Soho]

As a matlab user I really like having everything integrated (editor, 
console and figures). A lot of my colleagues uses different setups and I 
feel like they spend half of their day switching between windows and 
resizing them. I also have a hard time working without the cell feature of 
the matlab editor (sends blocks of code to the terminal without needing to 
select anything).

On Tuesday, October 14, 2014 1:28:55 AM UTC+2, Stefan Karpinski wrote:
>
> This may be off-topic, but I've always felt that in-editor semi-terminals 
> are incredibly awkward and seem to invariably lack features that real 
> terminals have – especially nice ones like are available on OS X and Linux. 
> What are the features that people really like from the terminal-in-editor 
> setup? Is it just the ability to send the current line or select to the 
> terminal to evaluate it without leaving the editor? Couldn't we make that 
> work without the terminal being inside the editor? Why not just send the 
> code to be evaluated over a socket to the REPL and have the REPL still be a 
> normal, functioning REPL for when you want to just interact with it 
> normally.
>
> On Mon, Oct 13, 2014 at 7:17 PM, Zac Cranko  > wrote:
>
>> I agree. I would also like a way to do this.
>>
>
>

[julia-users] Re: ANN: major upgrade to HDF5/JLD

[yuuki]

Just tested it on my windows 7 and the installation went relatively 
smoothly. This is really awesome, it makes Julia much more usable for 
everyday work. 

[julia-users] Re: how to set proxy attribute using httpClient.jl

[yuuki]

It seems there no example for setting a proxy, but you can easily setup 
libCurl options by using  LibCURL.curl_easy_setopt.

There's all the options here: 

http://curl.haxx.se/libcurl/c/curl_easy_setopt.html

I get the one you want is CURLOPT_PROXY and it seems you just need to 
provide an ip adress in a string.

Here is how you can setup options, you need to write a function that you 
pass to  HTTPC.get : 

function customize_curl(curl)
cc = LibCURL.curl_easy_setopt(curl, LibCURL.CURLOPT_USERAGENT, "Mozilla/5.0 
(Windows NT 6.1; rv:28.0) Gecko/20100101 Firefox/28.0")
if cc != LibCURL.CURLE_OK
error ("CURLOPT_USERAGENT failed: " * 
LibCURL.bytestring(curl_easy_strerror(cc)))
end
end

r = HTTPC.get(url,RequestOptions(
request_timeout=5.0,
callback=customize_curl
))

Re: [julia-users] Re: ANN: Gumbo.jl (HTML parsing library)

[Yuuki Soho]

I'm using it yes, but I'm not sure it can do what I want. Basically when I 
am on page, say "http://host.com/section1/section2/"; and I get relative 
links like "../section3" or "//section4"
then I have some dirty code to reconstruct the full urls. I mean it works 
alright, but it looks a bit brittle. Is there a clean way to do that with 
URIParser ?

[julia-users] Re: ANN: Gumbo.jl (HTML parsing library)

[Yuuki Soho]

I've used it a little bit and it's very nice, great job!

Just a related question, is there something to deal with html links 
currently in Julia, parsing all the ../, //, /, #, ? madness ? That would 
certainly be useful alongside Gumbo.

[julia-users] Re: GSoC: Julia IDE Progress update

[yuuki]

Cool, very nice work. I had no problem install it on windows 7, but I have 
to say I'm quite confused by Light Table, it seems interesting though.
I got these data display but I haven't find yet how to close them ... 
http://i.imgur.com/uNIxePy.png

How easy to contribute to the project, add features, etc ? I'm asking that 
because I feel like an IDE what it the easy to modify and to contribute to 
have more chances to succeed (Julia Studio for example doesn't seem to have 
a lot of contributions).

Re: [julia-users] Re: Help with writing a macro to for fetch, process, store pattern

[Yuuki Soho]

 This should do what you want:

>
>>> const circuitModelNames = names(CircuitModel)
>>>
>>>
>>> macro fetch(ex)
>>>
>>>   Expr(:block, [ :($(circuitModelNames[i]) = $ex.$(circuitModelNames[i]) 
>>> ) for i = 1:length(circuitModelNames) ]...) |> esc
>>>
>>> end
>>>
>>>
>>> julia> macroexpand (:(@fetch model))
>>>
>>> :(begin 
>>>
>>> v1 = model.v1
>>>
>>> v5 = model.v5
>>>
>>> v7 = model.v7
>>>
>>> v2 = model.v2
>>>
>>> v3 = model.v3
>>>
>>> v4 = model.v4
>>>
>>> v9 = model.v9
>>>
>>> v12 = model.v12
>>>
>>> v15 = model.v15
>>>
>>> v18 = model.v18
>>>
>>> v19 = model.v19
>>>
>>> v20 = model.v20
>>>
>>> v21 = model.v21
>>>
>>> v8 = model.v8
>>>
>>> v22 = model.v22
>>>
>>> v23 = model.v23
>>>
>>> v24 = model.v24
>>>
>>> i1 = model.i1
>>>
>>> i2 = model.i2
>>>
>>> i3 = model.i3
>>>
>>> i4 = model.i4
>>>
>>> i5 = model.i5
>>>
>>> i6 = model.i6
>>>
>>> i7 = model.i7
>>>
>>> ic1eq = model.ic1eq
>>>
>>> ic2eq = model.ic2eq
>>>
>>> ic3eq = model.ic3eq
>>>
>>> ic4eq = model.ic4eq
>>>
>>> ic5eq = model.ic5eq
>>>
>>> ic6eq = model.ic6eq
>>>
>>> ic7eq = model.ic7eq
>>>
>>> ic8eq = model.ic8eq
>>>
>>> sr = model.sr
>>>
>>> srinv = model.srinv
>>>
>>> pi = model.pi
>>>
>>> gmin = model.gmin
>>>
>>> is1 = model.is1
>>>
>>> nvtf1 = model.nvtf1
>>>
>>> is2 = model.is2
>>>
>>> nvtf2 = model.nvtf2
>>>
>>> nvtinvf1 = model.nvtinvf1
>>>
>>> vcritf1 = model.vcritf1
>>>
>>> nvtinvf2 = model.nvtinvf2
>>>
>>> vcritf2 = model.vcritf2
>>>
>>> gc1 = model.gc1
>>>
>>> gr1 = model.gr1
>>>
>>> gr2 = model.gr2
>>>
>>> itxr2 = model.itxr2
>>>
>>> gc2 = model.gc2
>>>
>>> gc3 = model.gc3
>>>
>>> gc4 = model.gc4
>>>
>>> gc5 = model.gc5
>>>
>>> gr7 = model.gr7
>>>
>>> gc6 = model.gc6
>>>
>>> gr3 = model.gr3
>>>
>>> itxr3 = model.itxr3
>>>
>>> gc7 = model.gc7
>>>
>>> gr4 = model.gr4
>>>
>>> gc8 = model.gc8
>>>
>>> gr5 = model.gr5
>>>
>>> vpos = model.vpos
>>>
>>> vneg = model.vneg
>>>
>>> gin = model.gin
>>>
>>> gininv = model.gininv
>>>
>>> vposcap = model.vposcap
>>>
>>> vnegcap = model.vnegcap
>>>
>>> ginotacore = model.ginotacore
>>>
>>> ginotares = model.ginotares
>>>
>>> ginotacoreinv = model.ginotacoreinv
>>>
>>> ginotaresinv = model.ginotaresinv
>>>
>>> vc3lo = model.vc3lo
>>>
>>> vc3hi = model.vc3hi
>>>
>>> a4a4c = model.a4a4c
>>>
>>> a5a5c = model.a5a5c
>>>
>>> a6a6c = model.a6a6c
>>>
>>> a14a14c = model.a14a14c
>>>
>>> a16a16c = model.a16a16c
>>>
>>> a17a17c = model.a17a17c
>>>
>>> a17a17nrmc = model.a17a17nrmc
>>>
>>> a12a17c = model.a12a17c
>>>
>>> a16a16nrmc = model.a16a16nrmc
>>>
>>> a15a16c = model.a15a16c
>>>
>>> a14a14nrmc = model.a14a14nrmc
>>>
>>> a13a14c = model.a13a14c
>>>
>>> a6a14c = model.a6a14c
>>>
>>> a13a6c = model.a13a6c
>>>
>>> a5a5nrmc = model.a5a5nrmc
>>>
>>> a4a5c = model.a4a5c
>>>
>>> a2a5c = model.a2a5c
>>>
>>> a2a4c = model.a2a4c
>>>
>>> a4a4nrmc = model.a4a4nrmc
>>>
>>> a1a4c = model.a1a4c
>>>
>>> v5c = model.v5c
>>>
>>> v7c = model.v7c
>>>
>>> end)
>>>
>>>

[julia-users] Re: Help with writing a macro to for fetch, process, store pattern

[yuuki]

I think that adapting the macro at the end of this thread: 
https://groups.google.com/forum/#!searchin/julia-users/unpack/julia-users/IQS2mT1ITwU/gEyj6JNJsuAJ
and getting the fields of your type by using names(CircuitModel) it should 
work. I'll give it a try tomorrow if I have the time.

Re: [julia-users] How to read and change the content of web pages to the vector ?

[Yuuki Soho]

So, you want to parse a web page and get the content out of it.

I'm not sure there's a very good way of doing it currently, because html 
pages are often messy ( 
http://programmers.stackexchange.com/questions/151739/getting-data-from-a-webpage-in-a-stable-and-efficient-way
 
). What you want is some kind of html parser like jsoup. I don't any are 
available in julia right know. You could try with an xml parser 
(https://github.com/lindahua/LightXML.jl) but I'm not sure it that will 
work very well.

Otherwise you can go take the dirty road and use regular expressions to 
extract what you want.

>

Re: [julia-users] REPL history-search prefix with up/down arrow key

[yuuki]

It seems it might come soon:

https://github.com/JuliaLang/julia/issues/6377

Personally I would prefer to have search bound to up key by default, simply 
because 90% of the time when I press up in a terminal I want to do a 
single-line prefix search. 

[julia-users] Re: How quickly and automatically find the cut-off point of vectors with long tails ?

[Yuuki Soho]

The simplest way to do it is probably to use a quantile:


a=[5 3 2 1.5 1.1 1 0.8 0.25 0.2 0.16]

q = quantile(vec(a),0.1)

a = a[1:cut] 

[julia-users] Re: Element-wise `and` for matrixes of booleans (i.e. BitArrays), and broadcasting

[Yuuki Soho]

Doesn't 

inside_R_box * inside_Z_box

gives what you want ?

[julia-users] Re: slicing a 2d array

[yuuki]

I think that the short answer is that a column vector is not the same thing 
than a row vector. Note that Array{Float64,1} (N-vector) is kind of 
equivalent to column vectors (Nx1 matrices) in Julia (meaning you can use 
them in linear algebra operations), that might be where the confusion comes 
from.

[julia-users] Re: GSOC 3D Visualizations plotting API - Make a wish!

[Yuuki Soho]

I wish for 2D and 3D plots to be unified in the same package and use the 
same syntax, as it would be quite awkward to have to change you code when 
going from plot to plot3.

Re: [julia-users] Question about 'dot' notation (and max/maximum)

[Yuuki Soho]

K leo, I think it's mainly a problem of consistency across operators (+ 
behave like * and .+ behave like .*) and across dimensions (adding a scalar 
to a vector behave the same as adding a vector to a matrix). 

The nice thing with Julia is that you can easily (re) define operators to 
do what you want, if you prefer + to work with scalar you can just define:

+{T,K}(a::Array{T,K},x::Number) = a .+ x

And it should work.

[julia-users] Re: Matlab urlread('URL','method',PARAMS), anyone knows the equivalent in Julia?

[yuuki]

I've been using  HTTPClient: 

https://github.com/amitmurthy/HTTPClient.jl

You can look in the tests for some examples:

https://github.com/amitmurthy/HTTPClient.jl/blob/master/test/tests.jl

[julia-users] Re: output sharing memory with input

[yuuki]

Even though I wasn't too aware of this shared memory thing, I have to say 
it never caused problems in practice (that I know of...).

I did a function that prints variables that points to the same place, just 
for fun (it's really cool you can do that easily):

julia> x = rand(10,10);

julia> y = vec(x);


julia> whosShared()

y <-> x


function whosShared()


m = Main

n = Array(String,0)

pts = Array(Ptr{Float64},0)


for v in names(m)

if isdefined(m,v) &&  typeof(eval(m,v)) <: Array

push!(n,string(v))

push!(pts, pointer(eval(m,v)))

end

end


for i=1:length(n)

for j=(i+1):length(n)

if pts[i] == pts[j]

println( string( n[i], " <-> ", n[j] ) )

end

end

end


end

Re: [julia-users] learning neural networks: optimization and memory allocation question

[yuuki]

It's nicer with indentation :)

http://pastebin.com/y3mt4QY6

>
>

[julia-users] Re: learning neural networks: optimization and memory allocation question

[yuuki]

Devectorizing is probably the simplest solution, this is what I get:

@time wb = test_batch(w0,x)
@time w1 = test_loop(w0,x,nd);
@time test_loop2!(w0,x,y,nd,ny,nx);

println( maximum( abs( w1 - wb ) ))
println( maximum( abs( w1 - w0 ) ))

elapsed time: 0.097451771 seconds (86417036 bytes allocated)
elapsed time: 10.216290059 seconds (16011425636 bytes allocated)
elapsed time: 2.0314269 seconds (36 bytes allocated)
0.19439111046738267
4.440892098500626e-16

I don't get the same result with  test_batch and the loops version, I don't 
know if I messed something up. 

Note I also passed the dimensions as arguments, running loops with globally 
defined dimensions was using a lot memory. The @inbounds remove the bounds 
check, it speeds things up quite a bit, but it will make Julia crash if 
there's a mistake somewhere.


nd = 200 # Number of input samples
nx = 1000 # Input layer dimension
ny = 2000 # Output layer dimension
x = randn(nx,nd) # Input data 
w = randn(nx,ny) # Weights
y = zeros(ny)

function test_loop(w::Array{Float64,2},x::Array{Float64,2},nd)
for i in 1:nd
y = x[:,i]'*w # Linear neuron. y_{1,ny} = xi_{nx,1}' * w_{nx,ny} 
w += 0.0001*(x[:,i]*y - w.*(y.*y)) # Oja's rule. w_{nx,ny} += 
x_{nx,1}*y_{1,ny} - w_{nx,ny}.*y_{1,ny}.^2
end
return w
end

function test_loop2!(w::Array{Float64,2},x::Array{Float64,2},y,nd,ny,nx)

for i = 1:nd
for j=1:ny
y[j] = 0.0 
for k = 1:nx 
@inbounds y[j] += + x[k,i]*w[k,j] # Linear neuron. y_{1,ny} = xi_{nx,1}' * 
w_{nx,ny} 
end
end 

for j=1:ny 
for k = 1:nx 
# Oja's rule. w_{nx,ny} += x_{nx,1}*y_{1,ny} - w_{nx,ny}.*y_{1,ny}.^2
@inbounds w[k,j] += 0.0001*(x[k,i]*y[j] - w[k,j]*(y[j]*y[j])) 
end
end
end
end

w0 = w

w1 = test_loop(w0,x,nd);
test_loop2!(w,x,y,nd,ny,nx);

@time w1 = test_loop(w0,x,nd);
@time test_loop2!(w0,x,y,nd,ny,nx);

println( maximum( abs( w1 - w ) ))

>
>
>

[julia-users] Re: Trying to understand how to structure parallel computation

[yuuki]

Maybe the pmap example from the doc can be useful, it feeds work to other 
processes:

http://julia.readthedocs.org/en/latest/manual/parallel-computing/#synchronization-with-remote-references

Re: [julia-users] Packing and unpacking parameters

[Yuuki Soho]

That was a slightly stupid question John, I should have thought about it 2 
minutes:)
I was hopping to do something like that, but it doesn't work because of the 
hygiene I guess.

macro unpack()

quote

a = p[1]

b = p[2]

end 

end 

macroexpand(:(@unpack))
#184#a = p[1] # line 1: #185#b = p[2]

This is what I get with yours Mike:

@unpack [1,2,3]
3-element Array{Expr,1}: :(a = [1,2,3][1]) :(b = [1,2,3][2]) :(c = 
[1,2,3][3])

Thanks for the answers!

Re: [julia-users] Packing and unpacking parameters

[Yuuki Soho]

Is it possible to do something like defining a piece of code, and then just 
insert it when needed ? Something like:

@insertcodehere somecode

Re: [julia-users] Packing and unpacking parameters

[Yuuki Soho]

I'm trying to write an unpack macro, just to learn a bit about 
meta-programming (I get it's probably not the best idea,
but sometimes you learn a lot doing stupid things), but I have to say than 
even after reading three times the doc, 
I have no idea how to do it. I wanted to do something like that:

n = [:a,:b]
p = [1 2]

@unpack n p

being transformed to:

a = p[1]
b = p[2]

But that doesn't seem to be possible because the unpack macro just get the 
expressions "n" and "p", so there's not much you can do there. 
It seems I need an unpack function first to create the correct expression:

@setvariables unpack(n,p)

Where unpack unpack(n,p) generate the expression "a b 1 2", any ideas how I 
can do that ?

[julia-users] Re: Packing and unpacking parameters

[Yuuki Soho]

Basically I want to define only once the name of my parameters, and never 
write them down in function's arguments or declare them in function's body,
such that I can add, remove, rename, reorder parameters without having to 
do anything, and still have all my parameters available in my functions 
body.

I was thinking of something like this: I define a list of variables name n 
= ["a","b","c"] in my main function
and then having a macro @unpack n p that would generate:

a = p[1]
b = p[2]
c = p[2]


Such that I can do something like :

function F1(n,p)

@unpack n p


return a*cos(b-c)

end 


Does that make any sense, or I am missing something obvious ? 

[julia-users] Packing and unpacking parameters

[Yuuki Soho]

It's a bit of a stupid question, but I don't really know how to deal with 
this efficiently.

So, in many application I have some model with parameters, and I want to 
the able to change the number of parameters, or they order easily.

For passing parameters to functions I want to pack them into a vector p, 
such that I don't have huge function definition, but inside

the function's body I'd prefer to have all the parameters given by their 
name, so I can use them in equations (instead of using p[1], p[2], ...).


I can write two functions p = pack(a,b,c) and (a,b,c) = unpack(p) but 
that's pretty restrictive because if you add or remove a parameters, I have 
to change all 

my function calls and definition. If I add another model I also need to 
write another pack and unpack pairs.


Is there an better approach to do this in Julia ? I was thinking maybe 
doing a macro @unpack p that would spawn all the variables needed, but I'm 
not

sure that's the right way to do it.