[julia-users] Re: Clustering with sklearn in Julia

[Vera Abaimova]

Ok, so the Solution to this was fairly simple actually. I imported 
sklearn.cluster as cluster, which seems to have fixed the problem. So now 
the code reads:

estimator = cluster.KMeans(n_clusters=k,init="k-means++")
estimator[:fit](dataTable)


On Wednesday, October 22, 2014 11:28:54 PM UTC-4, Vera Abaimova wrote:
>
> Hi, I'm trying to run a k-means clustering algorithm on my data using the 
> sklearn package. I'm using Julia so I'm using PyCall in order to have 
> access to all the sklearn functions. When I run my k-means I'm getting an 
> output of "cluster not defined". Any ideas on how to fix this?
>
> estimator = sklearn.cluster.KMeans(n_clusters=k,init="k-means++")
> estimator[:fit](dataTable)
>
> Output in IJulia:
> In [54]:
>
> evaluateVariousK()
>
> cluster not defined
> while loading In[54], in expression starting on line 1
>
>
>

Re: [julia-users] Re: 0.4 Roadmap for DataFrames, DataArrays, etc...

[John Myles White]

I’ll try to work on this in chunks starting soon, but there’s no way it’ll be 
finished before December.

 — John

On Oct 22, 2014, at 9:39 PM, Viral Shah  wrote:

> I would love to have this as soon as possible, and even work towards 
> parallelization. Perhaps if some first steps can be taken, I am sure more can 
> jump in and get this done.
> 
> Also, package help is the other big impediment here, and as soon as #8588 can 
> be merged - I think we can have something that a lot more people may be 
> comfortable using.
> 
> -viral
> 
> On Sunday, September 7, 2014 8:00:04 AM UTC+5:30, John Myles White wrote:
> I am hoping that the 0.4 release of Julia will coincide with a major cleanup 
> of the Data* world. I wrote up a very high level overview of my goals here: 
> https://gist.github.com/johnmyleswhite/ad5305ecaa9de01e317e 
> 
> There’s still more work to do to flesh out these ideas, but the basic 
> principles are pretty close to finalized. There’s also a rough draft of much 
> of the core functionality we’ll need to add to support this roadmap. 
> 
> I wanted to give everyone a heads up so that people understand where the 
> Data* packages are headed. The big idea is that we’ll be pushing more work 
> out into the type system, which will give substantial performance 
> improvements. 
> 
>  — John 
> 

[julia-users] Re: 0.4 Roadmap for DataFrames, DataArrays, etc...

[Viral Shah]

I would love to have this as soon as possible, and even work towards 
parallelization. Perhaps if some first steps can be taken, I am sure more 
can jump in and get this done.

Also, package help is the other big impediment here, and as soon as #8588 
can be merged - I think we can have something that a lot more people may be 
comfortable using.

-viral

On Sunday, September 7, 2014 8:00:04 AM UTC+5:30, John Myles White wrote:
>
> I am hoping that the 0.4 release of Julia will coincide with a major 
> cleanup of the Data* world. I wrote up a very high level overview of my 
> goals here: https://gist.github.com/johnmyleswhite/ad5305ecaa9de01e317e 
>
> There’s still more work to do to flesh out these ideas, but the basic 
> principles are pretty close to finalized. There’s also a rough draft of 
> much of the core functionality we’ll need to add to support this roadmap. 
>
> I wanted to give everyone a heads up so that people understand where the 
> Data* packages are headed. The big idea is that we’ll be pushing more work 
> out into the type system, which will give substantial performance 
> improvements. 
>
>  — John 
>
>

[julia-users] Clustering with sklearn in Julia

[Vera Abaimova]

Hi, I'm trying to run a k-means clustering algorithm on my data using the 
sklearn package. I'm using Julia so I'm using PyCall in order to have 
access to all the sklearn functions. When I run my k-means I'm getting an 
output of "cluster not defined". Any ideas on how to fix this?

estimator = sklearn.cluster.KMeans(n_clusters=k,init="k-means++")
estimator[:fit](dataTable)

Output in IJulia:
In [54]:

evaluateVariousK()

cluster not defined
while loading In[54], in expression starting on line 1

Re: [julia-users] Re: Geometry package

[Yakir Gagnon]

Woohoo!


Yakir Gagnon
The Queensland Brain Institute (Building #79)
The University of Queensland
Brisbane QLD 4072
Australia

cell +61 (0)424 393 332
work +61 (0)733 654 089

On Thu, Oct 23, 2014 at 7:18 AM, Ariel Keselman  wrote:

> maybe this can help:
>
> https://github.com/skariel/TriangleIntersect.jl
>
> it intersects rays with triangles in 3D...
>

Re: [julia-users] Terminating loops early based on user input

[Jameson Nash]

well, yes. on linux, “run(`kill -SIGUSR1 $(getpid())`)” is the way to get
the instant backtrace

window’s unfortunately has a limited number of signals available (SHUTDOWN,
LOGOFF, SIGKILL, SIGINT, and ctrl-break) which aren’t terribly helpful for
this

and just for fun, there’s an alternative implementation of John’s code:

begin
break_loop = @async while true # Pause logic
if read(stdin, Uint8) == 'p'
while true
@printf("Press 'c' to continue.\n")
@printf("Press 'a' to abort.\n")
input = read(stdin, Uint8)
if input == 'c'
break
elseif input == 'a'
return
end
end
end
end

for i in 1:100
# Original body of loop
println(i)
x -= 1
sleep(0.1)

# Pause logic
yield(); istaskdone(break_loop) && break
end
end

​

On Wed, Oct 22, 2014 at 9:47 PM, Pontus Stenetorp 
wrote:

> On 23 October 2014 01:31, Jameson Nash  wrote:
> >
> > As an extension of Steven's suggestion, you could use other signals too
> (eg
> > like how 'kill -SIGINFO' will give you a backtrace)
>
> SIGINFO is a lovely signal indeed.  For example, Ctrl+t when running
> cp gives you the status of the file currently being copied.
>
> Unfortunately it is a BSD-ism and will not work for Linux or Windows.
> I am not sure what one could use as a portable alternative, possibly
> SIGUSR1?
>
> Pontus
>

[julia-users] Re: Use of constant "pi" in rad2deg function causes convert error!

[Arch Call]

Thanks. I will probably just use 1pi when needed...Archie

On Wednesday, October 22, 2014 4:27:21 PM UTC-4, Arch Call wrote:
>
> Notice how the constant "pi" causes an error in rad2deg().
>
> I would think that this is a common use case for this function.
> Also notice how if it was multiplied by 1.0 and error did not happen.
>
> This is happening in in Version 0.3.1.
>
> ...Archie
>
>
> julia> pi
> π = 3.1415926535897...
>
> julia> rad2deg(pi)
> ERROR: `convert` has no method matching convert(::Type{MathConst{:π}}, 
> ::Float64
> )
>  in rad2deg at math.jl:81
>
> julia> rad2deg(1.0 * pi)
> 180.0
>

Re: [julia-users] Terminating loops early based on user input

[Pontus Stenetorp]

On 23 October 2014 01:31, Jameson Nash  wrote:
>
> As an extension of Steven's suggestion, you could use other signals too (eg
> like how 'kill -SIGINFO' will give you a backtrace)

SIGINFO is a lovely signal indeed.  For example, Ctrl+t when running
cp gives you the status of the file currently being copied.

Unfortunately it is a BSD-ism and will not work for Linux or Windows.
I am not sure what one could use as a portable alternative, possibly
SIGUSR1?

Pontus

[julia-users] Re: Gadfly plotting multiple lines with different colours

[Darwin Darakananda]

To specify colors with the layer approach, you can add a theme element to 
each layer:

colors = [color(c) for c in ["red", "blue", "green", ...]]
layers = Layer[]
for i=1:10
   push!(layers, layer(x=x, y=M[:,i], Geom.line, 
Theme(default_color=colors[i]))[1])
end

The only other way I'm aware of doing this is to convert the matrix to a 
DataFrame with an extra column to identify each line. So something like:

row,col = size(M)
data = vcat([DataFrame(x = x, M = M[:,i], line = i*ones(Int, row)) for i in 
1:col])
plot(data, x = :x, y = :M, color = :line, Geom.line)


On Tuesday, October 21, 2014 9:25:22 PM UTC-7, Dom Luna wrote:
>
> So I have a vector x and a matrix M.
>
> The vector x consists of the points on the x axis and the columns of M are 
> the respective y-cords for each line.
>
> I'm currently plotting all the lines using layers:
>
> layers = Layer[]
> for i=1:10
>push!(layers, layer(x=x, y=M[:,i], Geom.line)[1])
> end
>
> plot(layers)
>
> This gives me essentially what I want except all the lines are same 
> colour. What's the best way for Gadfly to uniquely colour each line?
>
> Also if the above layering approach isn't the best way to do what I'm 
> trying to do please let me know.
>
> Thanks.
>

Re: [julia-users] Use of constant "pi" in rad2deg function causes convert error!

[Isaiah Norton]

See https://github.com/JuliaLang/julia/issues/5561 and the other one linked
from there.

On Wed, Oct 22, 2014 at 4:27 PM, Arch Call  wrote:

> Notice how the constant "pi" causes an error in rad2deg().
>
> I would think that this is a common use case for this function.
> Also notice how if it was multiplied by 1.0 and error did not happen.
>
> This is happening in in Version 0.3.1.
>
> ...Archie
>
>
> julia> pi
> π = 3.1415926535897...
>
> julia> rad2deg(pi)
> ERROR: `convert` has no method matching convert(::Type{MathConst{:π}},
> ::Float64
> )
>  in rad2deg at math.jl:81
>
> julia> rad2deg(1.0 * pi)
> 180.0
>

Re: [julia-users] Embedding Julia in C on Windows - uv.h: No such file or directory

[Isaiah Norton]

>
> I guess so, except if you temporarily copy header files from here to a
> place where the compiler will find them (untested):


Or add `jlulia/deps/libuv/include` to your compiler include path.

On Tue, Oct 21, 2014 at 10:57 AM, Milan Bouchet-Valat 
wrote:

> Le mardi 21 octobre 2014 à 07:12 -0700, Stefan Babinec a écrit :
> > Hi Milan.
> >
> >
> > Thanks for reply.
> >
> >
> > Does it mean that until this is not fixed, I'm not able to use Julia's
> > "embeding c/c++ feature" ?
> I guess so, except if you temporarily copy header files from here to a
> place where the compiler will find them (untested):
> https://github.com/JuliaLang/libuv/tree/julia-uv0.11.26/include
>
> But it may get fixed soon -- and even sooner if you propose a patch
> implementing Jeff's suggestion from
> https://github.com/JuliaLang/julia/issues/8690#issuecomment-59272693
>
>
> Regards
>
>

[julia-users] Julia v0.3.2 released

[Elliot Saba]

Hello all!  The latest bugfix release of the 0.3.X Julia line has been
released.  Binaries are available from the usual place
, and as usual, please report all issues
to either the issue tracker , or
email this list.

As this is a bugfix release, there are no new big-item features to
announce, but if you are interested in the bugs fixed since 0.3.1, this
commit log 
should give you an idea of the effort put in by our team of backporters.

Happy Hacking,
-E

[julia-users] Re: Julia Integration in Paraview

[cdm]

the following links would suggest that paraview's "pvpython" application
could be reachable with pycall ...

   http://www.paraview.org/python/

   http://www.itk.org/Wiki/ParaView/Python_Scripting
   ( see "paraview.simple Module" section )


enjoy !!!

cdm

Re: [julia-users] -1^2

[John Myles White]

-1^2 == -(1^2)

 -- John

On Oct 22, 2014, at 2:39 PM, nikolai.mar...@icloud.com wrote:

> Could someone please explain to me how julia handles ^ operator. I'm confused 
> by this:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> $ julia
>_
>_   _ _(_)_ |  A fresh approach to technical computing
>   (_) | (_) (_)|  Documentation: http://docs.julialang.org
>_ _   _| |_  __ _   |  Type "help()" for help.
>   | | | | | | |/ _` |  |
>   | | |_| | | | (_| |  |  Version 0.3.1 (2014-09-21 21:30 UTC)
> 
> 
>  _/ |\__'_|_|_|\__'_|  |  
> |__/   |  x86_64-apple-darwin13.4.0
> 
> 
> julia> -1 ^ 2
> -1
> 
> 
> 
> 
> julia> (-1) ^ 2
> 1
> 
> 
> 
> 
> julia> -1 * -1
> 1
> 
> 

[julia-users] -1^2

[nikolai . markov]

Could someone please explain to me how julia handles ^ operator. I'm 
confused by this:









$ julia

   _

   _   _ _(_)_ |  A fresh approach to technical computing

  (_) | (_) (_)|  Documentation: http://docs.julialang.org

   _ _   _| |_  __ _   |  Type "help()" for help.

  | | | | | | |/ _` |  |

  | | |_| | | | (_| |  |  Version 0.3.1 (2014-09-21 21:30 UTC)

 _/ |\__'_|_|_|\__'_|  |  

|__/   |  x86_64-apple-darwin13.4.0



julia> -1 ^ 2

-1



julia> (-1) ^ 2

1



julia> -1 * -1

1

Re: [julia-users] Add new packages

[Markus Roth]

Seems that matplotlib was not installed. Installed it using "sudo apt-get 
install python-matplotlib". Sorry for wasting your time. Hope that this 
helps other beginners at their first installation of julia. Will start 
learning julia tomarrow (hopefully ;)).

Am Mittwoch, 22. Oktober 2014 23:10:53 UTC+2 schrieb Markus Roth:
>
>
> 
> Unfortunately got another error message (see screenshot) even though 
> PyPlot seems to be installed.
>
>  
>
> Am Mittwoch, 22. Oktober 2014 21:19:16 UTC+2 schrieb Elliot Saba:
>>
>> The nightlies should require `git` from now on.
>> -E
>>
>

Re: [julia-users] Add new packages

[Andreas Noack]

You probably don't have the Python package matplotlib installed. Please the
guide at https://github.com/stevengj/PyPlot.jl

Med venlig hilsen

Andreas Noack

2014-10-22 17:10 GMT-04:00 Markus Roth :

>
> 
> Unfortunately got another error message (see screenshot) even though
> PyPlot seems to be installed.
>
>
>
> Am Mittwoch, 22. Oktober 2014 21:19:16 UTC+2 schrieb Elliot Saba:
>>
>> The nightlies should require `git` from now on.
>> -E
>>
>

Re: [julia-users] Add new packages

[Elliot Saba]

PyPlot uses the python module "matplotlib".  You need to install that in
Python.

Something along the lines of "sudo pip install matplotlib", or "sudo
apt-get install python-matplotlib" should do the trick.
-E

Re: [julia-users] Re: Geometry package

[Ariel Keselman]

maybe this can help:

https://github.com/skariel/TriangleIntersect.jl

it intersects rays with triangles in 3D...

Re: [julia-users] Add new packages

[Markus Roth]

Unfortunately got another error message (see screenshot) even though PyPlot 
seems to be installed.

 

Am Mittwoch, 22. Oktober 2014 21:19:16 UTC+2 schrieb Elliot Saba:
>
> The nightlies should require `git` from now on.
> -E
>

[julia-users] Re: Bug in logdet?

[Adam Check]

Thank you both. Saving the matrix as Miles suggested fixes the 'issue' I 
was having.

On Saturday, October 18, 2014 7:03:41 PM UTC-7, Miles Lubin wrote:
>
> In 
>
> srand(1);a=rand(2,2);a=a*a';cholfact!(a);logdet(a)
>
> The issue is that cholfact!(a) destroys the data in a, you're not allowed 
> to re-use it afterwords. You still need to save the result of cholfact! 
> and use that object inside of logdet(), e.g.,
>
> srand(1);a=rand(2,2);a=a*a';afact = cholfact!(a);logdet(afact)
>
>
>
>
> On Saturday, October 18, 2014 7:47:59 PM UTC-4, Adam Check wrote:
>>
>> Hi,
>>
>> I am need to compute the cholesky decomposition of a matrix, and then use 
>> the log determinant of the decomposition. I get different answers depending 
>> on if I use cholfact vs. cholfact!
>>
>> The code below produces the same number for the logdet (-4.479)
>>
>> *srand(1);a=rand(2,2);a=a*a';logdet(cholfact(a))*
>>
>> *srand(1);a=rand(2,2);a=a*a';logdet(cholfact!(a))*
>>
>>
>> However, the following code results in different values being reported:
>>
>> *srand(1);a=rand(2,2);a=a*a';a=cholfact(a);logdet(a)*
>>
>> *srand(1);a=rand(2,2);a=a*a';cholfact!(a);logdet(a)*
>> with the second code reporting the (incorrect) value of -2.426, even 
>> though the Cholfact object "a" appears identical in each case.
>>
>> This error happens consistently when implementing cholfact!() as in the 
>> last line of code, and can result in an error when taking the logdet(): 
>> *ERROR: 
>> DomainError: determinant is negative*, even when no error results from 
>> using the cholfact() function.
>>
>> I am currently using julia version 0.3.1, but I believe the problem has 
>> been present in older versions as well.
>>
>> - Adam
>>
>

[julia-users] Re: Julia Integration in Paraview

[Viral Shah]

I have heard good things about paraview, and this does seem like a good 
match. However, not having ever used it myself, I do not have anything to 
add, except cheering from the sidelines!

-viral

On Wednesday, October 22, 2014 9:51:21 PM UTC+5:30, Brenda Make wrote:
>
> Hi All,
>
> I posted a suggestion, for integration of Julia in Paraview, but it could 
> use some votes/love.
>
> http://paraview.uservoice.com/forums/11350-general/suggestions/6580456-integrate-julia
>
> Paraview is a popular and important data 
> visualization program that's used for many scientific and technological 
> studies. It is often used for such things as Computation Fluid Dynamics, 
> Finite Element Analysts, Geology and other natural studies, as well as 
> Medical Imaging.
>
> Currently, Paraview is Python scriptable, but Julia would seem like a good 
> fit because of its speed and powerful matrix functions.
>
> Thank you for your consideration,
> --Brenda Make
>
>

[julia-users] Use of constant "pi" in rad2deg function causes convert error!

[Arch Call]

Notice how the constant "pi" causes an error in rad2deg().

I would think that this is a common use case for this function.
Also notice how if it was multiplied by 1.0 and error did not happen.

This is happening in in Version 0.3.1.

...Archie


julia> pi
π = 3.1415926535897...

julia> rad2deg(pi)
ERROR: `convert` has no method matching convert(::Type{MathConst{:π}}, 
::Float64
)
 in rad2deg at math.jl:81

julia> rad2deg(1.0 * pi)
180.0

[julia-users] Re: How can I get allocation information in Julia?

[Douglas Bates]

Those coming to Julia from languages like Python, R, Matlab/Octave, etc. 
have it ingrained in them to "fear the loop" and to perform loop-like 
calculations in "verbose and convoluted" ways.  It is actually quite easy 
to do this calculation in Julia with very little storage allocation by 
writing three nested loops.

function mindist(pos::Matrix{Float64}, Acp::Matrix{Float64})
m,n = size(pos)
r,s = size(Acp)
m == r || throw(DimensionMismatch(""))
res = fill(Inf,(n,))
for i in 1:n
for j in 1:s
sm = 0.
for k in 1:m
sm += abs2(pos[k,i] - Acp[k,j])
end
if sm < res[i]
res[i] = sm
end
end
end
res
end

On my machine timing the second execution (the first execution includes 
compile time) I get

julia> @time mindist(pos,Acp);
elapsed time: 0.002806999 seconds (744 bytes allocated)

[julia-users] Re: Julia Integration in Paraview

[Ivar Nesje]

It's kind of cheating if we as Julia people who has never heard of Paraview 
(or will ever use it) should vote on such a proposal.

I wish you the best of luck anyway. It's very nice to have people 
advocating Julia in various places.

kl. 18:21:21 UTC+2 onsdag 22. oktober 2014 skrev Brenda Make følgende:
>
> Hi All,
>
> I posted a suggestion, for integration of Julia in Paraview, but it could 
> use some votes/love.
>
> http://paraview.uservoice.com/forums/11350-general/suggestions/6580456-integrate-julia
>
> Paraview is a popular and important data 
> visualization program that's used for many scientific and technological 
> studies. It is often used for such things as Computation Fluid Dynamics, 
> Finite Element Analysts, Geology and other natural studies, as well as 
> Medical Imaging.
>
> Currently, Paraview is Python scriptable, but Julia would seem like a good 
> fit because of its speed and powerful matrix functions.
>
> Thank you for your consideration,
> --Brenda Make
>
>

Re: [julia-users] Add new packages

[Elliot Saba]

The nightlies should require `git` from now on.
-E

Re: [julia-users] Add new packages

[Jameson Nash]

Could git also just be specified as a requirement in the ppa?

On Wednesday, October 22, 2014, Elliot Saba  wrote:

> I think we should explicitly catch this.  I opened an issue
>  regarding this.
>
> On Wed, Oct 22, 2014 at 11:13 AM, Markus Roth  > wrote:
>
>> Just installed git via
>>
>> sudo apt-get install git-core
>> After that I was able to install PyPlot. Thank you very much! Maybe it is 
>> worth putting a note on git to your webiste (just for the less sophisticated 
>> users as I am ;))
>>
>>
>> Am Mittwoch, 22. Oktober 2014 19:53:18 UTC+2 schrieb Stefan Karpinski:
>>>
>>> Does git exist?
>>>
>>> On Wed, Oct 22, 2014 at 1:51 PM, Markus Roth  wrote:
>>>

 
 Just tried your suggestion. Got another, probably similar, error
 message (see attached).



 Am Mittwoch, 22. Oktober 2014 19:28:44 UTC+2 schrieb Jake Bolewski:

> run Pkg.init() first if the metadata does not exist.
>
> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>
>> Hey advanced julia users,
>>
>> I am new to the julia programming language and just started to learn
>> it. I just set up a new Linux Mint installation and installed julia by
>> adding the ubuntu repository as suggested on the website. As a first 
>> step I
>> wanted to install PyPlot by typing "Pkg.add("PyPlot")". I get an error
>> message (see attached file). Can you help me with my problem because I am
>> really interested in learning the language.
>>
>> Best,
>> Markus
>>
>>
>>
>> 
>>
>
>>>
>

Re: [julia-users] Re: Add new packages

[Elliot Saba]

I think we should explicitly catch this.  I opened an issue
 regarding this.

On Wed, Oct 22, 2014 at 11:13 AM, Markus Roth  wrote:

> Just installed git via
>
> sudo apt-get install git-core
> After that I was able to install PyPlot. Thank you very much! Maybe it is 
> worth putting a note on git to your webiste (just for the less sophisticated 
> users as I am ;))
>
>
> Am Mittwoch, 22. Oktober 2014 19:53:18 UTC+2 schrieb Stefan Karpinski:
>>
>> Does git exist?
>>
>> On Wed, Oct 22, 2014 at 1:51 PM, Markus Roth  wrote:
>>
>>>
>>> 
>>> Just tried your suggestion. Got another, probably similar, error message
>>> (see attached).
>>>
>>>
>>>
>>> Am Mittwoch, 22. Oktober 2014 19:28:44 UTC+2 schrieb Jake Bolewski:
>>>
 run Pkg.init() first if the metadata does not exist.

 On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>
> Hey advanced julia users,
>
> I am new to the julia programming language and just started to learn
> it. I just set up a new Linux Mint installation and installed julia by
> adding the ubuntu repository as suggested on the website. As a first step 
> I
> wanted to install PyPlot by typing "Pkg.add("PyPlot")". I get an error
> message (see attached file). Can you help me with my problem because I am
> really interested in learning the language.
>
> Best,
> Markus
>
>
>
> 
>

>>

Re: [julia-users] join(("1"," ")) removes the space in the end

[Stefan Karpinski]

Right, by default, the "joiner" string is the empty string.

On Wed, Oct 22, 2014 at 2:13 PM, Daniel Høegh  wrote:

> Sorry my mistake I should probably stop programing for to night:)
> Den onsdag den 22. oktober 2014 20.07.39 UTC+2 skrev John Myles White:
>>
>> I suspect Daniel may have thought that join adds "joining" spaces by
>> default:
>>
>> julia> length(join(("1"," ")))
>> 2
>>
>> julia> length(join(("1"," "), " "))
>> 3
>>
>>  -- John
>>
>> On Oct 22, 2014, at 11:05 AM, Stefan Karpinski 
>> wrote:
>>
>> > The space shouldn't be and isn't removed:
>> >
>> > julia> join(("1"," "))
>> > "1 "
>> >
>> > This seems like something that would not be different in any version of
>> Julia.
>> >
>> > On Wed, Oct 22, 2014 at 2:00 PM, Daniel Høegh 
>> wrote:
>> > Hi is there a way to concatenate turple of strings that if the last
>> string is a space that does not remove it?
>> > join(("1"," "))=="1 "
>> >
>> >
>> >
>>
>>

Re: [julia-users] join(("1"," ")) removes the space in the end

[Daniel Høegh]

Sorry my mistake I should probably stop programing for to night:)
Den onsdag den 22. oktober 2014 20.07.39 UTC+2 skrev John Myles White:
>
> I suspect Daniel may have thought that join adds "joining" spaces by 
> default: 
>
> julia> length(join(("1"," "))) 
> 2 
>
> julia> length(join(("1"," "), " ")) 
> 3 
>
>  -- John 
>
> On Oct 22, 2014, at 11:05 AM, Stefan Karpinski  > wrote: 
>
> > The space shouldn't be and isn't removed: 
> > 
> > julia> join(("1"," ")) 
> > "1 " 
> > 
> > This seems like something that would not be different in any version of 
> Julia. 
> > 
> > On Wed, Oct 22, 2014 at 2:00 PM, Daniel Høegh  > wrote: 
> > Hi is there a way to concatenate turple of strings that if the last 
> string is a space that does not remove it? 
> > join(("1"," "))=="1 " 
> >   
> > 
> > 
>
>

Re: [julia-users] Re: Add new packages

[Markus Roth]

Just installed git via 

sudo apt-get install git-core
After that I was able to install PyPlot. Thank you very much! Maybe it is worth 
putting a note on git to your webiste (just for the less sophisticated users as 
I am ;)) 


Am Mittwoch, 22. Oktober 2014 19:53:18 UTC+2 schrieb Stefan Karpinski:
>
> Does git exist?
>
> On Wed, Oct 22, 2014 at 1:51 PM, Markus Roth  > wrote:
>
>>
>> 
>> Just tried your suggestion. Got another, probably similar, error message 
>> (see attached).
>>
>>
>>
>> Am Mittwoch, 22. Oktober 2014 19:28:44 UTC+2 schrieb Jake Bolewski:
>>
>>> run Pkg.init() first if the metadata does not exist.
>>>
>>> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:

 Hey advanced julia users,

 I am new to the julia programming language and just started to learn 
 it. I just set up a new Linux Mint installation and installed julia by 
 adding the ubuntu repository as suggested on the website. As a first step 
 I 
 wanted to install PyPlot by typing "Pkg.add("PyPlot")". I get an error 
 message (see attached file). Can you help me with my problem because I am 
 really interested in learning the language.

 Best,
 Markus
  


 

>>>
>

Re: [julia-users] Julia Elliptic Curve Arithmetic slower than Python

[alexander maznev]

I appreciate the feedback I'm still just getting acquainted with Julia as 
you can tell, I'll post an updated version with some benchmarks in a bit.
One thing I'm not sure about is if this is the correct Method 
Parametrization syntax - it seems to work though...
  function +{T}(point_a::PointEC{T}, point_b::PointEC{T})
>Using dispatch to express some of the logic may seem appealing but I 
suspect it's not a good idea unless the result type only depends on the 
input types 
Do you mean the PointINF related logic? Python uses a Point(None, None, 
None, None) of the same class as the other points, one alternative would be 
a PointEC{Void} in that case the logic could go into the functions, 
although that might actually be slower than a unique PointINF type? 
>assert
It was just seemed quicker to write assert than to write proper error 
messages, that's obviously something that should be done if I were to port 
the entire library.

Still wondering about Mod seeming to run nearly 10x slower in Julia than 
Python. 

def test(n=10**7):
t= time.time()
p = 6277101735386680763835789423207666416083908700390324961279
Gx = 0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012
for i in range(n): 
(Gx+i) % p
return (time.time() - t)/ n

function test(n=10^7)
  t= time()
  Gx = 0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012
  p = 6277101735386680763835789423207666416083908700390324961279
  for i=1:n
mod((Gx+i), n)
  end
  return (time() -t)/n
end





On Wednesday, October 22, 2014 10:47:10 AM UTC-4, Stefan Karpinski wrote:
>
> This is what I was saying:
>
> immutable CurveFP{T<:Number}
> p::T
> a::T
> b::T
> function CurveFP(p::T, a::T, b::T)
> mod(4a^3 + 27b^2, p) != 0 || error("4*$a^3 + 27*$b^2 ≡ 0 mod $p")
> new(p, a, b)
> end
> end
> CurveFP{T<:Number}(p::T, a::T, b::T) = CurveFP{T}(p, a, b)
> CurveFP(p::Number, a::Number, b::Number) = CurveFP(promote(p, a, b)...)
>
> This allows specilized code be generated for CurveFP{T} for each T. The 
> version where the fields are abstractly typed defines a single type and 
> code that deals with that type has to generically deal with any possible 
> type of Number value for each field.
>
> Also note that the way you were using assert is not quite right: assert 
> should only be used to check internal correctness of your own code, not to 
> check that users of your code are doing the right thing. If you remove all 
> asserts, your code should continue to work identically from the user's 
> perspective.
>
>
> On Wed, Oct 22, 2014 at 10:14 AM, Erik Schnetter  > wrote:
>
>> What Stefan is probably trying to say:
>>
>> Instead of a function f(x, y) where x and y can have different types,
>> write f{T}(x::T, y::T), so that x and y are forced to have the same
>> type -- but this type can be any type. In this way, e.g. x+y can be
>> evaluated very efficiently, and without type conversion, independent
>> of whether x is Int or BigInt.
>>
>> -erik
>>
>>
>> On Tue, Oct 21, 2014 at 8:36 PM, Stefan Karpinski
>> > wrote:
>> > Oh yeah, those fields having abstract type is a doozy - you'll want to
>> > introduce a type parameter for that type and make all of the field of 
>> that
>> > type.
>> >
>> > On Oct 21, 2014, at 7:34 PM, alexander maznev > >
>> > wrote:
>> >
>> > I had issues with how Julia does not seem to do type coarsing even when 
>> a
>> > function will only take arguments of that one type. I.e. point_a * 10 
>> will
>> > fail because it expects a BigInt but receives an Int64, which i guess is
>> > solved by wrapping every single number passed around in the BigInt 
>> class, or
>> > duplicating all the methods. At any rate I tested it with BigInt 
>> instead of
>> > Number and the run times do not change much.
>> > Also I introduced a type when replacing divmod with divrem, it should 
>> be.
>> > (q,c),d = divrem(d,c), c
>> >
>> >
>> >
>> > On Tuesday, October 21, 2014 7:19:10 PM UTC-4, Kevin Squire wrote:
>> >>
>> >> One problem is that you're using an abstract type (Number) for all of 
>> the
>> >> variable members if your types.
>> >>
>> >> In function declarations, this is okay, because the function is
>> >> specialized on the concrete number type. But for types, abstractly 
>> typed
>> >> members are boxed (stored as pointers), because the exact type is not 
>> given
>> >> in the definition.
>> >>
>> >> You can get close to the same flexibility of the current code by using
>> >> parametric types, which should erase any performance gap.
>> >>
>> >> Cheers,
>> >>   Kevin
>> >>
>> >> On Tuesday, October 21, 2014, alexander maznev 
>> >> wrote:
>> >>>
>> >>> This should be an equivalent, or nearly there, implementation of 
>> Elliptic
>> >>> Curves mod p as found in the Python ecdsa library.
>> >>>
>> >>> https://gist.github.com/anonymous/a3799a5a2b0354022eac
>> >>>
>> >>> Noticeably, regular mod is 10x slower than python?
>> >>> Inverse_mod is 7x slower than python.
>> >>> Double is 7x slower than python
>> >

Re: [julia-users] join(("1"," ")) removes the space in the end

[John Myles White]

I suspect Daniel may have thought that join adds "joining" spaces by default:

julia> length(join(("1"," ")))
2

julia> length(join(("1"," "), " "))
3

 -- John

On Oct 22, 2014, at 11:05 AM, Stefan Karpinski  wrote:

> The space shouldn't be and isn't removed:
> 
> julia> join(("1"," "))
> "1 "
> 
> This seems like something that would not be different in any version of Julia.
> 
> On Wed, Oct 22, 2014 at 2:00 PM, Daniel Høegh  wrote:
> Hi is there a way to concatenate turple of strings that if the last string is 
> a space that does not remove it?
> join(("1"," "))=="1 "
>  
> 
> 

Re: [julia-users] join(("1"," ")) removes the space in the end

[Stefan Karpinski]

The space shouldn't be and isn't removed:

julia> join(("1"," "))
"1 "


This seems like something that would not be different in any version of
Julia.

On Wed, Oct 22, 2014 at 2:00 PM, Daniel Høegh  wrote:

> Hi is there a way to concatenate turple of strings that if the last string
> is a space that does not remove it?
>
> join(("1"," "))=="1 "
>
>
>

[julia-users] join(("1"," ")) removes the space in the end

[Daniel Høegh]

Hi is there a way to concatenate turple of strings that if the last string 
is a space that does not remove it?

join(("1"," "))=="1 "

 

Re: [julia-users] Re: Add new packages

[Stefan Karpinski]

Does git exist?

On Wed, Oct 22, 2014 at 1:51 PM, Markus Roth  wrote:

>
> 
> Just tried your suggestion. Got another, probably similar, error message
> (see attached).
>
>
>
> Am Mittwoch, 22. Oktober 2014 19:28:44 UTC+2 schrieb Jake Bolewski:
>
>> run Pkg.init() first if the metadata does not exist.
>>
>> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>>
>>> Hey advanced julia users,
>>>
>>> I am new to the julia programming language and just started to learn it.
>>> I just set up a new Linux Mint installation and installed julia by adding
>>> the ubuntu repository as suggested on the website. As a first step I wanted
>>> to install PyPlot by typing "Pkg.add("PyPlot")". I get an error message
>>> (see attached file). Can you help me with my problem because I am really
>>> interested in learning the language.
>>>
>>> Best,
>>> Markus
>>>
>>>
>>>
>>> 
>>>
>>

[julia-users] Re: Add new packages

[Markus Roth]

Just tried your suggestion. Got another, probably similar, error message 
(see attached).



Am Mittwoch, 22. Oktober 2014 19:28:44 UTC+2 schrieb Jake Bolewski:
>
> run Pkg.init() first if the metadata does not exist.
>
> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>
>> Hey advanced julia users,
>>
>> I am new to the julia programming language and just started to learn it. 
>> I just set up a new Linux Mint installation and installed julia by adding 
>> the ubuntu repository as suggested on the website. As a first step I wanted 
>> to install PyPlot by typing "Pkg.add("PyPlot")". I get an error message 
>> (see attached file). Can you help me with my problem because I am really 
>> interested in learning the language.
>>
>> Best,
>> Markus
>>  
>>
>>
>> 
>>
>

Re: [julia-users] Re: Add new packages

[Markus Roth]

The folder /home/markus/.julia is empty. Can I fix this somehow?

Am Mittwoch, 22. Oktober 2014 19:30:43 UTC+2 schrieb Stefan Karpinski:
>
> Although that should be done for you. Does /home/markus/.julia/v0.3 exist? 
> If so, what's in it?
>
> On Wed, Oct 22, 2014 at 1:28 PM, Jake Bolewski  > wrote:
>
>> run Pkg.init() first if the metadata does not exist.
>>
>>
>> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>>
>>> Hey advanced julia users,
>>>
>>> I am new to the julia programming language and just started to learn it. 
>>> I just set up a new Linux Mint installation and installed julia by adding 
>>> the ubuntu repository as suggested on the website. As a first step I wanted 
>>> to install PyPlot by typing "Pkg.add("PyPlot")". I get an error message 
>>> (see attached file). Can you help me with my problem because I am really 
>>> interested in learning the language.
>>>
>>> Best,
>>> Markus
>>>  
>>>
>>>
>>> 
>>>
>>
>

Re: [julia-users] Re: Add new packages

[Stefan Karpinski]

Ah, yes. Not having git could cause that error too.

On Wed, Oct 22, 2014 at 1:34 PM, Keno Fischer 
wrote:

> Also is git installed.
>
> On Wed, Oct 22, 2014 at 1:30 PM, Stefan Karpinski 
> wrote:
>
>> Although that should be done for you. Does /home/markus/.julia/v0.3
>> exist? If so, what's in it?
>>
>> On Wed, Oct 22, 2014 at 1:28 PM, Jake Bolewski 
>> wrote:
>>
>>> run Pkg.init() first if the metadata does not exist.
>>>
>>>
>>> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:

 Hey advanced julia users,

 I am new to the julia programming language and just started to learn
 it. I just set up a new Linux Mint installation and installed julia by
 adding the ubuntu repository as suggested on the website. As a first step I
 wanted to install PyPlot by typing "Pkg.add("PyPlot")". I get an error
 message (see attached file). Can you help me with my problem because I am
 really interested in learning the language.

 Best,
 Markus



 

>>>
>>
>

Re: [julia-users] Re: Add new packages

[Keno Fischer]

Also is git installed.

On Wed, Oct 22, 2014 at 1:30 PM, Stefan Karpinski 
wrote:

> Although that should be done for you. Does /home/markus/.julia/v0.3 exist?
> If so, what's in it?
>
> On Wed, Oct 22, 2014 at 1:28 PM, Jake Bolewski 
> wrote:
>
>> run Pkg.init() first if the metadata does not exist.
>>
>>
>> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>>
>>> Hey advanced julia users,
>>>
>>> I am new to the julia programming language and just started to learn it.
>>> I just set up a new Linux Mint installation and installed julia by adding
>>> the ubuntu repository as suggested on the website. As a first step I wanted
>>> to install PyPlot by typing "Pkg.add("PyPlot")". I get an error message
>>> (see attached file). Can you help me with my problem because I am really
>>> interested in learning the language.
>>>
>>> Best,
>>> Markus
>>>
>>>
>>>
>>> 
>>>
>>
>

Re: [julia-users] Re: Add new packages

[Stefan Karpinski]

Although that should be done for you. Does /home/markus/.julia/v0.3 exist?
If so, what's in it?

On Wed, Oct 22, 2014 at 1:28 PM, Jake Bolewski 
wrote:

> run Pkg.init() first if the metadata does not exist.
>
>
> On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>>
>> Hey advanced julia users,
>>
>> I am new to the julia programming language and just started to learn it.
>> I just set up a new Linux Mint installation and installed julia by adding
>> the ubuntu repository as suggested on the website. As a first step I wanted
>> to install PyPlot by typing "Pkg.add("PyPlot")". I get an error message
>> (see attached file). Can you help me with my problem because I am really
>> interested in learning the language.
>>
>> Best,
>> Markus
>>
>>
>>
>> 
>>
>

[julia-users] Re: Add new packages

[Jake Bolewski]

run Pkg.init() first if the metadata does not exist.

On Wednesday, October 22, 2014 1:20:42 PM UTC-4, Markus Roth wrote:
>
> Hey advanced julia users,
>
> I am new to the julia programming language and just started to learn it. I 
> just set up a new Linux Mint installation and installed julia by adding the 
> ubuntu repository as suggested on the website. As a first step I wanted to 
> install PyPlot by typing "Pkg.add("PyPlot")". I get an error message (see 
> attached file). Can you help me with my problem because I am really 
> interested in learning the language.
>
> Best,
> Markus
>  
>
>
> 
>

[julia-users] Add new packages

[Markus Roth]

Hey advanced julia users,

I am new to the julia programming language and just started to learn it. I 
just set up a new Linux Mint installation and installed julia by adding the 
ubuntu repository as suggested on the website. As a first step I wanted to 
install PyPlot by typing "Pkg.add("PyPlot")". I get an error message (see 
attached file). Can you help me with my problem because I am really 
interested in learning the language.

Best,
Markus
 

Re: [julia-users] Re: How can I get allocation information in Julia?

[Iain Dunning]

Yes, check the link Tim Holy sent - its a command line option.

On Wednesday, October 22, 2014 12:27:32 PM UTC-4, accou...@gmail.com wrote:
>
> This works for me: 
>>
>> julia> function mindists_sq(pos, dists_min, Acp) 
>> 
>>for i in 1:size(pos, 2) 
>>dists_min[i] = Inf 
>>for j in 1:size(Acp, 2) 
>>t = 0.0 
>>for k=1:size(pos,1) 
>>t += (pos[k, i]-Acp[k, j])^2 
>>end 
>>if t < dists_min[i] 
>>dists_min[i] = t 
>>end 
>>end 
>>end 
>>return dists_min 
>>end 
>> mindists_sq (generic function with 1 method) 
>>
>> julia> function test() 
>>const pos = rand(3, 64) 
>>const Acp = rand(3, 1200) 
>>const dists_min = zeros(64) 
>>const tmp = zeros(typeof(Acp[1]), 1) 
>>@time mindists_sq(pos, dists_min, Acp) 
>>end 
>> test (generic function with 1 method) 
>>
>> julia> test(); 
>> elapsed time: 0.001279041 seconds (0 bytes allocated) 
>>
>> Is this how you unrolled the innermost loop too? 
>>
>
> No, I've been even further:
> t += pos[k, i] 
> t -= Acp[k, j]
> t ^= 2 
>
> But it does not change anything (no allocations :D). Thank you. 
>
> I guess t in this case is allocated on the stack. I'm surprised that if I 
> allocate t on the heap I get such a large performance penalty (10-fold slow 
> down). I'm also surprised that t[:] += pos[k, i] allocates memory (and 
> incurs an even larger performance penalty). I really thought this was the 
> recommended way to perform inplace operations on arrays.
>
> So, is there any tool like Julia profiler but for memory allocation?
>
>

Re: [julia-users] Re: Learning to Type?

[John Myles White]

Yup. Maybe another way to say it: the function system can do so much ad hoc 
specialization precisely because the type system provides enough invariants for 
the compiler to depend upon.

  -- John

On Oct 22, 2014, at 9:37 AM, Stefan Karpinski  wrote:

> Right, the question is essentially whether you can implicitly introduce type 
> parameters for the programmer – which you can for immutable types but not for 
> mutable types. This example may make the difference clearer:
> 
> type Foo1
>   field::Number
> end
> 
> type Foo2{T<:Number}
>   field::T
> end
> 
> These types are pretty similar, but there's an important behavioral 
> difference:
> 
> julia> foo1 = Foo1(123)
> Foo1(123)
> 
> julia> foo2 = Foo2(123)
> Foo2{Int64}(123)
> 
> julia> foo1.field = 1.23
> 1.23
> 
> julia> foo1
> Foo1(1.23)
> 
> julia> foo2.field = 1.23
> ERROR: InexactError()
>  in convert at int.jl:185
> 
> Since the type of `.field` is encoded in the type of Foo2, you can change the 
> value of `.field` but not it's type. For Foo1, you can change the value and 
> the type of `.field.` Thus, if you have code that's specialized for a 
> particular kind of Foo2, it is specialized for the type of `.field` as well – 
> and in particular, that type cannot change over the course of the function. 
> If you've got code that's specialized for Foo1, you don't know anything about 
> the type of `.field` and moreover that type can change through the course of 
> a single function body. For immutables, of course, the value of a field can't 
> change, nevermind its type, so you could implicitly make the type of any 
> field part of the type.
> 
> On Wed, Oct 22, 2014 at 12:30 PM, John Myles White  
> wrote:
> The clean way to get that effect is via parametric types.
> 
>  -- John
> 
> On Oct 22, 2014, at 9:08 AM, Cedric St-Jean  
> wrote:
> 
>> Somewhat OT: since the JIT compiles a different function for each argument 
>> type combination, couldn't it compile a different type object for every 
>> field type combination?
>> 
>> On Wednesday, October 22, 2014 10:53:26 AM UTC-4, Iain Dunning wrote:
>> Do you feel like you are over-typing things?
>> 
>> In function definitions, you only need types if you actually need to them to 
>> pick a function to dispatch to. They don't make anything go faster, as a 
>> specific version of a function will be compiled for every different set of 
>> argument types you try calling it with.
>> 
>> So, check out:
>> https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
>> for example.
>> layout_spring_adj{T}(adj_matrix::Array{T,2}
>> So it takes any array of element type T. For every time you call this with a 
>> different element type, a different version of the function will be 
>> compiled. So this fancy looking code:
>> if adj_matrix[i,j] != zero(eltype(adj_matrix))
>> Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or  
>> and so on, pretty magic!
>> 
>> For Types, you should specify concrete types whenever you can, or use 
>> parametric types if you can't. If you don't, and use say, Number, or leave 
>> it blank, simple things like Float64s and Ints will be "boxed", so there 
>> will be overhead whenever you use them and the compiler may struggle to 
>> infer that some things are Float64s or Ints and not just Anys.
>> 
>> Hope that helps somewhat.
>> 
>> On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
>> Hi Andreas,
>> 
>> Don't know if you've seen this already and whether this is helpful at all, 
>> but I discovered this site once when I got confused by some TypeErrors, it 
>> basically lays out the entire type tree of Julia.
>> 
>> Best,
>> Nils
>> 
>> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:
>> Hello,
>> 
>> i'm now a julia user for some time, but still i have problems (comming from 
>> a c, f77, matlab, python background) to understand the type system. 
>> Especially when i try to integrate own code into packages (happened to Cairo 
>> + Compose, pending for Winston) i seem to choose always a more complicated 
>> way (my observation, code similar in py or c just looks shorter...) because 
>> somehow things do not match. 
>> 
>> So.
>> 
>> What to read to understand the julia approach to types and type interference?
>> 
>> Wishing a happy day,
>>Andreas
>> 
>> 
> 
> 

Re: [julia-users] Re: Learning to Type?

[Stefan Karpinski]

Right, the question is essentially whether you can implicitly introduce
type parameters for the programmer – which you can for immutable types but
not for mutable types. This example may make the difference clearer:

type Foo1
  field::Number
end

type Foo2{T<:Number}
  field::T
end


These types are pretty similar, but there's an important behavioral
difference:

julia> foo1 = Foo1(123)
Foo1(123)

julia> foo2 = Foo2(123)
Foo2{Int64}(123)

julia> foo1.field = 1.23
1.23

julia> foo1
Foo1(1.23)

julia> foo2.field = 1.23
ERROR: InexactError()
 in convert at int.jl:185


Since the type of `.field` is encoded in the type of Foo2, you can change
the value of `.field` but not it's type. For Foo1, you can change the value
and the type of `.field.` Thus, if you have code that's specialized for a
particular kind of Foo2, it is specialized for the type of `.field` as well
– and in particular, that type cannot change over the course of the
function. If you've got code that's specialized for Foo1, you don't know
anything about the type of `.field` and moreover that type can change
through the course of a single function body. For immutables, of course,
the value of a field can't change, nevermind its type, so you could
implicitly make the type of any field part of the type.

On Wed, Oct 22, 2014 at 12:30 PM, John Myles White  wrote:

> The clean way to get that effect is via parametric types.
>
>  -- John
>
> On Oct 22, 2014, at 9:08 AM, Cedric St-Jean 
> wrote:
>
> Somewhat OT: since the JIT compiles a different function for each argument
> type combination, couldn't it compile a different type object for every
> field type combination?
>
> On Wednesday, October 22, 2014 10:53:26 AM UTC-4, Iain Dunning wrote:
>>
>> Do you feel like you are over-typing things?
>>
>> In function definitions, you only need types if you actually need to them
>> to pick a function to dispatch to. They don't make anything go faster, as a
>> specific version of a function will be compiled for every different set of
>> argument types you try calling it with.
>>
>> So, check out:
>> https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
>> for example.
>> layout_spring_adj{T}(adj_matrix::Array{T,2}
>> So it takes any array of element type T. For every time you call this
>> with a different element type, a different version of the function will be
>> compiled. So this fancy looking code:
>> if adj_matrix[i,j] != zero(eltype(adj_matrix))
>> Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or 
>> and so on, pretty magic!
>>
>> For Types, you should specify concrete types whenever you can, or use
>> parametric types if you can't. If you don't, and use say, Number, or leave
>> it blank, simple things like Float64s and Ints will be "boxed", so there
>> will be overhead whenever you use them and the compiler may struggle to
>> infer that some things are Float64s or Ints and not just Anys.
>>
>> Hope that helps somewhat.
>>
>> On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
>>>
>>> Hi Andreas,
>>>
>>> Don't know if you've seen this already and whether this is helpful at
>>> all, but I discovered this site
>>> 
>>> once when I got confused by some TypeErrors, it basically lays out the
>>> entire type tree of Julia.
>>>
>>> Best,
>>> Nils
>>>
>>> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:

 Hello,

 i'm now a julia user for some time, but still i have problems (comming
 from a c, f77, matlab, python background) to understand the type system.
 Especially when i try to integrate own code into packages (happened to
 Cairo + Compose, pending for Winston) i seem to choose always a more
 complicated way (my observation, code similar in py or c just looks
 shorter...) because somehow things do not match.

 So.

 What to read to understand the julia approach to types and type
 interference?

 Wishing a happy day,
Andreas



>

Re: [julia-users] Terminating loops early based on user input

[Jameson Nash]

Note too that instead of readavailable, it seems like you want read(io,
Uint8)

As an extension of Steven's suggestion, you could use other signals too (eg
like how 'kill -SIGINFO' will give you a backtrace)

On Wednesday, October 22, 2014, Steven G. Johnson 
wrote:

> Couldn't you use InterruptExceptions here (ctrl-c handlers), disabling
> sigint during the loop body so that the exception only occurs when you are
> ready?
>
> for i in iterations
>try
>   disable_sigint() do
> do an iteration
>   end
>catch e
>   if isa(e, InterruptException)
>...exit gracefully
>   else
>  rethrow(e)
>   end
>end
> end
>
> ?   This kind of thing has a chance to work in an environment like IJulia
> too, whereas reading stdin will never work in a browser interface.
>
> Unfortunately, the above code may still have a race condition: if the user
> hits ctrl-c and the SIGINT signal is received in the loop body but before
> try...end then it won't be caught by the catch bock.   One way to hack
> around this would be to put the disable_sigint() outside the loop, and then
> check the C global jl_signal_pending != 0 inside the loop body to see if
> a SIGINT was received (but not handled yet).
>

Re: [julia-users] Re: Terminating loops early based on user input

[John Myles White]

I didn't know that disable_sigint existed. I'll experiment with it today.

 -- John

On Oct 22, 2014, at 9:09 AM, Steven G. Johnson  wrote:

> Couldn't you use InterruptExceptions here (ctrl-c handlers), disabling sigint 
> during the loop body so that the exception only occurs when you are ready?
> 
> for i in iterations
>try
>   disable_sigint() do
> do an iteration
>   end
>catch e
>   if isa(e, InterruptException)
>...exit gracefully
>   else
>  rethrow(e)
>   end
>end
> end
> 
> ?   This kind of thing has a chance to work in an environment like IJulia 
> too, whereas reading stdin will never work in a browser interface.
> 
> Unfortunately, the above code may still have a race condition: if the user 
> hits ctrl-c and the SIGINT signal is received in the loop body but before 
> try...end then it won't be caught by the catch bock.   One way to hack around 
> this would be to put the disable_sigint() outside the loop, and then check 
> the C global jl_signal_pending != 0 inside the loop body to see if a SIGINT 
> was received (but not handled yet).

Re: [julia-users] Re: Learning to Type?

[John Myles White]

The clean way to get that effect is via parametric types.

 -- John

On Oct 22, 2014, at 9:08 AM, Cedric St-Jean  wrote:

> Somewhat OT: since the JIT compiles a different function for each argument 
> type combination, couldn't it compile a different type object for every field 
> type combination?
> 
> On Wednesday, October 22, 2014 10:53:26 AM UTC-4, Iain Dunning wrote:
> Do you feel like you are over-typing things?
> 
> In function definitions, you only need types if you actually need to them to 
> pick a function to dispatch to. They don't make anything go faster, as a 
> specific version of a function will be compiled for every different set of 
> argument types you try calling it with.
> 
> So, check out:
> https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
> for example.
> layout_spring_adj{T}(adj_matrix::Array{T,2}
> So it takes any array of element type T. For every time you call this with a 
> different element type, a different version of the function will be compiled. 
> So this fancy looking code:
> if adj_matrix[i,j] != zero(eltype(adj_matrix))
> Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or  and 
> so on, pretty magic!
> 
> For Types, you should specify concrete types whenever you can, or use 
> parametric types if you can't. If you don't, and use say, Number, or leave it 
> blank, simple things like Float64s and Ints will be "boxed", so there will be 
> overhead whenever you use them and the compiler may struggle to infer that 
> some things are Float64s or Ints and not just Anys.
> 
> Hope that helps somewhat.
> 
> On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
> Hi Andreas,
> 
> Don't know if you've seen this already and whether this is helpful at all, 
> but I discovered this site once when I got confused by some TypeErrors, it 
> basically lays out the entire type tree of Julia.
> 
> Best,
> Nils
> 
> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:
> Hello,
> 
> i'm now a julia user for some time, but still i have problems (comming from a 
> c, f77, matlab, python background) to understand the type system. Especially 
> when i try to integrate own code into packages (happened to Cairo + Compose, 
> pending for Winston) i seem to choose always a more complicated way (my 
> observation, code similar in py or c just looks shorter...) because somehow 
> things do not match. 
> 
> So.
> 
> What to read to understand the julia approach to types and type interference?
> 
> Wishing a happy day,
>Andreas
> 
> 

Re: [julia-users] Re: How can I get allocation information in Julia?

[accoujulia]

>
> This works for me: 
>
> julia> function mindists_sq(pos, dists_min, Acp) 
> 
>for i in 1:size(pos, 2) 
>dists_min[i] = Inf 
>for j in 1:size(Acp, 2) 
>t = 0.0 
>for k=1:size(pos,1) 
>t += (pos[k, i]-Acp[k, j])^2 
>end 
>if t < dists_min[i] 
>dists_min[i] = t 
>end 
>end 
>end 
>return dists_min 
>end 
> mindists_sq (generic function with 1 method) 
>
> julia> function test() 
>const pos = rand(3, 64) 
>const Acp = rand(3, 1200) 
>const dists_min = zeros(64) 
>const tmp = zeros(typeof(Acp[1]), 1) 
>@time mindists_sq(pos, dists_min, Acp) 
>end 
> test (generic function with 1 method) 
>
> julia> test(); 
> elapsed time: 0.001279041 seconds (0 bytes allocated) 
>
> Is this how you unrolled the innermost loop too? 
>

No, I've been even further:
t += pos[k, i] 
t -= Acp[k, j]
t ^= 2 

But it does not change anything (no allocations :D). Thank you. 

I guess t in this case is allocated on the stack. I'm surprised that if I 
allocate t on the heap I get such a large performance penalty (10-fold slow 
down). I'm also surprised that t[:] += pos[k, i] allocates memory (and 
incurs an even larger performance penalty). I really thought this was the 
recommended way to perform inplace operations on arrays.

So, is there any tool like Julia profiler but for memory allocation?

[julia-users] Julia Integration in Paraview

[Brenda Make]

Hi All,

I posted a suggestion, for integration of Julia in Paraview, but it could 
use some votes/love.
http://paraview.uservoice.com/forums/11350-general/suggestions/6580456-integrate-julia

Paraview is a popular and important data 
visualization program that's used for many scientific and technological 
studies. It is often used for such things as Computation Fluid Dynamics, 
Finite Element Analysts, Geology and other natural studies, as well as 
Medical Imaging.

Currently, Paraview is Python scriptable, but Julia would seem like a good 
fit because of its speed and powerful matrix functions.

Thank you for your consideration,
--Brenda Make

Re: [julia-users] How can I get allocation information in Julia?

[Tim Holy]

accoujulia, see the "performance tips" section of the manual, which describes 
several tools.

--Tim

On Wednesday, October 22, 2014 07:56:11 AM accouju...@gmail.com wrote:
> Please consider this implementation of a naive nearest neighbors distance
> search:
> 
> function mindists_sq(pos, dists_min, Acp, t)
> for i in 1:size(pos, 2)
> dists_min[i] = Inf
> for j in 1:size(Acp, 2)
> sum!(t, (sub(pos, :, i) - sub(Acp, :, j)).^2)
> if t[1] < dists_min[i]
> dists_min[i] = t[1]
> end
> end
> end
> return dists_min
> end
> 
> 
> And this:
> 
> function test()
> const pos = rand(3, 64)
> const Acp = rand(3, 1200)
> const dists_min = zeros(64)
> const tmp = zeros(typeof(Acp[1]), 1)
> @time mindists_sq(pos, dists_min, Acp, tmp)
> end
> test()
> 
> 
> I get:
> 
> elapsed time: 1.075699957 seconds (130182144 bytes allocated, 10.60% gc time
> )
> 
> 
> There are two very surprising things here:
> 
>-  First I am not sure where the allocation comes from.
>(sub(pos, :, i) - sub(Acp, :, j)).^2
>most certainly generate a new array in the process. t[1] and
>dists_min[i] might too (but these are immutable bit types). I tried to
>unroll the loop, but I got even more allocation and it's twice slower.
>- Second, the execution time is insane. I tried to put @inbounds
>wherever possible but without success. More surprising, the following is
> 10 times faster and allocates 90MB less:
>function mindists_sq(pos, dists_min, Acp)
>for i in 1:size(pos, 2)
>dists_min[i] = Inf
>for j in 1:size(Acp, 2)
>t = sum((pos[:, i]-Acp[:, j]).^2)
>if t .< dists_min[i]
>dists_min[i] = t
>end
>end
>end
>return dists_min
>end
> 
> elapsed time: 0.11141394 seconds (37555212 bytes allocated, 48.96% gc time)
> 
> 
> Is there anything that can help me map new allocations with line numbers?
> Is there any way to get the allocation number to zero? Can you reproduce
> the described behavior?
> 
> Julia Version 0.3.1
> Commit c03f413 (2014-09-21 21:30 UTC)
> Platform Info:
>   System: Linux (x86_64-unknown-linux-gnu)
>   CPU: Intel(R) Core(TM)2 Duo CPU P8700  @ 2.53GHz
>   WORD_SIZE: 64
>   BLAS: libopenblas (NO_AFFINITY PENRYN)
>   LAPACK: liblapack
>   LIBM: libm
>   LLVM: libLLVM-3.3
> 
> Best regards

Re: [julia-users] equivalent to `import` or `using` for loading package only on one process

[Tim Holy]

I've run into the same problem. Once we can load packages quickly, this will 
be less of a concern, but it seems worth opening an issue.

--Tim

On Wednesday, October 22, 2014 08:33:11 AM Spencer Lyon wrote:
> I am doing some numerical work and then plotting some results.
> 
> I really don’t need to load the plotting packages on all processes that are
> used to do the computation. Is there a way to execute import or using on
> one process only?
> 
> I have tried include(joinpath(Pkg.dir("PackageName"), "PackageName.jl")),
> but this is only an ok way to do it because any time using, require, or
> import is called from within PackageName.jl it loads each of those
> dependencies on each process.
> 
> Anyone know of a better way to do this?
> ​

Re: [julia-users] Re: Learning to Type?

[Stefan Karpinski]

That's an option for immutable types and it amounts to making the types of
fields implicit (invisible) type parameters. For mutable types, the type of
an abstractly typed field can change at any point in time, so that's not an
option.

On Wed, Oct 22, 2014 at 12:08 PM, Cedric St-Jean  wrote:

> Somewhat OT: since the JIT compiles a different function for each argument
> type combination, couldn't it compile a different type object for every
> field type combination?
>
>
> On Wednesday, October 22, 2014 10:53:26 AM UTC-4, Iain Dunning wrote:
>>
>> Do you feel like you are over-typing things?
>>
>> In function definitions, you only need types if you actually need to them
>> to pick a function to dispatch to. They don't make anything go faster, as a
>> specific version of a function will be compiled for every different set of
>> argument types you try calling it with.
>>
>> So, check out:
>> https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
>> for example.
>> layout_spring_adj{T}(adj_matrix::Array{T,2}
>> So it takes any array of element type T. For every time you call this
>> with a different element type, a different version of the function will be
>> compiled. So this fancy looking code:
>> if adj_matrix[i,j] != zero(eltype(adj_matrix))
>> Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or 
>> and so on, pretty magic!
>>
>> For Types, you should specify concrete types whenever you can, or use
>> parametric types if you can't. If you don't, and use say, Number, or leave
>> it blank, simple things like Float64s and Ints will be "boxed", so there
>> will be overhead whenever you use them and the compiler may struggle to
>> infer that some things are Float64s or Ints and not just Anys.
>>
>> Hope that helps somewhat.
>>
>> On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
>>>
>>> Hi Andreas,
>>>
>>> Don't know if you've seen this already and whether this is helpful at
>>> all, but I discovered this site
>>> 
>>> once when I got confused by some TypeErrors, it basically lays out the
>>> entire type tree of Julia.
>>>
>>> Best,
>>> Nils
>>>
>>> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:

 Hello,

 i'm now a julia user for some time, but still i have problems (comming
 from a c, f77, matlab, python background) to understand the type system.
 Especially when i try to integrate own code into packages (happened to
 Cairo + Compose, pending for Winston) i seem to choose always a more
 complicated way (my observation, code similar in py or c just looks
 shorter...) because somehow things do not match.

 So.

 What to read to understand the julia approach to types and type
 interference?

 Wishing a happy day,
Andreas

[julia-users] Re: Terminating loops early based on user input

[Steven G. Johnson]

Couldn't you use InterruptExceptions here (ctrl-c handlers), disabling 
sigint during the loop body so that the exception only occurs when you are 
ready?

for i in iterations
   try
  disable_sigint() do
do an iteration
  end
   catch e
  if isa(e, InterruptException)
   ...exit gracefully
  else
 rethrow(e)
  end
   end
end

?   This kind of thing has a chance to work in an environment like IJulia 
too, whereas reading stdin will never work in a browser interface.

Unfortunately, the above code may still have a race condition: if the user 
hits ctrl-c and the SIGINT signal is received in the loop body but before 
try...end then it won't be caught by the catch bock.   One way to hack 
around this would be to put the disable_sigint() outside the loop, and then 
check the C global jl_signal_pending != 0 inside the loop body to see if a 
SIGINT was received (but not handled yet).

[julia-users] Re: Learning to Type?

[Cedric St-Jean]

Somewhat OT: since the JIT compiles a different function for each argument 
type combination, couldn't it compile a different type object for every 
field type combination?

On Wednesday, October 22, 2014 10:53:26 AM UTC-4, Iain Dunning wrote:
>
> Do you feel like you are over-typing things?
>
> In function definitions, you only need types if you actually need to them 
> to pick a function to dispatch to. They don't make anything go faster, as a 
> specific version of a function will be compiled for every different set of 
> argument types you try calling it with.
>
> So, check out:
> https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
> for example.
> layout_spring_adj{T}(adj_matrix::Array{T,2}
> So it takes any array of element type T. For every time you call this with 
> a different element type, a different version of the function will be 
> compiled. So this fancy looking code:
> if adj_matrix[i,j] != zero(eltype(adj_matrix))
> Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or  
> and so on, pretty magic!
>
> For Types, you should specify concrete types whenever you can, or use 
> parametric types if you can't. If you don't, and use say, Number, or leave 
> it blank, simple things like Float64s and Ints will be "boxed", so there 
> will be overhead whenever you use them and the compiler may struggle to 
> infer that some things are Float64s or Ints and not just Anys.
>
> Hope that helps somewhat.
>
> On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
>>
>> Hi Andreas,
>>
>> Don't know if you've seen this already and whether this is helpful at 
>> all, but I discovered this site 
>> 
>>  
>> once when I got confused by some TypeErrors, it basically lays out the 
>> entire type tree of Julia.
>>
>> Best,
>> Nils
>>
>> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:
>>>
>>> Hello,
>>>
>>> i'm now a julia user for some time, but still i have problems (comming 
>>> from a c, f77, matlab, python background) to understand the type system. 
>>> Especially when i try to integrate own code into packages (happened to 
>>> Cairo + Compose, pending for Winston) i seem to choose always a more 
>>> complicated way (my observation, code similar in py or c just looks 
>>> shorter...) because somehow things do not match. 
>>>
>>> So.
>>>
>>> What to read to understand the julia approach to types and type 
>>> interference?
>>>
>>> Wishing a happy day,
>>>Andreas
>>>
>>>
>>>

[julia-users] Terminating loops early based on user input

[John Myles White]

I've started trying to write code that would allow me to take an existing 
while-loop or for-loop and augment it with the ability for a user to pause the 
computation and terminate it early. This is really useful in Optim, where you 
might want to stop optimizing as soon as the solution hits a point you consider 
acceptable.

After a bunch of hacking based on suggestions that Jameson gave me a long time 
ago, I've gotten something that works when run outside of a REPL:

https://gist.github.com/johnmyleswhite/bb184e2fb202b262ef35

But I'm a little lost when thinking about the modifications I need to get this 
working in a REPL, where I seem to be fighting with the REPL for access to raw 
unbuffer input from STDIN.

I'd also love to know if the solution I've hit upon is something that's 
reasonable to hope will work in the future. The whole thing is clearly a hack, 
but I can't tell if the principle is sound or not.

 -- John

Re: [julia-users] Re: How can I get allocation information in Julia?

[Mauro]

This works for me:

julia> function mindists_sq(pos, dists_min, Acp)
   
   for i in 1:size(pos, 2)
   dists_min[i] = Inf
   for j in 1:size(Acp, 2)
   t = 0.0
   for k=1:size(pos,1)
   t += (pos[k, i]-Acp[k, j])^2
   end
   if t < dists_min[i]
   dists_min[i] = t
   end
   end
   end
   return dists_min
   end
mindists_sq (generic function with 1 method)

julia> function test()
   const pos = rand(3, 64)
   const Acp = rand(3, 1200)
   const dists_min = zeros(64)
   const tmp = zeros(typeof(Acp[1]), 1)
   @time mindists_sq(pos, dists_min, Acp)
   end
test (generic function with 1 method)

julia> test();
elapsed time: 0.001279041 seconds (0 bytes allocated)

Is this how you unrolled the innermost loop too?

On Wed, 2014-10-22 at 17:38, accouju...@gmail.com wrote:
> As a complement, I just tried with Julia master and in the first case, I got 
> an
> exec time around 0.7s. The second case is virtually the same.
>
> Julia Version 0.4.0-dev+1177
> Commit 16c3222* (2014-10-22 12:49 UTC)
> Platform Info:
>   System: Linux (x86_64-unknown-linux-gnu)
>   CPU: Intel(R) Core(TM)2 Duo CPU P8700  @ 2.53GHz
>   WORD_SIZE: 64
>   BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn)
>   LAPACK: libopenblas
>   LIBM: libopenlibm
>   LLVM: libLLVM-3.3
>
> And as a reference, the corresponding Python code is that one:
>
> def mindists_sq(B, Acp):
> return ((Acp - B[:, None])**2).sum(2).min(1)
>
> And get executed in 4-8 ms. 
>
>
> To Julia's honor, I can go down to 4-8 ms with it, but I need to emulate 
> Python
> behavior with broadcasts and large preallocated temporaries:
>
> function mindists_sq2(pos, dists_min, Acp, tmp, ntmp)
> for j in 1:size(pos, 2)
> broadcast!(.-, tmp, Acp, sub(pos, :, j))
> for i in 1:length(tmp[:])
> tmp[i] *= tmp[i]
> end
> ntmp[:] = sum(tmp, 1)
> dists_min[j] = minimum(ntmp)
> end
> return dists_min
> end
>
>
> function test(pos, Acp)
> const dists_min = zeros(typeof(Acp[1]), lenght(pos))
> const tmp = similar(Acp)
> const ntmp = similar(Acp[1,:,:])
> @time mindists_sq2(pos, dists_min, Acp, tmp, ntmp)
> end
> test(A, B)
>
> elapsed time: 0.004768065 seconds (3624960 bytes allocated)
>
> but it's a bit more verbose and convoluted.

[julia-users] Re: equivalent to `import` or `using` for loading package only on one process

[Nils Gudat]

I though the default behavior was for packages to be loaded only on one 
process? I.e. unless you do @everywhere using PyPlot, Pyplot will only be 
available on the first process?
Another somewhat hacky solution would be to simply add workers later, i.e. 
load the packages you need for plotting, then add workers, then load the 
rest.

[julia-users] Re: equivalent to `import` or `using` for loading package only on one process

[Spencer Lyon]

typo in my hacky function. Should have written 
include(joinpath(Pkg.dir("PackageName"), 
"src", "PackageName.jl"))

On Wednesday, October 22, 2014 11:33:11 AM UTC-4, Spencer Lyon wrote:

I am doing some numerical work and then plotting some results.
>
> I really don’t need to load the plotting packages on all processes that 
> are used to do the computation. Is there a way to execute import or using 
> on one process only?
>
> I have tried include(joinpath(Pkg.dir("PackageName"), "PackageName.jl")), 
> but this is only an ok way to do it because any time using, require, or 
> import is called from within PackageName.jl it loads each of those 
> dependencies on each process.
>
> Anyone know of a better way to do this?
> ​
>
​

[julia-users] Re: How can I get allocation information in Julia?

[accoujulia]

As a complement, I just tried with Julia master and in the first case, I 
got an exec time around 0.7s. The second case is virtually the same.

Julia Version 0.4.0-dev+1177
Commit 16c3222* (2014-10-22 12:49 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Core(TM)2 Duo CPU P8700  @ 2.53GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

And as a reference, the corresponding Python code is that one:

def mindists_sq(B, Acp):
return ((Acp - B[:, None])**2).sum(2).min(1)

And get executed in 4-8 ms. 


To Julia's honor, I can go down to 4-8 ms with it, but I need to emulate 
Python behavior with broadcasts and large preallocated temporaries:

function mindists_sq2(pos, dists_min, Acp, tmp, ntmp)
for j in 1:size(pos, 2)
broadcast!(.-, tmp, Acp, sub(pos, :, j))
for i in 1:length(tmp[:])
tmp[i] *= tmp[i]
end
ntmp[:] = sum(tmp, 1)
dists_min[j] = minimum(ntmp)
end
return dists_min
end


function test(pos, Acp)
const dists_min = zeros(typeof(Acp[1]), lenght(pos))
const tmp = similar(Acp)
const ntmp = similar(Acp[1,:,:])
@time mindists_sq2(pos, dists_min, Acp, tmp, ntmp)
end
test(A, B)

elapsed time: 0.004768065 seconds (3624960 bytes allocated)

but it's a bit more verbose and convoluted.

[julia-users] equivalent to `import` or `using` for loading package only on one process

[Spencer Lyon]

I am doing some numerical work and then plotting some results.

I really don’t need to load the plotting packages on all processes that are 
used to do the computation. Is there a way to execute import or using on 
one process only?

I have tried include(joinpath(Pkg.dir("PackageName"), "PackageName.jl")), 
but this is only an ok way to do it because any time using, require, or 
import is called from within PackageName.jl it loads each of those 
dependencies on each process.

Anyone know of a better way to do this?
​

[julia-users] How can I get allocation information in Julia?

[accoujulia]

Please consider this implementation of a naive nearest neighbors distance 
search:

function mindists_sq(pos, dists_min, Acp, t)
for i in 1:size(pos, 2)
dists_min[i] = Inf
for j in 1:size(Acp, 2)
sum!(t, (sub(pos, :, i) - sub(Acp, :, j)).^2)
if t[1] < dists_min[i]
dists_min[i] = t[1]
end
end
end
return dists_min
end


And this:

function test()
const pos = rand(3, 64)
const Acp = rand(3, 1200)
const dists_min = zeros(64)
const tmp = zeros(typeof(Acp[1]), 1)
@time mindists_sq(pos, dists_min, Acp, tmp)
end
test()


I get:

elapsed time: 1.075699957 seconds (130182144 bytes allocated, 10.60% gc time
)


There are two very surprising things here:

   -  First I am not sure where the allocation comes from.  
   (sub(pos, :, i) - sub(Acp, :, j)).^2
   most certainly generate a new array in the process. t[1] and 
   dists_min[i] might too (but these are immutable bit types). I tried to 
   unroll the loop, but I got even more allocation and it's twice slower.
   - Second, the execution time is insane. I tried to put @inbounds 
   wherever possible but without success. More surprising, the following is 10 
   times faster and allocates 90MB less:
   function mindists_sq(pos, dists_min, Acp)
   for i in 1:size(pos, 2)
   dists_min[i] = Inf
   for j in 1:size(Acp, 2)
   t = sum((pos[:, i]-Acp[:, j]).^2)
   if t .< dists_min[i]
   dists_min[i] = t
   end
   end
   end
   return dists_min
   end
   
elapsed time: 0.11141394 seconds (37555212 bytes allocated, 48.96% gc time)


Is there anything that can help me map new allocations with line numbers? 
Is there any way to get the allocation number to zero? Can you reproduce 
the described behavior?

Julia Version 0.3.1
Commit c03f413 (2014-09-21 21:30 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Core(TM)2 Duo CPU P8700  @ 2.53GHz
  WORD_SIZE: 64
  BLAS: libopenblas (NO_AFFINITY PENRYN)
  LAPACK: liblapack
  LIBM: libm
  LLVM: libLLVM-3.3

Best regards

[julia-users] Re: Learning to Type?

[Iain Dunning]

Do you feel like you are over-typing things?

In function definitions, you only need types if you actually need to them 
to pick a function to dispatch to. They don't make anything go faster, as a 
specific version of a function will be compiled for every different set of 
argument types you try calling it with.

So, check out:
https://github.com/IainNZ/GraphLayout.jl/blob/master/src/spring.jl
for example.
layout_spring_adj{T}(adj_matrix::Array{T,2}
So it takes any array of element type T. For every time you call this with 
a different element type, a different version of the function will be 
compiled. So this fancy looking code:
if adj_matrix[i,j] != zero(eltype(adj_matrix))
Simply gets turned into if adj_matrix[i,j] != 0.0  or  0 or False or  
and so on, pretty magic!

For Types, you should specify concrete types whenever you can, or use 
parametric types if you can't. If you don't, and use say, Number, or leave 
it blank, simple things like Float64s and Ints will be "boxed", so there 
will be overhead whenever you use them and the compiler may struggle to 
infer that some things are Float64s or Ints and not just Anys.

Hope that helps somewhat.

On Wednesday, October 22, 2014 7:28:12 AM UTC-4, Nils Gudat wrote:
>
> Hi Andreas,
>
> Don't know if you've seen this already and whether this is helpful at all, 
> but I discovered this site 
> 
>  
> once when I got confused by some TypeErrors, it basically lays out the 
> entire type tree of Julia.
>
> Best,
> Nils
>
> On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:
>>
>> Hello,
>>
>> i'm now a julia user for some time, but still i have problems (comming 
>> from a c, f77, matlab, python background) to understand the type system. 
>> Especially when i try to integrate own code into packages (happened to 
>> Cairo + Compose, pending for Winston) i seem to choose always a more 
>> complicated way (my observation, code similar in py or c just looks 
>> shorter...) because somehow things do not match. 
>>
>> So.
>>
>> What to read to understand the julia approach to types and type 
>> interference?
>>
>> Wishing a happy day,
>>Andreas
>>
>>
>>

Re: [julia-users] Julia Elliptic Curve Arithmetic slower than Python

[Stefan Karpinski]

It will also help to make your other types immutable. There's certainly no
reason for PointINF not to be immutable.

On Wed, Oct 22, 2014 at 10:47 AM, Steven G. Johnson 
wrote:

>
>
> On Wednesday, October 22, 2014 10:14:41 AM UTC-4, Erik Schnetter wrote:
>>
>> What Stefan is probably trying to say:
>>
>> Instead of a function f(x, y) where x and y can have different types,
>> write f{T}(x::T, y::T), so that x and y are forced to have the same
>> type -- but this type can be any type. In this way, e.g. x+y can be
>>
>
> No, f(x,y) is fine.  If x and y happen to have types such that x+y is
> fast, then it will be fast.   Remember that a *different* version of f gets
> compiled for *every* set of argument types.   So, when you call f(Int, Int)
> it will compile a type-specialized version.  You essentially never need to
> declare argument types in functions for performance, only to control
> dispatch.
>
> However, you *do* need concrete types in your structures, which you can do
> by type parameterization.  e.g.
>
> immutable CurveFP{T<:Number}
> p::T
> a::T
> b::T
> function CurveFP(p, a, b)
>   assert(mod(4*a^3+27*b^2,p) != 0)
>   new(p, a, b)
> end
> end
>
>

Re: [julia-users] Re: NLopt and Julia Inf

[Patrick O'Leary]

On Wednesday, October 22, 2014 9:40:14 AM UTC-5, Steven G. Johnson wrote:
>
>
>
> On Tuesday, October 21, 2014 10:44:30 PM UTC-4, Stefan Karpinski wrote:
>>
>> I think you can get it with 1.0/0.0 even if INFINITY isn't defined.
>>
>
> Even better, you can just use HUGE_VAL.  HUGE_VAL (defined in C89) is 
> equivalent to INFINITY on all IEEE machines (i.e. all modern CPUs); I don't 
> really understand why they bothered to add INFINITY in C99. 
>

HUGE_VAL sounds like the "up-goer five" version of INFINITY. I didn't know 
about it; thanks! 

Re: [julia-users] Julia Elliptic Curve Arithmetic slower than Python

[Steven G. Johnson]

On Wednesday, October 22, 2014 10:14:41 AM UTC-4, Erik Schnetter wrote:
>
> What Stefan is probably trying to say: 
>
> Instead of a function f(x, y) where x and y can have different types, 
> write f{T}(x::T, y::T), so that x and y are forced to have the same 
> type -- but this type can be any type. In this way, e.g. x+y can be 
>

No, f(x,y) is fine.  If x and y happen to have types such that x+y is fast, 
then it will be fast.   Remember that a *different* version of f gets 
compiled for *every* set of argument types.   So, when you call f(Int, Int) 
it will compile a type-specialized version.  You essentially never need to 
declare argument types in functions for performance, only to control 
dispatch.

However, you *do* need concrete types in your structures, which you can do 
by type parameterization.  e.g.

immutable CurveFP{T<:Number}
p::T
a::T
b::T
function CurveFP(p, a, b)
  assert(mod(4*a^3+27*b^2,p) != 0)
  new(p, a, b)
end
end

Re: [julia-users] Julia Elliptic Curve Arithmetic slower than Python

[Stefan Karpinski]

This is what I was saying:

immutable CurveFP{T<:Number}
p::T
a::T
b::T
function CurveFP(p::T, a::T, b::T)
mod(4a^3 + 27b^2, p) != 0 || error("4*$a^3 + 27*$b^2 ≡ 0 mod $p")
new(p, a, b)
end
end
CurveFP{T<:Number}(p::T, a::T, b::T) = CurveFP{T}(p, a, b)
CurveFP(p::Number, a::Number, b::Number) = CurveFP(promote(p, a, b)...)

This allows specilized code be generated for CurveFP{T} for each T. The
version where the fields are abstractly typed defines a single type and
code that deals with that type has to generically deal with any possible
type of Number value for each field.

Also note that the way you were using assert is not quite right: assert
should only be used to check internal correctness of your own code, not to
check that users of your code are doing the right thing. If you remove all
asserts, your code should continue to work identically from the user's
perspective.


On Wed, Oct 22, 2014 at 10:14 AM, Erik Schnetter 
wrote:

> What Stefan is probably trying to say:
>
> Instead of a function f(x, y) where x and y can have different types,
> write f{T}(x::T, y::T), so that x and y are forced to have the same
> type -- but this type can be any type. In this way, e.g. x+y can be
> evaluated very efficiently, and without type conversion, independent
> of whether x is Int or BigInt.
>
> -erik
>
>
> On Tue, Oct 21, 2014 at 8:36 PM, Stefan Karpinski
>  wrote:
> > Oh yeah, those fields having abstract type is a doozy - you'll want to
> > introduce a type parameter for that type and make all of the field of
> that
> > type.
> >
> > On Oct 21, 2014, at 7:34 PM, alexander maznev <
> alexander.maz...@gmail.com>
> > wrote:
> >
> > I had issues with how Julia does not seem to do type coarsing even when a
> > function will only take arguments of that one type. I.e. point_a * 10
> will
> > fail because it expects a BigInt but receives an Int64, which i guess is
> > solved by wrapping every single number passed around in the BigInt
> class, or
> > duplicating all the methods. At any rate I tested it with BigInt instead
> of
> > Number and the run times do not change much.
> > Also I introduced a type when replacing divmod with divrem, it should be.
> > (q,c),d = divrem(d,c), c
> >
> >
> >
> > On Tuesday, October 21, 2014 7:19:10 PM UTC-4, Kevin Squire wrote:
> >>
> >> One problem is that you're using an abstract type (Number) for all of
> the
> >> variable members if your types.
> >>
> >> In function declarations, this is okay, because the function is
> >> specialized on the concrete number type. But for types, abstractly typed
> >> members are boxed (stored as pointers), because the exact type is not
> given
> >> in the definition.
> >>
> >> You can get close to the same flexibility of the current code by using
> >> parametric types, which should erase any performance gap.
> >>
> >> Cheers,
> >>   Kevin
> >>
> >> On Tuesday, October 21, 2014, alexander maznev 
> >> wrote:
> >>>
> >>> This should be an equivalent, or nearly there, implementation of
> Elliptic
> >>> Curves mod p as found in the Python ecdsa library.
> >>>
> >>> https://gist.github.com/anonymous/a3799a5a2b0354022eac
> >>>
> >>> Noticeably, regular mod is 10x slower than python?
> >>> Inverse_mod is 7x slower than python.
> >>> Double is 7x slower than python
> >>> Multiply is more than 7X slower than python.
> >>>
> >
>
>
>
> --
> Erik Schnetter 
> http://www.perimeterinstitute.ca/personal/eschnetter/
>

Re: [julia-users] Re: NLopt and Julia Inf

[Steven G. Johnson]

On Tuesday, October 21, 2014 10:44:30 PM UTC-4, Stefan Karpinski wrote:
>
> I think you can get it with 1.0/0.0 even if INFINITY isn't defined.
>

Even better, you can just use HUGE_VAL.  HUGE_VAL (defined in C89) is 
equivalent to INFINITY on all IEEE machines (i.e. all modern CPUs); I don't 
really understand why they bothered to add INFINITY in C99. 

IIRC, some compilers will complain about 1.0/0.0 literals; they don't like 
constants that throw floating-point exceptions.

[julia-users] Re: Any function could detect if enviroment is REPL, IJulia or functional?

[Steven G. Johnson]

On Wednesday, October 22, 2014 9:20:01 AM UTC-4, xiong...@gmail.com wrote:
>
> It works, thank you!
> Is there any method to distinglish REPL or IJulia enviroment?
>

To detect whether you are running in IJulia,

 isdefined(Main, :IJulia) && Main.IJulia.inited 

should work.

Re: [julia-users] Re: Modified Gram-Schmidt: getting the best out of Julia

[Andreas Noack]

>
> I'd like to approach this speed at the end.


I don't think it is possible in Julia right now without using dirty tricks
such as passing pointers around. You'd like to get the speed from BLAS by
operating on panels of your matrix, but you'd like to avoid the copying and
reallocation of arrays. If you devectorised your code completely, there'd
be no allocation, but you wouldn't have the fast multiplications in BLAS.
You can get some of the way by using one of the ArrayViews packages, but
the garbage collector will charge a fee for each view you produce so LAPACK
speed is not attainable.

Med venlig hilsen

Andreas Noack

2014-10-22 10:09 GMT-04:00 Ján Dolinský :

>
>
> Dňa streda, 22. októbra 2014 11:56:31 UTC+2 Tim Holy napísal(-a):
>
>> On Wednesday, October 22, 2014 02:27:30 AM Ján Dolinský wrote:
>> > I wonder why my vectorized code is not fast. I assume operations like
>> "X' *
>> > a" are using BLAS (e.g. BLAS.gemv()) and thus it should be as fast as a
>> > devectorized code at least in this case.
>>
>> Yes, but their result requires allocating a temporary, and temporaries
>> mean
>> garbage collection. Try
>> - pre-allocating a couple of temp vectors to store the results of your
>> matrix
>> multiplies, and use functions like `Ac_mul_B!(dest, A, B)` to calculate
>> A'*B
>> and store the result in dest.
>> - Do the subtractions & divisions using a loop, as suggested by Steven.
>> - in each iteration you should call `view(X, :, l)` just once and store
>> the
>> result to a variable---no point recreating the same view 3 times per
>> iteration.
>>
>> --Tim
>>
>
> Hello Tim,
>
> Thanks for the tips. I rewrote a bit the mgs routine so that it is more
> readable and did profiling.
>
> using ArrayViews
>
> function mgs(X)
> # mgs speed project
>
>   nobs, nvars = size(X)
>   R = eye(nvars)
>
>   for l=1:nvars-1
> v_i = view(X,:,l+1:nvars) # --- line 10
> s = view(X,:,l);
>
> R[l,l+1:nvars]=(v_i'*s)/sumabs2(s) # --- line 13
> X[:,l+1:nvars]=v_i-(s*view(R,l,l+1:nvars)) # --- line 14
>   end
>
>   X, R
>
> end
>
> Profilling is as follows:
> 1 linalg/blas.jl; gemm!; line: 526
> 10455 task.jl; anonymous; line: 96
>  10455 REPL.jl; eval_user_input; line: 54
>   10455 profile.jl; anonymous; line: 14
>2...me/Projects/julia-trials/mgs.jl; mgs; line: 7
> 2 array.jl; eye; line: 176
>  2 array.jl; fill!; line: 155
>2383 ...me/Projects/julia-trials/mgs.jl; mgs; line: 10
>*41   ...me/Projects/julia-trials/mgs.jl; mgs; line: 13*
> 3  array.jl; ./; line: 756
> 7  array.jl; ./; line: 758
> 1  linalg/matmul.jl; gemv!; line: 217
>  1 pointer.jl; pointer; line: 29
> 16 multidimensional.jl; setindex!; line: 76
> 3  multidimensional.jl; setindex!; line: 295
> 6  reduce.jl; _mapreduce; line: 168
>*8027 ...me/Projects/julia-trials/mgs.jl; mgs; line: 14*
> 31   ...3/ArrayViews/src/ArrayViews.jl; strided_view; line: 72
> 1abstractarray.jl; copy!; line: 152
> 3109 array.jl; -; line: 719
> 2770 array.jl; -; line: 721
> 5linalg/matmul.jl; *; line: 74
>  1 abstractarray.jl; copy!; line: 150
> 1807 multidimensional.jl; setindex!; line: 76
> 1multidimensional.jl; setindex!; line: 80
> 303  multidimensional.jl; setindex!; line: 295
>1abstractarray.jl; stride; line: 30
>
> There is apparently a huge difference between line 13 and 14.
> Line 14 indeed deserves a loop as suggested by Steven. The culprit seems
> to be the minus operation. I'll explore the problem further.
> Line 13 seems to be already reasonably fast but preallocating a space for
> output of (v_i'*s) and using something like "Ac_mul_B!(dest, v_i, s)" might
> provide further speedup.
>
>
> It is very interesting, I'll play with it a bit further and post the
> improvements.
>
> Jan
>

Re: [julia-users] Julia Elliptic Curve Arithmetic slower than Python

[Erik Schnetter]

What Stefan is probably trying to say:

Instead of a function f(x, y) where x and y can have different types,
write f{T}(x::T, y::T), so that x and y are forced to have the same
type -- but this type can be any type. In this way, e.g. x+y can be
evaluated very efficiently, and without type conversion, independent
of whether x is Int or BigInt.

-erik


On Tue, Oct 21, 2014 at 8:36 PM, Stefan Karpinski
 wrote:
> Oh yeah, those fields having abstract type is a doozy - you'll want to
> introduce a type parameter for that type and make all of the field of that
> type.
>
> On Oct 21, 2014, at 7:34 PM, alexander maznev 
> wrote:
>
> I had issues with how Julia does not seem to do type coarsing even when a
> function will only take arguments of that one type. I.e. point_a * 10 will
> fail because it expects a BigInt but receives an Int64, which i guess is
> solved by wrapping every single number passed around in the BigInt class, or
> duplicating all the methods. At any rate I tested it with BigInt instead of
> Number and the run times do not change much.
> Also I introduced a type when replacing divmod with divrem, it should be.
> (q,c),d = divrem(d,c), c
>
>
>
> On Tuesday, October 21, 2014 7:19:10 PM UTC-4, Kevin Squire wrote:
>>
>> One problem is that you're using an abstract type (Number) for all of the
>> variable members if your types.
>>
>> In function declarations, this is okay, because the function is
>> specialized on the concrete number type. But for types, abstractly typed
>> members are boxed (stored as pointers), because the exact type is not given
>> in the definition.
>>
>> You can get close to the same flexibility of the current code by using
>> parametric types, which should erase any performance gap.
>>
>> Cheers,
>>   Kevin
>>
>> On Tuesday, October 21, 2014, alexander maznev 
>> wrote:
>>>
>>> This should be an equivalent, or nearly there, implementation of Elliptic
>>> Curves mod p as found in the Python ecdsa library.
>>>
>>> https://gist.github.com/anonymous/a3799a5a2b0354022eac
>>>
>>> Noticeably, regular mod is 10x slower than python?
>>> Inverse_mod is 7x slower than python.
>>> Double is 7x slower than python
>>> Multiply is more than 7X slower than python.
>>>
>



-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/

Re: [julia-users] import large matrix from file

[Tim Holy]

For people who use CSV, it would be helpful to know examples of elements that 
caused this error---I don't use CSV myself much, but it seems strange that it 
wouldn't be able to import a Float32 array.

If you're going binary, HDF5 would be a more portable choice.

--Tim

On Wednesday, October 22, 2014 05:55:39 AM Luca Rossetto wrote:
> I was able to generate an Array{Any, 2} or an Array{String, 2} but not an
> Array{Float32, 2}. convert() always failed because there was apparently at
> least one element in the array which caused a conversion error.
> 
> I now solved the problem using an entirely different data encoding. When I
> export the values as raw byte data from my other program, i managed to
> import is using:
> f = open(filename)
> matrix = read(f, Float32, a * a)
> matrix = reshape(matrix a, a)
> 
> Nevertheless, is there an efficient way to read a Array{FloatingPoint, 2}
> from a CSV file?
> 
> Am Mittwoch, 22. Oktober 2014 14:42:52 UTC+2 schrieb Andreas Noack:
> > What type had the "resulting object"?
> > 
> > Med venlig hilsen
> > 
> > Andreas Noack
> > 
> > 2014-10-22 7:27 GMT-04:00 Luca Rossetto >
> > 
> >> Hi
> >> 
> >> I have a (java) program which generates large amounts of numerical data
> >> in matrix form (square matrix, ~10'000 x 10'000) on which I want to
> >> perform
> >> some numerical analysis. Currently, I'm exporting the matrix data as CSV
> >> but I could change that to a different format if necessary. I tried
> >> importing such a file using readcsv, readdlm and readtable but was unable
> >> to convert the resulting object to the form Array{Float32, 2}. Any ideas
> >> on
> >> what I can do?

Re: [julia-users] Re: Modified Gram-Schmidt: getting the best out of Julia

[Ján Dolinský]

Dňa streda, 22. októbra 2014 11:56:31 UTC+2 Tim Holy napísal(-a):
>
> On Wednesday, October 22, 2014 02:27:30 AM Ján Dolinský wrote: 
> > I wonder why my vectorized code is not fast. I assume operations like 
> "X' * 
> > a" are using BLAS (e.g. BLAS.gemv()) and thus it should be as fast as a 
> > devectorized code at least in this case. 
>
> Yes, but their result requires allocating a temporary, and temporaries 
> mean 
> garbage collection. Try 
> - pre-allocating a couple of temp vectors to store the results of your 
> matrix 
> multiplies, and use functions like `Ac_mul_B!(dest, A, B)` to calculate 
> A'*B 
> and store the result in dest. 
> - Do the subtractions & divisions using a loop, as suggested by Steven. 
> - in each iteration you should call `view(X, :, l)` just once and store 
> the 
> result to a variable---no point recreating the same view 3 times per 
> iteration. 
>
> --Tim 
>

Hello Tim,

Thanks for the tips. I rewrote a bit the mgs routine so that it is more 
readable and did profiling.

using ArrayViews

function mgs(X)
# mgs speed project

  nobs, nvars = size(X)
  R = eye(nvars)

  for l=1:nvars-1
v_i = view(X,:,l+1:nvars) # --- line 10
s = view(X,:,l);

R[l,l+1:nvars]=(v_i'*s)/sumabs2(s) # --- line 13
X[:,l+1:nvars]=v_i-(s*view(R,l,l+1:nvars)) # --- line 14
  end

  X, R

end

Profilling is as follows:
1 linalg/blas.jl; gemm!; line: 526
10455 task.jl; anonymous; line: 96
 10455 REPL.jl; eval_user_input; line: 54
  10455 profile.jl; anonymous; line: 14
   2...me/Projects/julia-trials/mgs.jl; mgs; line: 7
2 array.jl; eye; line: 176
 2 array.jl; fill!; line: 155
   2383 ...me/Projects/julia-trials/mgs.jl; mgs; line: 10
   *41   ...me/Projects/julia-trials/mgs.jl; mgs; line: 13*
3  array.jl; ./; line: 756
7  array.jl; ./; line: 758
1  linalg/matmul.jl; gemv!; line: 217
 1 pointer.jl; pointer; line: 29
16 multidimensional.jl; setindex!; line: 76
3  multidimensional.jl; setindex!; line: 295
6  reduce.jl; _mapreduce; line: 168
   *8027 ...me/Projects/julia-trials/mgs.jl; mgs; line: 14*
31   ...3/ArrayViews/src/ArrayViews.jl; strided_view; line: 72
1abstractarray.jl; copy!; line: 152
3109 array.jl; -; line: 719
2770 array.jl; -; line: 721
5linalg/matmul.jl; *; line: 74
 1 abstractarray.jl; copy!; line: 150
1807 multidimensional.jl; setindex!; line: 76
1multidimensional.jl; setindex!; line: 80
303  multidimensional.jl; setindex!; line: 295
   1abstractarray.jl; stride; line: 30
 
There is apparently a huge difference between line 13 and 14. 
Line 14 indeed deserves a loop as suggested by Steven. The culprit seems to 
be the minus operation. I'll explore the problem further.
Line 13 seems to be already reasonably fast but preallocating a space for 
output of (v_i'*s) and using something like "Ac_mul_B!(dest, v_i, s)" might 
provide further speedup.


It is very interesting, I'll play with it a bit further and post the 
improvements.

Jan

[julia-users] Re: Any function could detect if enviroment is REPL, IJulia or functional?

[xiongjieyi]

It works, thank you!
Is there any method to distinglish REPL or IJulia enviroment?

On Wednesday, October 22, 2014 3:11:14 PM UTC+2, Patrick O'Leary wrote:
>
> On Wednesday, October 22, 2014 8:06:07 AM UTC-5, xiong...@gmail.com wrote:
>>
>> Is there any function could be used to detect if the current enviroment 
>> is REPL, IJulia or runing in a function? 
>>
>
> The isinteractive() function comes close. 
>

[julia-users] Re: Any function could detect if enviroment is REPL, IJulia or functional?

[Patrick O'Leary]

On Wednesday, October 22, 2014 8:06:07 AM UTC-5, xiong...@gmail.com wrote:
>
> Is there any function could be used to detect if the current enviroment is 
> REPL, IJulia or runing in a function? 
>

The isinteractive() function comes close. 

[julia-users] Any function could detect if enviroment is REPL, IJulia or functional?

[xiongjieyi]

Is there any function could be used to detect if the current enviroment is 
REPL, IJulia or runing in a function? 

Re: [julia-users] import large matrix from file

[Luca Rossetto]

I was able to generate an Array{Any, 2} or an Array{String, 2} but not an 
Array{Float32, 2}. convert() always failed because there was apparently at 
least one element in the array which caused a conversion error. 

I now solved the problem using an entirely different data encoding. When I 
export the values as raw byte data from my other program, i managed to 
import is using:
f = open(filename)
matrix = read(f, Float32, a * a)
matrix = reshape(matrix a, a)

Nevertheless, is there an efficient way to read a Array{FloatingPoint, 2} 
from a CSV file?

Am Mittwoch, 22. Oktober 2014 14:42:52 UTC+2 schrieb Andreas Noack:
>
> What type had the "resulting object"?
>
> Med venlig hilsen
>
> Andreas Noack
>
> 2014-10-22 7:27 GMT-04:00 Luca Rossetto >
> :
>
>> Hi
>>
>> I have a (java) program which generates large amounts of numerical data 
>> in matrix form (square matrix, ~10'000 x 10'000) on which I want to perform 
>> some numerical analysis. Currently, I'm exporting the matrix data as CSV 
>> but I could change that to a different format if necessary. I tried 
>> importing such a file using readcsv, readdlm and readtable but was unable 
>> to convert the resulting object to the form Array{Float32, 2}. Any ideas on 
>> what I can do?
>>
>
>

Re: [julia-users] import large matrix from file

[Andreas Noack]

What type had the "resulting object"?

Med venlig hilsen

Andreas Noack

2014-10-22 7:27 GMT-04:00 Luca Rossetto :

> Hi
>
> I have a (java) program which generates large amounts of numerical data in
> matrix form (square matrix, ~10'000 x 10'000) on which I want to perform
> some numerical analysis. Currently, I'm exporting the matrix data as CSV
> but I could change that to a different format if necessary. I tried
> importing such a file using readcsv, readdlm and readtable but was unable
> to convert the resulting object to the form Array{Float32, 2}. Any ideas on
> what I can do?
>

[julia-users] import large matrix from file

[Luca Rossetto]

Hi

I have a (java) program which generates large amounts of numerical data in 
matrix form (square matrix, ~10'000 x 10'000) on which I want to perform 
some numerical analysis. Currently, I'm exporting the matrix data as CSV 
but I could change that to a different format if necessary. I tried 
importing such a file using readcsv, readdlm and readtable but was unable 
to convert the resulting object to the form Array{Float32, 2}. Any ideas on 
what I can do?

Re: [julia-users] Re: catching an error in macro body

[Mauro]

I'm running into the problem in an example script, so Ivar's suggestion
works perfectly.  

Mauro

On Wed, 2014-10-22 at 12:11, Tim Holy  wrote:
> I frequently debug macros by writing them as functions that operate on 
> expressions, and then define the macro so it calls the function.
>
> --Tim
>
> On Wednesday, October 22, 2014 03:06:10 AM Ivar Nesje wrote:
>> The problem is that the error from the macro evaluation is an error that
>> gets raised when evaluating the source, so you need to put try outside the
>> include of the file or use eval:
>> 
>> try
>>eval(:(@a))
>> end
>> 
>> You can also put the try block inside the macro.
>> 
>> Ivar
>> 
>> kl. 11:59:39 UTC+2 onsdag 22. oktober 2014 skrev Mauro følgende:
>> > Is it possibly to catch an error happening during macro evaluation?
>> > 
>> > Example:
>> > ```
>> > macro a()
>> > 
>> > error("error in marco itself")
>> > 
>> > end
>> > 
>> > macro b()
>> > 
>> > :(error("error in marco generated code"))
>> > 
>> > end
>> > 
>> > # works:
>> > try
>> > 
>> > @b
>> > 
>> > end
>> > 
>> > # still errors:
>> > try
>> > 
>> > @a
>> > 
>> > end
>> > ```
>> > 
>> > How to `try` the error in `@a`?

[julia-users] Re: Learning to Type?

[Nils Gudat]

Hi Andreas,

Don't know if you've seen this already and whether this is helpful at all, 
but I discovered this site 

 
once when I got confused by some TypeErrors, it basically lays out the 
entire type tree of Julia.

Best,
Nils

On Wednesday, October 22, 2014 12:11:03 PM UTC+1, Andreas Lobinger wrote:
>
> Hello,
>
> i'm now a julia user for some time, but still i have problems (comming 
> from a c, f77, matlab, python background) to understand the type system. 
> Especially when i try to integrate own code into packages (happened to 
> Cairo + Compose, pending for Winston) i seem to choose always a more 
> complicated way (my observation, code similar in py or c just looks 
> shorter...) because somehow things do not match. 
>
> So.
>
> What to read to understand the julia approach to types and type 
> interference?
>
> Wishing a happy day,
>Andreas
>
>
>

[julia-users] Learning to Type?

[Andreas Lobinger]

Hello,

i'm now a julia user for some time, but still i have problems (comming from 
a c, f77, matlab, python background) to understand the type system. 
Especially when i try to integrate own code into packages (happened to 
Cairo + Compose, pending for Winston) i seem to choose always a more 
complicated way (my observation, code similar in py or c just looks 
shorter...) because somehow things do not match. 

So.

What to read to understand the julia approach to types and type 
interference?

Wishing a happy day,
   Andreas

Re: [julia-users] Re: catching an error in macro body

[Tim Holy]

I frequently debug macros by writing them as functions that operate on 
expressions, and then define the macro so it calls the function.

--Tim

On Wednesday, October 22, 2014 03:06:10 AM Ivar Nesje wrote:
> The problem is that the error from the macro evaluation is an error that
> gets raised when evaluating the source, so you need to put try outside the
> include of the file or use eval:
> 
> try
>eval(:(@a))
> end
> 
> You can also put the try block inside the macro.
> 
> Ivar
> 
> kl. 11:59:39 UTC+2 onsdag 22. oktober 2014 skrev Mauro følgende:
> > Is it possibly to catch an error happening during macro evaluation?
> > 
> > Example:
> > ```
> > macro a()
> > 
> > error("error in marco itself")
> > 
> > end
> > 
> > macro b()
> > 
> > :(error("error in marco generated code"))
> > 
> > end
> > 
> > # works:
> > try
> > 
> > @b
> > 
> > end
> > 
> > # still errors:
> > try
> > 
> > @a
> > 
> > end
> > ```
> > 
> > How to `try` the error in `@a`?

[julia-users] Re: catching an error in macro body

[Ivar Nesje]

The problem is that the error from the macro evaluation is an error that 
gets raised when evaluating the source, so you need to put try outside the 
include of the file or use eval:

try 
   eval(:(@a))
end

You can also put the try block inside the macro.

Ivar


kl. 11:59:39 UTC+2 onsdag 22. oktober 2014 skrev Mauro følgende:
>
> Is it possibly to catch an error happening during macro evaluation? 
>
> Example: 
> ``` 
> macro a() 
> error("error in marco itself") 
> end 
>
> macro b() 
> :(error("error in marco generated code")) 
> end 
>
> # works: 
> try 
> @b 
> end 
>
> # still errors: 
> try 
> @a 
> end 
> ``` 
>
> How to `try` the error in `@a`? 
>

[julia-users] catching an error in macro body

[Mauro]

Is it possibly to catch an error happening during macro evaluation?

Example:
```
macro a()
error("error in marco itself")
end

macro b()
:(error("error in marco generated code"))
end

# works:
try 
@b
end

# still errors:
try
@a
end
```

How to `try` the error in `@a`?

Re: [julia-users] Re: Modified Gram-Schmidt: getting the best out of Julia

[Tim Holy]

On Wednesday, October 22, 2014 02:27:30 AM Ján Dolinský wrote:
> I wonder why my vectorized code is not fast. I assume operations like "X' * 
> a" are using BLAS (e.g. BLAS.gemv()) and thus it should be as fast as a
> devectorized code at least in this case.

Yes, but their result requires allocating a temporary, and temporaries mean 
garbage collection. Try
- pre-allocating a couple of temp vectors to store the results of your matrix 
multiplies, and use functions like `Ac_mul_B!(dest, A, B)` to calculate A'*B 
and store the result in dest. 
- Do the subtractions & divisions using a loop, as suggested by Steven. 
- in each iteration you should call `view(X, :, l)` just once and store the 
result to a variable---no point recreating the same view 3 times per 
iteration.

--Tim

[julia-users] Re: Modified Gram-Schmidt: getting the best out of Julia

[Ján Dolinský]

Thanks. I'll try to use loops too. 

While still exploring my vectorized code I found out that matrix-vector 
multiplication is a lot faster when going column-wise, e.g.

X = rand(8000,8000);
a = rand(8000);
julia> @time r = a' * X;
elapsed time: 0.131689512 seconds (128224 bytes allocated)
going row-wise (slow)

julia> @time r = X' * a;
elapsed time: 0.057772737 seconds (64168 bytes allocated)
going column-wise (fast)

I therefore rewrote 
R[l,l+1:p]=(view(X,:,l)'*view(X,:,l+1:nvars))/sumabs2(view(X,:,l))
into
R[l,l+1:p]=(view(X,:,l+1:nvars)'*view(X,:,l))/sumabs2(view(X,:,l))

and I got a little increase in performance
julia> @time Q,R = mgs(X);
elapsed time: 6.815198093 seconds (8017005584 bytes allocated, 39.99% gc 
time)

Original qr function can do the job a lot more faster. I'd like to approach 
this speed at the end.
julia> @time Q,R = qr(X);
elapsed time: 0.542084044 seconds (3258 bytes allocated, 5.41% gc time)

I wonder why my vectorized code is not fast. I assume operations like "X' * 
a" are using BLAS (e.g. BLAS.gemv()) and thus it should be as fast as a 
devectorized code at least in this case.

Thanks,
Jan




Dňa utorok, 21. októbra 2014 17:16:10 UTC+2 Steven G. Johnson napísal(-a):
>
> Just write out the three nested loops of MGS, rather than using array 
> operations.  It will be a lot faster (in Julia).
>

Re: [julia-users] BLAS a' * b

[Tim Holy]

Two simple "rules":
- "Always" use concrete types for fields in types. When in doubt, use 
`isleaftype` to check.
- You "never" need to specify types in functions unless you want to control 
dispatch to different methods

Like most rules, there are circumstances where you can afford to/want to break 
them. But these are a pretty good starting point.

A final trick:

type MyType{AT<:AbstractMatrix}
mymatrix::AT
end

lets you use an AbstractMatrix in a type, concretely. See the FAQ for more 
detail, including any questions you're about to ask :) about specifying the 
element type in that matrix.

--Tim


On Tuesday, October 21, 2014 11:33:05 PM David van Leeuwen wrote:
> Hi,
> 
> Thanks for the explanation.  While writing the type abstract below I
> realized that I should have specified μ being Matrix{T} instead of
> Array{T}---but I hadn't suspected this was the culprit.  I still find it a
> difficult balance between over-specifying types in functions and type
> definitions and code generality.
> 
> Similarly, I am not sure if I win any performance pixy points by specifying
> Matrix{T} in types and functions rather than AbstractMatrix{T}---which is
> something I see a lot in other code.
> 
> Cheers,
> 
> ---david
> 
> On Wednesday, October 22, 2014 6:10:40 AM UTC+2, Simon Kornblith wrote:
> > On Tuesday, October 21, 2014 6:41:24 PM UTC-4, David van Leeuwen wrote:
> >> I replaced the `γ = broadcast()` with the lines below that.  No globals,
> >> but perhaps the field type gmm.μ is spoiling things.  I am not sure if
> >> this
> >> is a case of an abstractly typed field
> >> 
> >> type GMM{T<:FloatingPoint}
> >> ...
> >> 
> >>μ::Array{T}
> >> 
> >> ...
> >> end
> >> 
> >> Should I have written GMM{T2} in the declaration of stats()?
> > 
> > The declaration of stats() is fine, but μ is abstract because it's missing
> > the number of dimensions. It should be Array{T,2} or something like that,
> > or else GMM should be parametrized on the number of dimensions.
> > 
> > Simon

[julia-users] Re: Modified Gram-Schmidt: getting the best out of Julia

[Ján Dolinský]

Yes, indeed. It should be p. Thanks.

Dňa utorok, 21. októbra 2014 18:43:03 UTC+2 Douglas Bates napísal(-a):
>
> Also, in your Julia code you have used *nvars* without defining it.  You 
> probably intended
> n,nvars=size(X)
>
>
>
>

Re: [julia-users] return statement in inner expression

[Till Ehrengruber]

i just want to return from the function. i often encounter the problem 
where i need just one property out of a function result and want to return 
from the current function if the function contains "no data". this is e.g. 
usefull if you encapsulate data and status of your result into a special 
result type.

Am Dienstag, 21. Oktober 2014 23:28:31 UTC+2 schrieb Stefan Karpinski:
>
> Do you want the return to return from the function or do something else?
>
> On Tue, Oct 21, 2014 at 5:22 PM, Till Ehrengruber  > wrote:
>
>> what jacob already suggested as a working solution
>>
>> captures = ismatch(regex,str) ? match(regex, str).captures : return
>>
>> i can live with that but i would like to express this without the ismatch 
>> but still in a single line
>>
>> Am Dienstag, 21. Oktober 2014 21:41:35 UTC+2 schrieb Stefan Karpinski:
>>>
>>> What do you want your original example to do?
>>>
>>> On Oct 21, 2014, at 3:39 PM, Till Ehrengruber  
>>> wrote:
>>>
>>> ah i see. I'm a bit confused about conditionals since
>>>
>>> false || nothing
>>>
>>> for example works alright but
>>>
>>> nothing || false
>>>
>>> does not. Since the last one doesn't work like I expected my question is 
>>> pretty senseless ^^ But still i like the used syntax as this doesn't need 
>>> another function (like the `ismatch`) which can be pretty nasty to 
>>> implement when the computation is havy and you want to cache the result. Is 
>>> there any specific reason that conditionals aren't "symmetric"?
>>>
>>> Can't see the benefit of the nullable type in this case can you tell?
>>>
>>> Am Dienstag, 21. Oktober 2014 14:39:12 UTC+2 schrieb Jacob Quinn:

 Not sure what you’re asking here: do you have a case where you can’t 
 use a return within an expression? One thing to note with your example 
 above is that only boolean values can be used in conditionals (i.e. 
 if-statements, && and || operators, etc.), match returns the match 
 contents if there was a match, and the nothing value otherwise. So to 
 make your case work, you’d need something like:

 captures = ismatch(regex,str) ? match(regex, str).captures : return

 In Julia 0.4 dev branch, a Nullable type was recently merged and it’s 
 been discussed using a Nullable as the return type of regex operations 
 which would make it slightly easier to check for non-matches.

 -Jacob
 ​

 On Tue, Oct 21, 2014 at 3:52 AM, Till Ehrengruber  
 wrote:

> wouldn't it be nice to be able to use the return statement deep inside 
> your expression such that something like
>
> captures = (match(regex, str) || return).captures
>
> in this specific case i encountered a simple HttpRouter which just 
> skips the current handler and i don't really need the other results of 
> the 
> match expression
>
> regards till
>


>