[julia-users] Re: help understanding different ways of wrapping functions

[K leo]

Thanks so much for the tips.  The culprit is the keyword argument 
(xRat=0.).  Declaring it made the wrapped code twice as fast, but still way 
slower than the inline code.  But making it positional made the wrapped 
code just a little slower than the inline code - big improvement.

On Wednesday, September 28, 2016 at 2:50:40 PM UTC+8, Gunnar Farnebäck 
wrote:
>
> It's normal that manually inlined code of this kind is faster than wrapped 
> code unless the compiler manages to see the full inlining potential. In 
> this case the huge memory allocations for the wrapped solutions indicates 
> that it's nowhere near doing that at all. I doubt it will take you all the 
> way but start with modifying your inner M_CPS function to only take 
> positional arguments or declaring the type of the keyword argument as 
> suggested in the performance tips section of the manual.
>
> Den onsdag 28 september 2016 kl. 06:29:37 UTC+2 skrev K leo:
>>
>> I tested a few different ways of wrapping functions.  It looks different 
>> ways of wrapping has slightly different costs.  But the most confusing to 
>> me is that putting everything inline looks much faster than wrapping things 
>> up.  I would understand this in other languages, but I thought Julia 
>> advocates simple wrapping.  Can anyone help explain what is happening 
>> below, and how I can do most efficient wrapping in the demo code?
>>
>> Demo code is included below.
>>
>> julia> versioninfo()
>> Julia Version 0.5.0
>> Commit 3c9d753 (2016-09-19 18:14 UTC)
>> Platform Info:
>>   System: Linux (x86_64-pc-linux-gnu)
>>   CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
>>   WORD_SIZE: 64
>>   BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
>>   LAPACK: libopenblas64_
>>   LIBM: libopenlibm
>>   LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)
>>
>> julia> testFunc()
>> calling LoopCP (everything inline)
>>   0.097556 seconds (2.10 k allocations: 290.625 KB)
>> elapsed time (ns): 97555896
>> bytes allocated:   297600
>> pool allocs:   2100
>> [0.0,4200.0,0.0,0.0,4200.0,4200.0,4200.0,4200.0,0.0,4200.0,4200.0]
>>
>> calling LoopCP0 (slightly wrapped)
>>   4.173830 seconds (49.78 M allocations: 2.232 GB, 5.83% gc time)
>> elapsed time (ns): 4173830495
>> gc time (ns):  243516584
>> bytes allocated:   2396838538
>> pool allocs:   49783357
>> GC pauses: 104
>> full collections:  1
>> [4200.0,0.0,4200.0,4200.0,0.0,0.0,0.0,0.0,4200.0,0.0,0.0]
>>
>> calling LoopCP1 (wrapped one way)
>>   5.274723 seconds (59.59 M allocations: 2.378 GB, 3.62% gc time)
>> elapsed time (ns): 5274722983
>> gc time (ns):  191036337
>> bytes allocated:   2553752638
>> pool allocs:   59585834
>> GC pauses: 112
>> [8400.0,0.0,8400.0,8400.0,0.0,0.0,0.0,0.0,8400.0,0.0,0.0]
>>
>> calling LoopCP2 (wrapped another way)
>>   5.212895 seconds (59.58 M allocations: 2.378 GB, 3.60% gc time)
>> elapsed time (ns): 5212894550
>> gc time (ns):  187696529
>> bytes allocated:   2553577600
>> pool allocs:   59582100
>> GC pauses: 111
>> [0.0,8400.0,0.0,0.0,8400.0,8400.0,8400.0,8400.0,0.0,8400.0,8400.0]
>>
>> const dim=1000
>>>
>>>
 type Tech
>>>
>>> a::Array{Float64,1}
>>>
>>> c::Array{Int,1}
>>>
>>>
 function Tech()
>>>
>>> this = new()
>>>
>>> this.a = zeros(Float64, dim)
>>>
>>> this.c = rand([0,1;], dim)
>>>
>>> this
>>>
>>> end
>>>
>>> end
>>>
>>>
 function LoopCP(csign::Int, tech::Tech)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> @inbounds for i = 1:dim
>>>
>>> if csign == tech.c[i]
>>>
>>> tech.a[i] += 2.*xRat
>>>
>>> else
>>>
>>> tech.a[i] = 0.
>>>
>>> end
>>>
>>> end
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function M_CPS(i::Int, csign::Int, tech::Tech; xRat=0.)
>>>
>>> if csign == tech.c[i]
>>>
>>> tech.a[i] += 2.*xRat
>>>
>>> else
>>>
>>> tech.a[i] = 0.
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function LoopCP0(csign::Int, tech::Tech)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> @inbounds for i = 1:dim
>>>
>>> M_CPS(i, csign, tech, xRat=xRat)
>>>
>>> end
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function MoleculeWrapS(csign::Int, tech::Tech, molecule::Function, 
 xRat=0.)
>>>
>>> @inbounds for i = 1:dim
>>>
>>> molecule(i, csign, tech; xRat=xRat)
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function LoopRunnerM1(csign::Int, tech::Tech, molecule::Function)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> MoleculeWrapS(csign, tech, molecule, xRat)
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 LoopCP1(csign::Int, tech::Tech) = LoopRunnerM1(csign, tech, M_CPS)
>>>
>>>
 Wra

Re: [julia-users] Re: help understanding different ways of wrapping functions

[Mauro]

On Wed, 2016-09-28 at 08:50, Gunnar Farnebäck  wrote:
> It's normal that manually inlined code of this kind is faster than wrapped
> code unless the compiler manages to see the full inlining potential. In
> this case the huge memory allocations for the wrapped solutions indicates
> that it's nowhere near doing that at all. I doubt it will take you all the
> way but start with modifying your inner M_CPS function to only take
> positional arguments or declaring the type of the keyword argument as
> suggested in the performance tips section of the manual.

Even annotated keywords are slower than normal, positional ones (except
when their default value is used, as far as I recall).

> Den onsdag 28 september 2016 kl. 06:29:37 UTC+2 skrev K leo:
>>
>> I tested a few different ways of wrapping functions.  It looks different
>> ways of wrapping has slightly different costs.  But the most confusing to
>> me is that putting everything inline looks much faster than wrapping things
>> up.  I would understand this in other languages, but I thought Julia
>> advocates simple wrapping.  Can anyone help explain what is happening
>> below, and how I can do most efficient wrapping in the demo code?
>>
>> Demo code is included below.
>>
>> julia> versioninfo()
>> Julia Version 0.5.0
>> Commit 3c9d753 (2016-09-19 18:14 UTC)
>> Platform Info:
>>   System: Linux (x86_64-pc-linux-gnu)
>>   CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
>>   WORD_SIZE: 64
>>   BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
>>   LAPACK: libopenblas64_
>>   LIBM: libopenlibm
>>   LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)
>>
>> julia> testFunc()
>> calling LoopCP (everything inline)
>>   0.097556 seconds (2.10 k allocations: 290.625 KB)
>> elapsed time (ns): 97555896
>> bytes allocated:   297600
>> pool allocs:   2100
>> [0.0,4200.0,0.0,0.0,4200.0,4200.0,4200.0,4200.0,0.0,4200.0,4200.0]
>>
>> calling LoopCP0 (slightly wrapped)
>>   4.173830 seconds (49.78 M allocations: 2.232 GB, 5.83% gc time)
>> elapsed time (ns): 4173830495
>> gc time (ns):  243516584
>> bytes allocated:   2396838538
>> pool allocs:   49783357
>> GC pauses: 104
>> full collections:  1
>> [4200.0,0.0,4200.0,4200.0,0.0,0.0,0.0,0.0,4200.0,0.0,0.0]
>>
>> calling LoopCP1 (wrapped one way)
>>   5.274723 seconds (59.59 M allocations: 2.378 GB, 3.62% gc time)
>> elapsed time (ns): 5274722983
>> gc time (ns):  191036337
>> bytes allocated:   2553752638
>> pool allocs:   59585834
>> GC pauses: 112
>> [8400.0,0.0,8400.0,8400.0,0.0,0.0,0.0,0.0,8400.0,0.0,0.0]
>>
>> calling LoopCP2 (wrapped another way)
>>   5.212895 seconds (59.58 M allocations: 2.378 GB, 3.60% gc time)
>> elapsed time (ns): 5212894550
>> gc time (ns):  187696529
>> bytes allocated:   2553577600
>> pool allocs:   59582100
>> GC pauses: 111
>> [0.0,8400.0,0.0,0.0,8400.0,8400.0,8400.0,8400.0,0.0,8400.0,8400.0]
>>
>> const dim=1000
>>>
>>>
 type Tech
>>>
>>> a::Array{Float64,1}
>>>
>>> c::Array{Int,1}
>>>
>>>
 function Tech()
>>>
>>> this = new()
>>>
>>> this.a = zeros(Float64, dim)
>>>
>>> this.c = rand([0,1;], dim)
>>>
>>> this
>>>
>>> end
>>>
>>> end
>>>
>>>
 function LoopCP(csign::Int, tech::Tech)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> @inbounds for i = 1:dim
>>>
>>> if csign == tech.c[i]
>>>
>>> tech.a[i] += 2.*xRat
>>>
>>> else
>>>
>>> tech.a[i] = 0.
>>>
>>> end
>>>
>>> end
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function M_CPS(i::Int, csign::Int, tech::Tech; xRat=0.)
>>>
>>> if csign == tech.c[i]
>>>
>>> tech.a[i] += 2.*xRat
>>>
>>> else
>>>
>>> tech.a[i] = 0.
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function LoopCP0(csign::Int, tech::Tech)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> @inbounds for i = 1:dim
>>>
>>> M_CPS(i, csign, tech, xRat=xRat)
>>>
>>> end
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function MoleculeWrapS(csign::Int, tech::Tech, molecule::Function,
 xRat=0.)
>>>
>>> @inbounds for i = 1:dim
>>>
>>> molecule(i, csign, tech; xRat=xRat)
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 function LoopRunnerM1(csign::Int, tech::Tech, molecule::Function)
>>>
>>> for j=1:10
>>>
>>> for xRat in [1.:20.;]
>>>
>>> MoleculeWrapS(csign, tech, molecule, xRat)
>>>
>>> end #
>>>
>>> end
>>>
>>> nothing
>>>
>>> end
>>>
>>>
 LoopCP1(csign::Int, tech::Tech) = LoopRunnerM1(csign, tech, M_CPS)
>>>
>>>
 WrapCPS(csign::Int, tech::Tech, xRat=0.) = MoleculeWrapS(csign, tech,
 M_CPS, xRat)
>>>
>>>
 function LoopRunnerM2(csign::Int, tech::Tech, loop::Function)
>>>
>>> for j=1:10
>>>
>>>  

[julia-users] Re: help understanding different ways of wrapping functions

[Gunnar Farnebäck]

It's normal that manually inlined code of this kind is faster than wrapped 
code unless the compiler manages to see the full inlining potential. In 
this case the huge memory allocations for the wrapped solutions indicates 
that it's nowhere near doing that at all. I doubt it will take you all the 
way but start with modifying your inner M_CPS function to only take 
positional arguments or declaring the type of the keyword argument as 
suggested in the performance tips section of the manual.

Den onsdag 28 september 2016 kl. 06:29:37 UTC+2 skrev K leo:
>
> I tested a few different ways of wrapping functions.  It looks different 
> ways of wrapping has slightly different costs.  But the most confusing to 
> me is that putting everything inline looks much faster than wrapping things 
> up.  I would understand this in other languages, but I thought Julia 
> advocates simple wrapping.  Can anyone help explain what is happening 
> below, and how I can do most efficient wrapping in the demo code?
>
> Demo code is included below.
>
> julia> versioninfo()
> Julia Version 0.5.0
> Commit 3c9d753 (2016-09-19 18:14 UTC)
> Platform Info:
>   System: Linux (x86_64-pc-linux-gnu)
>   CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
>   WORD_SIZE: 64
>   BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
>   LAPACK: libopenblas64_
>   LIBM: libopenlibm
>   LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)
>
> julia> testFunc()
> calling LoopCP (everything inline)
>   0.097556 seconds (2.10 k allocations: 290.625 KB)
> elapsed time (ns): 97555896
> bytes allocated:   297600
> pool allocs:   2100
> [0.0,4200.0,0.0,0.0,4200.0,4200.0,4200.0,4200.0,0.0,4200.0,4200.0]
>
> calling LoopCP0 (slightly wrapped)
>   4.173830 seconds (49.78 M allocations: 2.232 GB, 5.83% gc time)
> elapsed time (ns): 4173830495
> gc time (ns):  243516584
> bytes allocated:   2396838538
> pool allocs:   49783357
> GC pauses: 104
> full collections:  1
> [4200.0,0.0,4200.0,4200.0,0.0,0.0,0.0,0.0,4200.0,0.0,0.0]
>
> calling LoopCP1 (wrapped one way)
>   5.274723 seconds (59.59 M allocations: 2.378 GB, 3.62% gc time)
> elapsed time (ns): 5274722983
> gc time (ns):  191036337
> bytes allocated:   2553752638
> pool allocs:   59585834
> GC pauses: 112
> [8400.0,0.0,8400.0,8400.0,0.0,0.0,0.0,0.0,8400.0,0.0,0.0]
>
> calling LoopCP2 (wrapped another way)
>   5.212895 seconds (59.58 M allocations: 2.378 GB, 3.60% gc time)
> elapsed time (ns): 5212894550
> gc time (ns):  187696529
> bytes allocated:   2553577600
> pool allocs:   59582100
> GC pauses: 111
> [0.0,8400.0,0.0,0.0,8400.0,8400.0,8400.0,8400.0,0.0,8400.0,8400.0]
>
> const dim=1000
>>
>>
>>> type Tech
>>
>> a::Array{Float64,1}
>>
>> c::Array{Int,1}
>>
>>
>>> function Tech()
>>
>> this = new()
>>
>> this.a = zeros(Float64, dim)
>>
>> this.c = rand([0,1;], dim)
>>
>> this
>>
>> end
>>
>> end
>>
>>
>>> function LoopCP(csign::Int, tech::Tech)
>>
>> for j=1:10
>>
>> for xRat in [1.:20.;]
>>
>> @inbounds for i = 1:dim
>>
>> if csign == tech.c[i]
>>
>> tech.a[i] += 2.*xRat
>>
>> else
>>
>> tech.a[i] = 0.
>>
>> end
>>
>> end
>>
>> end #
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> function M_CPS(i::Int, csign::Int, tech::Tech; xRat=0.)
>>
>> if csign == tech.c[i]
>>
>> tech.a[i] += 2.*xRat
>>
>> else
>>
>> tech.a[i] = 0.
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> function LoopCP0(csign::Int, tech::Tech)
>>
>> for j=1:10
>>
>> for xRat in [1.:20.;]
>>
>> @inbounds for i = 1:dim
>>
>> M_CPS(i, csign, tech, xRat=xRat)
>>
>> end
>>
>> end #
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> function MoleculeWrapS(csign::Int, tech::Tech, molecule::Function, 
>>> xRat=0.)
>>
>> @inbounds for i = 1:dim
>>
>> molecule(i, csign, tech; xRat=xRat)
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> function LoopRunnerM1(csign::Int, tech::Tech, molecule::Function)
>>
>> for j=1:10
>>
>> for xRat in [1.:20.;]
>>
>> MoleculeWrapS(csign, tech, molecule, xRat)
>>
>> end #
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> LoopCP1(csign::Int, tech::Tech) = LoopRunnerM1(csign, tech, M_CPS)
>>
>>
>>> WrapCPS(csign::Int, tech::Tech, xRat=0.) = MoleculeWrapS(csign, tech, 
>>> M_CPS, xRat)
>>
>>
>>> function LoopRunnerM2(csign::Int, tech::Tech, loop::Function)
>>
>> for j=1:10
>>
>> for xRat in [1.:20.;]
>>
>> loop(csign, tech, xRat)
>>
>> end #
>>
>> end
>>
>> nothing
>>
>> end
>>
>>
>>> LoopCP2(csign::Int, tech::Tech) = LoopRunnerM2(csign, tech, WrapCPS)
>>
>>
>>> function testFunc()
>>
>> tech = Tech()
>>
>> nloops = 100
>>
>>
>>> println("calling LoopCP (everything inline)")
>>
>> tech.a = zeros(tech.a)

[julia-users] Re: Help: Printing method in julia

[Weicheng Zhu]

OK, I see. Thank you!

On Wednesday, September 21, 2016 at 6:17:30 PM UTC-5, Steven G. Johnson 
wrote:
>
>
>
> On Wednesday, September 21, 2016 at 4:24:10 PM UTC-4, Weicheng Zhu wrote:
>>
>> Hi Dr. Johnson,
>> Thank you very much for your help! Not I understand how it works.
>> This problem has confused me a long time since I started learning julia.
>>
>> So if I want a `mytype` object to be printed in the pretty-printed lines 
>> by default, I have to define the display method, right?
>>
>
> No.  Only override show, not display.   And in Julia 0.4 the 3-argument 
> show is called writemime, not show.
>

[julia-users] Re: Help: Printing method in julia

[Steven G. Johnson]

On Wednesday, September 21, 2016 at 4:24:10 PM UTC-4, Weicheng Zhu wrote:
>
> Hi Dr. Johnson,
> Thank you very much for your help! Not I understand how it works.
> This problem has confused me a long time since I started learning julia.
>
> So if I want a `mytype` object to be printed in the pretty-printed lines 
> by default, I have to define the display method, right?
>

No.  Only override show, not display.   And in Julia 0.4 the 3-argument 
show is called writemime, not show.

[julia-users] Re: Help: Printing method in julia

[Weicheng Zhu]

OK, I see.
It seems that if I define the two show method directly in the REPL, it 
doesn't work. But it works when I wrap them into a module:)


On Wednesday, September 21, 2016 at 3:24:10 PM UTC-5, Weicheng Zhu wrote:
>
> Hi Dr. Johnson,
> Thank you very much for your help! Not I understand how it works.
> This problem has confused me a long time since I started learning julia.
>
> So if I want a `mytype` object to be printed in the pretty-printed lines 
> by default, I have to define the display method, right?
> I guess my definition of the display method is not quite correct, although 
> it works in REPL. 
>
>
>> import Base.show
>> type mytype
>>   x::Array{Float64,2}
>> end
>> function show(io::IO, x::mytype)
>>   show(io, x.x)
>> end
>> function show(io::IO, m::MIME"text/plain", x::mytype)
>>   show(io, m, x.x)
>> end
>> show(STDOUT, MIME("text/plain"), mytype(x))
>> function Base.display(x::mytype)
>>   println("mytype object:")
>>   show(STDOUT, MIME("text/plain"), x)
>> end
>> julia> x = rand(5,2)
>> julia> mytype(x)
>> mytype object:
>> 5×2 Array{Float64,2}:
>>  0.05127   0.908138
>>  0.527729  0.835109
>>  0.657212  0.275374
>>  0.119597  0.659259
>>  0.94996   0.36432 
>
>
>
> On Wednesday, September 21, 2016 at 2:27:18 PM UTC-5, Steven G. Johnson 
> wrote:
>>
>>
>>
>> On Wednesday, September 21, 2016 at 2:54:15 PM UTC-4, Weicheng Zhu wrote:
>>>
>>> Hi all,
>>>
>>> I have a few simple questions to ask.
>>>
>>> 1) What function is invoked when I type x in the following example?
>>>
>>
>> display(x), which calls show(STDOUT, MIME("text/plain"), x)  [or 
>> writemime in Julia ≤ 0.4]
>>
>>
>> (Under the hood, this eventually calls a function Base.showarray)
>>
>>
>> 2) When I define a type which takes a matrix as a member, how to define 
>>> the show method to print x as shown above in julia 0.5.
>>> julia> type mytype
>>>x::Array{Float64,2}
>>> end
>>> julia> mytype(x)
>>> mytype([0.923288 0.0157897; 0.439387 0.50823; … ; 0.605268 0.416877; 
>>> 0.223898 0.558542])
>>
>>
>> You want to define two show methods: show(io::IO, x::mytype), which calls 
>> show(io, x.x) and outputs everything on a single line, and show(io::IO, 
>> m::MIME"text/plain", x::mytype), which calls show(io, m, x.x) and outputs 
>> multiple pretty-printed lines.
>>
>>
>> (You can also define additional show methods, e.g. for HTML or Markdown 
>> output, for even nicer display in an environment like IJulia that supports 
>> other MIME types.)
>>
>

[julia-users] Re: Help: Printing method in julia

[Weicheng Zhu]

Hi Dr. Johnson,
Thank you very much for your help! Not I understand how it works.
This problem has confused me a long time since I started learning julia.

So if I want a `mytype` object to be printed in the pretty-printed lines by 
default, I have to define the display method, right?
I guess my definition of the display method is not quite correct, although 
it works in REPL. 


> import Base.show
> type mytype
>   x::Array{Float64,2}
> end
> function show(io::IO, x::mytype)
>   show(io, x.x)
> end
> function show(io::IO, m::MIME"text/plain", x::mytype)
>   show(io, m, x.x)
> end
> show(STDOUT, MIME("text/plain"), mytype(x))
> function Base.display(x::mytype)
>   println("mytype object:")
>   show(STDOUT, MIME("text/plain"), x)
> end
> julia> x = rand(5,2)
> julia> mytype(x)
> mytype object:
> 5×2 Array{Float64,2}:
>  0.05127   0.908138
>  0.527729  0.835109
>  0.657212  0.275374
>  0.119597  0.659259
>  0.94996   0.36432 



On Wednesday, September 21, 2016 at 2:27:18 PM UTC-5, Steven G. Johnson 
wrote:
>
>
>
> On Wednesday, September 21, 2016 at 2:54:15 PM UTC-4, Weicheng Zhu wrote:
>>
>> Hi all,
>>
>> I have a few simple questions to ask.
>>
>> 1) What function is invoked when I type x in the following example?
>>
>
> display(x), which calls show(STDOUT, MIME("text/plain"), x)  [or writemime 
> in Julia ≤ 0.4]
>
>
> (Under the hood, this eventually calls a function Base.showarray)
>
>
> 2) When I define a type which takes a matrix as a member, how to define 
>> the show method to print x as shown above in julia 0.5.
>> julia> type mytype
>>x::Array{Float64,2}
>> end
>> julia> mytype(x)
>> mytype([0.923288 0.0157897; 0.439387 0.50823; … ; 0.605268 0.416877; 
>> 0.223898 0.558542])
>
>
> You want to define two show methods: show(io::IO, x::mytype), which calls 
> show(io, x.x) and outputs everything on a single line, and show(io::IO, 
> m::MIME"text/plain", x::mytype), which calls show(io, m, x.x) and outputs 
> multiple pretty-printed lines.
>
>
> (You can also define additional show methods, e.g. for HTML or Markdown 
> output, for even nicer display in an environment like IJulia that supports 
> other MIME types.)
>

[julia-users] Re: Help: Printing method in julia

[Steven G. Johnson]

On Wednesday, September 21, 2016 at 2:54:15 PM UTC-4, Weicheng Zhu wrote:
>
> Hi all,
>
> I have a few simple questions to ask.
>
> 1) What function is invoked when I type x in the following example?
>

display(x), which calls show(STDOUT, MIME("text/plain"), x)  [or writemime 
in Julia ≤ 0.4]


(Under the hood, this eventually calls a function Base.showarray)


2) When I define a type which takes a matrix as a member, how to define the 
> show method to print x as shown above in julia 0.5.
> julia> type mytype
>x::Array{Float64,2}
> end
> julia> mytype(x)
> mytype([0.923288 0.0157897; 0.439387 0.50823; … ; 0.605268 0.416877; 
> 0.223898 0.558542])


You want to define two show methods: show(io::IO, x::mytype), which calls 
show(io, x.x) and outputs everything on a single line, and show(io::IO, 
m::MIME"text/plain", x::mytype), which calls show(io, m, x.x) and outputs 
multiple pretty-printed lines.


(You can also define additional show methods, e.g. for HTML or Markdown 
output, for even nicer display in an environment like IJulia that supports 
other MIME types.)

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Thanks for the information. I will install 0.5.0-rc4 binary and try a small 
benchmark among Julia, Matlab (forgot which version I have probably 2013), 
also Anaconda Python 2.7 with and without MKL. I will avoid loops in doing 
so.

Will drop an email regarding commercial solution over at Julia Computing.



On Saturday, September 17, 2016 at 1:00:26 AM UTC-5, Tony Kelman wrote:
>
> That benchmark doesn't say what they were using for "normal backends" - 
> was it openblas or atlas or the reference blas, which set of kernels, were 
> they using multithreading, etc. The open source Julia build will almost 
> certainly have faster FFT's than SciPy without MKL, since Julia uses FFTW 
> but SciPy uses a slower non-GPL-licensed implementation. Note that we had a 
> bug on our buildbots regarding optimization flags in some dependencies so 
> you may want to wait until 0.4.7 binaries or test with 0.5.0-rc4 or newer 
> for comparing FFT performance.
>
> Julia Computing offers custom solutions that aren't necessarily listed on 
> the products page of our website yet, you can inquire at the contact 
> address there.
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

That benchmark doesn't say what they were using for "normal backends" - was it 
openblas or atlas or the reference blas, which set of kernels, were they using 
multithreading, etc. The open source Julia build will almost certainly have 
faster FFT's than SciPy without MKL, since Julia uses FFTW but SciPy uses a 
slower non-GPL-licensed implementation. Note that we had a bug on our buildbots 
regarding optimization flags in some dependencies so you may want to wait until 
0.4.7 binaries or test with 0.5.0-rc4 or newer for comparing FFT performance.

Julia Computing offers custom solutions that aren't necessarily listed on the 
products page of our website yet, you can inquire at the contact address there.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

I saw benchmarks showing impressive speedup using Anaconda + MKL vs. open 
source library. For 
example, https://github.com/ContinuumIO/mkl-optimizations-benchmarks

For my application, a representative problem would be split-step Fourier 
method (https://en.wikipedia.org/wiki/Split-step_method). This would 
involve multiple arrays (e.g. 8), each containing tens of thousands to 
hundreds of thousands of complex values, going through FFT - array 
calculation - inverse FFT - array calculation... repeated many many times. 
There could be dozens of parameter combinations to run the calculation on, 
which could be parallelized. Currently the calculation is done in Matlab 
with some use of GPU, which is reasonably fast but not fast enough. Python 
would be slow for this situation as each step depends on previous step in a 
nonlinear way so the loop cannot be vectorized. I was hoping to convert the 
Matlab project into Julia to gain significant speedup (fast Julia base + 
MKL + GPU) on a single 6-core machine. If then I can get a few machines to 
work together to test different parameter sets that'd be even better - 
realistic limitation is we have to stick with Windows and won't have a real 
cluster.

Based on the instructions, building Julia with MKL seems to be easy on 
Linux. I didn't realize it could be this hard for Windows, at least for me.

Commercial distribution of Julia could be an option if the license fee is 
low (people got spoiled by Anaconda). However, I don't see one that 
supports MKL yet. 



On Friday, September 16, 2016 at 5:46:39 PM UTC-5, Tony Kelman wrote:
>
> mic probably means xeon phi, so that's likely for cross compiling to a 
> xeon phi device from a windows host. Based on the sizes of the .lib files, 
> I take what I said back, those probably are static libraries. I did this 
> many months ago and have a Make.user somewhere.
>
> Before you spend too much time on this, how sure are you that MKL will 
> make a major difference over openblas on your application? openblas is 
> close in performance to MKl on many, but not quite all operations. In some 
> cases it may even be faster. Open source Julia is much better supported 
> against openblas, especially on Windows. There are avenues for commercial 
> supported Julia as well.
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

mic probably means xeon phi, so that's likely for cross compiling to a xeon phi 
device from a windows host. Based on the sizes of the .lib files, I take what I 
said back, those probably are static libraries. I did this many months ago and 
have a Make.user somewhere.

Before you spend too much time on this, how sure are you that MKL will make a 
major difference over openblas on your application? openblas is close in 
performance to MKl on many, but not quite all operations. In some cases it may 
even be faster. Open source Julia is much better supported against openblas, 
especially on Windows. There are avenues for commercial supported Julia as well.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

I completely restarted the build process from fresh (git clone). In 
addition to the renaming and copying .lib and .dll files, I also did some 
many configuration of environmental variables before the build. 
These are mostly based on 
what 
C:\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\bin\mklvars.bat 
would have done.

export 
MKLROOT=/c/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl
export MKL_TARGET_ARCH=intel64
export 
REDIST=/c/IntelSWTools/compilers_and_libraries_2017.0.109/windows/redist
export MKL_LP64_ILP64=lp64
export 
LIB=/c/IntelSWTools/compilers_and_libraries_2017.0.109/windows/tbb/lib/intel64/vc_mt:/C/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64:/C/IntelSWTools/compilers_and_libraries_2017.0.109/windows/compiler/lib/intel64
export 
INCLUDE=/C/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/include

Now I am stuck at a different place:

Making install in SVD
Making install in SYM
libtool: compile:  gcc -m64 -DTIME_WITH_SYS_TIME=1 -DHAVE_INTTYPES_H=1 
-DHAVE_STDINT_H=1 -DHAVE_LOCALE_H=1 -DHAVE_WCHAR_H=1 -DHAVE_STDARG=1 
-DHAVE_SYS_TIME_H=1 -DHAVE_STDINT_H=1 -DHAVE_VA_COPY=1 -DHAVE_SETLOCALE=1 
-DHAVE_GETTIMEOFDAY=1 -DHAVE_LONG_LONG=1 -DHAVE_INTMAX_T=1 
-DMPFR_HAVE_INTMAX_MAX=1 -DMPFR_HAVE_FESETROUND=1 -DHAVE_DENORMS=1 
-DHAVE_ROUND=1 -DHAVE_TRUNC=1 -DHAVE_FLOOR=1 -DHAVE_CEIL=1 
-DHAVE_NEARBYINT=1 -DHAVE_LDOUBLE_IEEE_EXT_LITTLE=1 -DLT_OBJDIR=\".libs/\" 
-DHAVE_ATTRIBUTE_MODE=1 -I. -I../src -I../src -DSRCDIR=\".\" 
-I/c/users/zpan/Documents/GitHub/julia/usr/include -DNPRINTF_L -DNPRINTF_T 
-DNPRINTF_J -MT random2.lo -MD -MP -MF .deps/random2.Tpo -c random2.c 
 -DDLL_EXPORT -DPIC -o .libs/random2.o
mv: cannot stat 
'/c/users/zpan/Documents/GitHub/julia/usr/bin/libarpack-2.dll': No such 
file or directory
make[1]: *** [Makefile:1331: 
/c/users/zpan/Documents/GitHub/julia/usr/bin/libarpack.dll] Error 1
make[1]: *** Waiting for unfinished jobs

I cannot find libarpack-2.dll or libarpack.dll anywhere.

BTW, I bumped into a folder under MKL installation: 
C:\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win_mic
The files inside seems to be Linux-like library files, but I don't know how 
they should be used:

zpan@WSPWORK 
C:\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win_mic
> dir
 Volume in drive C is Windows7_OS
 Volume Serial Number is 4131-0400

 Directory of 
C:\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win_mic

09/16/2016  15:02  .
09/16/2016  15:02  ..
08/01/2016  18:3619,309,845 libmkl_ao_worker.so
08/01/2016  18:30 2,119,960 libmkl_blas95_ilp64.a
08/01/2016  18:30 2,147,752 libmkl_blas95_lp64.a
08/01/2016  18:34   153,164,198 libmkl_core.a
08/01/2016  18:36   122,863,997 libmkl_core.so
08/01/2016  18:3424,628,804 libmkl_intel_ilp64.a
08/01/2016  18:3610,850,199 libmkl_intel_ilp64.so
08/01/2016  18:3426,277,006 libmkl_intel_lp64.a
08/01/2016  18:3612,012,951 libmkl_intel_lp64.so
08/01/2016  18:3452,875,578 libmkl_intel_thread.a
08/01/2016  18:3640,692,525 libmkl_intel_thread.so
08/01/2016  18:3225,226,176 libmkl_lapack95_ilp64.a
08/01/2016  18:3125,654,272 libmkl_lapack95_lp64.a
08/01/2016  18:36 5,128,544 libmkl_rt.so
08/01/2016  18:3430,265,120 libmkl_sequential.a
08/01/2016  18:3624,301,227 libmkl_sequential.so
08/01/2016  18:3432,158,354 libmkl_tbb_thread.a
08/01/2016  18:3625,593,814 libmkl_tbb_thread.so
09/16/2016  15:02  locale
  18 File(s)635,270,322 bytes
   3 Dir(s)  31,684,468,736 bytes free

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

BTW I have re-installed Intel MKL in a folder without any spaces in the 
path name, so you may notice the location changed. Not sure if it helps, 
just trying.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

Something's broken, you'll probably have to run that in gdb and try to get 
a backtrace. I don't think the .lib files are full-fledged static libraries 
(how big are they?), I suspect they are import libraries that go along with 
the corresponding dll's. So the mingw naming convention from them would be 
libmkl_rt.dll.a for mkl_rt.lib. The main thing you want to do is make sure 
the blas and lapack library ccall's are getting pointed to the mkl_rt.dll.


On Wednesday, September 14, 2016 at 3:14:52 PM UTC-7, Zhong Pan wrote:
>
> Thanks, Tony, with your help I think I am pretty close.
>
> Yes you are right, I searched for *.dll and found them in a folder that I 
> didn't notice before.
>
> Here's what I did last based on your comments to move things forward a few 
> steps more:
>
> (1) Copied all the 17 .lib files from "C:\Program Files 
> (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win"
>  
> to 
> "C:\Users\zpan\Documents\GitHub\julia\usr\x86_64-w64-mingw32\sys-root\mingw\lib".
>  
> Changed them from .lib files to .a files and added prefix "lib". So now the 
> 17 files look like:
>
> libmkl_blas95_ilp64.a
> libmkl_blas95_lp64.a
> libmkl_core.a
> libmkl_core_dll.a
> libmkl_intel_ilp64.a
> libmkl_intel_ilp64_dll.a
> libmkl_intel_lp64.a
> libmkl_intel_lp64_dll.a
> libmkl_intel_thread.a
> libmkl_intel_thread_dll.a
> libmkl_lapack95_ilp64.a
> libmkl_lapack95_lp64.a
> libmkl_rt.a
> libmkl_sequential.a
> libmkl_sequential_dll.a
> libmkl_tbb_thread.a
> libmkl_tbb_thread_dll.a
>
>
>
> (2) Copied all the .dll files from "C:\Program Files 
> (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\redist\intel64_win\mkl"
>  
> to 
> "C:\Users\zpan\Documents\GitHub\julia\usr\x86_64-w64-mingw32\sys-root\mingw\bin".
>  
> This includes the folder "33". Here are all the files that got copied:
>
> 1033/mkl_msg.dll
> libimalloc.dll
> mkl_avx.dll
> mkl_avx2.dll
> mkl_avx512.dll
> mkl_avx512_mic.dll
> mkl_core.dll
> mkl_def.dll
> mkl_intel_thread.dll
> mkl_mc.dll
> mkl_mc3.dll
> mkl_rt.dll
> mkl_sequential.dll
> mkl_tbb_thread.dll
> mkl_vml_avx.dll
> mkl_vml_avx2.dll
> mkl_vml_avx512.dll
> mkl_vml_avx512_mic.dll
> mkl_vml_cmpt.dll
> mkl_vml_def.dll
> mkl_vml_mc.dll
> mkl_vml_mc2.dll
> mkl_vml_mc3.dll
>
>
> (3) "make -j 4" again. This time, there was no "Cannot find BLAS 
> libraries" error anymore. The build went on for a while until here:
>
> floatfuncs.jl
>
> Please submit a bug report with steps to reproduce this fault, and any 
> error messages that follow (in their entirety). Thanks.
> Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: 
> )
> unknown function (ip: )
> *** This error is usually fixed by running `make clean`. If the error 
> persists, try `make cleanall`. ***
> make[1]: *** [Makefile:200: 
> /c/users/zpan/Documents/GitHub/julia/usr/lib/julia/sys.o] Error 1
> make: *** [Makefile:69: julia-sysimg-release] Error 2
>
>
> I tried both "make clean" and "make cleanall", and the same problem 
> persisted. I also tried renaming the .dll and .a files in a couple ways in 
> vain. 
>
> Again, any help appreciated!! Hope I am close to success. When I am done I 
> will repeat the process and document it here for others to see.
>
>
>
>
> On Wednesday, September 14, 2016 at 3:02:40 PM UTC-5, Tony Kelman wrote:
>>
>> Which configure file was this from? Julia doesn't need the .lib files, it 
>> needs the dll, and I assure you there are dlls in MKL. However some of the 
>> dependencies may need the .lib files, and/or you can try copying them to a 
>> .dll.a file name if that helps libtool or configure work better.
>>
>>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Thanks, Tony, with your help I think I am pretty close.

Yes you are right, I searched for *.dll and found them in a folder that I 
didn't notice before.

Here's what I did last based on your comments to move things forward a few 
steps more:

(1) Copied all the 17 .lib files from "C:\Program Files 
(x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win"
 
to 
"C:\Users\zpan\Documents\GitHub\julia\usr\x86_64-w64-mingw32\sys-root\mingw\lib".
 
Changed them from .lib files to .a files and added prefix "lib". So now the 
17 files look like:

libmkl_blas95_ilp64.a
libmkl_blas95_lp64.a
libmkl_core.a
libmkl_core_dll.a
libmkl_intel_ilp64.a
libmkl_intel_ilp64_dll.a
libmkl_intel_lp64.a
libmkl_intel_lp64_dll.a
libmkl_intel_thread.a
libmkl_intel_thread_dll.a
libmkl_lapack95_ilp64.a
libmkl_lapack95_lp64.a
libmkl_rt.a
libmkl_sequential.a
libmkl_sequential_dll.a
libmkl_tbb_thread.a
libmkl_tbb_thread_dll.a



(2) Copied all the .dll files from "C:\Program Files 
(x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\redist\intel64_win\mkl"
 
to 
"C:\Users\zpan\Documents\GitHub\julia\usr\x86_64-w64-mingw32\sys-root\mingw\bin".
 
This includes the folder "33". Here are all the files that got copied:

1033/mkl_msg.dll
libimalloc.dll
mkl_avx.dll
mkl_avx2.dll
mkl_avx512.dll
mkl_avx512_mic.dll
mkl_core.dll
mkl_def.dll
mkl_intel_thread.dll
mkl_mc.dll
mkl_mc3.dll
mkl_rt.dll
mkl_sequential.dll
mkl_tbb_thread.dll
mkl_vml_avx.dll
mkl_vml_avx2.dll
mkl_vml_avx512.dll
mkl_vml_avx512_mic.dll
mkl_vml_cmpt.dll
mkl_vml_def.dll
mkl_vml_mc.dll
mkl_vml_mc2.dll
mkl_vml_mc3.dll


(3) "make -j 4" again. This time, there was no "Cannot find BLAS libraries" 
error anymore. The build went on for a while until here:

floatfuncs.jl

Please submit a bug report with steps to reproduce this fault, and any 
error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: 
)
unknown function (ip: )
*** This error is usually fixed by running `make clean`. If the error 
persists, try `make cleanall`. ***
make[1]: *** [Makefile:200: 
/c/users/zpan/Documents/GitHub/julia/usr/lib/julia/sys.o] Error 1
make: *** [Makefile:69: julia-sysimg-release] Error 2


I tried both "make clean" and "make cleanall", and the same problem 
persisted. I also tried renaming the .dll and .a files in a couple ways in 
vain. 

Again, any help appreciated!! Hope I am close to success. When I am done I 
will repeat the process and document it here for others to see.




On Wednesday, September 14, 2016 at 3:02:40 PM UTC-5, Tony Kelman wrote:
>
> Which configure file was this from? Julia doesn't need the .lib files, it 
> needs the dll, and I assure you there are dlls in MKL. However some of the 
> dependencies may need the .lib files, and/or you can try copying them to a 
> .dll.a file name if that helps libtool or configure work better.
>
>
>>>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

Which configure file was this from? Julia doesn't need the .lib files, it 
needs the dll, and I assure you there are dlls in MKL. However some of the 
dependencies may need the .lib files, and/or you can try copying them to a 
.dll.a file name if that helps libtool or configure work better.

On Wednesday, September 14, 2016 at 12:28:16 PM UTC-7, Zhong Pan wrote:
>
> BTW I did try Tony's idea of adding "lib" as prefix to each .lib filename. 
> That didn't solve the problem.
>
> On Wednesday, September 14, 2016 at 2:07:14 PM UTC-5, Zhong Pan wrote:
>>
>> Just to report my progress and ask for help again if anybody succeeded in 
>> building Julia 0.4 or 0.5 on Windows with MKL.
>>
>> I backed off a little and tried building release-0.5 on Windows following 
>> the standard instructions below. That is, without any MKL related 
>> configurations.
>> https://github.com/JuliaLang/julia/blob/master/README.windows.md 
>> 
>> But the build failed with Error 2 (some dependency failed to build). 
>>
>> Since my main focus is to get MKL to work rather than trying out 0.5, I 
>> tried building release-0.4 first following the same instructions (no MKL). 
>> And that was a success!
>> Encouraged, I added the following lines to Make.user and copied the 17 
>> .lib files from the MKL library to julia/usr/bin and julia/usr/lib and 
>> C:\mkllib
>> USE_INTEL_MKL = 1
>> USE_INTEL_MKL_FFT = 1
>> USE_INTEL_LIBM = 1
>>
>> I also did this, though I know it probably won't matter:
>> export MKLLIB=/c/mkllib
>>
>> "make -j 4" failed with the expected error:
>>
>> ...
>> configure: error: Cannot find BLAS libraries
>> ...
>> make: *** [Makefile:51: julia-deps] Error 2
>>
>> I know CMake probably won't be able to figure out Windows style libraries 
>> by itself, but I don't know how to make this work. Could anyone help? 
>>
>> P.S. the 17 MKL .lib files are listed below:
>>
>> mkl_blas95_ilp64.lib
>> mkl_blas95_lp64.lib
>> mkl_core.lib
>> mkl_core_dll.lib
>> mkl_intel_ilp64.lib
>> mkl_intel_ilp64_dll.lib
>> mkl_intel_lp64.lib
>> mkl_intel_lp64_dll.lib
>> mkl_intel_thread.lib
>> mkl_intel_thread_dll.lib
>> mkl_lapack95_ilp64.lib
>> mkl_lapack95_lp64.lib
>> mkl_rt.lib
>> mkl_sequential.lib
>> mkl_sequential_dll.lib
>> mkl_tbb_thread.lib
>> mkl_tbb_thread_dll.lib
>>
>>
>>
>>
>>
>>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

BTW I did try Tony's idea of adding "lib" as prefix to each .lib filename. 
That didn't solve the problem.

On Wednesday, September 14, 2016 at 2:07:14 PM UTC-5, Zhong Pan wrote:
>
> Just to report my progress and ask for help again if anybody succeeded in 
> building Julia 0.4 or 0.5 on Windows with MKL.
>
> I backed off a little and tried building release-0.5 on Windows following 
> the standard instructions below. That is, without any MKL related 
> configurations.
> https://github.com/JuliaLang/julia/blob/master/README.windows.md 
> 
> But the build failed with Error 2 (some dependency failed to build). 
>
> Since my main focus is to get MKL to work rather than trying out 0.5, I 
> tried building release-0.4 first following the same instructions (no MKL). 
> And that was a success!
> Encouraged, I added the following lines to Make.user and copied the 17 
> .lib files from the MKL library to julia/usr/bin and julia/usr/lib and 
> C:\mkllib
> USE_INTEL_MKL = 1
> USE_INTEL_MKL_FFT = 1
> USE_INTEL_LIBM = 1
>
> I also did this, though I know it probably won't matter:
> export MKLLIB=/c/mkllib
>
> "make -j 4" failed with the expected error:
>
> ...
> configure: error: Cannot find BLAS libraries
> ...
> make: *** [Makefile:51: julia-deps] Error 2
>
> I know CMake probably won't be able to figure out Windows style libraries 
> by itself, but I don't know how to make this work. Could anyone help? 
>
> P.S. the 17 MKL .lib files are listed below:
>
> mkl_blas95_ilp64.lib
> mkl_blas95_lp64.lib
> mkl_core.lib
> mkl_core_dll.lib
> mkl_intel_ilp64.lib
> mkl_intel_ilp64_dll.lib
> mkl_intel_lp64.lib
> mkl_intel_lp64_dll.lib
> mkl_intel_thread.lib
> mkl_intel_thread_dll.lib
> mkl_lapack95_ilp64.lib
> mkl_lapack95_lp64.lib
> mkl_rt.lib
> mkl_sequential.lib
> mkl_sequential_dll.lib
> mkl_tbb_thread.lib
> mkl_tbb_thread_dll.lib
>
>
>
>
>
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Just to report my progress and ask for help again if anybody succeeded in 
building Julia 0.4 or 0.5 on Windows with MKL.

I backed off a little and tried building release-0.5 on Windows following 
the standard instructions below. That is, without any MKL related 
configurations.
https://github.com/JuliaLang/julia/blob/master/README.windows.md 

But the build failed with Error 2 (some dependency failed to build). 

Since my main focus is to get MKL to work rather than trying out 0.5, I 
tried building release-0.4 first following the same instructions (no MKL). 
And that was a success!
Encouraged, I added the following lines to Make.user and copied the 17 .lib 
files from the MKL library to julia/usr/bin and julia/usr/lib and C:\mkllib
USE_INTEL_MKL = 1
USE_INTEL_MKL_FFT = 1
USE_INTEL_LIBM = 1

I also did this, though I know it probably won't matter:
export MKLLIB=/c/mkllib

"make -j 4" failed with the expected error:

...
configure: error: Cannot find BLAS libraries
...
make: *** [Makefile:51: julia-deps] Error 2

I know CMake probably won't be able to figure out Windows style libraries 
by itself, but I don't know how to make this work. Could anyone help? 

P.S. the 17 MKL .lib files are listed below:

mkl_blas95_ilp64.lib
mkl_blas95_lp64.lib
mkl_core.lib
mkl_core_dll.lib
mkl_intel_ilp64.lib
mkl_intel_ilp64_dll.lib
mkl_intel_lp64.lib
mkl_intel_lp64_dll.lib
mkl_intel_thread.lib
mkl_intel_thread_dll.lib
mkl_lapack95_ilp64.lib
mkl_lapack95_lp64.lib
mkl_rt.lib
mkl_sequential.lib
mkl_sequential_dll.lib
mkl_tbb_thread.lib
mkl_tbb_thread_dll.lib

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

The "Can't open perl script "scripts/config.pl": No such file or directory" 
error has been solved. See: https://github.com/ARMmbed/mbedtls/issues/541
What I did was to open the CMakeList.txt file under 
"C:\Users\zpan\Documents\GitHub\julia\deps\srccache\mbedtls-2.3.0-gpl", and 
replaced ${PERL_EXECUTABLE} with ${CMAKE_CURRENT_SOURCE_DIR} on line 34.
 
Trying to solve the "Could not create symbolic link" issue.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Tony,

Thanks for your reply. I did solve the CMAKE_C_COMPILER issue - see my 
previous post (just 1 min after yours). However, it's not by appending to 
$PATH though - I had tried adding the gcc path to $PATH and the same error 
still showed up.

Thanks for the hint about adding "lib" prefix. I don't have .dll files 
though - the MKL library consists of 17 .lib files (the file list I gave 
missed a few letters at the end for some files so they become .l or .li - 
my apologies). 

I will try adding the prefix anyway... After other road blocks are removed. 
Please see my previous post for details.

On Tuesday, September 13, 2016 at 3:45:16 PM UTC-5, Tony Kelman wrote:
>
> maybe better to just temporarily add gcc's location to your working path 
> for the duration of the make (don't leave it there longer though).
>
> I think that linker error is from arpack. Try making a copy of mkl_rt.dll 
> called libmkl_rt.dll and see if that helps. libtool isn't able to link 
> against dlls that don't have a lib prefix unless you have a corresponding 
> .dll.a import library (maybe copying and renaming one od the .lib files to 
> libmkl_rt.dll.a could also work?) because being annoying is what libtool 
> does.
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Tony,

Thanks for your reply. I did solve the CMAKE_C_COMPILER issue - see my 
previous post (just 1 min after yours). However, it's not by appending to 
$PATH though - I had tried adding the gcc path to $PATH and the same error 
still showed up.

Thanks for the hint about adding "lib" prefix. I don't have .dll files 
though - the MKL library consists of 17 .lib files. I will try adding the 
prefix anyway... After other road blocks are removed. Please see my 
previous post for details.


On Tuesday, September 13, 2016 at 3:45:16 PM UTC-5, Tony Kelman wrote:
>
> maybe better to just temporarily add gcc's location to your working path 
> for the duration of the make (don't leave it there longer though).
>
> I think that linker error is from arpack. Try making a copy of mkl_rt.dll 
> called libmkl_rt.dll and see if that helps. libtool isn't able to link 
> against dlls that don't have a lib prefix unless you have a corresponding 
> .dll.a import library (maybe copying and renaming one od the .lib files to 
> libmkl_rt.dll.a could also work?) because being annoying is what libtool 
> does.
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

I realize I may be documenting a bunch of build issues that more 
experienced programmer may overcome more easily. Anyway since I started, 
please bear with me. :-)

So I solved the problem of incorrect CMAKE_C_COMPILER. What I did was I 
found the make file that was causing Error 1:
/c/users/zpan/documents/github/julia/deps/mbedtls.mk 
I opened it, scrolled to line 40, and inserted a line before line 38:
MBEDTLS_OPTS += 
-DCMAKE_C_COMPILER="/C/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe"
 

After saving this and trying "make install" again, the compiler seems to be 
found, but another issue popped up, which I cannot solve yet. 

The error message:

zpan@WSPWork MSYS /c/users/zpan/documents/github/julia
$ make install
-- The C compiler identification is GNU 6.2.0
-- Check for working C compiler: 
C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe
-- Check for working C compiler: 
C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe
 
-- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Perl: C:/msys64/usr/bin/perl.exe (found version "5.22.1")
Can't open perl script "scripts/config.pl": No such file or directory
CMake Error at tests/CMakeLists.txt:122 (message):
  Could not create symbolic link for:
  
C:/Users/zpan/Documents/GitHub/julia/deps/srccache/mbedtls-2.3.0-gpl/tests/data_files
  --> Invalid switch - "Users".



-- Configuring incomplete, errors occurred!
See also 
"C:/Users/zpan/Documents/GitHub/julia/deps/build/mbedtls-2.3.0/CMakeFiles/CMakeOutput.log".
make[2]: *** [/c/users/zpan/documents/github/julia/deps/mbedtls.mk:41: 
build/mbedtls-2.3.0/Makefile] Error 1
make[1]: *** [Makefile:81: julia-deps] Error 2
make: *** [Makefile:331: install] Error 2

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

maybe better to just temporarily add gcc's location to your working path for 
the duration of the make (don't leave it there longer though).

I think that linker error is from arpack. Try making a copy of mkl_rt.dll 
called libmkl_rt.dll and see if that helps. libtool isn't able to link against 
dlls that don't have a lib prefix unless you have a corresponding .dll.a import 
library (maybe copying and renaming one od the .lib files to libmkl_rt.dll.a 
could also work?) because being annoying is what libtool does.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

OK my non-parallel build bumped into a different problem. It seems the 
CMAKE_C_COMPILER PATH was not understood. 
I am not sure what is the correct one, so I tried a few variations:
C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe
/C/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc
/C/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe

For example, I tried modifying CMakeCache.txt by modifying this line:
CMAKE_C_COMPILER:UNINITIALIZED=C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc
into
CMAKE_C_COMPILER:UNINITIALIZED=C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe
(and also tried the other 2 variations)

Or creating a new line:
CMAKE_C_COMPILER=C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc.exe
(and also tried the other 2 variations)

Problem is everytime I run "make" or "make install", CMakeCache.txt is 
overwritten.

I also tried the environmental variable way:
$ export 
CC=/C/Users/zpan/Documents/GitHub/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc
(and also tried the other 2 variations)

None of the solutions worked. So if I do "make" or "make install" I am 
stuck at this error message. 

I am not experienced in building projects with MinGW, so maybe the problem 
is a trivial one. But, for me it's tough. :-) Any help appreciated!


See full error message below:

zpan@WSPWork MSYS /c/users/zpan/documents/github/julia
$ make
-- The C compiler identification is GNU 6.2.0
CMake Error at CMakeLists.txt:2 (project):
  The CMAKE_C_COMPILER:


C:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/gcc

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path 
to
  the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
See also 
"C:/Users/zpan/Documents/GitHub/julia/deps/build/mbedtls-2.3.0/CMakeFiles/CMakeOutput.log".
make[1]: *** [/c/users/zpan/documents/github/julia/deps/mbedtls.mk:40: 
build/mbedtls-2.3.0/Makefile] Error 1
make: *** [Makefile:81: julia-deps] Error 2





On Tuesday, September 13, 2016 at 2:47:07 PM UTC-5, Zhong Pan wrote:
>
> Thank you Tony for the information. I still had some problem.
>
> I tried building Julia (release-0.5) using MinGW following the 
> instructions here: 
> https://github.com/JuliaLang/julia/blob/master/README.windows.md
>
> The only difference is, after creating the Make.user file, I edited it 
> using a text editor and added the options to use MKL, so the complete 
> Make.user now looks like:
> override CMAKE=/C/CMake/bin/cmake.exe
> USE_INTEL_MKL = 1
> USE_INTEL_MKL_FFT = 1
> USE_INTEL_LIBM = 1
>
> The addition of the above lines is per build instructions about MKL at 
> https://github.com/JuliaLang/julia, except that I removed "USEICC = 1" 
> and "USEIFC = 1" since I am not using Intel compiler. But I do have Intel 
> MKL with the free Community License installed on my computer at "C:\Program 
> Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl"
>
> After I started the build with "make -j 3", the build would run for a 
> while then stop with 
> make: *** [Makefile:81: julia-deps] Error 2
>
> I went through the messages and found this to be the most suspicious for a 
> root cause:
> c:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>  
> cannot find -lmkl_rt
>
> So I realized that MinGW was not able to find the MKL libraries. I tried 
> the following 2:
>
> (1) Copied all the 17 .lib files from "C:\Program Files 
> (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win"
>  
> to "C:\mkllib" (so the path does not contain a space), then I did
> $ export MKLLIB=/C/mkllib
>
> (2) Copied all the 17 .lib files to 
> "C:\Users\zpan\Documents\GitHub\julia\usr\lib"
>
> But I was still getting the same Error 2 in the end, and I could still 
> find the error message saying "cannot find -lmkl_rt".  Well, mkl_rt.lib was 
> indeed part of the 17 files, which are:
>
> mkl_blas95_ilp64.lib
> mkl_blas95_lp64.lib
> mkl_core.lib
> mkl_core_dll.lib
> mkl_intel_ilp64.lib
> mkl_intel_ilp64_dll.li
> mkl_intel_lp64.lib
> mkl_intel_lp64_dll.lib
> mkl_intel_thread.lib
> mkl_intel_thread_dll.l
> mkl_lapack95_ilp64.lib
> mkl_lapack95_lp64.lib
> mkl_rt.lib
> mkl_sequential.lib
> mkl_sequential_dll.lib
> mkl_tbb_thread.lib
> mkl_tbb_thread_dll.lib
>
> I am currently trying the command "make install" to build it without 
> parallel option - this is very slow but I can see the messages in proper 
> order.
>
> Did I miss something to let the linker see the .lib files? Any help 
> greatly appreciated! 
>
>
>
> On 

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Thank you Tony for the information. I still had some problem.

I tried building Julia (release-0.5) using MinGW following the instructions 
here: 
https://github.com/JuliaLang/julia/blob/master/README.windows.md

The only difference is, after creating the Make.user file, I edited it 
using a text editor and added the options to use MKL, so the complete 
Make.user now looks like:
override CMAKE=/C/CMake/bin/cmake.exe
USE_INTEL_MKL = 1
USE_INTEL_MKL_FFT = 1
USE_INTEL_LIBM = 1

The addition of the above lines is per build instructions about MKL 
at https://github.com/JuliaLang/julia, except that I removed "USEICC = 1" 
and "USEIFC = 1" since I am not using Intel compiler. But I do have Intel 
MKL with the free Community License installed on my computer at "C:\Program 
Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl"

After I started the build with "make -j 3", the build would run for a while 
then stop with 
make: *** [Makefile:81: julia-deps] Error 2

I went through the messages and found this to be the most suspicious for a 
root cause:
c:/users/zpan/documents/github/julia/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
 
cannot find -lmkl_rt

So I realized that MinGW was not able to find the MKL libraries. I tried 
the following 2:

(1) Copied all the 17 .lib files from "C:\Program Files 
(x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\mkl\lib\intel64_win"
 
to "C:\mkllib" (so the path does not contain a space), then I did
$ export MKLLIB=/C/mkllib

(2) Copied all the 17 .lib files to 
"C:\Users\zpan\Documents\GitHub\julia\usr\lib"

But I was still getting the same Error 2 in the end, and I could still find 
the error message saying "cannot find -lmkl_rt".  Well, mkl_rt.lib was 
indeed part of the 17 files, which are:

mkl_blas95_ilp64.lib
mkl_blas95_lp64.lib
mkl_core.lib
mkl_core_dll.lib
mkl_intel_ilp64.lib
mkl_intel_ilp64_dll.li
mkl_intel_lp64.lib
mkl_intel_lp64_dll.lib
mkl_intel_thread.lib
mkl_intel_thread_dll.l
mkl_lapack95_ilp64.lib
mkl_lapack95_lp64.lib
mkl_rt.lib
mkl_sequential.lib
mkl_sequential_dll.lib
mkl_tbb_thread.lib
mkl_tbb_thread_dll.lib

I am currently trying the command "make install" to build it without 
parallel option - this is very slow but I can see the messages in proper 
order.

Did I miss something to let the linker see the .lib files? Any help greatly 
appreciated! 



On Monday, September 12, 2016 at 3:59:19 PM UTC-5, Tony Kelman wrote:
>
> Intel compilers on windows are MSVC style, which our build system is not 
> really set up to handle. There is experimental partial support (search for 
> "MSVC support tracking issue" if you're interested) but it would really 
> require rewriting the build system to use cmake to work smoothly.
>
> You can build Julia from source with mingw but direct the build system to 
> use an mkl library instead of building openblas from source. At one point 
> mkl on windows didn't export gfortran style naming conventions for blas and 
> lapack functions, but recent versions do last I checked. Could even be able 
> to switch from an existing Julia binary by rebuilding the system image.
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Tony Kelman]

Intel compilers on windows are MSVC style, which our build system is not really 
set up to handle. There is experimental partial support (search for "MSVC 
support tracking issue" if you're interested) but it would really require 
rewriting the build system to use cmake to work smoothly.

You can build Julia from source with mingw but direct the build system to use 
an mkl library instead of building openblas from source. At one point mkl on 
windows didn't export gfortran style naming conventions for blas and lapack 
functions, but recent versions do last I checked. Could even be able to switch 
from an existing Julia binary by rebuilding the system image.

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Zhong Pan]

Thanks! I am still a bit confused about this part:

To build Julia for Windows, this page says I need to use MinGW compiler 
either under MSYS2 or 
Cygwin: https://github.com/JuliaLang/julia/blob/master/README.windows.md
However, to build Julia with MKL BLAS and LAPACK libraries, the first link 
you referenced says I need to use Intel Compiler 15 or above. The example 
given is for POSIX system.

So, can I directly use the Windows version of Intel Compiler to compile 
Julia with the options turned on to use MKL, and forget about MinGW 
completely?

Thanks for leading me to VML.jl. I assume this module is independent of MKL 
BLAS and LAPACK, so even if I didn't link Julia with MKL, as long as I have 
MKL binary installed, this would work?



On Monday, September 12, 2016 at 12:35:35 AM UTC-5, Zhong Pan wrote:
>
> Anybody knows how to build Julia with Intel MKL on Windows? 
>
> I found the article below but it is for Linux.
>
> https://software.intel.com/en-us/articles/julia-with-intel-mkl-for-improved-performance
>
> Thanks!
>
>

[julia-users] Re: Help on building Julia with Intel MKL on Windows?

[Chris Rackauckas]

You just do what it says here: https://github.com/JuliaLang/julia. Then you 
can replace a lot of the functions using VML.jl 


On Sunday, September 11, 2016 at 10:35:35 PM UTC-7, Zhong Pan wrote:
>
> Anybody knows how to build Julia with Intel MKL on Windows? 
>
> I found the article below but it is for Linux.
>
> https://software.intel.com/en-us/articles/julia-with-intel-mkl-for-improved-performance
>
> Thanks!
>
>

[julia-users] Re: Help: how to improve the function performance

[Weicheng Zhu]

Hi Kristoffer,
Thank you for your reply.
I was used to the R language and I didn't even realize that the norm(x[:,i] 
- x[:,j]) part is the problem.
To get speed boost, I add another for loop to calculate the norm manually 
to avoid the memory allocation and it works very well.
Thank you again for your help!

Best,
Weicheng

On Wednesday, September 7, 2016 at 6:51:08 AM UTC-5, Kristoffer Carlsson 
wrote:
>
> The code in Distances.jl is quite heavily optimized and uses BLAS calls 
> when possible (which it is for Euclidean metric). Your code has many 
> allocations like x = x' and norm(x[:,i] - x[:,j]).
>
> On Wednesday, September 7, 2016 at 1:43:11 PM UTC+2, Weicheng Zhu wrote:
>>
>> Hi there,
>> I write a function to calculate the distance for each row of a two 
>> dimensional array and I compared it with the `pairwise` function in the 
>> Distance module.
>> Does anyone can help me to find out the reason why my function is slower 
>> than the pairwise function? I only keep the triangle elements of the 
>> distance matrix which I thought should be faster. Thanks in advance for any 
>> help:)
>>
>> Here is the code:
>>
>> Module Tmp
>>
>> import DataFrames: DataFrame
>>
>> function dist(x::Matrix)
>>
>> x = x'
>>
>> n = size(x, 2)
>>
>> ij::UInt = 0
>>
>> d = zeros(convert(Int, (n-1)*n/2))
>>
>> for i in 1:n
>>
>> for j in (i+1):n
>>
>> ij += 1
>>
>> d[ij] = norm(x[:,i] - x[:,j])
>>
>> end
>>
>> end
>>
>> return d
>>
>> end
>>
>>
>> function dist(x::DataFrame)
>>
>> dist(convert(Array, dat))
>>
>> end
>>
>> export dist
>>
>> end
>>
>>
>> using Tmp
>>
>> using Distances
>>
>> x = rand(100,2)
>>
>> @time dist(x)
>>
>> # 0.001581 seconds (29.71 k allocations: 1.399 MB)
>>
>> @time pairwise(Euclidean(), x')
>>
>> # 0.000318 seconds (310 allocations: 91.984 KB)
>>
>>
>>   
>> 
>> 
>> 
>>
>>
>

[julia-users] Re: Help: how to improve the function performance

[Kristoffer Carlsson]

The code in Distances.jl is quite heavily optimized and uses BLAS calls 
when possible (which it is for Euclidean metric). Your code has many 
allocations like x = x' and norm(x[:,i] - x[:,j]).

On Wednesday, September 7, 2016 at 1:43:11 PM UTC+2, Weicheng Zhu wrote:
>
> Hi there,
> I write a function to calculate the distance for each row of a two 
> dimensional array and I compared it with the `pairwise` function in the 
> Distance module.
> Does anyone can help me to find out the reason why my function is slower 
> than the pairwise function? I only keep the triangle elements of the 
> distance matrix which I thought should be faster. Thanks in advance for any 
> help:)
>
> Here is the code:
>
> Module Tmp
>
> import DataFrames: DataFrame
>
> function dist(x::Matrix)
>
> x = x'
>
> n = size(x, 2)
>
> ij::UInt = 0
>
> d = zeros(convert(Int, (n-1)*n/2))
>
> for i in 1:n
>
> for j in (i+1):n
>
> ij += 1
>
> d[ij] = norm(x[:,i] - x[:,j])
>
> end
>
> end
>
> return d
>
> end
>
>
> function dist(x::DataFrame)
>
> dist(convert(Array, dat))
>
> end
>
> export dist
>
> end
>
>
> using Tmp
>
> using Distances
>
> x = rand(100,2)
>
> @time dist(x)
>
> # 0.001581 seconds (29.71 k allocations: 1.399 MB)
>
> @time pairwise(Euclidean(), x')
>
> # 0.000318 seconds (310 allocations: 91.984 KB)
>
>
>   
> 
> 
> 
>
>

[julia-users] Re: [HELP] Doubts in Julia Language with Strings.. I've condensed my doubts into one Simple Program

[randmstring]

Note that the above solutions won't work if your input happens to be 
non-ascii -- g's code will give something like "3:3-5:2-4:�-�" as the 
output for "abcabü".

Imho it's better to build the string you want in the end from scratch 
instead of treating it as a mutable object:

input = "abcabü"

tmp = IOBuffer()
ind = 1
for char in input
  if isodd(ind)
print(tmp, Int(char) + 2 - Int('a') + 1 )
ind == length(input) || print(tmp, ':')
  else
print(tmp, Int(char) + 1 - Int('a') + 1)
ind == length(input) || print(tmp, '-')
  end
  ind += 1
end

out = takebuf_string(tmp)

This will give you a sensible result even for unicode input 
("3:3-5:2-4:157" in the case I mentioned above).


Am Mittwoch, 24. August 2016 16:35:58 UTC+2 schrieb Rishabh Raghunath:
>
> *Hello Julia Users !!..*
>
> I have asked this question before in the forum but am still a bit unclear 
> on how to do the below.. I have condensed a few of my doubts in the 
> question below.
>
> Being from a C background where strings are just character Arrays, 
> manipulating individual characters using the array index was easy.. However 
> In julia that definitely does not seem to be the recommended way to go 
> about doing things and I have a lot of doubts about the small details on 
> the language. Here's a small example of what I am trying to achieve with my 
> program .. This is a fairly simple program while doing it in C but I have 
> absolutely no idea how to implement the same in Julia as strings cannot be 
> treated like character arrays and i cannot just  increment an index of the 
> string to increase the ASCII value of the character in Julia. Please try to 
> solve the problem or guide me on how to implement this in Julia.. I promise 
> the question is very simple and isn't as complex as It may seem at a 
> glance. 
>
> *My Aim*:
>
> To write a program a program which accepts a string via the input 
> and encrypts it to the following simple algorithm ..
>
> *Algorithm:*
>
> *Example Input String: abcabc*
>
> 1. Increase the ASCII value of the character at the *ODD index* of the 
> string  by 5 steps( example increased the ASCII value of '  *a*  ' by 2 
> thus making it   ' * c*  '  
> 2. Increase the ASCII value of the character at the *EVEN index* of the 
> string  by 1 step( example increased the ASCII value of ' * b*  ' by 1 
> thus making it   '  *c*  '
> 3. Print the new encrypted string on Screen. I need it stored in a new 
> string variable say z *(OUTPUT : ccebdd  )*
> 3. Insert a ' - ' between each pair of characters of the string and store 
> in new string in another string variable say y.
> 4. Print string y on the screen  * (OUTPUT : cc-eb-dd  ) *
> 5. Change all the characters in the string into its number equivalent. ( 
> a=1,b=2,c=3,.) and separate the numbers representing each character by 
>  '  :  ' 
> 6. Store the new string from the above operation in variable v and print v 
>  *(OUTPUT : 3:3-5:2-4:4)*
>
> Guys .. Try to help me out !!.. I tried searching everywhere on the 
> internet for steps in doing the above operations in Julia but haven't found 
> anything.. I always get the index errors for treating Julia the C way.. 
> Please try to create a program that does the above.. Just so that Its 
> easier for me to understand what you are doing in the program rather than 
> type the directives in English and then later misunderstanding stuff... I 
> need to learn doing the above procedure the right way without messing with 
> stuff like using indexes in strings for character manipulation that may get 
> deprecated in the future..
>
> Thanks !! Waiting eagerly for a reply !!  
>
>

[julia-users] Re: [HELP] Doubts in Julia Language with Strings.. I've condensed my doubts into one Simple Program

[Tim Wheeler]

str = "abcabc"
arr = convert(Vector{UInt8}, str)

# you can increase indeces with addition now.
arr[1] += 5

# convert back
str2 = convert(typeof(str), arr)
println(str2) # -> should print "fbcabc"


On Wednesday, August 24, 2016 at 7:35:58 AM UTC-7, Rishabh Raghunath wrote:
>
> *Hello Julia Users !!..*
>
> I have asked this question before in the forum but am still a bit unclear 
> on how to do the below.. I have condensed a few of my doubts in the 
> question below.
>
> Being from a C background where strings are just character Arrays, 
> manipulating individual characters using the array index was easy.. However 
> In julia that definitely does not seem to be the recommended way to go 
> about doing things and I have a lot of doubts about the small details on 
> the language. Here's a small example of what I am trying to achieve with my 
> program .. This is a fairly simple program while doing it in C but I have 
> absolutely no idea how to implement the same in Julia as strings cannot be 
> treated like character arrays and i cannot just  increment an index of the 
> string to increase the ASCII value of the character in Julia. Please try to 
> solve the problem or guide me on how to implement this in Julia.. I promise 
> the question is very simple and isn't as complex as It may seem at a 
> glance. 
>
> *My Aim*:
>
> To write a program a program which accepts a string via the input 
> and encrypts it to the following simple algorithm ..
>
> *Algorithm:*
>
> *Example Input String: abcabc*
>
> 1. Increase the ASCII value of the character at the *ODD index* of the 
> string  by 5 steps( example increased the ASCII value of '  *a*  ' by 2 
> thus making it   ' * c*  '  
> 2. Increase the ASCII value of the character at the *EVEN index* of the 
> string  by 1 step( example increased the ASCII value of ' * b*  ' by 1 
> thus making it   '  *c*  '
> 3. Print the new encrypted string on Screen. I need it stored in a new 
> string variable say z *(OUTPUT : ccebdd  )*
> 3. Insert a ' - ' between each pair of characters of the string and store 
> in new string in another string variable say y.
> 4. Print string y on the screen  * (OUTPUT : cc-eb-dd  ) *
> 5. Change all the characters in the string into its number equivalent. ( 
> a=1,b=2,c=3,.) and separate the numbers representing each character by 
>  '  :  ' 
> 6. Store the new string from the above operation in variable v and print v 
>  *(OUTPUT : 3:3-5:2-4:4)*
>
> Guys .. Try to help me out !!.. I tried searching everywhere on the 
> internet for steps in doing the above operations in Julia but haven't found 
> anything.. I always get the index errors for treating Julia the C way.. 
> Please try to create a program that does the above.. Just so that Its 
> easier for me to understand what you are doing in the program rather than 
> type the directives in English and then later misunderstanding stuff... I 
> need to learn doing the above procedure the right way without messing with 
> stuff like using indexes in strings for character manipulation that may get 
> deprecated in the future..
>
> Thanks !! Waiting eagerly for a reply !!  
>
>

[julia-users] Re: [HELP] Doubts in Julia Language with Strings.. I've condensed my doubts into one Simple Program

[ggggg]

I haven't worked in C, but my impression is that it doesn't really 
distinguish between a UInt8 and a Character. In Julia, you can convert your 
string to a Vector{UInt8} and you should be able to work like you are used 
to. 

This gist has one way to implement your algorithm:
https://gist.github.com/g/97451aad24996687ca754052082a239d

>
>

Re: [julia-users] Re: help with deepcopy_internal

[Chris Stook]

I think this fixes it:


type SelfReferential
  obj::SelfReferential

  SelfReferential() = (x = new(); x.obj = x)
end

function Base.deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
  if haskey(oidd,sr)
print("X")
return oidd[sr]
  end
  print("Y")
  new_sr = SelfReferential() # fix .obj after recursive call
  oidd[sr] = new_sr
  new_obj = Base.deepcopy_internal(sr,oidd)
  new_sr.obj = new_obj
  return new_sr
end

sr = SelfReferential()
deepcopy_sr = deepcopy(sr)
sr == deepcopy_sr   # false
sr === deepcopy_sr  # false

I'll have to go back and rethink what I am doing in my module now that I 
have a better understanding.

Thank all,
Chris


On Wednesday, August 24, 2016 at 4:34:20 AM UTC-4, Tommy Hofmann wrote:
>
> OK, I now see that I misunderstood the question. After rereading it, I 
> noticed something important. I think your deepcopy_internal version is 
> broken, but you did not notice it since you did _not_ define 
> Base.deepcopy_internal(::SelfReferential, ::ObjectIdDict).
>
> You should do the following:
>
> import Base: deepcopy_internal
>
> type SelfReferential
>   obj::SelfReferential
>
>   SelfReferential() = (x = new(); x.obj = x)
> end
>
> function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
> #  if haskey(oidd,sr)
> #   return oidd[sr]
> # else
> new_obj = deepcopy_internal(sr,oidd)
> new_sr = sr(new_obj)
> oidd[sr] = new_sr
> return new_sr
> #  end
> end
>
> sr = SelfReferential()
> deepcopy_sr = deepcopy(sr)
>
> Now this will break, since it runs in into an infinite recursion. Even if 
> you enable the lines which are commented out, it still breaks, but I don't 
> know why.
>
> But as Yichao Yu has pointed out, the builtin deepcopy function works out 
> of the box for your custom type.
>
> On Wednesday, August 24, 2016 at 9:02:19 AM UTC+2, Yichao Yu wrote:
>>
>>
>>
>> On Tue, Aug 23, 2016 at 5:19 PM, Tommy Hofmann  wrote:
>>
>>> Why do you think that deepcopy works on SelfReferential without defining 
>>> deepcopy_internal? Line 8 of deepcopy.jl is: deepcopy(x) = 
>>> deepcopy_internal(x, ObjectIdDict())
>>>
>>
>> Because `deepcopy_internal` already handles this.
>>
>> You shouldn't need to define deepcopy_internal for custom types in 
>> general, unless your type needs special serialization support (e.g. if it 
>> has pointer to C memory)
>>  
>>
>>>
>>>
>>> On Tuesday, August 23, 2016 at 10:52:35 AM UTC+2, Chris Stook wrote:

 I created this simple example to try to understand deepcopy_internal.

 type SelfReferential
   obj::SelfReferential

   SelfReferential() = (x = new(); x.obj = x)
 end

 function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
 #  if haskey(oidd,sr)
 #return oidd[sr]
 #  else
 new_obj = deepcopy_internal(sr,oidd)
 new_sr = sr(new_obj)
 oidd[sr] = new_sr
 return new_sr
 #  end
 end

 sr = SelfReferential()
 deepcopy_sr = deepcopy(sr)

 This works, but deepcopy works on SelfReferential without defining 
 deepcopy_internal, so this isn't a good 
 example.  

 How should this be modified so deepcopy_internal is required?

 Are the commented out lines of code necessary?

 Thanks,
 Chris


>>

Re: [julia-users] Re: help with deepcopy_internal

[Tommy Hofmann]

OK, I now see that I misunderstood the question. After rereading it, I 
noticed something important. I think your deepcopy_internal version is 
broken, but you did not notice it since you did _not_ define 
Base.deepcopy_internal(::SelfReferential, ::ObjectIdDict).

You should do the following:

import Base: deepcopy_internal

type SelfReferential
  obj::SelfReferential

  SelfReferential() = (x = new(); x.obj = x)
end

function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
#  if haskey(oidd,sr)
#   return oidd[sr]
# else
new_obj = deepcopy_internal(sr,oidd)
new_sr = sr(new_obj)
oidd[sr] = new_sr
return new_sr
#  end
end

sr = SelfReferential()
deepcopy_sr = deepcopy(sr)

Now this will break, since it runs in into an infinite recursion. Even if 
you enable the lines which are commented out, it still breaks, but I don't 
know why.

But as Yichao Yu has pointed out, the builtin deepcopy function works out 
of the box for your custom type.

On Wednesday, August 24, 2016 at 9:02:19 AM UTC+2, Yichao Yu wrote:
>
>
>
> On Tue, Aug 23, 2016 at 5:19 PM, Tommy Hofmann  > wrote:
>
>> Why do you think that deepcopy works on SelfReferential without defining 
>> deepcopy_internal? Line 8 of deepcopy.jl is: deepcopy(x) = 
>> deepcopy_internal(x, ObjectIdDict())
>>
>
> Because `deepcopy_internal` already handles this.
>
> You shouldn't need to define deepcopy_internal for custom types in 
> general, unless your type needs special serialization support (e.g. if it 
> has pointer to C memory)
>  
>
>>
>>
>> On Tuesday, August 23, 2016 at 10:52:35 AM UTC+2, Chris Stook wrote:
>>>
>>> I created this simple example to try to understand deepcopy_internal.
>>>
>>> type SelfReferential
>>>   obj::SelfReferential
>>>
>>>   SelfReferential() = (x = new(); x.obj = x)
>>> end
>>>
>>> function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
>>> #  if haskey(oidd,sr)
>>> #return oidd[sr]
>>> #  else
>>> new_obj = deepcopy_internal(sr,oidd)
>>> new_sr = sr(new_obj)
>>> oidd[sr] = new_sr
>>> return new_sr
>>> #  end
>>> end
>>>
>>> sr = SelfReferential()
>>> deepcopy_sr = deepcopy(sr)
>>>
>>> This works, but deepcopy works on SelfReferential without defining 
>>> deepcopy_internal, so this isn't a good 
>>> example.  
>>>
>>> How should this be modified so deepcopy_internal is required?
>>>
>>> Are the commented out lines of code necessary?
>>>
>>> Thanks,
>>> Chris
>>>
>>>
>

Re: [julia-users] Re: help with deepcopy_internal

[Yichao Yu]

On Tue, Aug 23, 2016 at 5:19 PM, Tommy Hofmann  wrote:

> Why do you think that deepcopy works on SelfReferential without defining
> deepcopy_internal? Line 8 of deepcopy.jl is: deepcopy(x) =
> deepcopy_internal(x, ObjectIdDict())
>

Because `deepcopy_internal` already handles this.

You shouldn't need to define deepcopy_internal for custom types in general,
unless your type needs special serialization support (e.g. if it has
pointer to C memory)


>
>
> On Tuesday, August 23, 2016 at 10:52:35 AM UTC+2, Chris Stook wrote:
>>
>> I created this simple example to try to understand deepcopy_internal.
>>
>> type SelfReferential
>>   obj::SelfReferential
>>
>>   SelfReferential() = (x = new(); x.obj = x)
>> end
>>
>> function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
>> #  if haskey(oidd,sr)
>> #return oidd[sr]
>> #  else
>> new_obj = deepcopy_internal(sr,oidd)
>> new_sr = sr(new_obj)
>> oidd[sr] = new_sr
>> return new_sr
>> #  end
>> end
>>
>> sr = SelfReferential()
>> deepcopy_sr = deepcopy(sr)
>>
>> This works, but deepcopy works on SelfReferential without defining
>> deepcopy_internal, so this isn't a good
>> example.
>>
>> How should this be modified so deepcopy_internal is required?
>>
>> Are the commented out lines of code necessary?
>>
>> Thanks,
>> Chris
>>
>>

[julia-users] Re: help with deepcopy_internal

[Tommy Hofmann]

Why do you think that deepcopy works on SelfReferential without defining 
deepcopy_internal? Line 8 of deepcopy.jl is: deepcopy(x) = 
deepcopy_internal(x, ObjectIdDict())

On Tuesday, August 23, 2016 at 10:52:35 AM UTC+2, Chris Stook wrote:
>
> I created this simple example to try to understand deepcopy_internal.
>
> type SelfReferential
>   obj::SelfReferential
>
>   SelfReferential() = (x = new(); x.obj = x)
> end
>
> function deepcopy_internal(sr::SelfReferential, oidd::ObjectIdDict)
> #  if haskey(oidd,sr)
> #return oidd[sr]
> #  else
> new_obj = deepcopy_internal(sr,oidd)
> new_sr = sr(new_obj)
> oidd[sr] = new_sr
> return new_sr
> #  end
> end
>
> sr = SelfReferential()
> deepcopy_sr = deepcopy(sr)
>
> This works, but deepcopy works on SelfReferential without defining 
> deepcopy_internal, so this isn't a good 
> example.  
>
> How should this be modified so deepcopy_internal is required?
>
> Are the commented out lines of code necessary?
>
> Thanks,
> Chris
>
>

[julia-users] Re: Help needed with excessive memory allocation for Tuples

[vavasis]

Just so you know, a similar issue exists in some of the Base types in 
Julia.  For example "sub" (to extract a subarray from an array that does 
not copy the elements but rather provides a view of the original array) and 
"Ref" (to create a pointer that can be passed to a C subroutine) both 
allocate small reference objects on the heap rather than the stack.  This 
means that there is a significant loss of performance, e.g., if a code 
repeatedly uses "sub" to extract a small subarray in an inner loop. In this 
scenario, it is faster to copy the subarray into temporary preallocated 
storage and then copy it back after modification.  I believe that the core 
Julia developers are working on optimizations so that functions like this 
will allocate on the stack instead of the heap if the compiler can prove to 
itself that the "sub" or "Ref" object will never escape from an inner 
loop.  I don't know what is the state of these optimizations.

Stack allocation in the general case (i.e., in the case when the object may 
escape from a loop) does not seem feasible for "sub" objects, "Ref" objects 
or tuples similar to your example.  The reason is that the garbage 
collector needs to know whether there is a live reference to a mutable data 
structure, and if references to mutable data structures were copied around 
like bits objects, the run-time system could not keep track of this.

-- Steve Vavasis


 


On Wednesday, August 17, 2016 at 9:09:31 AM UTC-4, Bill Hart wrote:
>
> We have a (time critical function) with the following prototype:
>
> heapinsert!{N}(xs::Array{Tuple{NTuple{N, UInt}, heap_s}, 1}, exp::NTuple{N
> , UInt}, x::heap_s)
>
>
> However, one line of code in the implementation is allocating gigabytes of 
> memory, which is causing a massive performance deficit in our program:
>
> xs[1] = (exp, x)
>
> Note heap_s is just an ordinary Julia type here, which contains a couple 
> of Ints and another heap_s. As far as I can see it is irrelevant. It should 
> just be a pointer at the machine level.
>
> It seems that Julia allocates space for the tuple (exp, x) and then writes 
> it into the array xs.
>
> Does anyone have any ideas how I can stop it from making this unnecessary 
> allocation? It's slowing down multivariate polynomial multiplication by a 
> factor of at least 2 (and possibly 10) and causing a massive blowout in 
> memory allocation (this is really performance critical code).
>
> I've already tried making using an immutable rather than a tuple. Same 
> problem. I'm tracing this using julia --track-allocation=all.
>
> Bill.
>

[julia-users] Re: Help please: how to expand dictionary into (local ?) variables with the same name as the keys and the corresponding values

[Chris Rackauckas]

There's a PR for this on Parameters.jl. See 
this: https://github.com/mauro3/Parameters.jl/pull/13



On Saturday, August 13, 2016 at 1:04:19 AM UTC-7, Adrian Salceanu wrote:
>
> Hi, 
>
> This seems to be a recurring question per my googlings, but still I could 
> not find a satisfactory answer. 
>
> I'm looking for a generic way of doing this: 
>
> render_template(template_content, Dict(:greeting => "Hello", :greeted => 
> "World", :other => Dict("foo" => 1)))
>
> where 
>
> function render_template(template_content, template_vars = Dict{Symbol,Any
> }()) 
>   # here instantiate template_vars keys with corresponding values, so 
> that 
>   # greeting = "Hello"
>   # greeted = "World"
>   # other = Dict("foo" => 1)
> end
>
> Thanks! 
>

[julia-users] Re: Help with `promote_op` in v0.5

[j verzani]

Thanks!

On Friday, July 29, 2016 at 11:22:47 AM UTC-4, Pablo Zubieta wrote:
>
> This is due to issue https://github.com/JuliaLang/julia/issues/17314, and 
> should get fixed if https://github.com/JuliaLang/julia/pull/17389 gets 
> merged.
>
> On Friday, July 29, 2016 at 5:00:14 PM UTC+2, j verzani wrote:
>>
>> The test for the Polynomials package are now failing in v0.5. Before
>>
>> ```
>> p1  = Poly([1, 2])
>> p = [p, p] # 2-element Array{Polynomials.Poly{Float64},1}:
>> p + 3  # 2-element Array{Polynomials.Poly{Float64},1}:
>> ```
>>
>> But now, `p+3` is an `Array{Any,1}`. I think the solution is related to 
>> how `promote_array_type` uses `promote_op`, which in turn returns `Any` for 
>> `Base.promote_op(:+, eltype(p), eltype(3))`. There is a comment in 
>> promotion.jl about adding methods to `promote_op`, but as `promote_op` 
>> isn't exported this seems like the wrong solution for this case. (As well, 
>> I couldn't figure out exactly how to do so.) I could also fix this by 
>> defining `.+` to directly call `broadcast`, but that defeats the work in 
>> `arraymath,jl`.
>>
>> Any hints?
>>
>

[julia-users] Re: Help with `promote_op` in v0.5

[Pablo Zubieta]

This is due to issue https://github.com/JuliaLang/julia/issues/17314, and 
should get fixed if https://github.com/JuliaLang/julia/pull/17389 gets 
merged.

On Friday, July 29, 2016 at 5:00:14 PM UTC+2, j verzani wrote:
>
> The test for the Polynomials package are now failing in v0.5. Before
>
> ```
> p1  = Poly([1, 2])
> p = [p, p] # 2-element Array{Polynomials.Poly{Float64},1}:
> p + 3  # 2-element Array{Polynomials.Poly{Float64},1}:
> ```
>
> But now, `p+3` is an `Array{Any,1}`. I think the solution is related to 
> how `promote_array_type` uses `promote_op`, which in turn returns `Any` for 
> `Base.promote_op(:+, eltype(p), eltype(3))`. There is a comment in 
> promotion.jl about adding methods to `promote_op`, but as `promote_op` 
> isn't exported this seems like the wrong solution for this case. (As well, 
> I couldn't figure out exactly how to do so.) I could also fix this by 
> defining `.+` to directly call `broadcast`, but that defeats the work in 
> `arraymath,jl`.
>
> Any hints?
>

[julia-users] Re: Help Julia win a performance comparison!

[Chris Rackauckas]

Nevermind. You have a non-zero probability of having zero offspring since 
it's Poisson. This works if every element is at least 1. However, you can 
have the population size decrease, which then causes errors if you resize 
first. But then you still want to put the new elements in the first n slots 
of the array (which also contains the values you want to write over), so I 
don't think you can do that directly in place unless someone has a clever 
trick.

However, I could just keep a second array around for writing into, and then 
write into x. That saves allocations with a few more writes, but is a net 
speedup. I posted an Edit 3 explaining that.

On Thursday, July 21, 2016 at 8:38:42 AM UTC-7, Chris Rackauckas wrote:
>
> I see it now. Sum the elements to resize the array, and then loop through 
> backwards adding the values (so that way you don't overwrite what 
> you haven't used).
>
> On Thursday, July 21, 2016 at 8:34:11 AM UTC-7, Kristoffer Carlsson wrote:
>>
>> Sum the elements and resize the array to that length? 
>
>

[julia-users] Re: Help Julia win a performance comparison!

[Chris Rackauckas]

I see it now. Sum the elements to resize the array, and then loop through 
backwards adding the values (so that way you don't overwrite what 
you haven't used).

On Thursday, July 21, 2016 at 8:34:11 AM UTC-7, Kristoffer Carlsson wrote:
>
> Sum the elements and resize the array to that length? 

[julia-users] Re: Help Julia win a performance comparison!

[Kristoffer Carlsson]

Sum the elements and resize the array to that length? 

[julia-users] Re: Help Julia win a performance comparison!

[Chris Rackauckas]

Let me explain. The easy place to add an in-place operation with resize! 
would be with the RNG call, rpois. I used resize! to make the Poisson RNG 
go a little faster. It's now:

  function rpois!(n::Int,p::Vector{Float64},out::Vector{Int})
resize!(out,n)
for i in 1:n
  @inbounds out[i] = StatsFuns.RFunctions.poisrand(p[i]) 
#rand(Poisson(p[i]))
end
  end

and then I change the script to use that. However, it doesn't help very 
much, the time IN the RNG calls is still the major timing consuming 
element, and the allocation wasn't much of a factor.

The other place that has an allocation is in the R.rep function. I am not 
sure how to make that an in-place function though. For reference, this 
function takes an array like [0.1 0.2 0.3] and an array of ints [2 3 1] and 
duplicates each element that many times: [0.1 0.1 0.2 0.2 0.2 0.3]. With it 
being in-place, you don't have an easy way to reference what values should 
be duplicated how many times.  In this case, the allocation is a (pretty 
small, but still shows up) measurable part of the timing, but is harder to 
deal with.

So we're stuck with RNG time as the major issue, but can chop off a little 
more if this allocation can be dealt with better (the R code has both of 
these same issues, which is why we're virtually tied).

On Thursday, July 21, 2016 at 5:49:23 AM UTC-7, Steven G. Johnson wrote:
>
>
>
> On Thursday, July 21, 2016 at 5:37:12 AM UTC-4, Chris Rackauckas wrote:
>>
>> Maybe. I thought about that, but I don't think that satisfies the 
>> "elegant and compactness" requirement, unless there's an easy way to do the 
>> growing without too much extra code hanging around. 
>>
>
> Why is `resize!` so much harder than allocating a new array? 
>

[julia-users] Re: Help Julia win a performance comparison!

[Steven G. Johnson]

On Thursday, July 21, 2016 at 5:37:12 AM UTC-4, Chris Rackauckas wrote:
>
> Maybe. I thought about that, but I don't think that satisfies the "elegant 
> and compactness" requirement, unless there's an easy way to do the growing 
> without too much extra code hanging around. 
>

Why is `resize!` so much harder than allocating a new array? 

[julia-users] Re: Help Julia win a performance comparison!

[Christoph Ortner]

feels like one may want a little auxiliary package that can make available 
small chunks from a long pre-allocated vector.

On Thursday, 21 July 2016 10:37:12 UTC+1, Chris Rackauckas wrote:
>
> Maybe. I thought about that, but I don't think that satisfies the "elegant 
> and compactness" requirement, unless there's an easy way to do the growing 
> without too much extra code hanging around. 
>
> On Thursday, July 21, 2016 at 1:54:10 AM UTC-7, Christoph Ortner wrote:
>>
>> could still preallocate and grow as needed?
>>
>> On Thursday, 21 July 2016 02:48:58 UTC+1, Chris Rackauckas wrote:
>>>
>>> Most of the arrays are changing size each time though, since they 
>>> represent a population which changes each timestep.
>>>
>>> On Wednesday, July 20, 2016 at 6:47:39 PM UTC-7, Steven G. Johnson wrote:

 It looks like you are allocating lots of arrays in your doStep 
 inner-loop function, so I'm sure you could improve it by moving the 
 allocations out of the inner loop.  (In general, vectorized routines are 
 convenient but they aren't the fastest way to do things.)

>>>

[julia-users] Re: Help Julia win a performance comparison!

[Chris Rackauckas]

Maybe. I thought about that, but I don't think that satisfies the "elegant 
and compactness" requirement, unless there's an easy way to do the growing 
without too much extra code hanging around. 

On Thursday, July 21, 2016 at 1:54:10 AM UTC-7, Christoph Ortner wrote:
>
> could still preallocate and grow as needed?
>
> On Thursday, 21 July 2016 02:48:58 UTC+1, Chris Rackauckas wrote:
>>
>> Most of the arrays are changing size each time though, since they 
>> represent a population which changes each timestep.
>>
>> On Wednesday, July 20, 2016 at 6:47:39 PM UTC-7, Steven G. Johnson wrote:
>>>
>>> It looks like you are allocating lots of arrays in your doStep 
>>> inner-loop function, so I'm sure you could improve it by moving the 
>>> allocations out of the inner loop.  (In general, vectorized routines are 
>>> convenient but they aren't the fastest way to do things.)
>>>
>>

[julia-users] Re: Help Julia win a performance comparison!

[Christoph Ortner]

could still preallocate and grow as needed?

On Thursday, 21 July 2016 02:48:58 UTC+1, Chris Rackauckas wrote:
>
> Most of the arrays are changing size each time though, since they 
> represent a population which changes each timestep.
>
> On Wednesday, July 20, 2016 at 6:47:39 PM UTC-7, Steven G. Johnson wrote:
>>
>> It looks like you are allocating lots of arrays in your doStep inner-loop 
>> function, so I'm sure you could improve it by moving the allocations out of 
>> the inner loop.  (In general, vectorized routines are convenient but they 
>> aren't the fastest way to do things.)
>>
>

[julia-users] Re: Help Julia win a performance comparison!

[Chris Rackauckas]

Most of the arrays are changing size each time though, since they represent 
a population which changes each timestep.

On Wednesday, July 20, 2016 at 6:47:39 PM UTC-7, Steven G. Johnson wrote:
>
> It looks like you are allocating lots of arrays in your doStep inner-loop 
> function, so I'm sure you could improve it by moving the allocations out of 
> the inner loop.  (In general, vectorized routines are convenient but they 
> aren't the fastest way to do things.)
>

[julia-users] Re: Help Julia win a performance comparison!

[Steven G. Johnson]

It looks like you are allocating lots of arrays in your doStep inner-loop 
function, so I'm sure you could improve it by moving the allocations out of 
the inner loop.  (In general, vectorized routines are convenient but they 
aren't the fastest way to do things.)

Re: [julia-users] Re: Help catting

[J Luis]

Hmm, thanks ... but too mysterious for me. What is 'dest' and 'i' (columns 
I guess, but how to loop on it?)? 

segunda-feira, 18 de Julho de 2016 às 01:34:11 UTC+1, Yichao Yu escreveu:
>
> On Sun, Jul 17, 2016 at 8:05 PM, J Luis > 
> wrote: 
> > Ghrrr, this works 
> > 
> > julia> [pointer_to_array(C[1], DS.n_rows) pointer_to_array(C[2], 
> DS.n_rows)] 
> > 186x2 Array{Float64,2}: 
> >  7.0  53.3242 
> >  6.99583  53.325 
> >  6.99249  53.3264 
> > ... 
>
> You'll be better off to just create the Matrix and copy over the data 
> with `unsafe_copy!(pointer(dest, DS.n_rows * (i - 1) + 1), 
> unsafe_load(DS.data, i), DS.n_rows)` 
>
> > 
> > but I need a generic solution for a variable number of columns. 
> > 
> > 
> > segunda-feira, 18 de Julho de 2016 às 00:40:25 UTC+1, J Luis escreveu: 
> >> 
> >> Hi, need help reshaping an output from C that comes out in N (columns) 
> >> vectors into a 2-D matrix 
> >> 
> >> julia> C = pointer_to_array(DS.data, 2) 
> >> 2-element Array{Ptr{Float64},1}: 
> >>  Ptr{Float64} @0x1bf17360 
> >>  Ptr{Float64} @0x1bf17970 
> >> 
> >> and 
> >> 
> >> julia> pointer_to_array(C[1], DS.n_rows) 
> >> 186-element Array{Float64,1}: 
> >>  7.0 
> >>  6.99583 
> >>  6.99249 
> >> ... 
> >> 
> >> and I need to reshape these 2 columns into a 186x2 matrix but that's 
> where 
> >> all my attempts fails. I tried for example 
> >> 
> >> julia> hcat([pointer_to_array(C[k], DS.n_rows) for k=1:2]) 
> >> 2x1 Array{Array{T,N},2}: 
> >> 
> >> 
> [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111 
>
> >> … 
> >> 
> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>  
>
> >> 
> >> 
> [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338 
>
> >> … 
> >> 
> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>  
>
> >> 
> >> or 
> >> 
> >> julia> cat(2,[pointer_to_array(C[k], DS.n_rows) for k=1:2]) 
> >> 2x1 Array{Array{T,N},2}: 
> >> 
> >> 
> [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111 
>
> >> … 
> >> 
> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>  
>
> >> 
> >> 
> [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338 
>
> >> … 
> >> 
> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>  
>
> >> 
> >> but that is not what I need,  as mentioned I need a Mx2 matrix. 
> >> 
> >> Thanks 
> >> 
> >> Joaquim 
>

Re: [julia-users] Re: Help catting

[Yichao Yu]

On Sun, Jul 17, 2016 at 8:05 PM, J Luis  wrote:
> Ghrrr, this works
>
> julia> [pointer_to_array(C[1], DS.n_rows) pointer_to_array(C[2], DS.n_rows)]
> 186x2 Array{Float64,2}:
>  7.0  53.3242
>  6.99583  53.325
>  6.99249  53.3264
> ...

You'll be better off to just create the Matrix and copy over the data
with `unsafe_copy!(pointer(dest, DS.n_rows * (i - 1) + 1),
unsafe_load(DS.data, i), DS.n_rows)`

>
> but I need a generic solution for a variable number of columns.
>
>
> segunda-feira, 18 de Julho de 2016 às 00:40:25 UTC+1, J Luis escreveu:
>>
>> Hi, need help reshaping an output from C that comes out in N (columns)
>> vectors into a 2-D matrix
>>
>> julia> C = pointer_to_array(DS.data, 2)
>> 2-element Array{Ptr{Float64},1}:
>>  Ptr{Float64} @0x1bf17360
>>  Ptr{Float64} @0x1bf17970
>>
>> and
>>
>> julia> pointer_to_array(C[1], DS.n_rows)
>> 186-element Array{Float64,1}:
>>  7.0
>>  6.99583
>>  6.99249
>> ...
>>
>> and I need to reshape these 2 columns into a 186x2 matrix but that's where
>> all my attempts fails. I tried for example
>>
>> julia> hcat([pointer_to_array(C[k], DS.n_rows) for k=1:2])
>> 2x1 Array{Array{T,N},2}:
>>
>> [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111
>> …
>> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>>
>> [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338
>> …
>> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>>
>> or
>>
>> julia> cat(2,[pointer_to_array(C[k], DS.n_rows) for k=1:2])
>> 2x1 Array{Array{T,N},2}:
>>
>> [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111
>> …
>> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>>
>> [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338
>> …
>> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>>
>> but that is not what I need,  as mentioned I need a Mx2 matrix.
>>
>> Thanks
>>
>> Joaquim

[julia-users] Re: Help catting

[J Luis]

Ghrrr, this works

julia> [pointer_to_array(C[1], DS.n_rows) pointer_to_array(C[2], DS.n_rows)]
186x2 Array{Float64,2}:
 7.0  53.3242
 6.99583  53.325
 6.99249  53.3264
...

but I need a generic solution for a variable number of columns.

segunda-feira, 18 de Julho de 2016 às 00:40:25 UTC+1, J Luis escreveu:
>
> Hi, need help reshaping an output from C that comes out in N (columns) 
> vectors into a 2-D matrix
>
> julia> C = pointer_to_array(DS.data, 2)
> 2-element Array{Ptr{Float64},1}:
>  Ptr{Float64} @0x1bf17360
>  Ptr{Float64} @0x1bf17970
>
> and
>
> julia> pointer_to_array(C[1], DS.n_rows)
> 186-element Array{Float64,1}:
>  7.0
>  6.99583
>  6.99249
> ...
>
> and I need to reshape these 2 columns into a 186x2 matrix but that's where 
> all my attempts fails. I tried for example 
>
> julia> hcat([pointer_to_array(C[k], DS.n_rows) for k=1:2])
> 2x1 Array{Array{T,N},2}:
>  [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111 
>  … 
>  
> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>  [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338 
>  … 
>  
> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>
> or
>
> julia> cat(2,[pointer_to_array(C[k], DS.n_rows) for k=1:2])
> 2x1 Array{Array{T,N},2}:
>  [7.0,6.99583,6.99249,6.98917,6.98639,6.98306,6.98029,6.97694,6.97417,6.97111 
>  … 
>  
> 6.44916,6.44472,6.44056,6.43612,6.43139,6.42611,6.42167,6.41694,6.41306,6.4089]
>  [53.3242,53.325,53.3264,53.3281,53.33,53.3314,53.3331,53.3344,53.3364,53.338 
>  … 
>  
> 53.6008,53.6017,53.6028,53.6033,53.6042,53.6047,53.6053,53.6061,53.6072,53.6075]
>
> but that is not what I need,  as mentioned I need a Mx2 matrix.
>
> Thanks
>
> Joaquim
>

Re: [julia-users] Re: Help on functions in a package

[Yichao Yu]

On Sat, Jul 16, 2016 at 7:50 AM, Phil_n  wrote:
> I have tried the ? command i REPL. But the information about functions with
> this help i in most cases very meager. Does anyone know a better option to
> get information about a function?

Ask the package author to write more documentations.

>
> On Friday, July 15, 2016 at 8:04:33 PM UTC+2, Phil_n wrote:
>>
>> There does not seem to be a help() function in Julia
>>
>> On Friday, July 15, 2016 at 7:18:11 PM UTC+2, Phil_n wrote:
>>>
>>> How do I invoke help on a certain function in a package? That is I would
>>> like an explanatory text of how to use a certain function in a package.
>>>
>>> For example:  I would like to know how to use the function rotationmatrix
>>> in package Quaternions

[julia-users] Re: Help on functions in a package

[Phil_n]

I have tried the ? command i REPL. But the information about functions with 
this help i in most cases very meager. Does anyone know a better option to 
get information about a function? 

On Friday, July 15, 2016 at 8:04:33 PM UTC+2, Phil_n wrote:
>
> There does not seem to be a help() function in Julia
>
> On Friday, July 15, 2016 at 7:18:11 PM UTC+2, Phil_n wrote:
>>
>> How do I invoke help on a certain function in a package? That is I would 
>> like an explanatory text of how to use a certain function in a package.
>>
>> For example:  I would like to know how to use the function rotationmatrix 
>> in package Quaternions
>>
>

[julia-users] Re: Help on functions in a package

[Phil_n]

There does not seem to be a help() function in Julia

On Friday, July 15, 2016 at 7:18:11 PM UTC+2, Phil_n wrote:
>
> How do I invoke help on a certain function in a package? That is I would 
> like an explanatory text of how to use a certain function in a package.
>
> For example:  I would like to know how to use the function rotationmatrix 
> in package Quaternions
>

Re: [julia-users] Re: Help with a macro

[Mosè Giordano]

Hi Mauro,

2016-06-22 21:13 GMT+02:00 Mauro:
>
> A REPL session of mine trying to figure out how to make that macro would 
> look something like this: 
>
> julia> ex = :(f(x, y, z, zz)) # this is what is passed into the macro 
> :(f(x,y,z,zz)) 
>
> julia> xdump(ex)  # xdump is nice to get an overview of nested 
> datastructures 
> Expr 
>   head: Symbol call 
>   args: Array(Any,(5,)) 
> 1: Symbol f 
> 2: Symbol x 
> 3: Symbol y 
> 4: Symbol z 
> 5: Symbol zz 
>   typ: Any::DataType  <: Any 
>

I knew and used "dump" but not "xdump".  I've seen that the latter has been 
deprecated in favor of the former in Julia 0.5.
 

> julia> target = :(f([x.re, y.re, z.re, zz.re].^2 + [x.im, y.im, z.im, 
> zz.im])) # this is your target 
> :(f([x.re,y.re,z.re,zz.re] .^ 2 + [x.im,y.im,z.im,zz.im])) 
>
> julia> xdump(target) 
> Expr 
>   head: Symbol call 
>   args: Array(Any,(2,)) 
> 1: Symbol f 
> 2: Expr 
>   head: Symbol call 
>   args: Array(Any,(3,)) 
> 1: Symbol + 
> 2: Expr 
>   head: Symbol call 
>   args: Array(Any,(3,)) 
>   typ: Any::DataType  <: Any 
> 3: Expr 
>   head: Symbol vect 
>   args: Array(Any,(4,)) 
>   typ: Any::DataType  <: Any 
>   typ: Any::DataType  <: Any 
>   typ: Any::DataType  <: Any 
>
>
> Now, writing the macro is really an exercise in manipulating nested data 
> structures.  With the added bonus that there are convenient constructors 
> for those datastructures, namely expression such as e.g. :(z = 4 +7). 
>
> julia> fn = ex.args[1] # extract the name of the function 
> :f 
>
> julia> args = ex.args[2:end] # its arguments 
> 4-element Array{Any,1}: 
>  :x 
>  :y 
>  :z 
>  :zz 
>
> julia> ar1 =:([])  # build up the first array 
> :([]) 
>
> julia> ar1.head # check 
> :vect 
>
> julia> [push!(ar1.args, :($(args[i]).re)) for i=1:length(args)] #add the 
> elements 
> 4-element Array{Any,1}: 
>  Any[:(x.re),:(y.re),:(z.re),:(zz.re)] 
>  Any[:(x.re),:(y.re),:(z.re),:(zz.re)] 
>  Any[:(x.re),:(y.re),:(z.re),:(zz.re)] 
>  Any[:(x.re),:(y.re),:(z.re),:(zz.re)] 
>
> julia> ar1 
> :([x.re,y.re,z.re,zz.re]) 
>
> julia> :($ar1^2) # square 
> :([x.re,y.re,z.re,zz.re] ^ 2) 
>
> etc. 
>
> It takes a bit of practice and patience.  Important, as Kristoffer said, 
> write out and test the code you expect the macro to form (at least until 
> you become a macro ninja), then write the macro.  For instance in one of 
> my projects I hand wrote all of the code I later built macros to 
> generate: 
> https://github.com/mauro3/Traits.jl/blob/master/test/manual-traitdef.jl 
>

Thank you so much, I've been able to change my "@uncertain" macro for 
Measurements.jl  package 
following your hints.  In the end, the trick was to quote rather than to 
evaluate the array of arguments.  Probably this is a good rule of thumb 
when writing macros.

Bye,
Mosè

Re: [julia-users] Re: Help with a macro

[Mauro]

A REPL session of mine trying to figure out how to make that macro would
look something like this:

julia> ex = :(f(x, y, z, zz)) # this is what is passed into the macro
:(f(x,y,z,zz))

julia> xdump(ex)  # xdump is nice to get an overview of nested datastructures
Expr
  head: Symbol call
  args: Array(Any,(5,))
1: Symbol f
2: Symbol x
3: Symbol y
4: Symbol z
5: Symbol zz
  typ: Any::DataType  <: Any

julia> target = :(f([x.re, y.re, z.re, zz.re].^2 + [x.im, y.im, z.im, zz.im])) 
# this is your target
:(f([x.re,y.re,z.re,zz.re] .^ 2 + [x.im,y.im,z.im,zz.im]))

julia> xdump(target)
Expr
  head: Symbol call
  args: Array(Any,(2,))
1: Symbol f
2: Expr
  head: Symbol call
  args: Array(Any,(3,))
1: Symbol +
2: Expr
  head: Symbol call
  args: Array(Any,(3,))
  typ: Any::DataType  <: Any
3: Expr
  head: Symbol vect
  args: Array(Any,(4,))
  typ: Any::DataType  <: Any
  typ: Any::DataType  <: Any
  typ: Any::DataType  <: Any


Now, writing the macro is really an exercise in manipulating nested data
structures.  With the added bonus that there are convenient constructors
for those datastructures, namely expression such as e.g. :(z = 4 +7).

julia> fn = ex.args[1] # extract the name of the function
:f

julia> args = ex.args[2:end] # its arguments
4-element Array{Any,1}:
 :x
 :y
 :z
 :zz

julia> ar1 =:([])  # build up the first array
:([])

julia> ar1.head # check
:vect

julia> [push!(ar1.args, :($(args[i]).re)) for i=1:length(args)] #add the 
elements
4-element Array{Any,1}:
 Any[:(x.re),:(y.re),:(z.re),:(zz.re)]
 Any[:(x.re),:(y.re),:(z.re),:(zz.re)]
 Any[:(x.re),:(y.re),:(z.re),:(zz.re)]
 Any[:(x.re),:(y.re),:(z.re),:(zz.re)]

julia> ar1
:([x.re,y.re,z.re,zz.re])

julia> :($ar1^2) # square
:([x.re,y.re,z.re,zz.re] ^ 2)

etc.

It takes a bit of practice and patience.  Important, as Kristoffer said,
write out and test the code you expect the macro to form (at least until
you become a macro ninja), then write the macro.  For instance in one of
my projects I hand wrote all of the code I later built macros to
generate:
https://github.com/mauro3/Traits.jl/blob/master/test/manual-traitdef.jl

On Wed, 2016-06-22 at 13:00, Mosè Giordano  wrote:
> Ok, here is a simpler example, not specific to Measuments.jl.
>
> From
>
> @foo f(x, y, z, ...)
>
> get
>
> f([x.re, y.re, z.re, ...].^2 + [x.im, y.im, z.im, ...])
>
> The expression provided to the macro should can take any number of arguments.
> For example:
>
> x = complex(3.2, -5.2)
> y = complex(8.9, 0.4)
> @foo(log(x, y)) == log([3.2, 8.9].^2 + [-5.2, 0.4])
>
> So my original question boils down to: how to get specific fields of input
> arguments to a macro without evaluating them?
>
> Bye,
> Mosè

[julia-users] Re: Help with a macro

[Mosè Giordano]

Ok, here is a simpler example, not specific to Measuments.jl.

From

@foo f(x, y, z, ...)

get

f([x.re, y.re, z.re, ...].^2 + [x.im, y.im, z.im, ...])

The expression provided to the macro should can take any number of 
arguments.  For example:

x = complex(3.2, -5.2)
y = complex(8.9, 0.4)
@foo(log(x, y)) == log([3.2, 8.9].^2 + [-5.2, 0.4])

So my original question boils down to: how to get specific fields of input 
arguments to a macro without evaluating them?

Bye,
Mosè

[julia-users] Re: Help with a macro

[Mosè Giordano]

2016-06-21 23:02 GMT+02:00 Kristoffer Carlsson:
>
>
> Your macro should not evaluate anything per say. It should generate 
> expressions that when the code is run will call functions. 
>
> Say a user types @uncertain log(x,y), what do you want the resulting 
> expression after macro expansion has run to be?
>

This should be the expression to be run in the end:

result(log(x.val, y.val),
   (Calculus.gradient(x -> log(x...), [x.val, y.val])),
   (x, y))

Thank you,
bye,
Mosè

[julia-users] Re: Help with a macro

[Kristoffer Carlsson]

On Tuesday, June 21, 2016 at 4:35:09 PM UTC-4, Mosè Giordano wrote:
>
> Hi Kristoffer,
>
> 2016-06-21 22:19 GMT+02:00 Kristoffer Carlsson:
>>
>> Using @eval in a macro is generally the wrong thing to do. A macro should 
>> simply transform an expression to another expression. 
>
>
> Ok, that's probably what I'm missing, thank you!
>  
>
>> If you give an example function body and what you want the macro to 
>> transform it to it would be easier to visualize what you want to do.
>>
>
> What the macro does is to evaluate the function at the nominal values of 
> the Measurement objects provided, calculate the derivative or gradient 
> (depending on the arity of the function), and then pass the results to the 
> "result" function defined within the package (you don't need to know what 
> it does, but if you're curios it's widely commented a few lines above in 
> the source code of the package).
>

Your macro should not evaluate anything per say. It should generate 
expressions that when the code is run will call functions. 

Say a user types @uncertain log(x,y), what do you want the resulting 
expression after macro expansion has run to be?


 

>
> The "a" variable in the "then" branch holds the list of arguments, all of 
> Measurement type.  I need to get from all of them the nominal value, that 
> is the "val" field, in order to evaluate the provided function for those 
> values.
>
> The user can pass to the "uncertain" macro any function taking all 
> arguments of type "Measurement".  So something like
>
> @uncertain log(x, y)
> @uncertain hypot(a, b, c, d)
>
> with a, b, c, d, x, ,y of type Measurement, should work.
>
> Is is clearer now or do you need more information?
>
> Bye,
> Mosè
>

[julia-users] Re: Help with a macro

[Mosè Giordano]

Hi Kristoffer,

2016-06-21 22:19 GMT+02:00 Kristoffer Carlsson:
>
> Using @eval in a macro is generally the wrong thing to do. A macro should 
> simply transform an expression to another expression. 


Ok, that's probably what I'm missing, thank you!
 

> If you give an example function body and what you want the macro to 
> transform it to it would be easier to visualize what you want to do.
>

What the macro does is to evaluate the function at the nominal values of 
the Measurement objects provided, calculate the derivative or gradient 
(depending on the arity of the function), and then pass the results to the 
"result" function defined within the package (you don't need to know what 
it does, but if you're curios it's widely commented a few lines above in 
the source code of the package).

The "a" variable in the "then" branch holds the list of arguments, all of 
Measurement type.  I need to get from all of them the nominal value, that 
is the "val" field, in order to evaluate the provided function for those 
values.

The user can pass to the "uncertain" macro any function taking all 
arguments of type "Measurement".  So something like

@uncertain log(x, y)
@uncertain hypot(a, b, c, d)

with a, b, c, d, x, ,y of type Measurement, should work.

Is is clearer now or do you need more information?

Bye,
Mosè

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Thanks again for the all of the advice. The Julia community is very 
helpful.  I should be able to figure out a viable solution.

[julia-users] Re: Help creating user-defined types and constructors

[Ford O.]

I want to discourage you from using symbols because although it perfectly 
works for you right now, but you might get into trouble later.

Lets say you want to add an array into Population and you want to keep 
symbol indexing:
Population
  persons
  newarray
end
#You can't do population[:newarray, index]
#Unless you rewrite your code and change population[:persons, :attribute, 
index]

Or you decide, that you wanna make new interface:
population[:persons, index]
population[:attribute, index]
population[index] # throws error 
This also wont be possible, so you end up rewriting all your code.

I wish it would be possible to do this in julia
setindex!(population, value, ::Individual, index...) = population.
individuals[index...] = value
setindex!(population, value, ::Infected, index...) = population.individuals[
index...].infected = value
population[::Individual, 1] = Individual()
population[::Infected,   1] = true
# Note that I have not declared Infected type anywhere

Which would eliminate all kind of problems, but I am not sure it's codeable 
or wanted in the first place.

So since this is not possible, I would stick to classical functions, as I 
showed you in the very first post in this discussion.


On Sunday, June 5, 2016 at 7:06:32 AM UTC+2, Glenn Fulford wrote:
>
> Hi Christopher, 
> You are right, there are not many examples out there of using Julia for 
> agent based models, at least as far as I can see. 
>
> I am not sure if you know about this one, but in the excellent QuantEcon 
> website, they give an example of Schelling's model of segregation with some 
> example code.
>
> http://nbviewer.jupyter.org/github/QuantEcon/QuantEcon.applications/blob/master/schelling/schelling_solutions_jl.ipynb
>
> But I think maybe what you are trying to do is a little bit more than 
> this, but just thought I would mention this example in case you hadn't seen 
> it. 
> Glenn
>
>
> On Saturday, 4 June 2016 22:19:02 UTC+10, Christopher Fisher wrote:
>>
>> I was wondering if someone would be willing to help me with creating 
>> user-defined types. I've been using Julia for about two years now but I am 
>> new to the idea of creating custom types. I'm trying to create a population 
>> of agents/individuals in a simple epidemiological simulation. 
>>
>> type Person
>> infected::Int64
>> vaccinated::Int64
>> dead::Int64
>>history::Array{Int64,1}
>> end
>>
>> Any help would be greatly appreciated. 
>>
>

[julia-users] Re: Help creating user-defined types and constructors

[Glenn Fulford]

Hi Christopher, 
You are right, there are not many examples out there of using Julia for 
agent based models, at least as far as I can see. 

I am not sure if you know about this one, but in the excellent QuantEcon 
website, they give an example of Schelling's model of segregation with some 
example code.
http://nbviewer.jupyter.org/github/QuantEcon/QuantEcon.applications/blob/master/schelling/schelling_solutions_jl.ipynb

But I think maybe what you are trying to do is a little bit more than this, 
but just thought I would mention this example in case you hadn't seen it. 
Glenn


On Saturday, 4 June 2016 22:19:02 UTC+10, Christopher Fisher wrote:
>
> I was wondering if someone would be willing to help me with creating 
> user-defined types. I've been using Julia for about two years now but I am 
> new to the idea of creating custom types. I'm trying to create a population 
> of agents/individuals in a simple epidemiological simulation. 
>
> type Person
> infected::Int64
> vaccinated::Int64
> dead::Int64
>history::Array{Int64,1}
> end
>
> Any help would be greatly appreciated. 
>

[julia-users] Re: Help creating user-defined types and constructors

[Scott Jones]

You might also want to think about using BitVectors to store the 
information about whether a person was infected, vaccinated, or deceased 
(which would take only 3 bits per person), and then you could easily 
perform operations to find out the number of people infected, etc. using 
the bit vectors.

On Saturday, June 4, 2016 at 10:48:29 AM UTC-4, Christopher Fisher wrote:
>
> Thank you for your help. 
>
> So my basic goal is to be able to perform operations on the individual and 
> population level. In this case, population.infected would return an 
> Array{Int64,1} indicating which individuals in the population are infected. 
> For example, population.infected[1]  = 1 would indicate the first 
> individual is infected. Something like mean(population.infected .== 1) 
> would indicate the percentage infected in the population. Similarly, if I 
> queried population[1], this would allow me to inspect all of the properties 
> assigned to individual 1. 
>
> Does that make sense?
>
> On Saturday, June 4, 2016 at 10:26:52 AM UTC-4, Ford Ox wrote:
>>
>> Could you please specify what should population.infected return? Could 
>> you provide an interface?
>>
>> Right now it seems like you want something like this:
>>
>> type Individual end
>>
>> type Population
>> individuals::Vector{Individual}
>> end
>>
>> setindex!(p::Population, i::Individual, index...) = p.individuals[index
>> ...] = i
>> getindex(p::Population, index) = p.individuals[index]
>> infected(i::Individual) = i.infected
>> vaccinated(i::Individual) = i.vaccinated
>>
>> Usage:
>> population[index] = Individual() # set new individual
>> infected(population[index])  # gets the infected property of 
>> individual at index
>> vaccinated(population[index])# ...
>>
>> But you don't really need functions for these things...
>>
>>
>> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>>
>>> I was wondering if someone would be willing to help me with creating 
>>> user-defined types. I've been using Julia for about two years now but I am 
>>> new to the idea of creating custom types. I'm trying to create a population 
>>> of agents/individuals in a simple epidemiological simulation. I would like 
>>> the population of individuals to be structured as a  2 dimensional array 
>>> with rows as individuals and columns as properties. This would be somewhat 
>>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>>> index an individual like so: population[1]. This woud list all of the 
>>> information for individual 1.  I would also like to be able to look at an 
>>> attribute across individuals: population.infected or population[:infected]. 
>>> At the same time, I would like to have to flexibility of using an array to 
>>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>>> Based on existing documentation and examples, I have only been able to 
>>> create individuals but cannot figure out how to create a population as 
>>> described above:
>>>
>>> type Person
>>> infected::Int64
>>> vaccinated::Int64
>>> dead::Int64
>>>history::Array{Int64,1}
>>> end
>>>
>>> Any help would be greatly appreciated. 
>>>
>>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

You might be right. I previously coded a simple simulation similar to what 
you described. In this  simulation,  individuals were stored in a vector 
and their properties were represented as integers (e.g. recovered: 3, 
infected: 1, not infected: 0 etc.). I used an adjacency matrix to represent 
the social network through which the infection spread probabilistically 
between connected individuals.  This approach works fine as long as the 
properties are mutually exclusive. My goal was to explore more flexible 
data structures for scaling up the simulation to something more complex. 
It's certainly possible a better approach exists. Without many agent based 
models available in Julia, its hard for me to know what the best approach 
might be. 

On Saturday, June 4, 2016 at 8:16:27 PM UTC-4, David P. Sanders wrote:
>
>
>
> El sábado, 4 de junio de 2016, 18:18:00 (UTC-4), Christopher Fisher 
> escribió:
>>
>> Ok. I didn't anticipate this would be a drawn out process.  I just Ford's 
>> suggestion fixed the issue of changing values but recreated the problem of 
>> accessing an individual's information like so:  population[2] . 
>> population[:,2] did not work either. I tried other changes to the syntax 
>> but to no avail. It fails at the last line.  Any ideas would be 
>> appreciated. 
>>
>> type Person
>> infected::Bool
>> vaccinated::Bool
>> dead::Bool
>> history::Vector{Int}
>> end
>>
>> type Population <: AbstractArray
>> individuals::Vector{Person}
>> 
>> end
>>
>>
>> Population() = Population(Person[])
>> Base.push!(p::Population, x::Person) = push!(p.individuals, x)
>>
>>
>> function Base.getindex(pop::Population, field::Symbol)
>> [getfield(x, field) for x in pop.individuals]
>> end
>>
>> setindex!(p::Population, value, field::Symbol, index...) = 
>> p.individuals[index...].(field) = value
>> getindex(p::Population, field::Symbol, index...) = 
>> p.individuals[index...].(field)
>>
>> Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, 
>> ", vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)
>>
>>
>> population = Population()
>> push!(population, Person(true, false, true, [3,4,5]))
>> push!(population, Person(false, true, false, Int64[3,4,5]))
>>
>> #Show that it works
>> population[:infected]
>> population[:infected,1]
>> population[2]
>>
>
> The syntax `population[x]` calls  `getindex(population, x)`.
> You are using two different *types* of object x here: integers and symbols.
> If you really want to do this, you would thus need to define both
>
> getindex(p::Population, n::Integer)
>
> to return the nth individual,
>
> and
>
> getindex(p::Population, s::Symbol)
>
> to return the list of those with status s.
>
> However, this does not seem like a very good idea to me.
>
> What it sounds like you want to do is something like "find the first 
> healthy individual and infect it".
> For this, you are better off writing a function called something like 
> infect!(p::Population)
> which will choose an individual to infect. 
>
> If you are planning to make this into an individual-based simulator, then 
> you probably want to go back
> to just thinking of a vector of individuals, and maintain separately a 
> list of those which are currently infected for example.
> Even though this will be extra book-keeping when an individual becomes 
> infected or stops being infected, it will almost certainly
> be much faster. 
>
> I suggest you try fleshing out your model a bit just thinking of functions 
> that operate on the population as I have suggested.
>
>  
>
>>
>> LoadError: MethodError: `size` has no method matching size(::Population)
>> Closest candidates are:
>>   size(::Any, !Matched::Integer, !Matched::Integer, !Matched::Integer...)
>> while loading In[1], in expression starting on line 36
>>
>>  in getindex at abstractarray.jl:488
>>
>>
>>
>> On Saturday, June 4, 2016 at 3:11:18 PM UTC-4, Ford O. wrote:
>>>
>>> I would like to discourage you from passing symbols as an array index 
>>> but I guess you gonna do it anyway...
>>>
>>>
>>> So if you really want it, here it is:
>>> setindex!(p::Population, value, field::Symbol, index...) = p.individuals
>>> [index...].(field) = value
>>> getindex(p::Population, field::Symbol, index...) = p.individuals[index
>>> ...].(field)
>>>
>>> Usage
>>> population[:infected, 1]
>>> population[:infected, 1] = false
>>>
>>>
>>>
>>>
>>> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:

 I was wondering if someone would be willing to help me with creating 
 user-defined types. I've been using Julia for about two years now but I am 
 new to the idea of creating custom types. I'm trying to create a 
 population 
 of agents/individuals in a simple epidemiological simulation. I would like 
 the population of individuals to be structured as a  2 dimensional array 
 with rows as individuals and columns as properties. This would be somewhat 
 similar to a D

[julia-users] Re: Help creating user-defined types and constructors

[David P. Sanders]

El sábado, 4 de junio de 2016, 18:18:00 (UTC-4), Christopher Fisher 
escribió:
>
> Ok. I didn't anticipate this would be a drawn out process.  I just Ford's 
> suggestion fixed the issue of changing values but recreated the problem of 
> accessing an individual's information like so:  population[2] . 
> population[:,2] did not work either. I tried other changes to the syntax 
> but to no avail. It fails at the last line.  Any ideas would be 
> appreciated. 
>
> type Person
> infected::Bool
> vaccinated::Bool
> dead::Bool
> history::Vector{Int}
> end
>
> type Population <: AbstractArray
> individuals::Vector{Person}
> 
> end
>
>
> Population() = Population(Person[])
> Base.push!(p::Population, x::Person) = push!(p.individuals, x)
>
>
> function Base.getindex(pop::Population, field::Symbol)
> [getfield(x, field) for x in pop.individuals]
> end
>
> setindex!(p::Population, value, field::Symbol, index...) = 
> p.individuals[index...].(field) = value
> getindex(p::Population, field::Symbol, index...) = 
> p.individuals[index...].(field)
>
> Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, 
> ", vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)
>
>
> population = Population()
> push!(population, Person(true, false, true, [3,4,5]))
> push!(population, Person(false, true, false, Int64[3,4,5]))
>
> #Show that it works
> population[:infected]
> population[:infected,1]
> population[2]
>

The syntax `population[x]` calls  `getindex(population, x)`.
You are using two different *types* of object x here: integers and symbols.
If you really want to do this, you would thus need to define both

getindex(p::Population, n::Integer)

to return the nth individual,

and

getindex(p::Population, s::Symbol)

to return the list of those with status s.

However, this does not seem like a very good idea to me.

What it sounds like you want to do is something like "find the first 
healthy individual and infect it".
For this, you are better off writing a function called something like 
infect!(p::Population)
which will choose an individual to infect. 

If you are planning to make this into an individual-based simulator, then 
you probably want to go back
to just thinking of a vector of individuals, and maintain separately a list 
of those which are currently infected for example.
Even though this will be extra book-keeping when an individual becomes 
infected or stops being infected, it will almost certainly
be much faster. 

I suggest you try fleshing out your model a bit just thinking of functions 
that operate on the population as I have suggested.

 

>
> LoadError: MethodError: `size` has no method matching size(::Population)
> Closest candidates are:
>   size(::Any, !Matched::Integer, !Matched::Integer, !Matched::Integer...)
> while loading In[1], in expression starting on line 36
>
>  in getindex at abstractarray.jl:488
>
>
>
> On Saturday, June 4, 2016 at 3:11:18 PM UTC-4, Ford O. wrote:
>>
>> I would like to discourage you from passing symbols as an array index but 
>> I guess you gonna do it anyway...
>>
>>
>> So if you really want it, here it is:
>> setindex!(p::Population, value, field::Symbol, index...) = p.individuals[
>> index...].(field) = value
>> getindex(p::Population, field::Symbol, index...) = p.individuals[index
>> ...].(field)
>>
>> Usage
>> population[:infected, 1]
>> population[:infected, 1] = false
>>
>>
>>
>>
>> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>>
>>> I was wondering if someone would be willing to help me with creating 
>>> user-defined types. I've been using Julia for about two years now but I am 
>>> new to the idea of creating custom types. I'm trying to create a population 
>>> of agents/individuals in a simple epidemiological simulation. I would like 
>>> the population of individuals to be structured as a  2 dimensional array 
>>> with rows as individuals and columns as properties. This would be somewhat 
>>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>>> index an individual like so: population[1]. This woud list all of the 
>>> information for individual 1.  I would also like to be able to look at an 
>>> attribute across individuals: population.infected or population[:infected]. 
>>> At the same time, I would like to have to flexibility of using an array to 
>>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>>> Based on existing documentation and examples, I have only been able to 
>>> create individuals but cannot figure out how to create a population as 
>>> described above:
>>>
>>> type Person
>>> infected::Int64
>>> vaccinated::Int64
>>> dead::Int64
>>>history::Array{Int64,1}
>>> end
>>>
>>> Any help would be greatly appreciated. 
>>>
>>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Ok. I didn't anticipate this would be a drawn out process.  I just Ford's 
suggestion fixed the issue of changing values but recreated the problem of 
accessing an individual's information like so:  population[2] . 
population[:,2] did not work either. I tried other changes to the syntax 
but to no avail. It fails at the last line.  Any ideas would be 
appreciated. 

type Person
infected::Bool
vaccinated::Bool
dead::Bool
history::Vector{Int}
end

type Population <: AbstractArray
individuals::Vector{Person}

end


Population() = Population(Person[])
Base.push!(p::Population, x::Person) = push!(p.individuals, x)


function Base.getindex(pop::Population, field::Symbol)
[getfield(x, field) for x in pop.individuals]
end

setindex!(p::Population, value, field::Symbol, index...) = 
p.individuals[index...].(field) = value
getindex(p::Population, field::Symbol, index...) = 
p.individuals[index...].(field)

Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, ", 
vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)


population = Population()
push!(population, Person(true, false, true, [3,4,5]))
push!(population, Person(false, true, false, Int64[3,4,5]))

#Show that it works
population[:infected]
population[:infected,1]
population[2]

LoadError: MethodError: `size` has no method matching size(::Population)
Closest candidates are:
  size(::Any, !Matched::Integer, !Matched::Integer, !Matched::Integer...)
while loading In[1], in expression starting on line 36

 in getindex at abstractarray.jl:488



On Saturday, June 4, 2016 at 3:11:18 PM UTC-4, Ford O. wrote:
>
> I would like to discourage you from passing symbols as an array index but 
> I guess you gonna do it anyway...
>
>
> So if you really want it, here it is:
> setindex!(p::Population, value, field::Symbol, index...) = p.individuals[
> index...].(field) = value
> getindex(p::Population, field::Symbol, index...) = p.individuals[index
> ...].(field)
>
> Usage
> population[:infected, 1]
> population[:infected, 1] = false
>
>
>
>
> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>
>> I was wondering if someone would be willing to help me with creating 
>> user-defined types. I've been using Julia for about two years now but I am 
>> new to the idea of creating custom types. I'm trying to create a population 
>> of agents/individuals in a simple epidemiological simulation. I would like 
>> the population of individuals to be structured as a  2 dimensional array 
>> with rows as individuals and columns as properties. This would be somewhat 
>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>> index an individual like so: population[1]. This woud list all of the 
>> information for individual 1.  I would also like to be able to look at an 
>> attribute across individuals: population.infected or population[:infected]. 
>> At the same time, I would like to have to flexibility of using an array to 
>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>> Based on existing documentation and examples, I have only been able to 
>> create individuals but cannot figure out how to create a population as 
>> described above:
>>
>> type Person
>> infected::Int64
>> vaccinated::Int64
>> dead::Int64
>>history::Array{Int64,1}
>> end
>>
>> Any help would be greatly appreciated. 
>>
>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Got it. Thanks for clarifying. 

On Saturday, June 4, 2016 at 5:49:39 PM UTC-4, Steven G. Johnson wrote:
>
>
>
> On Saturday, June 4, 2016 at 5:05:47 PM UTC-4, Christopher Fisher wrote:
>>
>> Thanks Ford. Is there a reason you discourage using symbols? I opted for 
>> the symbols partly because I couldn't get your example to work. So I 
>> combined code from the various answers until something worked. 
>>
>> I also noticed some odd behavior with the present implementation and I 
>> think its part of the reason I am having difficulty understanding all of 
>> the syntax. For example, overwriting values works both ways: 
>> population[:history,1][1] = 20 or population[:history][1][1] = 20. However 
>> the other properties do not work with both indexing schemes. It only works 
>> with population[:infected,1] = 20.
>>
>
> That's because population[:infected] makes a copy of the data.   With 
> history, it works because Arrays are mutable types (so basically you are 
> just getting a pointer to the same data).
>

[julia-users] Re: Help creating user-defined types and constructors

[Steven G. Johnson]

On Saturday, June 4, 2016 at 5:05:47 PM UTC-4, Christopher Fisher wrote:
>
> Thanks Ford. Is there a reason you discourage using symbols? I opted for 
> the symbols partly because I couldn't get your example to work. So I 
> combined code from the various answers until something worked. 
>
> I also noticed some odd behavior with the present implementation and I 
> think its part of the reason I am having difficulty understanding all of 
> the syntax. For example, overwriting values works both ways: 
> population[:history,1][1] = 20 or population[:history][1][1] = 20. However 
> the other properties do not work with both indexing schemes. It only works 
> with population[:infected,1] = 20.
>

That's because population[:infected] makes a copy of the data.   With 
history, it works because Arrays are mutable types (so basically you are 
just getting a pointer to the same data).

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Thanks Ford. Is there a reason you discourage using symbols? I opted for 
the symbols partly because I couldn't get your example to work. So I 
combined code from the various answers until something worked. 

I also noticed some odd behavior with the present implementation and I 
think its part of the reason I am having difficulty understanding all of 
the syntax. For example, overwriting values works both ways: 
population[:history,1][1] = 20 or population[:history][1][1] = 20. However 
the other properties do not work with both indexing schemes. It only works 
with population[:infected,1] = 20. 

On Saturday, June 4, 2016 at 3:11:18 PM UTC-4, Ford O. wrote:
>
> I would like to discourage you from passing symbols as an array index but 
> I guess you gonna do it anyway...
>
>
> So if you really want it, here it is:
> setindex!(p::Population, value, field::Symbol, index...) = p.individuals[
> index...].(field) = value
> getindex(p::Population, field::Symbol, index...) = p.individuals[index
> ...].(field)
>
> Usage
> population[:infected, 1]
> population[:infected, 1] = false
>
>
>
>
> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>
>> I was wondering if someone would be willing to help me with creating 
>> user-defined types. I've been using Julia for about two years now but I am 
>> new to the idea of creating custom types. I'm trying to create a population 
>> of agents/individuals in a simple epidemiological simulation. I would like 
>> the population of individuals to be structured as a  2 dimensional array 
>> with rows as individuals and columns as properties. This would be somewhat 
>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>> index an individual like so: population[1]. This woud list all of the 
>> information for individual 1.  I would also like to be able to look at an 
>> attribute across individuals: population.infected or population[:infected]. 
>> At the same time, I would like to have to flexibility of using an array to 
>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>> Based on existing documentation and examples, I have only been able to 
>> create individuals but cannot figure out how to create a population as 
>> described above:
>>
>> type Person
>> infected::Int64
>> vaccinated::Int64
>> dead::Int64
>>history::Array{Int64,1}
>> end
>>
>> Any help would be greatly appreciated. 
>>
>

[julia-users] Re: Help creating user-defined types and constructors

[Ford Ox]

I would like to discourage you from passing symbols as an array index but I 
guess you gonna do it anyway...


So if you really want it, here it is:
setindex!(p::Population, value, field::Symbol, index...) = p.individuals[
index...].(field) = value
getindex(p::Population, field::Symbol, index...) = p.individuals[index...].(
field)

Usage
population[:infected, 1]
population[:infected, 1] = false




On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>
> I was wondering if someone would be willing to help me with creating 
> user-defined types. I've been using Julia for about two years now but I am 
> new to the idea of creating custom types. I'm trying to create a population 
> of agents/individuals in a simple epidemiological simulation. I would like 
> the population of individuals to be structured as a  2 dimensional array 
> with rows as individuals and columns as properties. This would be somewhat 
> similar to a DataFrame, but potentially more flexible. I want to be able to 
> index an individual like so: population[1]. This woud list all of the 
> information for individual 1.  I would also like to be able to look at an 
> attribute across individuals: population.infected or population[:infected]. 
> At the same time, I would like to have to flexibility of using an array to 
> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
> Based on existing documentation and examples, I have only been able to 
> create individuals but cannot figure out how to create a population as 
> described above:
>
> type Person
> infected::Int64
> vaccinated::Int64
> dead::Int64
>history::Array{Int64,1}
> end
>
> Any help would be greatly appreciated. 
>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Thanks everyone for your helpful suggestions. Although a few details seem 
fuzzy, I think I am learning quite a bit and I am very close to the final 
product. 

Combining  Ford's initial code, Steve's code and David's code allowed me to 
access the information from specific individuals (e.g. population[1]) and 
access information across individuals (e.g. population[:infected] ). One 
thing is perplexing to me. I cannot change the values once the have been 
created. For example, after running the code below, I get:

population[:infected][1]

true


population[:infected][1] = falsepopulation[:infected][1]


true


That seems odd, but obviously I am missing something. The other question I have 
is whether my naive combining of code is the best implementation?







Combined code: 

type Person
infected::Bool
vaccinated::Bool
dead::Bool
history::Vector{Int}
end

type Population <: AbstractArray
individuals::Vector{Person}

end


Population() = Population(Person[])
Base.push!(p::Population, x::Person) = push!(p.individuals, x)
length(p::Population) = length(p.individuals)
size(p::Population) = size(p.individuals)

function Base.getindex(pop::Population, field::Symbol)
[getfield(x, field) for x in pop.individuals]
end

Base.setindex!(p::Population, i::Person, index...) = 
p.individuals[index...] = i
Base.getindex(p::Population, index) = p.individuals[index]
infected(i::Person) = i.infected
vaccinated(i::Person) = i.vaccinated

Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, ", 
vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)


population = Population()
push!(population, Person(true, false, true, [3,4,5]))
push!(population, Person(false, true, false, Int64[3,4,5]))

#Show that it works
population[:infected]
population[2]

[julia-users] Re: Help creating user-defined types and constructors

[Ford Ox]

FWIW

[individual.infected for individual in population] #list of all infected 
individuals 

typealias Population Vector{Individual}
population = Population(0)

On Saturday, June 4, 2016 at 4:48:29 PM UTC+2, Christopher Fisher wrote:
>
> Thank you for your help. 
>
> So my basic goal is to be able to perform operations on the individual and 
> population level. In this case, population.infected would return an 
> Array{Int64,1} indicating which individuals in the population are infected. 
> For example, population.infected[1]  = 1 would indicate the first 
> individual is infected. Something like mean(population.infected .== 1) 
> would indicate the percentage infected in the population. Similarly, if I 
> queried population[1], this would allow me to inspect all of the properties 
> assigned to individual 1. 
>
> Does that make sense?
>
> On Saturday, June 4, 2016 at 10:26:52 AM UTC-4, Ford Ox wrote:
>>
>> Could you please specify what should population.infected return? Could 
>> you provide an interface?
>>
>> Right now it seems like you want something like this:
>>
>> type Individual end
>>
>> type Population
>> individuals::Vector{Individual}
>> end
>>
>> setindex!(p::Population, i::Individual, index...) = p.individuals[index
>> ...] = i
>> getindex(p::Population, index) = p.individuals[index]
>> infected(i::Individual) = i.infected
>> vaccinated(i::Individual) = i.vaccinated
>>
>> Usage:
>> population[index] = Individual() # set new individual
>> infected(population[index])  # gets the infected property of 
>> individual at index
>> vaccinated(population[index])# ...
>>
>> But you don't really need functions for these things...
>>
>>
>> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>>
>>> I was wondering if someone would be willing to help me with creating 
>>> user-defined types. I've been using Julia for about two years now but I am 
>>> new to the idea of creating custom types. I'm trying to create a population 
>>> of agents/individuals in a simple epidemiological simulation. I would like 
>>> the population of individuals to be structured as a  2 dimensional array 
>>> with rows as individuals and columns as properties. This would be somewhat 
>>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>>> index an individual like so: population[1]. This woud list all of the 
>>> information for individual 1.  I would also like to be able to look at an 
>>> attribute across individuals: population.infected or population[:infected]. 
>>> At the same time, I would like to have to flexibility of using an array to 
>>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>>> Based on existing documentation and examples, I have only been able to 
>>> create individuals but cannot figure out how to create a population as 
>>> described above:
>>>
>>> type Person
>>> infected::Int64
>>> vaccinated::Int64
>>> dead::Int64
>>>history::Array{Int64,1}
>>> end
>>>
>>> Any help would be greatly appreciated. 
>>>
>>

[julia-users] Re: Help creating user-defined types and constructors

[David P. Sanders]

You can define a `Population` type and overload `getindex` for it to do 
what you want, something like:

type Person
infected::Bool
vaccinated::Bool
dead::Bool
history::Vector{Int}
end

type Population <: AbstractArray
individuals::Vector{Person}
end


Population() = Population(Person[])
Base.push!(p::Population, x::Person) = push!(p.individuals, x)
length(p::Population) = length(p.individuals)
size(p::Population) = size(p.individuals)

function Base.getindex(pop::Population, field::Symbol)
[getfield(x, field) for x in pop.individuals]
end

population = Population()
push!(population, Person(true, false, true, [3,4,5]))
push!(population, Person(false, true, false, Int64[3,4,5]))

population[:infected]


Note that this is not very efficient; I believe the following package may 
be an alternative / better solution:

 https://github.com/simonster/StructsOfArrays.jl



El sábado, 4 de junio de 2016, 11:13:21 (UTC-4), Christopher Fisher 
escribió:
>
> Thanks for your suggestions Steven. You are correct. I could many of those 
> fields as Boolean and maybe there is a reason to prefer that rather than 
> using 1 and 0s. 
>
> Initially, I also tried to push the individuals into an empty array as you 
> suggested. This worked well, except I was not able to query across 
> individuals:
>
> type Person
> infected::Int64
> vaccinated::Int64
> dead::Int64
>history::Array{Int64,1}
> end
> population = Person[]   # create an empty population
> push!(population, Person(0, 2, 0, Int64[3,4,5]))   # add a person to the 
> population
> push!(population, Person(0, 2, 0, Int64[3,4,5]))
> population.infected
>
> LoadError: type Array has no field infected
> while loading In[16], in expression starting on line 10
>
>
>
> What I was hoping is that it would list the infection status for all 
> members of the population. Is there any way to do that?
>
>
> On Saturday, June 4, 2016 at 10:47:55 AM UTC-4, Steven G. Johnson wrote:
>>
>>
>>
>> On Saturday, June 4, 2016 at 8:19:02 AM UTC-4, Christopher Fisher wrote:
>>>
>>> I was wondering if someone would be willing to help me with creating 
>>> user-defined types. I've been using Julia for about two years now but I am 
>>> new to the idea of creating custom types. I'm trying to create a population 
>>> of agents/individuals in a simple epidemiological simulation. I would like 
>>> the population of individuals to be structured as a  2 dimensional array 
>>> with rows as individuals and columns as properties. This would be somewhat 
>>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>>> index an individual like so: population[1]. This woud list all of the 
>>> information for individual 1.  I would also like to be able to look at an 
>>> attribute across individuals: population.infected or population[:infected]. 
>>> At the same time, I would like to have to flexibility of using an array to 
>>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>>> Based on existing documentation and examples, I have only been able to 
>>> create individuals but cannot figure out how to create a population as 
>>> described above:
>>>
>>> type Person
>>> infected::Int64
>>> vaccinated::Int64
>>> dead::Int64
>>>history::Array{Int64,1}
>>> end
>>>
>>
>> It sounds like you just want an array of your Person type.
>>
>> e.g.
>>
>> population = Person[]   # create an empty population
>> push!(population, Person(0, 2, 0, Int64[3,4,5]))   # add a person to the 
>> population
>> population[1].dead # access the "dead" field of the first person in the 
>> population
>> population[1].history[3]  # access history[3] from person 1
>>
>> Of course, you can make working with the Person type a lot nicer by 
>> defining more methods.   e.g. you probably want to have a "show" method to 
>> pretty-print a person, for example:
>>
>> Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, 
>> ", vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)
>>
>> When designing your types, you also want to think carefully about your 
>> fields.   e.g. aren't infected, dead, etc. really Boolean fields?  If you 
>> can encode "history" into a fixed-width field (e.g. a single 64-bit 
>> integer), it will be much more efficient.  And arrays of persons will be 
>> more efficient if you can make the person immutable --- you only need a 
>> (mutable) type if you need to have multiple references to the same 
>> individual, where modifying one reference's fields will change the data 
>> seen from all the references.
>>
>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Thanks for your suggestions Steven. You are correct. I could many of those 
fields as Boolean and maybe there is a reason to prefer that rather than 
using 1 and 0s. 

Initially, I also tried to push the individuals into an empty array as you 
suggested. This worked well, except I was not able to query across 
individuals:

type Person
infected::Int64
vaccinated::Int64
dead::Int64
   history::Array{Int64,1}
end
population = Person[]   # create an empty population
push!(population, Person(0, 2, 0, Int64[3,4,5]))   # add a person to the 
population
push!(population, Person(0, 2, 0, Int64[3,4,5]))
population.infected

LoadError: type Array has no field infected
while loading In[16], in expression starting on line 10



What I was hoping is that it would list the infection status for all 
members of the population. Is there any way to do that?


On Saturday, June 4, 2016 at 10:47:55 AM UTC-4, Steven G. Johnson wrote:
>
>
>
> On Saturday, June 4, 2016 at 8:19:02 AM UTC-4, Christopher Fisher wrote:
>>
>> I was wondering if someone would be willing to help me with creating 
>> user-defined types. I've been using Julia for about two years now but I am 
>> new to the idea of creating custom types. I'm trying to create a population 
>> of agents/individuals in a simple epidemiological simulation. I would like 
>> the population of individuals to be structured as a  2 dimensional array 
>> with rows as individuals and columns as properties. This would be somewhat 
>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>> index an individual like so: population[1]. This woud list all of the 
>> information for individual 1.  I would also like to be able to look at an 
>> attribute across individuals: population.infected or population[:infected]. 
>> At the same time, I would like to have to flexibility of using an array to 
>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>> Based on existing documentation and examples, I have only been able to 
>> create individuals but cannot figure out how to create a population as 
>> described above:
>>
>> type Person
>> infected::Int64
>> vaccinated::Int64
>> dead::Int64
>>history::Array{Int64,1}
>> end
>>
>
> It sounds like you just want an array of your Person type.
>
> e.g.
>
> population = Person[]   # create an empty population
> push!(population, Person(0, 2, 0, Int64[3,4,5]))   # add a person to the 
> population
> population[1].dead # access the "dead" field of the first person in the 
> population
> population[1].history[3]  # access history[3] from person 1
>
> Of course, you can make working with the Person type a lot nicer by 
> defining more methods.   e.g. you probably want to have a "show" method to 
> pretty-print a person, for example:
>
> Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, 
> ", vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)
>
> When designing your types, you also want to think carefully about your 
> fields.   e.g. aren't infected, dead, etc. really Boolean fields?  If you 
> can encode "history" into a fixed-width field (e.g. a single 64-bit 
> integer), it will be much more efficient.  And arrays of persons will be 
> more efficient if you can make the person immutable --- you only need a 
> (mutable) type if you need to have multiple references to the same 
> individual, where modifying one reference's fields will change the data 
> seen from all the references.
>

[julia-users] Re: Help creating user-defined types and constructors

[Christopher Fisher]

Thank you for your help. 

So my basic goal is to be able to perform operations on the individual and 
population level. In this case, population.infected would return an 
Array{Int64,1} indicating which individuals in the population are infected. 
For example, population.infected[1]  = 1 would indicate the first 
individual is infected. Something like mean(population.infected .== 1) 
would indicate the percentage infected in the population. Similarly, if I 
queried population[1], this would allow me to inspect all of the properties 
assigned to individual 1. 

Does that make sense?

On Saturday, June 4, 2016 at 10:26:52 AM UTC-4, Ford Ox wrote:
>
> Could you please specify what should population.infected return? Could you 
> provide an interface?
>
> Right now it seems like you want something like this:
>
> type Individual end
>
> type Population
> individuals::Vector{Individual}
> end
>
> setindex!(p::Population, i::Individual, index...) = p.individuals[index
> ...] = i
> getindex(p::Population, index) = p.individuals[index]
> infected(i::Individual) = i.infected
> vaccinated(i::Individual) = i.vaccinated
>
> Usage:
> population[index] = Individual() # set new individual
> infected(population[index])  # gets the infected property of 
> individual at index
> vaccinated(population[index])# ...
>
> But you don't really need functions for these things...
>
>
> On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>>
>> I was wondering if someone would be willing to help me with creating 
>> user-defined types. I've been using Julia for about two years now but I am 
>> new to the idea of creating custom types. I'm trying to create a population 
>> of agents/individuals in a simple epidemiological simulation. I would like 
>> the population of individuals to be structured as a  2 dimensional array 
>> with rows as individuals and columns as properties. This would be somewhat 
>> similar to a DataFrame, but potentially more flexible. I want to be able to 
>> index an individual like so: population[1]. This woud list all of the 
>> information for individual 1.  I would also like to be able to look at an 
>> attribute across individuals: population.infected or population[:infected]. 
>> At the same time, I would like to have to flexibility of using an array to 
>> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
>> Based on existing documentation and examples, I have only been able to 
>> create individuals but cannot figure out how to create a population as 
>> described above:
>>
>> type Person
>> infected::Int64
>> vaccinated::Int64
>> dead::Int64
>>history::Array{Int64,1}
>> end
>>
>> Any help would be greatly appreciated. 
>>
>

[julia-users] Re: Help creating user-defined types and constructors

[Steven G. Johnson]

On Saturday, June 4, 2016 at 8:19:02 AM UTC-4, Christopher Fisher wrote:
>
> I was wondering if someone would be willing to help me with creating 
> user-defined types. I've been using Julia for about two years now but I am 
> new to the idea of creating custom types. I'm trying to create a population 
> of agents/individuals in a simple epidemiological simulation. I would like 
> the population of individuals to be structured as a  2 dimensional array 
> with rows as individuals and columns as properties. This would be somewhat 
> similar to a DataFrame, but potentially more flexible. I want to be able to 
> index an individual like so: population[1]. This woud list all of the 
> information for individual 1.  I would also like to be able to look at an 
> attribute across individuals: population.infected or population[:infected]. 
> At the same time, I would like to have to flexibility of using an array to 
> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
> Based on existing documentation and examples, I have only been able to 
> create individuals but cannot figure out how to create a population as 
> described above:
>
> type Person
> infected::Int64
> vaccinated::Int64
> dead::Int64
>history::Array{Int64,1}
> end
>

It sounds like you just want an array of your Person type.

e.g.

population = Person[]   # create an empty population
push!(population, Person(0, 2, 0, Int64[3,4,5]))   # add a person to the 
population
population[1].dead # access the "dead" field of the first person in the 
population
population[1].history[3]  # access history[3] from person 1

Of course, you can make working with the Person type a lot nicer by 
defining more methods.   e.g. you probably want to have a "show" method to 
pretty-print a person, for example:

Base.show(io::IO, p::Person) = print(io, "Person(infected=", p.infected, ", 
vaccinated=", p.vaccinated, ", dead=", p.dead, ", history=", p.history)

When designing your types, you also want to think carefully about your 
fields.   e.g. aren't infected, dead, etc. really Boolean fields?  If you 
can encode "history" into a fixed-width field (e.g. a single 64-bit 
integer), it will be much more efficient.  And arrays of persons will be 
more efficient if you can make the person immutable --- you only need a 
(mutable) type if you need to have multiple references to the same 
individual, where modifying one reference's fields will change the data 
seen from all the references.

[julia-users] Re: Help creating user-defined types and constructors

[Ford Ox]

Could you please specify what should population.infected return? Could you 
provide an interface?

Right now it seems like you want something like this:

type Individual end

type Population
individuals::Vector{Individual}
end

setindex!(p::Population, i::Individual, index...) = p.individuals[index...] 
= i
getindex(p::Population, index) = p.individuals[index]
infected(i::Individual) = i.infected
vaccinated(i::Individual) = i.vaccinated

Usage:
population[index] = Individual() # set new individual
infected(population[index])  # gets the infected property of individual 
at index
vaccinated(population[index])# ...

But you don't really need functions for these things...


On Saturday, June 4, 2016 at 2:19:02 PM UTC+2, Christopher Fisher wrote:
>
> I was wondering if someone would be willing to help me with creating 
> user-defined types. I've been using Julia for about two years now but I am 
> new to the idea of creating custom types. I'm trying to create a population 
> of agents/individuals in a simple epidemiological simulation. I would like 
> the population of individuals to be structured as a  2 dimensional array 
> with rows as individuals and columns as properties. This would be somewhat 
> similar to a DataFrame, but potentially more flexible. I want to be able to 
> index an individual like so: population[1]. This woud list all of the 
> information for individual 1.  I would also like to be able to look at an 
> attribute across individuals: population.infected or population[:infected]. 
> At the same time, I would like to have to flexibility of using an array to 
> keep track of individuals: typeof(population.history[1]) is Array{Int64,1}. 
> Based on existing documentation and examples, I have only been able to 
> create individuals but cannot figure out how to create a population as 
> described above:
>
> type Person
> infected::Int64
> vaccinated::Int64
> dead::Int64
>history::Array{Int64,1}
> end
>
> Any help would be greatly appreciated. 
>

[julia-users] Re: Help! Problem with repeated calling of a function that creates SharedArray.

[Nils Gudat]

This code runs for me without problems (Julia 0.4.5 64bit on an 8GB Win8.1 
laptop)

[julia-users] Re: Help with MXNet AssertionError: get_batch_size(opts.eval_data) == batch_size

[Iain Dunning]

Try filing an issue on the MXNet.jl repository - the developers might not 
(probably don't) read this list.

On Friday, April 15, 2016 at 9:47:46 AM UTC-4, kleinsplash wrote:
>
> still having this issue - any ideas?
>
> On Monday, 11 April 2016 15:58:03 UTC+2, kleinsplash wrote:
>>
>> $ julia G3DB_cnn.jl 
>> (128,128,1,800)
>> (128,128,1,200)
>> INFO: Start training on [GPU0]
>> INFO: Initializing parameters...
>> INFO: Creating KVStore...
>> INFO: Start training...
>> INFO: == Epoch 001 ==
>> INFO: ## Training summary
>> INFO:   accuracy = 0.5638
>> INFO:   time = 1.9078 seconds
>> ERROR: LoadError: AssertionError: get_batch_size(opts.eval_data) == 
>> batch_size
>>  in fit at /home/ashley/.julia/v0.4/MXNet/src/model.jl:482
>>  in include at ./boot.jl:261
>>  in include_from_node1 at ./loading.jl:304
>>  in process_options at ./client.jl:280
>>  in _start at ./client.jl:378
>>
>>
>> Could someone help me understand the above? I dont have many data so my 
>> batch_size is 0. Trying to get to the predicted values.. 
>>
>> Thx
>>
>

[julia-users] Re: Help with MXNet AssertionError: get_batch_size(opts.eval_data) == batch_size

[Iain Dunning]

Try filing an issue on the MXNet.jl repository - the developers might not 
(probably don't) read this list.

On Friday, April 15, 2016 at 9:47:46 AM UTC-4, kleinsplash wrote:
>
> still having this issue - any ideas?
>
> On Monday, 11 April 2016 15:58:03 UTC+2, kleinsplash wrote:
>>
>> $ julia G3DB_cnn.jl 
>> (128,128,1,800)
>> (128,128,1,200)
>> INFO: Start training on [GPU0]
>> INFO: Initializing parameters...
>> INFO: Creating KVStore...
>> INFO: Start training...
>> INFO: == Epoch 001 ==
>> INFO: ## Training summary
>> INFO:   accuracy = 0.5638
>> INFO:   time = 1.9078 seconds
>> ERROR: LoadError: AssertionError: get_batch_size(opts.eval_data) == 
>> batch_size
>>  in fit at /home/ashley/.julia/v0.4/MXNet/src/model.jl:482
>>  in include at ./boot.jl:261
>>  in include_from_node1 at ./loading.jl:304
>>  in process_options at ./client.jl:280
>>  in _start at ./client.jl:378
>>
>>
>> Could someone help me understand the above? I dont have many data so my 
>> batch_size is 0. Trying to get to the predicted values.. 
>>
>> Thx
>>
>

[julia-users] Re: Help with MXNet AssertionError: get_batch_size(opts.eval_data) == batch_size

[kleinsplash]

still having this issue - any ideas?

On Monday, 11 April 2016 15:58:03 UTC+2, kleinsplash wrote:
>
> $ julia G3DB_cnn.jl 
> (128,128,1,800)
> (128,128,1,200)
> INFO: Start training on [GPU0]
> INFO: Initializing parameters...
> INFO: Creating KVStore...
> INFO: Start training...
> INFO: == Epoch 001 ==
> INFO: ## Training summary
> INFO:   accuracy = 0.5638
> INFO:   time = 1.9078 seconds
> ERROR: LoadError: AssertionError: get_batch_size(opts.eval_data) == 
> batch_size
>  in fit at /home/ashley/.julia/v0.4/MXNet/src/model.jl:482
>  in include at ./boot.jl:261
>  in include_from_node1 at ./loading.jl:304
>  in process_options at ./client.jl:280
>  in _start at ./client.jl:378
>
>
> Could someone help me understand the above? I dont have many data so my 
> batch_size is 0. Trying to get to the predicted values.. 
>
> Thx
>

[julia-users] Re: Help with parallel computing for optimization.

[Jiahao Chen]

Looks like the bug report ClusterManagers.jl#31. Can you try

Pkg.checkout("ClusterManagers")


and see if that works for you? 

[julia-users] Re: Help with convoluted types and Vararg

['Greg Plowman' via julia-users]

Pair is a parametric type, and in Julia these are invariant, 
meaning element subtyping does not imply pair subtyping.

In your case, the pair elements are subtypes:

Tuple{Function,Int,Int,Int} <: Tuple{Function,Vararg{Int}} # true
Int <: Int # true

but the Pair is not:

Pair{Tuple{Function,Int,Int,Int}, Int} <: Pair{Tuple{Function,Vararg{Int}}, 
Int} # false


On Wednesday, April 6, 2016 at 4:51:39 AM UTC+10, Seth wrote:

> Hi all,
>
> I have the following on 0.4.6-pre+18:
>
> z = [Pair((+,1,5,7), 3), Pair((-,6,5,3,5,8), 1)]
> type Foo
> x::Array{Pair{Tuple{Function, Vararg{Int}}, Int}}
> end
>
>
> and I'm getting
>
> julia> Foo(z)
> ERROR: MethodError: `convert` has no method matching 
> convert(::Type{Pair{Tuple{Function,Vararg{Int64}},Int64}}, 
> ::Pair{Tuple{Function,Int64,Int64,Int64},Int64})
> This may have arisen from a call to the constructor 
> Pair{Tuple{Function,Vararg{Int64}},Int64}(...),
> since type constructors fall back to convert methods.
> Closest candidates are:
>   Pair{A,B}(::Any, ::Any)
>   call{T}(::Type{T}, ::Any)
>   convert{T}(::Type{T}, ::T)
>  in copy! at abstractarray.jl:310
>  in call at none:2
>
>
> It's probably a stupid oversight, but I'm stuck. Can someone point me to 
> the error?
>

[julia-users] Re: Help with convoluted types and Vararg

[Jeffrey Sarnoff]

this works -- more than than .. well no

type Foo
   x::Vector{}
end

z = [Pair((+,1,5,7), 3), Pair((-,6,5,3,5,8), 1)]

Foo(z)
Foo( Pair{A,Int64}[ (+,1,5,7)=>3, (-,6,5,3,5,8)=>1 ] )


On Tuesday, April 5, 2016 at 2:51:39 PM UTC-4, Seth wrote:
>
> Hi all,
>
> I have the following on 0.4.6-pre+18:
>
> z = [Pair((+,1,5,7), 3), Pair((-,6,5,3,5,8), 1)]
> type Foo
> x::Array{Pair{Tuple{Function, Vararg{Int}}, Int}}
> end
>
>
> and I'm getting
>
> julia> Foo(z)
> ERROR: MethodError: `convert` has no method matching 
> convert(::Type{Pair{Tuple{Function,Vararg{Int64}},Int64}}, 
> ::Pair{Tuple{Function,Int64,Int64,Int64},Int64})
> This may have arisen from a call to the constructor 
> Pair{Tuple{Function,Vararg{Int64}},Int64}(...),
> since type constructors fall back to convert methods.
> Closest candidates are:
>   Pair{A,B}(::Any, ::Any)
>   call{T}(::Type{T}, ::Any)
>   convert{T}(::Type{T}, ::T)
>  in copy! at abstractarray.jl:310
>  in call at none:2
>
>
> It's probably a stupid oversight, but I'm stuck. Can someone point me to 
> the error?
>

[julia-users] Re: Help with PyCall syntax

[Chris]

Ok. I tried some variation of a get() call, but I had no idea I needed to 
use pybuiltin(:slice). Thanks!

On Tuesday, March 15, 2016 at 6:52:58 PM UTC-4, Steven G. Johnson wrote:
>
> s = pybuiltin(:slice)
>
> get(np.mgrid, (s(0,5), s(0,5)))
>
>
> works.   (In general, foo[bar, baz] in Python needs to be get(foo, 
> (bar,baz)) in PyCall at the moment, because foo[...] is used for 
> attributes.)
>
>
> It would be nice if you could just use 0:5 here, but 0:5 is currently 
> translated into an xrange object in Python, not a slice.   I'm not sure if 
> slice would be a better default translation?   (I don't understand why 
> Python has slice, xrange, and itertools.islice, all of which seem like they 
> could be collapsed into a single type.)
>

[julia-users] Re: Help with PyCall syntax

[Steven G. Johnson]

 

s = pybuiltin(:slice)

get(np.mgrid, (s(0,5), s(0,5)))


works.   (In general, foo[bar, baz] in Python needs to be get(foo, 
(bar,baz)) in PyCall at the moment, because foo[...] is used for 
attributes.)


It would be nice if you could just use 0:5 here, but 0:5 is currently 
translated into an xrange object in Python, not a slice.   I'm not sure if 
slice would be a better default translation?   (I don't understand why 
Python has slice, xrange, and itertools.islice, all of which seem like they 
could be collapsed into a single type.)

[julia-users] Re: Help with PyCall syntax

[Chris]

Thanks! This helped me with the problem at hand, but it would still be nice 
to know if there's a more general way of doing this type of evaluation in 
PyCall.

On Tuesday, March 15, 2016 at 12:23:55 PM UTC-4, Robin Deits wrote:
>
> This isn't pretty, but it might do the job: 
>
> julia> np.mgrid[:__getitem__]((pybuiltin(:slice)(0,5), 
> pybuiltin(:slice)(0,5)))
> 2x5x5 Array{Int64,3}:
> [:, :, 1] =
>  0  1  2  3  4
>  0  0  0  0  0
>
> [:, :, 2] =
>  0  1  2  3  4
>  1  1  1  1  1
>
> [:, :, 3] =
>  0  1  2  3  4
>  2  2  2  2  2
>
> [:, :, 4] =
>  0  1  2  3  4
>  3  3  3  3  3
>
> [:, :, 5] =
>  0  1  2  3  4
>  4  4  4  4  4
>
> It's just a deconstruction of the actual call that happens in python: 
>
> In [7]: np.mgrid.__getitem__((slice(0,5), slice(0,5)))
> Out[7]:
> array([[[0, 0, 0, 0, 0],
> [1, 1, 1, 1, 1],
> [2, 2, 2, 2, 2],
> [3, 3, 3, 3, 3],
> [4, 4, 4, 4, 4]],
>
>[[0, 1, 2, 3, 4],
> [0, 1, 2, 3, 4],
> [0, 1, 2, 3, 4],
> [0, 1, 2, 3, 4],
> [0, 1, 2, 3, 4]]])
>
> On Tuesday, March 15, 2016 at 12:01:06 PM UTC-4, Chris wrote:
>>
>> I'm trying to use numpy's mgrid (
>> https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.mgrid.html)
>>  
>> via PyCall, but I can't figure out what the syntax should be. I've tried 
>> every permutation I can think of, could someone help me out?
>>
>> The python call looks like:
>>
>> >>> np.mgrid[0:5,0:5]array([[[0, 0, 0, 0, 0],[1, 1, 1, 1, 1],
>> >>> [2, 2, 2, 2, 2],[3, 3, 3, 3, 3],[4, 4, 4, 4, 4]],   
>> >>> [[0, 1, 2, 3, 4],[0, 1, 2, 3, 4],[0, 1, 2, 3, 4],
>> >>> [0, 1, 2, 3, 4],[0, 1, 2, 3, 4]]])
>>
>>
>> Thanks in advance.
>>
>

[julia-users] Re: Help with PyCall syntax

[Robin Deits]

This isn't pretty, but it might do the job: 

julia> np.mgrid[:__getitem__]((pybuiltin(:slice)(0,5), 
pybuiltin(:slice)(0,5)))
2x5x5 Array{Int64,3}:
[:, :, 1] =
 0  1  2  3  4
 0  0  0  0  0

[:, :, 2] =
 0  1  2  3  4
 1  1  1  1  1

[:, :, 3] =
 0  1  2  3  4
 2  2  2  2  2

[:, :, 4] =
 0  1  2  3  4
 3  3  3  3  3

[:, :, 5] =
 0  1  2  3  4
 4  4  4  4  4

It's just a deconstruction of the actual call that happens in python: 

In [7]: np.mgrid.__getitem__((slice(0,5), slice(0,5)))
Out[7]:
array([[[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2],
[3, 3, 3, 3, 3],
[4, 4, 4, 4, 4]],

   [[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]])

On Tuesday, March 15, 2016 at 12:01:06 PM UTC-4, Chris wrote:
>
> I'm trying to use numpy's mgrid (
> https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.mgrid.html) 
> via PyCall, but I can't figure out what the syntax should be. I've tried 
> every permutation I can think of, could someone help me out?
>
> The python call looks like:
>
> >>> np.mgrid[0:5,0:5]array([[[0, 0, 0, 0, 0],[1, 1, 1, 1, 1],
> >>> [2, 2, 2, 2, 2],[3, 3, 3, 3, 3],[4, 4, 4, 4, 4]],   
> >>> [[0, 1, 2, 3, 4],[0, 1, 2, 3, 4],[0, 1, 2, 3, 4],
> >>> [0, 1, 2, 3, 4],[0, 1, 2, 3, 4]]])
>
>
> Thanks in advance.
>

[julia-users] Re: Help with pipes

[J Luis]

Created issue 15479 

domingo, 13 de Março de 2016 às 00:59:28 UTC, ele...@gmail.com escreveu:
>
> Ok, the docs 
> http://docs.julialang.org/en/release-0.4/stdlib/io-network/#Base.IOBuffer 
> seem to say it creates an IOStream, and pipeline says it accepts one 
> http://docs.julialang.org/en/release-0.4/stdlib/base/?highlight=pipeline#Base.pipeline
>  
> so one of them is wrong.
>
> On Sunday, 13 March 2016 10:49:55 UTC+10, J Luis wrote:
>>
>> Thanks but doesn't work either
>>
>> a = IOBuffer();
>> print(a, P.ps);
>>
>> julia> run(pipeline(a, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH 
>> -sDEVICE=bbox -`))
>> ERROR: MethodError: `uvtype` has no method matching uvtype(::Base.
>> AbstractIOBuffer{Array{UInt8,1}})
>>  in _jl_spawn at process.jl:253
>>
>>
>>
>> domingo, 13 de Março de 2016 às 00:36:19 UTC, ele...@gmail.com escreveu:
>>>
>>> Can't test ATM, but argument one IOBuffer(ps_string) ?
>>>
>>> On Sunday, 13 March 2016 09:52:52 UTC+10, J Luis wrote:

 Hi,

 I need to reproduce a similar behavior of this on command line. 

 cat barco.eps | gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -
 %%BoundingBox: 0 0 71 64
 %%HiResBoundingBox: 0.00 0.00 70.865998 63.305998

 What it does is to send the contents of a PostScript file (ascii file) 
 through ghostscript and get the result printed on stdout

 Now, instead of a file I have in Julia the contents of that file as a 
 long string and wanted to feed it to ghostscript, first to do the 
 BoundingBox query and later on to convert the ps into a raster. So I tried

  run(pipeline(P.ps, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH 
 -sDEVICE=bbox -`))
 ERROR: open: no such file or directory (ENOENT)
  in open at fs.jl:82

 I guess the reason lies on the *pipeline *manual mention when it says 
 that "Strings refer to filenames". So given that P.ps above is a 
 string it tries to open it as a file.
 So my question is, how can I put the string inpipeline's first argument 
 and get the output in a Julia variable?

 Thanks

 Joaquim

>>>

[julia-users] Re: Help with pipes

[elextr]

Ok, the 
docs http://docs.julialang.org/en/release-0.4/stdlib/io-network/#Base.IOBuffer 
seem to say it creates an IOStream, and pipeline says it accepts 
one 
http://docs.julialang.org/en/release-0.4/stdlib/base/?highlight=pipeline#Base.pipeline
 
so one of them is wrong.

On Sunday, 13 March 2016 10:49:55 UTC+10, J Luis wrote:
>
> Thanks but doesn't work either
>
> a = IOBuffer();
> print(a, P.ps);
>
> julia> run(pipeline(a, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH 
> -sDEVICE=bbox -`))
> ERROR: MethodError: `uvtype` has no method matching uvtype(::Base.
> AbstractIOBuffer{Array{UInt8,1}})
>  in _jl_spawn at process.jl:253
>
>
>
> domingo, 13 de Março de 2016 às 00:36:19 UTC, ele...@gmail.com escreveu:
>>
>> Can't test ATM, but argument one IOBuffer(ps_string) ?
>>
>> On Sunday, 13 March 2016 09:52:52 UTC+10, J Luis wrote:
>>>
>>> Hi,
>>>
>>> I need to reproduce a similar behavior of this on command line. 
>>>
>>> cat barco.eps | gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -
>>> %%BoundingBox: 0 0 71 64
>>> %%HiResBoundingBox: 0.00 0.00 70.865998 63.305998
>>>
>>> What it does is to send the contents of a PostScript file (ascii file) 
>>> through ghostscript and get the result printed on stdout
>>>
>>> Now, instead of a file I have in Julia the contents of that file as a 
>>> long string and wanted to feed it to ghostscript, first to do the 
>>> BoundingBox query and later on to convert the ps into a raster. So I tried
>>>
>>>  run(pipeline(P.ps, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH 
>>> -sDEVICE=bbox -`))
>>> ERROR: open: no such file or directory (ENOENT)
>>>  in open at fs.jl:82
>>>
>>> I guess the reason lies on the *pipeline *manual mention when it says 
>>> that "Strings refer to filenames". So given that P.ps above is a string 
>>> it tries to open it as a file.
>>> So my question is, how can I put the string inpipeline's first argument 
>>> and get the output in a Julia variable?
>>>
>>> Thanks
>>>
>>> Joaquim
>>>
>>

[julia-users] Re: Help with pipes

[J Luis]

Thanks but doesn't work either

a = IOBuffer();
print(a, P.ps);

julia> run(pipeline(a, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox 
-`))
ERROR: MethodError: `uvtype` has no method matching uvtype(::Base.
AbstractIOBuffer{Array{UInt8,1}})
 in _jl_spawn at process.jl:253



domingo, 13 de Março de 2016 às 00:36:19 UTC, ele...@gmail.com escreveu:
>
> Can't test ATM, but argument one IOBuffer(ps_string) ?
>
> On Sunday, 13 March 2016 09:52:52 UTC+10, J Luis wrote:
>>
>> Hi,
>>
>> I need to reproduce a similar behavior of this on command line. 
>>
>> cat barco.eps | gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -
>> %%BoundingBox: 0 0 71 64
>> %%HiResBoundingBox: 0.00 0.00 70.865998 63.305998
>>
>> What it does is to send the contents of a PostScript file (ascii file) 
>> through ghostscript and get the result printed on stdout
>>
>> Now, instead of a file I have in Julia the contents of that file as a 
>> long string and wanted to feed it to ghostscript, first to do the 
>> BoundingBox query and later on to convert the ps into a raster. So I tried
>>
>>  run(pipeline(P.ps, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox 
>> -`))
>> ERROR: open: no such file or directory (ENOENT)
>>  in open at fs.jl:82
>>
>> I guess the reason lies on the *pipeline *manual mention when it says 
>> that "Strings refer to filenames". So given that P.ps above is a string 
>> it tries to open it as a file.
>> So my question is, how can I put the string inpipeline's first argument 
>> and get the output in a Julia variable?
>>
>> Thanks
>>
>> Joaquim
>>
>

[julia-users] Re: Help with pipes

[elextr]

Can't test ATM, but argument one IOBuffer(ps_string) ?

On Sunday, 13 March 2016 09:52:52 UTC+10, J Luis wrote:
>
> Hi,
>
> I need to reproduce a similar behavior of this on command line. 
>
> cat barco.eps | gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -
> %%BoundingBox: 0 0 71 64
> %%HiResBoundingBox: 0.00 0.00 70.865998 63.305998
>
> What it does is to send the contents of a PostScript file (ascii file) 
> through ghostscript and get the result printed on stdout
>
> Now, instead of a file I have in Julia the contents of that file as a long 
> string and wanted to feed it to ghostscript, first to do the BoundingBox 
> query and later on to convert the ps into a raster. So I tried
>
>  run(pipeline(P.ps, `gswin64c -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox 
> -`))
> ERROR: open: no such file or directory (ENOENT)
>  in open at fs.jl:82
>
> I guess the reason lies on the *pipeline *manual mention when it says 
> that "Strings refer to filenames". So given that P.ps above is a string 
> it tries to open it as a file.
> So my question is, how can I put the string inpipeline's first argument 
> and get the output in a Julia variable?
>
> Thanks
>
> Joaquim
>

[julia-users] Re: Help with some code where we would normally use inheritance

[Bryan Rivera]

I think I almost got it - just need to figure out how to inject 
onCompletedInterval with the proper vars in macro.

macro gen_intervalize(onCompletedInterval)

  return quote
  if(isNextInterval(i.intervalState, time))

onCompletedInterval(getCurrentIntervalTime(i.intervalState), i.
currentValue)

# Set next interval.
shiftInterval!(i)

# Reset Value.
resetValue!(i)
  end

  # Increment current value.
  incrementCurrentValue!(value)
   end
end


function intervalize{T, V}(hi::HistoricalIntervalizer{T,V}, time::T, value::
V)
  @gen_intervalize quote
minuteOfDay = (time / (60  * 1000))
hi.array[minuteOfDay + 1] = value
  end
end

That's powerful.  

[julia-users] Re: help with using Julia + Cilk

[Kevin Lin]

Thanks for the quick response.  Missing python lib does not seem to be 
problem, because (as I said) PyPlot etc have all been working just fine 
except when a c module using cilk (such as the hello code above) is called 
*before* the "using PyPlot".  Indeed, if I do

using PyPlot
include("hello.jl")
plot(...)

i.e., reverse the order of PyPlot and hello in my earlier example, then it 
works fine on one of my machines but not on the others.  This behavior 
seems a bit odd.

Still, it is quite possibly a problem with my installation.  Can anyone 
reproduce this behavior?

Thanks,
Kevin

On Friday, January 22, 2016 at 5:18:25 AM UTC-7, Lutfullah Tomak wrote:
>
> It is unrelated to Julia because you are missing a python library that is 
> needed by PyPlot. From error, I think you're missing numpy.

[julia-users] Re: Help: PyPlot cannot be included in ~/.juliarc.jl?

[Kaj Wiik]

Yes, calling ion() does help, thanks!


On Wednesday, January 20, 2016 at 9:01:36 PM UTC+2, cdm wrote:
>
> maybe interactive mode is off somehow ...
>
> does a call to
>
>ion()
>
> help ... ?
>
>
>
> On Wednesday, January 20, 2016 at 12:05:51 AM UTC-8, Daniel Carrera wrote:
>>
>> Hello,
>>
>> Here is an odd problem. If I put in "using PyPlot" inside my 
>> ~/.juliarc.jl file, when I start the Julia shell I cannot get any plots. 
>> There is no error message, just no plot:
>>
>> $ julia 
>> ...
>> julia> x = linspace(0,2pi);
>> julia> y = sin(x);
>> julia> plot(x,y)
>> 1-element Array{Any,1}:
>>  PyObject 
>>
>>
>> But if I remove PyPlot from ~/.juliarc.jl and insert it manually in the 
>> Julia shell, everything works great. Does anyone know what's happening?
>>
>> Cheers,
>> Daniel.
>>
>