[julia-users] Re: push! performance

[Dan]

AFAIK the implementation of `push!` in Julia only occasionally allocates a 
bigger buffer for the new vector. This results in a very low average time 
for adding multiple elements (O(1) amortized time in complexity jargon).

Notice, `a = [a new_element ]` would in Julia do something else than 
`push!` - it would create a new array (with allocation), put the old `a` 
and `new_element` in it, and then rebind the symbol `a` to the new array, 
and let the garbage collector free the old array. This would be costly, so 
it's better to use push!.

Even better, is to do the experiment and possibly report the answers on the 
list with working code, timings and versions. Hope this helps.

On Sunday, November 15, 2015 at 6:51:30 AM UTC+2, Achu wrote:
>
> So in Matlab there's a pretty big difference in performance between 
> preallocating an array, and growing it in a loop with a=[a new_element];
> How big is the performance difference in Julia between preallocating and 
> using push! to grow the array in a loop?
>
> Thanks!
> Achu
>

[julia-users] ANN: MXNet.jl v0.0.5, LSTM char-rnn tutorial for generating random sentences

[Chiyuan Zhang]

Please check out this new tutorial 
 on 
training a char-level LSTM recurrent neural network (RNN) to generate 
random sentences from arbitrary domain.

MXNet.jl  is a flexible and efficient 
deep learning library in Julia. Here are the main changes since the last 
announced version:

v0.0.5 (2015.11.14)
   
   - char-lstm example.
   - Network visualization via GraphViz.
   - NN-factory for common models.
   - Convenient @nd_as_jl macro to work with NDArray as Julia Arrays.
   - Refactoring: Symbol -> SymbolicNode.
   - More evaluation metrics (@vchuravy, @Andy-P)

v0.0.4 
(2015.11.09)
   
   - ADAM optimizer (@cbecker)
   - Improved data provider API.
   - More documentation.
   - Fix a bug in array data iterator (@vchuravy)

enjoy!
-- pluskid

[julia-users] Julia #1 on Hacker News

[Mauro]

The Moore Foundation's sponsorship of Julia is currently #1 on Hacker
News:
https://news.ycombinator.com/item?id=10565940

Thanks to Mike for posting!

[julia-users] Progress meter for parallel workers

[Jason Eckstein]

I was playing around with the ProgressMeter.jl package by Tim Holy and it 
works great for a single threaded loop.  I was wondering if there's any way 
to run several loops on multiple workers and have a set of progress bars 
update in real time.  When I tried this with Tim's package it just said 
Progress: 100% Time:  after each was complete but nothing during 
intermediate progress.  What I had in mind was a display like
Worker 1:  Progress: X%...
Worker 2: Progress: Y% 
that update in real time but have multiple lines to represent the different 
workers.

Re: [julia-users] Re: indexing with non Integer Reals is deprecated

[Sheehan Olver]

I didn't realize ÷ was div!  I think people would be happier if they knew 
they could use A[n÷2] instead of A[div(n,2)].

On Sunday, November 15, 2015 at 12:48:54 PM UTC+11, Steven G. Johnson wrote:
>
> See also
>
> https://github.com/JuliaLang/julia/issues/10154
>
> where a lot of this was discussed.
>
> Non-integer indices often indicate a mistake on the part of the programmer 
> (e.g. using / rather than ÷, i.e. div), and it is more reliable to force 
> the programmer to be explicit about what was intended (both to indicate 
> whether they want round/floor/ceil, and to correct unintentional type 
> instabilities that often accrue from accidental floating-point indices).   
> It is also a headache for people implementing AbstractArray subtypes, 
> because it creates an expectation that they will implement non-obvious 
> getindex methods for Real types, with calls to the to_index method as 
> needed.
>
> I feel like 99% of the problems that people have with deprecating 
> floating-point indices arise because they used / rather than ÷ for an 
> integer division.   I wonder if there is some way to give a helpful warning 
> in such cases?
>

[julia-users] Re: Meshgrid

[digxx]

Hey Thanks for ur input...I have one question regarding plotting:
Initially Plots (I just installed it) tried to call PyPlots and spit out 
this error:

while loading C:\Users\Diger\Documents\Julia\timecompare.jl, in expression 
starting on line 27 in include_from_node1 at loading.jl:304 in include at 
boot.jl:261 in plot at 
C:\cygwin64\home\Diger\.julia\v0.4\Plots\src\plot.jl:46 in backend at 
C:\cygwin64\home\Diger\.julia\v0.4\Plots\src\plotter.jl:167 in pyimport at 
C:\cygwin64\home\Diger\.julia\v0.4\PyCall\src\PyCall.jl:79 [inlined code] 
from C:\cygwin64\home\Diger\.julia\v0.4\PyCall\src\exception.jl:81from 
genericpath import _unicode  File 
"C:\cygwin64\home\Diger\.julia\v0.4\Conda\deps\usr\lib\posixpath.py", line 
19, in import posixpath  File 
"C:\cygwin64\home\Diger\.julia\v0.4\Conda\deps\usr\lib\urllib2.py", line 
97, in from urllib2 import urlopen  File 
"C:\cygwin64\home\Diger\.julia\v0.4\Conda\deps\usr\lib\site-packages\matplotlib\__init__.py",
 
line 169, in ImportError('cannot import name _unicode',)ERROR: 
LoadError: PyError (:PyImport_ImportModule) WARNING: Couldn't import PyPlot.  Install it with: 
Pkg.add("PyPlot").[Plots.jl] Initializing backend: pyplot

After removing the Pkg PyPlots, Plots initialized the backend Gadfly with 
which everything worked fine...
Do you know why this error occurs with PyPlot?

Re: [julia-users] Re: The growth of Julia userbase

[Ben Ward]

Thanks, for the responses everyone, it was incredibly helpful! :)

On Thursday, November 12, 2015 at 9:21:33 AM UTC, Viral Shah wrote:
>
> Attching the github traffic data. Doesn't have history though, but the 
> numbers were bigger than what I expected when I first saw this.
>
>
> Some other interesting numbers:
>
> 1. ~10,000 pageviews on the website daily
>
> 2. 1.7M "users" and 8.6M pageviews over the lifetime of julialang.org - 
> as per google analytics
>
> 3. The mailing lists have ~4000 subscribers combined
>
> 4. Github page gets 74000 views every fortnight, ~10,000 uniques. ~3000 
> clones every fortnight, with ~700 uniques.
>
>
> Iain posted the packages data, and the github stars and watch data is 
> visible to everyone. 
>
>
> -viral
>
>
>
>
>
> 
> On Thursday, November 5, 2015 at 12:57:22 AM UTC+5:30, Patrick O'Leary 
> wrote:
>>
>> It's not in the repo, it's a GitHub feature. But it may only be visible 
>> if you have Collaborator or Owner status on the repository.
>>
>> On Wednesday, November 4, 2015 at 10:51:34 AM UTC-6, Ben Ward wrote:
>>>
>>> I don't think the /graphs folder is part of the julia repo any-more :(
>>>
>>> On Wed, Nov 4, 2015 at 4:48 PM, Tony Kelman >> > wrote:
>>>
 There are some interesting numbers at 
 https://github.com/JuliaLang/julia/graphs/traffic

 Elliot Saba also did some scraping of the AWS download logs for 
 binaries and shared the aggregate numbers (broken down by platform) 
 privately with a few people, it may be worth sharing those publicly.


 On Wednesday, November 4, 2015 at 8:26:19 AM UTC-8, Ben Ward wrote:
>
> Hi all,
>
> I was wondering are there any metrics or stats available that show how 
> the user-base of Julia has grown over the last few years, and what it's 
> size is now?
>
> Many Thanks,
> Ben W.
>

>>>

Re: [julia-users] display precision

[Yichao Yu]

On Sun, Nov 15, 2015 at 1:25 PM, digxx  wrote:
> Is it possible to display the precision of a number to arbitrary decimals
> points?
> For example if I want to display pi (or any other result) up to 1e6
> decimals?!

For arbitrary result, you can't print arbitrary precision if your
calculation does not have that precision (unless you just want to see
1e6 zeros printed). You want to perform the calculation on BigFloat[1]
if you need that precision (and then printing it out should give you
the expected number of digits).

In theory, there could be a way to print out pi to a high precision
without converting it to a BigFloat first although I also don't see
why that's a useful thing to do in general.

[1] 
http://julia.readthedocs.org/en/latest/manual/integers-and-floating-point-numbers/#arbitrary-precision-arithmetic

Re: [julia-users] using a JSON REST API from Julia

[Avik Sengupta]

Below is some recent code to access a REST API. This is possibly the 
simplest way to do this, using Dict's as inputs and outputs. 

https://github.com/aviks/Mandrill.jl/blob/2dbce6dc6b91a56a350b29a02d072b9b90416771/src/Mandrill.jl#L39

Regards
-
Avik

On Saturday, 14 November 2015 14:42:31 UTC, Jacob Quinn wrote:
>
> You got it. Requests.jl is the current standard, unless you felt more 
> comfortable calling a python library with PyCall. Requests.jl has gotten a 
> lot of love lately from Jon Malmaud and is 0.4 ready.
>
> -Jacob
>
>
> On Sat, Nov 14, 2015 at 7:10 AM, Christof Stocker  > wrote:
>
>> What is the current best practice to handle JSON and to use a REST API 
>> from Julia?
>>
>> From preliminary search I would assume JuliaLang/JSON.jl and 
>> JuliaWeb/Requests.jl, but I am unsure
>>
>> Doe anyone have any suggestions what to look at?
>>
>
>

Re: [julia-users] General question on indexing

[Tim Holy]

I don't think there's an obvious "policy" in place for this, but in general 
I'd agree that it's better foo[i] == collect(foo)[i].

But there's no reason you can't have different types of iterators that visit 
different types of edges:

for edge in all_edges(graph)
# visits symmetric edges twice
end

for edge in unique_edges(graph)
# visits only the ones with d Is it always expected that foo[i] == collect(foo)[i]? I'm running into a
> bit of an issue with undirected graphs where this may not be the case for
> collections of graph edges, and I'm wondering how much time I should sink
> into fixing a problem that might not even exist.
> 
> The discrepancy is with an edge (s,d) where s > d - in the iterator they're
> ignored in undirected graphs since we represent it as d,s in this case
> (smaller vertex first), but in the adjacency list that is used to build the
> getindex, they're both there. So we can have Edge(20,10) in edges(graph) be
> true, but Edge(20,10) in collect(edges(graph)) be false (Edge(10,20) will
> be in the collection).
> 
> Thanks for any input.

Re: [julia-users] Re: General question on indexing

[Tim Holy]

This may also be a case where writing an iterator that just visits the 
elements with i >= j might be what you want. You can study the implementation 
of CartesianIndex in Base for inspiration.

--Tim

On Saturday, November 14, 2015 02:18:45 PM hustf wrote:
> I believe this work-in-progress type may be adapted? I wanted to make this
> faster by using tuples or immutable fixed vectors to store the data or
> whatnot, but I guess I'm currently stuck. Help wanted.
> 
> Compared to looping unnecessarily over a full matrix, this speeds things up
> by around 35%.
> 
> immutable Maq7 <:AbstractMatrix{Int}
> data::Array{Int64,2}   # upper triangular except diagonal, stored as a
> horizontal vector
> width::Int64
> function Maq7(mat::Matrix{Int64})
> if issym(mat)
> n=size(mat,1)
> if n>1
> v=mat[2:n,1]
> for col = 2:(n-1)
> append!(v,mat[(col+1):n,col])
> end
> new(v', size(mat,1))
> else
> error("Ouch, not > (1x1)")
> end
> else
> error("Ouch, not square symmetric")
> end
> end
> end
> Base.size(A::Maq7)= A.width, A.width
> Base.length(A::Maq7)= div(A.width*(A.width-1), 2)
> Base.getindex(A::Maq7, i::Int) = Base.getindex(A.data, i)
> function Base.getindex(A::Maq7, r::Int , c::Int )
> if c > r
> return getindex(A.data, div( (r-1)*(2*A.width -r) , 2 ) + c - r )
> elseif r > c
> return getindex(A.data, div( (c-1)*(2*A.width -c) , 2 ) + r - c )
> else
> return zero(eltype(A.data))
> end
> end
> Base.linearindexing(::Type{Maq7}) = Base.LinearFast()
> 
> 
> 
> Example:
> julia> ms
> 4x4 Array{Int64,2}:
>  7  1  2   3
>  1  8  4   5
>  2  4  9   6
>  3  5  6  10
> 
> 
> julia> m=Maq7(ms)
> 4x4 Maq7:
>  0  1  2  3
>  1  0  4  5
>  2  4  0  6
>  3  5  6  0
> 
> julia> collect(m)
> 6-element Array{Int64,1}:
>  1
>  2
>  3
>  4
>  5
>  6
> 
> 
> julia> length(m)
> 6

Re: [julia-users] display precision

[David P. Sanders]

El domingo, 15 de noviembre de 2015, 12:45:11 (UTC-6), Yichao Yu escribió:
>
> On Sun, Nov 15, 2015 at 1:25 PM, digxx  
> wrote: 
> > Is it possible to display the precision of a number to arbitrary 
> decimals 
> > points? 
> > For example if I want to display pi (or any other result) up to 1e6 
> > decimals?! 
>
>
Note that the BigFloat precision is given in binary digits, not decimal 
digits,
so you have to multiply the 10^6 that you want by log2(10) = 3.32...:

num_binary_digits = ceil(Int, 10^6 * log2(10))
set_bigfloat_precision(num_binary_digits)

@time s = string(big(pi))
length(s)

The results are 103 digits and

0.184317 seconds


Of course, actually *displaying* the string takes a lot longer (output is slow).

[julia-users] Re: push! performance

[Steven G. Johnson]

function prealloc(n)
a = Array(Int, n)
for i = 1:n
a[i] = i
end
return a
end
function dynamic(n)
a = Int[]
for i = 1:n
push!(a, i)
end
return a
end
@time prealloc(10^7);
@time dynamic(10^7);


On my machine, the preallocated version is 2.5–3x faster.  A significant 
but not overwhelming margin.

Re: [julia-users] Re: General question on indexing

[Seth]

Thank you, Tim - I'll definitely check it out. Right now I can't seem to 
find an efficient way of doing it, but there are a few tricks in 
Iterators.jl that might work.

On Sunday, November 15, 2015 at 1:33:29 PM UTC-8, Tim Holy wrote:
>
> This may also be a case where writing an iterator that just visits the 
> elements with i >= j might be what you want. You can study the 
> implementation 
> of CartesianIndex in Base for inspiration. 
>
> --Tim 
>
> On Saturday, November 14, 2015 02:18:45 PM hustf wrote: 
> > I believe this work-in-progress type may be adapted? I wanted to make 
> this 
> > faster by using tuples or immutable fixed vectors to store the data or 
> > whatnot, but I guess I'm currently stuck. Help wanted. 
> > 
> > Compared to looping unnecessarily over a full matrix, this speeds things 
> up 
> > by around 35%. 
> > 
> > immutable Maq7 <:AbstractMatrix{Int} 
> > data::Array{Int64,2}   # upper triangular except diagonal, stored as a 
> > horizontal vector 
> > width::Int64 
> > function Maq7(mat::Matrix{Int64}) 
> > if issym(mat) 
> > n=size(mat,1) 
> > if n>1 
> > v=mat[2:n,1] 
> > for col = 2:(n-1) 
> > append!(v,mat[(col+1):n,col]) 
> > end 
> > new(v', size(mat,1)) 
> > else 
> > error("Ouch, not > (1x1)") 
> > end 
> > else 
> > error("Ouch, not square symmetric") 
> > end 
> > end 
> > end 
> > Base.size(A::Maq7)= A.width, A.width 
> > Base.length(A::Maq7)= div(A.width*(A.width-1), 2) 
> > Base.getindex(A::Maq7, i::Int) = Base.getindex(A.data, i) 
> > function Base.getindex(A::Maq7, r::Int , c::Int ) 
> > if c > r 
> > return getindex(A.data, div( (r-1)*(2*A.width -r) , 2 ) + c - r ) 
> > elseif r > c 
> > return getindex(A.data, div( (c-1)*(2*A.width -c) , 2 ) + r - c ) 
> > else 
> > return zero(eltype(A.data)) 
> > end 
> > end 
> > Base.linearindexing(::Type{Maq7}) = Base.LinearFast() 
> > 
> > 
> > 
> > Example: 
> > julia> ms 
> > 4x4 Array{Int64,2}: 
> >  7  1  2   3 
> >  1  8  4   5 
> >  2  4  9   6 
> >  3  5  6  10 
> > 
> > 
> > julia> m=Maq7(ms) 
> > 4x4 Maq7: 
> >  0  1  2  3 
> >  1  0  4  5 
> >  2  4  0  6 
> >  3  5  6  0 
> > 
> > julia> collect(m) 
> > 6-element Array{Int64,1}: 
> >  1 
> >  2 
> >  3 
> >  4 
> >  5 
> >  6 
> > 
> > 
> > julia> length(m) 
> > 6 
>
>

[julia-users] Re: push! performance

[Seth]

What happens if you use sizehint!() with dynamic()?

On Sunday, November 15, 2015 at 3:35:45 PM UTC-8, Steven G. Johnson wrote:
>
> function prealloc(n)
> a = Array(Int, n)
> for i = 1:n
> a[i] = i
> end
> return a
> end
> function dynamic(n)
> a = Int[]
> for i = 1:n
> push!(a, i)
> end
> return a
> end
> @time prealloc(10^7);
> @time dynamic(10^7);
>
>
> On my machine, the preallocated version is 2.5–3x faster.  A significant 
> but not overwhelming margin.
>

[julia-users] Creating a stable version of Julia + Packages for a semester long course?

[Sheehan Olver]

I'm trying to figure out the "best" way to create a stable version of Julia 
+ Gadfly + PyPlot + IJulia (+ other packages?) for a semester long course. 
 I don't want to have the students run Pkg.add(...)/Pkg.update(), as 
packages have a tendency to occasionally break on updates, and it's a 
headache dealing with this during the lecture.

Two possible solutions I can think of of are:

1)  Prebake a .julia folder that contains all the necessary resources, with 
a script to reset in case the students break it with Pkg.update().
2)  Use system image

http://docs.julialang.org/en/release-0.4/devdocs/sysimg/

that includes all the necessary packages.   It's not really clear how to do 
this from the documentation, though.   I'm also not sure how that would 
interact with Pkg.update() though, so probably instructions to delete 
.julia would also need to be given.


Any other options I'm missing?  If 2 is recommended, any tutorial how to do 
this?

[julia-users] Re: push! performance

[David P. Sanders]

El domingo, 15 de noviembre de 2015, 18:00:09 (UTC-6), David P. Sanders 
escribió:
>
>
>
> El domingo, 15 de noviembre de 2015, 17:35:45 (UTC-6), Steven G. Johnson 
> escribió:
>>
>> function prealloc(n)
>> a = Array(Int, n)
>> for i = 1:n
>> a[i] = i
>> end
>> return a
>> end
>> function dynamic(n)
>> a = Int[]
>> for i = 1:n
>> push!(a, i)
>> end
>> return a
>> end
>> @time prealloc(10^7);
>> @time dynamic(10^7);
>>
>>
>> On my machine, the preallocated version is 2.5–3x faster.  A significant 
>> but not overwhelming margin.
>>
>
> However, if you are doing actual calculations inside the loop 
> (rather than just storing the values), then in my experience the 
> performance difference is much smaller.
>

For example,

function prealloc2(n)
a = Array(Float64, n)
   for i = 1:n
a[i] = sqrt(i)
   end
   return a
   end

function dynamic2(n)
a = Float64[]
   for i = 1:n
push!(a, sqrt(i))
   end
   return a
   end

prealloc2(1)
dynamic2(1)

N = 10^8
@time prealloc2(N);
@time dynamic2(N); 

0.973452 seconds (6 allocations: 762.940 MB, 15.12% gc time)
1.042340 seconds (33 allocations: 912.361 MB, 8.61% gc time)


>  
>

Re: [julia-users] Re: General question on indexing

[Tim Holy]

Haven't tested timing carefully, but something like the following should be 
pretty decent:

julia> immutable LowerTriangularIterator{A<:AbstractMatrix}
   data::A
   end

julia> Base.start(iter::LowerTriangularIterator) = (1,1)
start (generic function with 49 methods)

julia> Base.done(iter::LowerTriangularIterator, state) = state[2] > 
size(iter.data, 2)
done (generic function with 51 methods)

julia> @inline function Base.next(iter::LowerTriangularIterator, state)
   item = iter.data[state[1], state[2]]
   newstate = ifelse(state[1] == size(iter.data, 1), (state[2]+1, 
state[2]+1), (state[1]+1, state[2]))
   item, newstate
   end


To make it vectorize (i.e., SIMD), you might also need definitions like these:
https://github.com/JuliaLang/julia/blob/aa24ee0d0f334e8d9ecbced64bd975a6cc5f240d/base/multidimensional.jl#L134-L150

but perhaps vectorization isn't applicable to your problem anyway.

--Tim

On Sunday, November 15, 2015 05:06:23 PM Seth wrote:
> Thank you, Tim - I'll definitely check it out. Right now I can't seem to
> find an efficient way of doing it, but there are a few tricks in
> Iterators.jl that might work.
> 
> On Sunday, November 15, 2015 at 1:33:29 PM UTC-8, Tim Holy wrote:
> > This may also be a case where writing an iterator that just visits the
> > elements with i >= j might be what you want. You can study the
> > implementation
> > of CartesianIndex in Base for inspiration.
> > 
> > --Tim
> > 
> > On Saturday, November 14, 2015 02:18:45 PM hustf wrote:
> > > I believe this work-in-progress type may be adapted? I wanted to make
> > 
> > this
> > 
> > > faster by using tuples or immutable fixed vectors to store the data or
> > > whatnot, but I guess I'm currently stuck. Help wanted.
> > > 
> > > Compared to looping unnecessarily over a full matrix, this speeds things
> > 
> > up
> > 
> > > by around 35%.
> > > 
> > > immutable Maq7 <:AbstractMatrix{Int}
> > > data::Array{Int64,2}   # upper triangular except diagonal, stored as a
> > > horizontal vector
> > > width::Int64
> > > function Maq7(mat::Matrix{Int64})
> > > if issym(mat)
> > > n=size(mat,1)
> > > if n>1
> > > v=mat[2:n,1]
> > > for col = 2:(n-1)
> > > append!(v,mat[(col+1):n,col])
> > > end
> > > new(v', size(mat,1))
> > > else
> > > error("Ouch, not > (1x1)")
> > > end
> > > else
> > > error("Ouch, not square symmetric")
> > > end
> > > end
> > > end
> > > Base.size(A::Maq7)= A.width, A.width
> > > Base.length(A::Maq7)= div(A.width*(A.width-1), 2)
> > > Base.getindex(A::Maq7, i::Int) = Base.getindex(A.data, i)
> > > function Base.getindex(A::Maq7, r::Int , c::Int )
> > > if c > r
> > > return getindex(A.data, div( (r-1)*(2*A.width -r) , 2 ) + c - r )
> > > elseif r > c
> > > return getindex(A.data, div( (c-1)*(2*A.width -c) , 2 ) + r - c )
> > > else
> > > return zero(eltype(A.data))
> > > end
> > > end
> > > Base.linearindexing(::Type{Maq7}) = Base.LinearFast()
> > > 
> > > 
> > > 
> > > Example:
> > > julia> ms
> > > 
> > > 4x4 Array{Int64,2}:
> > >  7  1  2   3
> > >  1  8  4   5
> > >  2  4  9   6
> > >  3  5  6  10
> > > 
> > > julia> m=Maq7(ms)
> > > 
> > > 4x4 Maq7:
> > >  0  1  2  3
> > >  1  0  4  5
> > >  2  4  0  6
> > >  3  5  6  0
> > > 
> > > julia> collect(m)
> > > 
> > > 6-element Array{Int64,1}:
> > >  1
> > >  2
> > >  3
> > >  4
> > >  5
> > >  6
> > > 
> > > julia> length(m)
> > > 6

Re: [julia-users] Re: General question on indexing

[Seth]

That's very cool and close to what I need. Thanks. I'm using adjacency 
lists (vectors of vectors) but I'm sure I can adapt this code.

On Sunday, November 15, 2015 at 6:12:19 PM UTC-8, Tim Holy wrote:
>
> Haven't tested timing carefully, but something like the following should 
> be 
> pretty decent: 
>
> julia> immutable LowerTriangularIterator{A<:AbstractMatrix} 
>data::A 
>end 
>
> julia> Base.start(iter::LowerTriangularIterator) = (1,1) 
> start (generic function with 49 methods) 
>
> julia> Base.done(iter::LowerTriangularIterator, state) = state[2] > 
> size(iter.data, 2) 
> done (generic function with 51 methods) 
>
> julia> @inline function Base.next(iter::LowerTriangularIterator, state) 
>item = iter.data[state[1], state[2]] 
>newstate = ifelse(state[1] == size(iter.data, 1), (state[2]+1, 
> state[2]+1), (state[1]+1, state[2])) 
>item, newstate 
>end 
>
>
> To make it vectorize (i.e., SIMD), you might also need definitions like 
> these: 
>
> https://github.com/JuliaLang/julia/blob/aa24ee0d0f334e8d9ecbced64bd975a6cc5f240d/base/multidimensional.jl#L134-L150
>  
>
> but perhaps vectorization isn't applicable to your problem anyway. 
>
> --Tim 
>
> On Sunday, November 15, 2015 05:06:23 PM Seth wrote: 
> > Thank you, Tim - I'll definitely check it out. Right now I can't seem to 
> > find an efficient way of doing it, but there are a few tricks in 
> > Iterators.jl that might work. 
> > 
> > On Sunday, November 15, 2015 at 1:33:29 PM UTC-8, Tim Holy wrote: 
> > > This may also be a case where writing an iterator that just visits the 
> > > elements with i >= j might be what you want. You can study the 
> > > implementation 
> > > of CartesianIndex in Base for inspiration. 
> > > 
> > > --Tim 
> > > 
> > > On Saturday, November 14, 2015 02:18:45 PM hustf wrote: 
> > > > I believe this work-in-progress type may be adapted? I wanted to 
> make 
> > > 
> > > this 
> > > 
> > > > faster by using tuples or immutable fixed vectors to store the data 
> or 
> > > > whatnot, but I guess I'm currently stuck. Help wanted. 
> > > > 
> > > > Compared to looping unnecessarily over a full matrix, this speeds 
> things 
> > > 
> > > up 
> > > 
> > > > by around 35%. 
> > > > 
> > > > immutable Maq7 <:AbstractMatrix{Int} 
> > > > data::Array{Int64,2}   # upper triangular except diagonal, stored as 
> a 
> > > > horizontal vector 
> > > > width::Int64 
> > > > function Maq7(mat::Matrix{Int64}) 
> > > > if issym(mat) 
> > > > n=size(mat,1) 
> > > > if n>1 
> > > > v=mat[2:n,1] 
> > > > for col = 2:(n-1) 
> > > > append!(v,mat[(col+1):n,col]) 
> > > > end 
> > > > new(v', size(mat,1)) 
> > > > else 
> > > > error("Ouch, not > (1x1)") 
> > > > end 
> > > > else 
> > > > error("Ouch, not square symmetric") 
> > > > end 
> > > > end 
> > > > end 
> > > > Base.size(A::Maq7)= A.width, A.width 
> > > > Base.length(A::Maq7)= div(A.width*(A.width-1), 2) 
> > > > Base.getindex(A::Maq7, i::Int) = Base.getindex(A.data, i) 
> > > > function Base.getindex(A::Maq7, r::Int , c::Int ) 
> > > > if c > r 
> > > > return getindex(A.data, div( (r-1)*(2*A.width -r) , 2 ) + c - r ) 
> > > > elseif r > c 
> > > > return getindex(A.data, div( (c-1)*(2*A.width -c) , 2 ) + r - c ) 
> > > > else 
> > > > return zero(eltype(A.data)) 
> > > > end 
> > > > end 
> > > > Base.linearindexing(::Type{Maq7}) = Base.LinearFast() 
> > > > 
> > > > 
> > > > 
> > > > Example: 
> > > > julia> ms 
> > > > 
> > > > 4x4 Array{Int64,2}: 
> > > >  7  1  2   3 
> > > >  1  8  4   5 
> > > >  2  4  9   6 
> > > >  3  5  6  10 
> > > > 
> > > > julia> m=Maq7(ms) 
> > > > 
> > > > 4x4 Maq7: 
> > > >  0  1  2  3 
> > > >  1  0  4  5 
> > > >  2  4  0  6 
> > > >  3  5  6  0 
> > > > 
> > > > julia> collect(m) 
> > > > 
> > > > 6-element Array{Int64,1}: 
> > > >  1 
> > > >  2 
> > > >  3 
> > > >  4 
> > > >  5 
> > > >  6 
> > > > 
> > > > julia> length(m) 
> > > > 6 
>
>

[julia-users] Re: push! performance

[Alex Ames]

Looks like push! reduces memory allocation by a constant-ish factor of 
10^3, but reduces execution time exponentially. push!ing 1 elements 
takes the same time as copying 10. (gist 
)







On Sunday, November 15, 2015 at 3:42:18 AM UTC-6, Dan wrote:
>
> AFAIK the implementation of `push!` in Julia only occasionally allocates a 
> bigger buffer for the new vector. This results in a very low average time 
> for adding multiple elements (O(1) amortized time in complexity jargon).
>
> Notice, `a = [a new_element ]` would in Julia do something else than 
> `push!` - it would create a new array (with allocation), put the old `a` 
> and `new_element` in it, and then rebind the symbol `a` to the new array, 
> and let the garbage collector free the old array. This would be costly, so 
> it's better to use push!.
>
> Even better, is to do the experiment and possibly report the answers on 
> the list with working code, timings and versions. Hope this helps.
>
> On Sunday, November 15, 2015 at 6:51:30 AM UTC+2, Achu wrote:
>>
>> So in Matlab there's a pretty big difference in performance between 
>> preallocating an array, and growing it in a loop with a=[a new_element];
>> How big is the performance difference in Julia between preallocating and 
>> using push! to grow the array in a loop?
>>
>> Thanks!
>> Achu
>>
>

[julia-users] Re: push! performance

[David P. Sanders]

El domingo, 15 de noviembre de 2015, 17:35:45 (UTC-6), Steven G. Johnson 
escribió:
>
> function prealloc(n)
> a = Array(Int, n)
> for i = 1:n
> a[i] = i
> end
> return a
> end
> function dynamic(n)
> a = Int[]
> for i = 1:n
> push!(a, i)
> end
> return a
> end
> @time prealloc(10^7);
> @time dynamic(10^7);
>
>
> On my machine, the preallocated version is 2.5–3x faster.  A significant 
> but not overwhelming margin.
>

However, if you are doing actual calculations inside the loop 
(rather than just storing the values), then in my experience the 
performance difference is much smaller.

 

[julia-users] Re: General question on indexing

[Seth]

Hustf, I don't know why my reply didn't show up here yesterday, but I 
wanted to thank you for the code. I'm looking at how to make that work. 
Thanks for your response.

On Saturday, November 14, 2015 at 2:18:45 PM UTC-8, hustf wrote:
>
> I believe this work-in-progress type may be adapted? I wanted to make this 
> faster by using tuples or immutable fixed vectors to store the data or 
> whatnot, but I guess I'm currently stuck. Help wanted.
>
> Compared to looping unnecessarily over a full matrix, this speeds things 
> up by around 35%.
>
> immutable Maq7 <:AbstractMatrix{Int}
> data::Array{Int64,2}   # upper triangular except diagonal, stored as a 
> horizontal vector
> width::Int64
> function Maq7(mat::Matrix{Int64}) 
> if issym(mat)
> n=size(mat,1)
> if n>1
> v=mat[2:n,1]
> for col = 2:(n-1)
> append!(v,mat[(col+1):n,col])
> end
> new(v', size(mat,1))
> else
> error("Ouch, not > (1x1)")
> end
> else 
> error("Ouch, not square symmetric")
> end
> end
> end
> Base.size(A::Maq7)= A.width, A.width 
> Base.length(A::Maq7)= div(A.width*(A.width-1), 2)
> Base.getindex(A::Maq7, i::Int) = Base.getindex(A.data, i) 
> function Base.getindex(A::Maq7, r::Int , c::Int )
> if c > r 
> return getindex(A.data, div( (r-1)*(2*A.width -r) , 2 ) + c - r )
> elseif r > c
> return getindex(A.data, div( (c-1)*(2*A.width -c) , 2 ) + r - c )
> else
> return zero(eltype(A.data))
> end
> end
> Base.linearindexing(::Type{Maq7}) = Base.LinearFast()
>
>
>
> Example:
> julia> ms
> 4x4 Array{Int64,2}:
>  7  1  2   3
>  1  8  4   5
>  2  4  9   6
>  3  5  6  10
>
>
> julia> m=Maq7(ms)
> 4x4 Maq7:
>  0  1  2  3
>  1  0  4  5
>  2  4  0  6
>  3  5  6  0
>
> julia> collect(m)
> 6-element Array{Int64,1}:
>  1
>  2
>  3
>  4
>  5
>  6
>
>
> julia> length(m)
> 6
>
>
>
>
>

[julia-users] Re: Creating a stable version of Julia + Packages for a semester long course?

[David P. Sanders]

El domingo, 15 de noviembre de 2015, 21:05:45 (UTC-6), Sheehan Olver 
escribió:
>
>
> I'm trying to figure out the "best" way to create a stable version of 
> Julia + Gadfly + PyPlot + IJulia (+ other packages?) for a semester long 
> course.  I don't want to have the students run Pkg.add(...)/Pkg.update(), 
> as packages have a tendency to occasionally break on updates, and it's a 
> headache dealing with this during the lecture.
>
> Two possible solutions I can think of of are:
>
> 1)  Prebake a .julia folder that contains all the necessary resources, 
> with a script to reset in case the students break it with Pkg.update().
> 2)  Use system image
>
> http://docs.julialang.org/en/release-0.4/devdocs/sysimg/
>
> that includes all the necessary packages.   It's not really clear how to 
> do this from the documentation, though.   I'm also not sure how that would 
> interact with Pkg.update() though, so probably instructions to delete 
> .julia would also need to be given.
>
>
> Any other options I'm missing?  If 2 is recommended, any tutorial how to 
> do this?
>

Another option is 

https://github.com/rened/DeclarativePackages.jl 

which allows you to run Julia with a given set of versions of given 
packages.

Possibly another option would be a Docker container image.

[julia-users] Re: Creating a stable version of Julia + Packages for a semester long course?

[Eric Forgy]

I have a similar situation.

I am building up a team of modelers and would like to make sure we're all 
using the same versions of everything (Julia + packages). 
DeclarativePackages.jl looks promising/interesting and I'm curious what 
other things people have tried.

On Monday, November 16, 2015 at 12:21:31 PM UTC+8, David P. Sanders wrote:
>
>
>
> El domingo, 15 de noviembre de 2015, 21:05:45 (UTC-6), Sheehan Olver 
> escribió:
>>
>>
>> I'm trying to figure out the "best" way to create a stable version of 
>> Julia + Gadfly + PyPlot + IJulia (+ other packages?) for a semester long 
>> course.  I don't want to have the students run Pkg.add(...)/Pkg.update(), 
>> as packages have a tendency to occasionally break on updates, and it's a 
>> headache dealing with this during the lecture.
>>
>> Two possible solutions I can think of of are:
>>
>> 1)  Prebake a .julia folder that contains all the necessary resources, 
>> with a script to reset in case the students break it with Pkg.update().
>> 2)  Use system image
>>
>> http://docs.julialang.org/en/release-0.4/devdocs/sysimg/
>>
>> that includes all the necessary packages.   It's not really clear how to 
>> do this from the documentation, though.   I'm also not sure how that would 
>> interact with Pkg.update() though, so probably instructions to delete 
>> .julia would also need to be given.
>>
>>
>> Any other options I'm missing?  If 2 is recommended, any tutorial how to 
>> do this?
>>
>
> Another option is 
>
> https://github.com/rened/DeclarativePackages.jl 
>
> which allows you to run Julia with a given set of versions of given 
> packages.
>
> Possibly another option would be a Docker container image.
>

[julia-users] Re: run parallel code in two machine with julia

[mahdi jamshidian]

thank you very much

On Sunday, November 15, 2015 at 1:33:13 AM UTC+3:30, André Lage wrote:
>
> Hi Mahdi,
>
> Just configure these machines with SSH passwordless login and run your 
> code.
>
> More info at: 
> http://docs.julialang.org/en/release-0.4/manual/parallel-computing/
>
> A tutorial on Distributed Computing with Julia: 
> https://github.com/proflage/2015-julia-hands-on
>
>
> André Lage.
>
> On Friday, November 13, 2015 at 7:00:53 PM UTC-3, mahdi jamshidian wrote:
>>
>> i am new user in julia .
>> i have a project for my university class and i must run a parallel code 
>> in two machine.my two machine os is ubuntu and i want  run julia parallel 
>> code in this two machine i must cluster these two machine  or connect them 
>> via network is enough . and any one have a tutorial for run parallel code 
>> in multiple machine with julia.
>> thank 's a lot
>>
>

[julia-users] Re: Write an array of tuple to csv

[Grey Marsh]

Thanks for the help.

On Saturday, November 14, 2015 at 9:28:05 PM UTC+5:30, Steven G. Johnson 
wrote:
>
> I've pushed a fix: https://github.com/JuliaLang/julia/pull/13989
>

[julia-users] display precision

[digxx]

Is it possible to display the precision of a number to arbitrary decimals 
points?
For example if I want to display pi (or any other result) up to 1e6 
decimals?!