Re: [julia-users] Re: Adding items into a tuple

[Alexei Serdiuk]

Gred, thank you.
This is exactly want I looked for.

среда, 31 августа 2016 г., 4:26:02 UTC+3 пользователь Greg Plowman написал:
>
>
>> I could use an array, but one product can correspond to different number 
>> of bases.
>>
>> That's why I decided to use tuples.
>>
>> You could use an Array of (different length) Arrays, similar to Array of 
> Tuples.
>
>
> Another strategy might be to construct a vector of (Product, Base) pairs, 
> and then iterate over this vector:
>
> product_bases = Tuple{ProductType, BaseType}[]
>
> for k in Products, b in Bases
> if accordance[k,b]
> push!(product_bases, (k,b))
> end
> end
>
> for (k,b) in product_bases
> ...
> end
>
>
>

[julia-users] Re: @threads all vs. @parallel ???

[Chris Rackauckas]

That's pretty much it. Threads are shared memory, which have less overhead 
(and are thus faster), and can share variables between them. @parallel is 
multiprocessing, i.e. each worker process has its own set of defined 
variables which do not overlap, and data has to be transferred between 
them. @parallel has the advantage that it does not have to be local: 
different processes can be on completely different computers/nodes. But it 
has a higher startup cost and is thus better suited to larger 
parallelizable problems.

However, as Yichao states, @threads is still experimental. For one, since 
the memory is shared, you have have to make sure everything is 
"thread-safe" in order to be correct and not fail (example: two threads 
can't write to the same spot at once or else it can be non-deterministic as 
to what the result is). But also, the internals still have some rough 
edges. Just check out the repo for bug reports and you'll see that things 
can still go wrong, or that your performance can even decrease due to 
type-inference bugs.  Thus 
it is something to play around with, but it definitely isn't something that 
you should put into production yet (though in many cases it is already 
looking pretty good!).

On Tuesday, August 30, 2016 at 5:46:57 PM UTC-7, Andrew wrote:
>
> I have also been wondering this. I tried @threads yesterday and it got me 
> around a 4-fold speedup on a loop which applied a function to each element 
> in an array, and I conveniently didn't need to bother using SharedArrays as 
> I would with @parallel.
>
> On Tuesday, August 30, 2016 at 7:20:36 PM UTC-4, digxx wrote:
>>
>> Sorry if there is already some information on this though I didnt find 
>> it...
>> So: What is the difference between these?
>> I have used @parallel so far for parallel loops but recently saw this 
>> @threads all in some video and I was wondering what the difference is?
>> Could anyone elaborate or give me a link with some info?
>> Thanks digxx
>>
>

[julia-users] Re: How to publish a package

[Chris Rackauckas]

Some of this might be a little old now, but it'll guide you step-by-step on 
how to take your functions into a registered module with unit 
tests: 
http://www.stochasticlifestyle.com/finalizing-julia-package-documentation-testing-coverage-publishing/
 
. Hope it helps!

On Tuesday, August 30, 2016 at 10:11:41 PM UTC-7, M Hashmi wrote:
>
> Hi,
> I need to know that how to publish a package for Julia.
> Regards,
> Mudassar
>

[julia-users] How to publish a package

[M Hashmi]

Hi,
I need to know that how to publish a package for Julia.
Regards,
Mudassar

[julia-users] SharedArray fails to gc() when called within a sequence of functions? 0.5.0-rc3

[Rafael Menegassi]

Dear all
Quite new in julia so sorry if made something wrong;
Reduced the case to simplest possible;

Using SharedArray within a sequence of functions:

addprocs(4)

function chisq(n::Integer)
A=SharedArray(Float64, n)
@sync @parallel for i in 1:n
A[i]=(rand()-rand())^2
end
sumsq=sum(A)
end

function calculate(n::Integer)
b=0.0
for j in 1:n
b+=chisq(n)
end
return b
end

chisq(500^2)  #ok no failure

calculate(500) # fails


Calculating the same number of evaluations (500 x 500)  it does not fail 
while it crashes before the same function is called 500 times

And the failure is:

> *ERROR: SystemError: shm_open() failed for /jl005889eze42OrPYHS9RKjHZihQ: 
> Too many open files*
>
> * in uv_error at ./libuv.jl:68 [inlined]*
>
> * in _link_pipe(::Ptr{Void}, ::Ptr{Void}) at ./stream.jl:596*
>
> * in link_pipe(::Base.PipeEndpoint, ::Bool, ::Base.PipeEndpoint, ::Bool) 
> at ./stream.jl:652*
>
> * in setup_stdio(::Pipe, ::Bool) at ./process.jl:419*
>
> * in setup_stdio(::Base.##412#413{Cmd,Ptr{Void},Base.Process}, 
> ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./process.jl:464*
>
> * in #spawn#411(::Nullable{Base.ProcessChain}, ::Function, ::Cmd, 
> ::Tuple{Base.DevNullStream,Pipe,Base.TTY}, ::Bool, ::Bool) at 
> ./process.jl:477*
>
> * in (::Base.#kw##spawn)(::Array{Any,1}, ::Base.#spawn, ::Cmd, 
> ::Tuple{Base.DevNullStream,Pipe,Base.TTY}, ::Bool, ::Bool) at ./:0*
>
> * in open(::Cmd, ::String, ::Base.DevNullStream) at ./process.jl:539*
>
> * in read(::Cmd, ::Base.DevNullStream) at ./process.jl:574*
>
> * in readstring at ./process.jl:581 [inlined] (repeats 2 times)*
>
> * in print_shmem_limits(::Int64) at ./sharedarray.jl:488*
>
> * in shm_mmap_array(::Type{T}, ::Tuple{Int64}, ::String, ::UInt16) at 
> ./sharedarray.jl:515*
>
> * in #SharedArray#786(::Bool, ::Array{Int64,1}, ::Type{T}, 
> ::Type{Float64}, ::Tuple{Int64}) at ./sharedarray.jl:70*
>
> * in SharedArray{T,N}(::Type{Float64}, ::Tuple{Int64}) at 
> ./sharedarray.jl:57*
>
> * in #SharedArray#793(::Array{Any,1}, ::Type{T}, ::Type{T}, ::Int64, 
> ::Vararg{Int64,N}) at ./sharedarray.jl:113*
>
> * in chisq(::Int64) at ./REPL[2]:2*
>
> * in calculate(::Int64) at ./REPL[3]:4*
>

It also happens at  0.4.6, albeit a little different error:

> *ERROR: On worker 3:*
>
> *SystemError: shm_open() failed for /jl006428a6fpOftDBFr087xQnY6F: Too 
> many open files*
>
> * in remotecall_fetch at multi.jl:747*
>
> * in remotecall_fetch at multi.jl:750*
>
> * in call_on_owner at multi.jl:793*
>
> * in wait at multi.jl:808*
>
> * in __SharedArray#138__ at sharedarray.jl:74*
>
> * in SharedArray at sharedarray.jl:117*
>
> * in chisq at none:2*
>
> * in calculate at none:4*
>

In fact, even without the @sync @parallel in the for o function chisq() it 
still crashes; it crashes even without addprocs()

if @everywhere gc() called in the second function (at each function 
calling), it doesn't crash (but long gc() time).

Is garbage collection not recognizing function creating SharedArrays being 
called many times and hitting system's limit of open files?

This might be a common case, for example, when adjusting parameters by 
optimization of a chisquare function - and each simulation being done in 
parallel, whereas optimization method calling chisquare many times...

Or I made something wrong?

Best regards
Rafael

p.s.: could reproduce also in juliabox 0.5.0-dev (below) and 0.4.6, but not 
in a julia 0.4.5 32 bits system:

> In [4]:
>
> calculate(500)
>
> LoadError: On worker 2:
> SystemError: shm_open() failed for /jl34opVp2HcAjt3ix2bbeW5A: Too many 
> open files
>  in _jl_spawn at ./process.jl:321
>  in #293 at ./process.jl:474 [inlined]
>  in setup_stdio at ./process.jl:462
>  in #spawn#292 at ./process.jl:473
>  in #spawn at ./:0
>  in ip:0x7f5f467573de at /opt/julia-0.5.0-dev/lib/julia/sys.so:? (repeats 2 
> times)
>  in readstring at ./process.jl:577 [inlined] (repeats 2 times)
>  in print_shmem_limits at ./sharedarray.jl:488
>  in shm_mmap_array at ./sharedarray.jl:515
>  in #657 at ./sharedarray.jl:80
>  in #494 at ./multi.jl:1189
>  in run_work_thunk at ./multi.jl:844
>  in run_work_thunk at ./multi.jl:853 [inlined]
>  in #474 at ./task.jl:54
> while loading In[4], in expression starting on line 1
>
>  in #remotecall_fetch#482(::Array{Any,1}, ::Function, ::Function, 
> ::Base.Worker, ::Base.RRID, ::Vararg{Any,N}) at ./multi.jl:904
>  in remotecall_fetch(::Function, ::Base.Worker, ::Base.RRID, ::Vararg{Any,N}) 
> at ./multi.jl:898
>  in #remotecall_fetch#483(::Array{Any,1}, ::Function, ::Function, ::Int64, 
> ::Base.RRID, ::Vararg{Any,N}) at ./multi.jl:907
>  in remotecall_fetch(::Function, ::Int64, ::Base.RRID, ::Vararg{Any,N}) at 
> ./multi.jl:907
>  in call_on_owner(::Function, ::Future, ::Int64, ::Vararg{Int64,N}) at 
> ./multi.jl:950
>  in wait(::Future) at ./multi.jl:965
>  in #SharedArray#654(::Bool, ::Array{Int64,1}, ::Type{T}, ::Type{Float64}, 
> ::Tuple{Int64}) at ./sharedarray.jl:89
>  in SharedArray{T,N}(::Type{Float64}, ::Tuple{Int64}) at 

[julia-users] Re: Saving and Loading data (when JLD is not suitable)

[Lyndon White]

On Wednesday, 31 August 2016 12:47:37 UTC+8, Lyndon White wrote:
>
>
> So I have one of the models, saved using their custom format (
> https://github.com/sbos/AdaGram.jl/blob/master/src/util.jl#L94)
> It is *6Gb*, and takes ages to load and save.
> I switch to JLD, *13Gb* even worse.
> I used Base.serialize, Its just *13Mb* -- That is a full 3 orders of 
> magnitude different.
> And it is fast to read and write from disk.
>
>
Actually, the figure for Base,Serialize is off there.
Turns out there was a ShareArray that did not serialize to disk.
As so that 13 MB file can't be deserialised.
I thought it was a bit extreme.

But the point stands, that in general there is a large category of types 
for which Base.Serialise is much much faster than JLD.

[julia-users] Re: Running Julia in Ubuntu

[Angshuman Goswami]

When i performed build again errors cropped up.

Pkg.build("PyCall")
WARNING: unable to determine host cpu name.
INFO: Building PyCall
INFO: No system-wide Python was found; got the following error:
could not spawn `/usr/local/lib/python2.7 -c "import distutils.sysconfig; 
print(distutils.sysconfig.get_config_var('VERSION'))"`: permission denied 
(EACCES)
using the Python distribution in the Conda package
INFO: Downloading miniconda installer ...
  % Total% Received % Xferd  Average Speed   TimeTime Time  
Current
 Dload  Upload   Total   SpentLeft  
Speed
100 24.7M  100 24.7M0 0  2401k  0  0:00:10  0:00:10 --:--:-- 
2743k
INFO: Installing miniconda ...
PREFIX=/home/odroid/.julia/v0.4/Conda/deps/usr
installing: _cache-0.0-py27_x0 ...
installing: python-2.7.11-0 ...
installing: conda-env-2.4.5-py27_0 ...
installing: openssl-1.0.2g-0 ...
installing: pycosat-0.6.1-py27_0 ...
installing: pyyaml-3.11-py27_1 ...
installing: readline-6.2-2 ...
installing: requests-2.9.1-py27_0 ...
installing: sqlite-3.9.2-0 ...
installing: tk-8.5.18-0 ...
installing: yaml-0.1.6-0 ...
installing: zlib-1.2.8-0 ...
installing: conda-4.0.5-py27_0 ...
installing: pycrypto-2.6.1-py27_0 ...
installing: pip-8.1.1-py27_1 ...
installing: wheel-0.29.0-py27_0 ...
installing: setuptools-20.3-py27_0 ...
/home/odroid/.julia/v0.4/Conda/deps/usr/installer.sh: line 288: 
/home/odroid/.julia/v0.4/Conda/deps/usr/pkgs/python-2.7.11-0/bin/python: 
cannot execute binary file: Exec format error
ERROR:
cannot execute native linux-32 binary, output from 'uname -a' is:
Linux odroid 3.10.69 #1 SMP PREEMPT Thu Feb 12 15:22:14 BRST 2015 armv7l 
armv7l armv7l GNU/Linux
===[ ERROR: PyCall 
]

LoadError: failed process: 
Process(`/home/odroid/.julia/v0.4/Conda/deps/usr/installer.sh -b -f -p 
/home/odroid/.julia/v0.4/Conda/deps/usr`, ProcessExited(1)) [1]
while loading /home/odroid/.julia/v0.4/PyCall/deps/build.jl, in expression 
starting on line 17



[ BUILD ERRORS 
]

WARNING: PyCall had build errors.

 - packages with build errors remain installed in /home/odroid/.julia/v0.4
 - build the package(s) and all dependencies with `Pkg.build("PyCall")`
 - build a single package by running its `deps/build.jl` script




On Wednesday, August 31, 2016 at 12:08:33 AM UTC-4, Angshuman Goswami wrote:
>
> julia> Pkg.status()
> 7 required packages:
>  - AmplNLWriter  0.2.2
>  - CoinOptServices   0.1.2
>  - IJulia1.2.0
>  - Ipopt 0.2.4
>  - JuMP  0.14.0
>  - PyCall1.7.1
>  - RobotOS   0.4.1
> 19 additional packages:
>  - BinDeps   0.4.3
>  - Calculus  0.1.15
>  - Cbc   0.2.3
>  - Clp   0.2.2
>  - Compat0.8.8
>  - Conda 0.2.3
>  - DataStructures0.4.5
>  - ForwardDiff   0.2.4
>  - JSON  0.7.0
>  - Lazy  0.11.0
>  - LightXML  0.3.0
>  - MacroTools0.3.2
>  - MathProgBase  0.5.4
>  - NaNMath   0.2.1
>  - Nettle0.2.4
>  - ReverseDiffSparse 0.5.8
>  - SHA   0.2.1
>  - URIParser 0.1.6
>  - ZMQ   0.3.4
>
>
> On Tuesday, August 30, 2016 at 10:58:39 PM UTC-4, Angshuman Goswami wrote:
>>
>> I am running julia on a 32 bit system and I made sure the version i 
>> downloaded is 32 bit
>>
>> On Tuesday, August 30, 2016 at 10:34:25 PM UTC-4, Angshuman Goswami wrote:
>>>
>>> I did that and now I am getting this error when I type julia to run in 
>>> the command line 
>>> bash: /usr/local/bin/julia: cannot execute binary file: Exec format error
>>>
>>>
>>> On Tuesday, August 30, 2016 at 4:55:54 AM UTC-4, Kaj Wiik wrote:

 I have been using the third route very successfully:

 Download the binary from e.g.

 https://julialang.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-linux-x86_64.tar.gz
 or

 https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc3-linux-x86_64.tar.gz

 (http://julialang.org/downloads/)

 cd /opt

 sudo tar xzvf tarball.tgz
 A directory like julia-2e358ce975 will be created.

 Then make a symlink
 sudo ln -s /opt/julia-2e358ce975/bin/julia /usr/local/bin

 That's it, very easy.

 Note that now you can support multiple versions by making symlinks e.g. 
 julia-v0.5, that's not possible (or very 

[julia-users] Saving and Loading data (when JLD is not suitable)

[Lyndon White]

I like JLD. I really do.

But I have a problem that it can't solve -- not due to a limitation of JLD, 
but due to HDF5.
HDF5 is (AFAIK) designed with rectangular arrays in mind -- everything else 
is secondary.
If your the type you are trying to serialize is more or less a  Structure 
of Arrays, your fine.
If it is an array of structures, you are not fine.
If it is a linked list/tree, you are *really* not fine.


So I have been working with some models from AdaGrams.jl 
(https://github.com/sbos/AdaGram)
Very cool stuff in my area. 

So I have one of the models, saved using their custom format 
(https://github.com/sbos/AdaGram.jl/blob/master/src/util.jl#L94)
It is *6Gb*, and takes ages to load and save.
I switch to JLD, *13Gb* even worse.
I used Base.serialize, Its just *13Mb* -- That is a full 3 orders of 
magnitude different.
And it is fast to read and write from disk.

This happens to me again and again.
Apparently the types in tend to need, do not make JLD happy.



There are 3 problems with using Base,serialize as a data storage format.

   1. *It is not stable* -- the format is evolving as the language evolves, 
   and it breaks the ability to load files
   2. *It is only usable from Julia* -- vs JLD which is, in the endm fancy 
   HDF5, anything can read it after a little work
   3. *It is not safe from a security perspective*  -- Maliciously crafted 
   ".jsz" files can allow arbitrary code execution to occur during the 
   deserialize step.



Points 2, and 3 are also true of Python's Pickle.


I haven't actually heard of anyone doing Point 3, but it seems really 
likely.
Deserialisation is a standard vector of attack, and this is not a security 
hardened part of the codebase
Here is how it can be done using Pickle 



Point 2 is pretty serious, it leads to bad open science.


Point 1 though is what we tend to focus on, at least from what I've 
observed on SO, Github issues, Juliausers, and in the docs.

I have been bit by point 1, when I fully knew I was doing the wrong thing 
by using Base.serialize as a storage format, but did it anyway,
because it change things from hours to seconds.
I updated the Julia nightly and then couldn't load any of my files.
So I ended up having to clone a version earlier, and then wrote a script to 
resave everything in to JLD.

One thing that would go a way towards solving points 2, and 3 would be a 
standard script distributed with JLD (or julia),
that does convert ".jsz" into JLD files; in a sandboxed enviroment.

Solving point 1, would be to have a script/function (possibly as part of 
compat) that can converts between version of Base.Serialize.
This might be bit tricky, given changes to Base.Serialize are often tied to 
deep changes in how the language works internally.
In ways that mean that the Base.Serialize code from before the commit the 
changed it, can not even be run after that commit.
Such a script may need to do what I did and check out two versions of julia 
and convert to an intermediary format.


The other option would be to try and create a new data storage package 
based on the same logic as Base.Serialize,
but without it breaking when the language changes. 
This would be hard, since I suspect what makes it fast, is also what makes 
it incompatible with changes: 

that the representation on disk is very close to the representation in 
memory.
Still, even if it is 100x worse that Base.Serialize, it would still (For my 
kinda data) be 10x faster that JLD.



I had hoped Protobuf.jl might be a package that I could use as an 
alternative:
But the Protobuf spec doesn't really make it good for general data storage
See https://github.com/tanmaykm/ProtoBuf.jl/issues/73



So what to do?

Right now, I try to use Base.Serialize internally, and to convert to JLD 
when I am done messing with it, and also before updating julia.

[julia-users] Re: Running Julia in Ubuntu

[Angshuman Goswami]

julia> Pkg.status()
7 required packages:
 - AmplNLWriter  0.2.2
 - CoinOptServices   0.1.2
 - IJulia1.2.0
 - Ipopt 0.2.4
 - JuMP  0.14.0
 - PyCall1.7.1
 - RobotOS   0.4.1
19 additional packages:
 - BinDeps   0.4.3
 - Calculus  0.1.15
 - Cbc   0.2.3
 - Clp   0.2.2
 - Compat0.8.8
 - Conda 0.2.3
 - DataStructures0.4.5
 - ForwardDiff   0.2.4
 - JSON  0.7.0
 - Lazy  0.11.0
 - LightXML  0.3.0
 - MacroTools0.3.2
 - MathProgBase  0.5.4
 - NaNMath   0.2.1
 - Nettle0.2.4
 - ReverseDiffSparse 0.5.8
 - SHA   0.2.1
 - URIParser 0.1.6
 - ZMQ   0.3.4


On Tuesday, August 30, 2016 at 10:58:39 PM UTC-4, Angshuman Goswami wrote:
>
> I am running julia on a 32 bit system and I made sure the version i 
> downloaded is 32 bit
>
> On Tuesday, August 30, 2016 at 10:34:25 PM UTC-4, Angshuman Goswami wrote:
>>
>> I did that and now I am getting this error when I type julia to run in 
>> the command line 
>> bash: /usr/local/bin/julia: cannot execute binary file: Exec format error
>>
>>
>> On Tuesday, August 30, 2016 at 4:55:54 AM UTC-4, Kaj Wiik wrote:
>>>
>>> I have been using the third route very successfully:
>>>
>>> Download the binary from e.g.
>>>
>>> https://julialang.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-linux-x86_64.tar.gz
>>> or
>>>
>>> https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc3-linux-x86_64.tar.gz
>>>
>>> (http://julialang.org/downloads/)
>>>
>>> cd /opt
>>>
>>> sudo tar xzvf tarball.tgz
>>> A directory like julia-2e358ce975 will be created.
>>>
>>> Then make a symlink
>>> sudo ln -s /opt/julia-2e358ce975/bin/julia /usr/local/bin
>>>
>>> That's it, very easy.
>>>
>>> Note that now you can support multiple versions by making symlinks e.g. 
>>> julia-v0.5, that's not possible (or very difficult) with the deb packages.
>>>
>>> Cheers,
>>> Kaj
>>>
>>>
>>> On Tuesday, August 30, 2016 at 8:26:29 AM UTC+3, Angshuman Goswami wrote:

 I was running Julia to run my MPC code. I needed to upgrade and hence i 
 deleted the folder i cloned from git hub. Now I have two problems:

 1) Installing julia by sudo get-apt install julia, I get the following 
 message:

 Reading package lists... Done
 Building dependency tree   
 Reading state information... Done
 Package julia is not available, but is referred to by another package.
 This may mean that the package is missing, has been obsoleted, or
 is only available from another source

 E: Package 'julia' has no installation candidate

 2) When I cloned the github link by  git clone 
 https://github.com/JuliaLang/julia.git

 I tried make -j N

 it didn't work

 3) I then used 

 git pull && make

 Now Julia was updated to 0.4.7 
 And now I thought it will finally work.
 But now when I do i) using PyCall or  ii) using RobotOS
 I get the following error:
 julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, 
 jl_codectx_t*, bool, bool): Assertion `ctx->gensym_assigned.at(idx)' 
 failed.

 signal (6): Aborted
 ERROR: LoadError: Failed to precompile PyCall to 
 /home/odroid/.julia/lib/v0.4/PyCall.ji
 while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in 
 expression starting on line 3

 M stuck

Re: [julia-users] Set Data Structure

[Jared Crean]

That's nifty, thanks.

  Jared Crean

On Tuesday, August 30, 2016 at 11:51:02 AM UTC-4, Kevin Squire wrote:
>
> Note that you can also write this as
>
> julia> s = Set([2,3,1])
> Set([2,3,1])
>
> julia> 2 in s
> true
>
> julia> s in 2
> false
>
> Which might make it easier to understand why the second one fails.
>
> Cheers,
>Kevin
>
>
> On Mon, Aug 29, 2016 at 3:00 PM, Erik Schnetter  > wrote:
>
>> On Mon, Aug 29, 2016 at 3:59 PM, Jared Crean > > wrote:
>>
>>> Here is an oddity:
>>>
>>> julia> s
>>> Set([2,3,1])
>>>
>>> julia> in(s, 2)
>>> false
>>>
>>> julia> in(2, s)
>>> true
>>>
>>> I would have though the first use of in would be an error because asking 
>>> if a set is contained in a number is not defined.  Is there some other 
>>> interpretation of the operation?
>>>
>>
>> In Julia, every number is a collection that contains just this number. 
>> Thus you are essentially asking whether the set `s` is equal to this 
>> number, which it is not.
>>
>> -erik
>>
>> On Monday, August 29, 2016 at 3:27:30 PM UTC-4, Jared Crean wrote:

 Ah, yes.  That's it.

   Thanks,
 Jared Crean

 On Monday, August 29, 2016 at 3:11:02 PM UTC-4, Erik Schnetter wrote:
>
> Jared
>
> Are you looking for the function `in`?
>
> -erik
>
> On Mon, Aug 29, 2016 at 3:06 PM, Jared Crean  
> wrote:
>
>> I'm looking for a data structure that allows O(1) querying if a value 
>> is contained in the data structure, and reasonably fast construction of 
>> the 
>> data structure given that the initial size is unknown (although this 
>> criteria is not that strict).  I was looking at the Set in base, but I 
>> can't find a way to test if a Set contains contains a particular value.  
>> I 
>> also don't know about the efficiency of appending to a Set.  Any 
>> suggestions or information about Set would be appreciated.
>>
>> Jared Crean
>>
>
>
>
> -- 
> Erik Schnetter  
> http://www.perimeterinstitute.ca/personal/eschnetter/
>

>>
>>
>> -- 
>> Erik Schnetter  
>> http://www.perimeterinstitute.ca/personal/eschnetter/
>>
>
>

[julia-users] Re: Running Julia in Ubuntu

[Angshuman Goswami]

I am running julia on a 32 bit system and I made sure the version i 
downloaded is 32 bit

On Tuesday, August 30, 2016 at 10:34:25 PM UTC-4, Angshuman Goswami wrote:
>
> I did that and now I am getting this error when I type julia to run in the 
> command line 
> bash: /usr/local/bin/julia: cannot execute binary file: Exec format error
>
>
> On Tuesday, August 30, 2016 at 4:55:54 AM UTC-4, Kaj Wiik wrote:
>>
>> I have been using the third route very successfully:
>>
>> Download the binary from e.g.
>>
>> https://julialang.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-linux-x86_64.tar.gz
>> or
>>
>> https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc3-linux-x86_64.tar.gz
>>
>> (http://julialang.org/downloads/)
>>
>> cd /opt
>>
>> sudo tar xzvf tarball.tgz
>> A directory like julia-2e358ce975 will be created.
>>
>> Then make a symlink
>> sudo ln -s /opt/julia-2e358ce975/bin/julia /usr/local/bin
>>
>> That's it, very easy.
>>
>> Note that now you can support multiple versions by making symlinks e.g. 
>> julia-v0.5, that's not possible (or very difficult) with the deb packages.
>>
>> Cheers,
>> Kaj
>>
>>
>> On Tuesday, August 30, 2016 at 8:26:29 AM UTC+3, Angshuman Goswami wrote:
>>>
>>> I was running Julia to run my MPC code. I needed to upgrade and hence i 
>>> deleted the folder i cloned from git hub. Now I have two problems:
>>>
>>> 1) Installing julia by sudo get-apt install julia, I get the following 
>>> message:
>>>
>>> Reading package lists... Done
>>> Building dependency tree   
>>> Reading state information... Done
>>> Package julia is not available, but is referred to by another package.
>>> This may mean that the package is missing, has been obsoleted, or
>>> is only available from another source
>>>
>>> E: Package 'julia' has no installation candidate
>>>
>>> 2) When I cloned the github link by  git clone 
>>> https://github.com/JuliaLang/julia.git
>>>
>>> I tried make -j N
>>>
>>> it didn't work
>>>
>>> 3) I then used 
>>>
>>> git pull && make
>>>
>>> Now Julia was updated to 0.4.7 
>>> And now I thought it will finally work.
>>> But now when I do i) using PyCall or  ii) using RobotOS
>>> I get the following error:
>>> julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, 
>>> bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.
>>>
>>> signal (6): Aborted
>>> ERROR: LoadError: Failed to precompile PyCall to 
>>> /home/odroid/.julia/lib/v0.4/PyCall.ji
>>> while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in 
>>> expression starting on line 3
>>>
>>> M stuck
>>>
>>>

Re: [julia-users] Re: Return type of eye()

[Christoph Ortner]

On Wednesday, 31 August 2016 00:14:30 UTC+1, Sheehan Olver wrote:
>
> I agree with Chris, though I prefer Matrix(eye(5)) to collect(eye(5)).
>
>>
Matrix(eye(5)) looks reasonably pleasing to the eyes :).  (but still 8 
keystrokes more than needed)

[julia-users] Re: Running Julia in Ubuntu

[Angshuman Goswami]

I did that and now I am getting this error when I type julia to run in the 
command line 
bash: /usr/local/bin/julia: cannot execute binary file: Exec format error


On Tuesday, August 30, 2016 at 4:55:54 AM UTC-4, Kaj Wiik wrote:
>
> I have been using the third route very successfully:
>
> Download the binary from e.g.
>
> https://julialang.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-linux-x86_64.tar.gz
> or
>
> https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc3-linux-x86_64.tar.gz
>
> (http://julialang.org/downloads/)
>
> cd /opt
>
> sudo tar xzvf tarball.tgz
> A directory like julia-2e358ce975 will be created.
>
> Then make a symlink
> sudo ln -s /opt/julia-2e358ce975/bin/julia /usr/local/bin
>
> That's it, very easy.
>
> Note that now you can support multiple versions by making symlinks e.g. 
> julia-v0.5, that's not possible (or very difficult) with the deb packages.
>
> Cheers,
> Kaj
>
>
> On Tuesday, August 30, 2016 at 8:26:29 AM UTC+3, Angshuman Goswami wrote:
>>
>> I was running Julia to run my MPC code. I needed to upgrade and hence i 
>> deleted the folder i cloned from git hub. Now I have two problems:
>>
>> 1) Installing julia by sudo get-apt install julia, I get the following 
>> message:
>>
>> Reading package lists... Done
>> Building dependency tree   
>> Reading state information... Done
>> Package julia is not available, but is referred to by another package.
>> This may mean that the package is missing, has been obsoleted, or
>> is only available from another source
>>
>> E: Package 'julia' has no installation candidate
>>
>> 2) When I cloned the github link by  git clone 
>> https://github.com/JuliaLang/julia.git
>>
>> I tried make -j N
>>
>> it didn't work
>>
>> 3) I then used 
>>
>> git pull && make
>>
>> Now Julia was updated to 0.4.7 
>> And now I thought it will finally work.
>> But now when I do i) using PyCall or  ii) using RobotOS
>> I get the following error:
>> julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, 
>> bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.
>>
>> signal (6): Aborted
>> ERROR: LoadError: Failed to precompile PyCall to 
>> /home/odroid/.julia/lib/v0.4/PyCall.ji
>> while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in expression 
>> starting on line 3
>>
>> M stuck
>>
>>

Re: [julia-users] Running Julia in Ubuntu

[Angshuman Goswami]

14.04

On Tuesday, August 30, 2016 at 2:55:30 AM UTC-4, Henri Girard wrote:
>
> WHICH UBUNTU VERSION ?
>
> Le 30/08/2016 à 00:30, Angshuman Goswami a écrit :
>
> I was running Julia to run my MPC code. I needed to upgrade and hence i 
> deleted the folder i cloned from git hub. Now I have two problems:
>
> 1) Installing julia by sudo get-apt install julia, I get the following 
> message:
>
> Reading package lists... Done
> Building dependency tree   
> Reading state information... Done
> Package julia is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
>
> E: Package 'julia' has no installation candidate
>
> 2) When I cloned the github link by  git clone 
> https://github.com/JuliaLang/julia.git
>
> I tried make -j N
>
> it didn't work
>
> 3) I then used 
>
> git pull && make
>
> Now Julia was updated to 0.4.7 
> And now I thought it will finally work.
> But now when I do i) using PyCall or  ii) using RobotOS
> I get the following error:
> julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, 
> bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.
>
> signal (6): Aborted
> ERROR: LoadError: Failed to precompile PyCall to 
> /home/odroid/.julia/lib/v0.4/PyCall.ji
> while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in expression 
> starting on line 3
>
> M stuck
>
>
>
>

[julia-users] issues with Pycall

[Angshuman Goswami]

I am trying to use Julia for running an MPC but I am getting the following 
error:
All was running fine in the Julia 0.4.4 but when I updated my system to 
0.4.7 this happened
julia> using RobotOS
INFO: Precompiling module PyCall...
WARNING: unable to determine host cpu name.
julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, 
bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.

signal (6): Aborted
ERROR: LoadError: Failed to precompile PyCall to 
/home/odroid/.julia/lib/v0.4/PyCall.ji
while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in 
expression starting on line 3

Re: [julia-users] code_native complex for simple snippet

[Erik Schnetter]

With unsigned shift counts the generated code should be shorter, since
Julia then knows in which direction to shift.

-erik

On Tue, Aug 30, 2016 at 7:24 PM, mmh  wrote:

> Is there an unsafe version of >> and <<  that does not do range checks?
>
>
> On Tuesday, August 30, 2016 at 6:46:36 PM UTC-4, Yichao Yu wrote:
>>
>>
>>
>> On Tue, Aug 30, 2016 at 6:25 PM, mmh  wrote:
>>
>>>
>>> 
>>>
>>>
>>>
>> FWIW, your "simple" function has 10 operations that generates ~40
>> instructions when fully inlined including the range check for the shifts,
>> prologue, epilogue, struct return etc, that seems like a reasonable number
>> for me.
>>
>>>
>>>
>>>
>>> 
>>>
>>> The
>>> output for the Int32 and Int64 case
>>> 
>>>
>>> The
>>> following looks particularly bad for the Int32 case
>>> 
>>>
>>> 
>>>  movabsq $">>", %rax
>>> 
>>>  movl$6, %edx
>>> 
>>>  callq   *%rax
>>> 
>>>
>>>
>>>
>>> 
>>>
>>> 
>>>
>>>
>>


-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/

Re: [julia-users] Re: Adding items into a tuple

['Greg Plowman' via julia-users]

>
>
> I could use an array, but one product can correspond to different number 
> of bases.
>
> That's why I decided to use tuples.
>
> You could use an Array of (different length) Arrays, similar to Array of 
Tuples.


Another strategy might be to construct a vector of (Product, Base) pairs, 
and then iterate over this vector:

product_bases = Tuple{ProductType, BaseType}[]

for k in Products, b in Bases
if accordance[k,b]
push!(product_bases, (k,b))
end
end

for (k,b) in product_bases
...
end

[julia-users] Re: @threads all vs. @parallel ???

[Andrew]

I have also been wondering this. I tried @threads yesterday and it got me 
around a 4-fold speedup on a loop which applied a function to each element 
in an array, and I conveniently didn't need to bother using SharedArrays as 
I would with @parallel.

On Tuesday, August 30, 2016 at 7:20:36 PM UTC-4, digxx wrote:
>
> Sorry if there is already some information on this though I didnt find 
> it...
> So: What is the difference between these?
> I have used @parallel so far for parallel loops but recently saw this 
> @threads all in some video and I was wondering what the difference is?
> Could anyone elaborate or give me a link with some info?
> Thanks digxx
>

Re: [julia-users] High memory consumption with loops

[Venil Noronha]

Thanks Steve!

The problem was finally found to be with some matrix operations. We've used 
an alternate approach and it seems to be fixed now.

Venil

On Thursday, August 25, 2016 at 6:41:25 PM UTC-7, vav...@uwaterloo.ca wrote:
>
> I don't have any suggestions about the performance, but I do have a code 
> correctness suggestion.  It appears from your code snippet that you are 
> using dictionaries called "effectiveness" and "resource_fines", and that 
> the keys to these dictionaries are a mutable type.  This can lead to subtle 
> program bugs.  For example, the following code sequence:
>
> d = Dict{Array{Int,1},Float64}()
> a = [4,5,6]
> d[a] = 9.1
> a[2] = 4
>
> will leave internal indexing structure of d in a corrupted state.  It is 
> safer to use immutables as keys for Dict.  If you must use mutables, then 
> whenever you assign a dictionary entry it is safest to insert a 'deepcopy' 
> operation; e.g., the above snippet can be fixed by substituting the 
> following for d[a]=9.1:
>d[deepcopy(a)] = 9.1
>
> -- Steve Vavasis
>
>
> On Thursday, August 25, 2016 at 2:21:30 AM UTC-4, Venil Noronha wrote:
>>
>> I've also tried to @profile, however, julia libraries seem to be executed 
>> the most.
>>
>> Venil
>>
>> On Wednesday, August 24, 2016 at 11:16:53 PM UTC-7, Venil Noronha wrote:
>>>
>>> I've just been able to @time the loop so far to see allocations at 
>>> iteration level. I haven't yet tried @code_warntype; I'll probably do that 
>>> next and see if I can get somewhere.
>>>
>>> Thanks,
>>> Venil
>>>
>>> On Wednesday, August 24, 2016 at 5:37:17 PM UTC-7, Tim Holy wrote:

 What have you tried so far? See 
 http://docs.julialang.org/en/latest/manual/ 

 performance-tips/#tools 
 , 
 esp. @code_warntype. 

 --Tim 

 On Wednesday, August 24, 2016 1:55:23 PM CDT Venil Noronha wrote: 
 > The following code seems to consume a lot of memory. Could anyone 
 spot what 
 > I'm doing wrong here? I'm also using the JuMP library. 
 > 
 > function initUtilityConstraint() 
 > c = categories[1] 
 > me = attack_methods[1] 
 > t = teams[1] 
 > tuple = Simulate.cmt(c, me, t) 
 > w = windows[1] 
 > r = resources[1] 
 > wrtuple = Simulate.wr(w, r) 
 > for ni in 1:size(list,1), c in categories,* f in flights* 
 > performloop(ni, c, f, tuple, wrtuple) 
 > end 
 > end 
 > 
 > function performloop(ni, c, f, tuple, wrtuple) 
 > fi = findfirst(flights, f) 
 > for w in windows, me in attack_methods 
 > tuple.c = c 
 > tuple.m = me 
 > total = 0.0 
 > for t in teams 
 > tuple.t = t 
 > strat = linearizeStrategyS(f, c, t, w, ni) 
 > total = total + effectiveness[tuple]*strat 
 > end 
 > 
 > total = ( total*(flight_vals[fi]*(-1)) + flight_vals[fi] ) 
 > 
 > for w2 in owindows, r2 in resources 
 > wrtuple.w = w2 
 > wrtuple.r = r2 
 > over = linearizeOverflow(w2, r2, ni) 
 > total = total - resource_fines[wrtuple]*over 
 > end 
 > # println(string( sc[c], "<=", ( total*(flight_vals[fi]*(-1)) 
 + 
 > flight_vals[fi] ))) 
 > @addConstraint(m, sc[c] <= total) 
 > end 
 > end 
 > 
 > Following are the stats for piece this code. Stats were recorded 
 using 
 > @time. 
 > 
 > For 1 item in the flights array, it takes about 620KB to execute the 
 > performloop method - peak memory consumption by the program is 
 8.84GBs. 
 > 2 flights - 1.02MB per iteration of performloop - peak 8.88GBs. 
 > 3 flights - 3.45MB - 9.60GBs 
 > 4 flights - 4.35MB - 10.24GBs 
 > 5 flights - 10.8MB - 15.63GBs 
 > 
 > It'd be great if someone could help me with this asap! 
 > 
 > Thanks. 

[julia-users] Re: I want to publish a package into the Julia Ecosystem

[Jeffrey Sarnoff]

Got it sorted. For the next one to find doing this to be as a treacle oriel:
  
http://docs.julialang.org/en/latest/manual/packages/#man-manual-publish
  do what it says slowly and deliberately (read twice, press enter 
once),
  

On Tuesday, August 30, 2016 at 7:15:00 PM UTC-4, Jeffrey Sarnoff wrote:
>
> I tried the way given in the new latest [docs] (
> http://docs.julialang.org/en/latest/manual/packages/#man-manual-publish) 
> to deal with the METADATA manually.
> It seems METADATA has outgrown github defaults:  Sorry, we had to 
> truncate this directory to 1,000 files. 126 entries were omitted from the 
> list.
>
> I think it wrong to try to do that with missing METADATA, no?
>
>
> On Tuesday, August 30, 2016 at 4:51:09 PM UTC-4, Jeffrey Sarnoff wrote:
>>
>> Please give me a way that requires no imagination (mourning Gene Wilder).
>>
>> The github repository is here: 
>> https://github.com/JuliaArbTypes/ArbFloats.jl
>> I have read related: v0.4 does not help me, and generalities are 
>> unfamiliar to me.
>>
>>
>> julia> PkgDev.publish()
>> INFO: Validating METADATA
>> INFO: Pushing ArbFloats permanent tags: v0.1.0, v0.2.0, v0.3.0, v0.4.0, 
>> v0.5.0, v0.6.0
>> ERROR: GitError(Code:EAUTH, Class:None, No errors)
>>  in macro expansion at ./libgit2/error.jl:99 [inlined]
>>  in #push#53(::Bool, ::Base.LibGit2.PushOptions, ::Function, 
>> ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./libgit2/remote.jl:84
>>  in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
>> ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./:0
>>  in #push#94(::String, ::String, ::Array{String,1}, ::Bool, 
>> ::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Function, 
>> ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:185
>>  in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
>> ::Base.LibGit2.GitRepo) at ./:0
>>  in 
>> (::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo)
>>  
>> at /home/jas/.julia/v0.5/PkgDev/src/entry.jl:114
>>  in with(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}}, 
>> ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:638
>>  in publish(::String, ::String) at /home/jas/.julia/v0.5/PkgDev/src/entry
>> .jl:97
>>  in publish() at /home/jas/.julia/v0.5/PkgDev/src/PkgDev.jl:70
>>
>>
>>
>>
>>
>>
>>
>>
>>

Re: [julia-users] code_native complex for simple snippet

[mmh]

Is there an unsafe version of >> and <<  that does not do range checks?


On Tuesday, August 30, 2016 at 6:46:36 PM UTC-4, Yichao Yu wrote:
>
>
>
> On Tue, Aug 30, 2016 at 6:25 PM, mmh  
> wrote:
>
>>
>> 
>>
>>
>>
> FWIW, your "simple" function has 10 operations that generates ~40 
> instructions when fully inlined including the range check for the shifts, 
> prologue, epilogue, struct return etc, that seems like a reasonable number 
> for me.
>
>>
>>
>>
>> 
>>
>> The
>>  
>> output for the Int32 and Int64 case 
>> 
>>
>> The
>>  
>> following looks particularly bad for the Int32 case  
>> 
>>
>> 
>>  movabsq $">>", %rax 
>> 
>>   
>>  movl$6, %edx 
>> 
>>   
>>  callq   *%rax 
>> 
>>   
>>
>>
>> 
>>
>> 
>>
>>
>

Re: [julia-users] @threads all vs. @parallel ???

[Yichao Yu]

On Tue, Aug 30, 2016 at 7:20 PM, digxx  wrote:

> Sorry if there is already some information on this though I didnt find
> it...
> So: What is the difference between these?
> I have used @parallel so far for parallel loops but recently saw this
> @threads all in some video and I was wondering what the difference is?
> Could anyone elaborate or give me a link with some info?
> Thanks digxx
>

@parallel uses worker processes (can also be on a different machine)

@threads uses threads. You shouldn't use @threads now unless you are
working on threading support in julia since it is really experimental at
the moment.

[julia-users] @threads all vs. @parallel ???

[digxx]

Sorry if there is already some information on this though I didnt find it...
So: What is the difference between these?
I have used @parallel so far for parallel loops but recently saw this 
@threads all in some video and I was wondering what the difference is?
Could anyone elaborate or give me a link with some info?
Thanks digxx

[julia-users] Re: I want to publish a package into the Julia Ecosystem

[Jeffrey Sarnoff]

I tried the way given in the new latest [docs] 
(http://docs.julialang.org/en/latest/manual/packages/#man-manual-publish) 
to deal with the METADATA manually.
It seems METADATA has outgrown github defaults:  Sorry, we had to truncate 
this directory to 1,000 files. 126 entries were omitted from the list.

I think it wrong to try to do that with missing METADATA, no?


On Tuesday, August 30, 2016 at 4:51:09 PM UTC-4, Jeffrey Sarnoff wrote:
>
> Please give me a way that requires no imagination (mourning Gene Wilder).
>
> The github repository is here: 
> https://github.com/JuliaArbTypes/ArbFloats.jl
> I have read related: v0.4 does not help me, and generalities are 
> unfamiliar to me.
>
>
> julia> PkgDev.publish()
> INFO: Validating METADATA
> INFO: Pushing ArbFloats permanent tags: v0.1.0, v0.2.0, v0.3.0, v0.4.0, v0
> .5.0, v0.6.0
> ERROR: GitError(Code:EAUTH, Class:None, No errors)
>  in macro expansion at ./libgit2/error.jl:99 [inlined]
>  in #push#53(::Bool, ::Base.LibGit2.PushOptions, ::Function, 
> ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./libgit2/remote.jl:84
>  in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
> ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./:0
>  in #push#94(::String, ::String, ::Array{String,1}, ::Bool, 
> ::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Function, 
> ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:185
>  in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
> ::Base.LibGit2.GitRepo) at ./:0
>  in 
> (::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo) 
> at /home/jas/.julia/v0.5/PkgDev/src/entry.jl:114
>  in with(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}}, 
> ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:638
>  in publish(::String, ::String) at /home/jas/.julia/v0.5/PkgDev/src/entry.
> jl:97
>  in publish() at /home/jas/.julia/v0.5/PkgDev/src/PkgDev.jl:70
>
>
>
>
>
>
>
>
>

Re: [julia-users] Re: Return type of eye()

[Sheehan Olver]

I agree with Chris, though I prefer Matrix(eye(5)) to collect(eye(5)).

I've found teaching-wise that saying "Matlab has Dense and Sparse.  Julia 
has a lot more special types."  worked pretty well to explain the 
situation.  

The issue though is with rand:  it seems like overkill for rand(5,5) to 
return a special type, but sometimes you want to create it on a GPU.



On Wednesday, August 31, 2016 at 3:26:22 AM UTC+10, Chris Rackauckas wrote:
>
> Even then, creating the 5x5 dense matrix to then copy it into A[1:5,1:5] 
> is not what you'd want to do. Ideally would just have eye(5) return 
> something like I which has a size, and just sets A[i,j]=1 if i=j 0 
> otherwise, with checks that it's the right size. Actually, the current I 
> would do it if you defined a dispatch on setindex!. You still don't need 
> the dense matrix there...
>
> Tim Holy had the only example where you need the dense matrix, and it's 
> when you want to make a dense matrix to start with and them modify the off 
> diagonals of that same matrix. Even then, collect(eye(5)) where eye 
> returned some smart type instead of a matrix wouldn't be bad syntax to do 
> so, and it would be consistent with the rest of the library.
>
> I agree that there can be a teaching problem with Julia. A lot of things 
> happen like fast magic to "mathematicians not trained in CS". I don't think 
> the right way to go is to fill Base with methods that shouldn't be used in 
> most cases: someone could make their own "helper libraries" if they really 
> think it's an issue (another example is "EasyFloats" which are not type 
> stable, so they will just turn into complex numbers when needed like in 
> MATLAB. It might help new users from MATLAB see less errors, but I'd never 
> want to see it in Base.) Instead I think it comes down to documentation and 
> extensive tutorials to fix the problem (and filling up StackOverflow with 
> answers).
>
> On Tuesday, August 30, 2016 at 10:15:08 AM UTC-7, Christoph Ortner wrote:
>>
>>
>> If this is your only use-case of I, then you don't need it anyways. Just 
>> write 1.0 * A instead; same effect, independent of what type of array A is.
>>
>> But what if I use I in a different way? Suppose I want to A[1:5,1:5] = 
>> eye(5); I can't do that with I. Of course we could give it another type 
>> parameters, and then do the whole collect thing again. But then we are back 
>> to what I said before above about needless distractions.
>>
>> I don't understand why there is such a resistance to providing both the 
>> explicit arrays and the lazy functionalities in Julia.
>>
>> Christoph
>>
>>
>> On Tuesday, 30 August 2016 17:32:37 UTC+1, Júlio Hoffimann wrote:
>>>
>>> I don't think there is anything like pushing the language to computer 
>>> scientists, it's the exact opposite, making it seamlessly fast without 
>>> forcing the user to manipulate types. Again, you write B = I*A and get B = 
>>> copy(A) performance. That is the original proposal.
>>>
>>> Most of us follow the same development strategy while doing science, we 
>>> write something quick and profile later. The fact that Julia can be fast 
>>> from the beginning is the best thing about it, no need for rewriting code.
>>>
>>> -Júlio
>>>
>>

Re: [julia-users] Re: Return type of eye()

[Sheehan Olver]

I agree with 

On Wednesday, August 31, 2016 at 3:26:22 AM UTC+10, Chris Rackauckas wrote:
>
> Even then, creating the 5x5 dense matrix to then copy it into A[1:5,1:5] 
> is not what you'd want to do. Ideally would just have eye(5) return 
> something like I which has a size, and just sets A[i,j]=1 if i=j 0 
> otherwise, with checks that it's the right size. Actually, the current I 
> would do it if you defined a dispatch on setindex!. You still don't need 
> the dense matrix there...
>
> Tim Holy had the only example where you need the dense matrix, and it's 
> when you want to make a dense matrix to start with and them modify the off 
> diagonals of that same matrix. Even then, collect(eye(5)) where eye 
> returned some smart type instead of a matrix wouldn't be bad syntax to do 
> so, and it would be consistent with the rest of the library.
>
> I agree that there can be a teaching problem with Julia. A lot of things 
> happen like fast magic to "mathematicians not trained in CS". I don't think 
> the right way to go is to fill Base with methods that shouldn't be used in 
> most cases: someone could make their own "helper libraries" if they really 
> think it's an issue (another example is "EasyFloats" which are not type 
> stable, so they will just turn into complex numbers when needed like in 
> MATLAB. It might help new users from MATLAB see less errors, but I'd never 
> want to see it in Base.) Instead I think it comes down to documentation and 
> extensive tutorials to fix the problem (and filling up StackOverflow with 
> answers).
>
> On Tuesday, August 30, 2016 at 10:15:08 AM UTC-7, Christoph Ortner wrote:
>>
>>
>> If this is your only use-case of I, then you don't need it anyways. Just 
>> write 1.0 * A instead; same effect, independent of what type of array A is.
>>
>> But what if I use I in a different way? Suppose I want to A[1:5,1:5] = 
>> eye(5); I can't do that with I. Of course we could give it another type 
>> parameters, and then do the whole collect thing again. But then we are back 
>> to what I said before above about needless distractions.
>>
>> I don't understand why there is such a resistance to providing both the 
>> explicit arrays and the lazy functionalities in Julia.
>>
>> Christoph
>>
>>
>> On Tuesday, 30 August 2016 17:32:37 UTC+1, Júlio Hoffimann wrote:
>>>
>>> I don't think there is anything like pushing the language to computer 
>>> scientists, it's the exact opposite, making it seamlessly fast without 
>>> forcing the user to manipulate types. Again, you write B = I*A and get B = 
>>> copy(A) performance. That is the original proposal.
>>>
>>> Most of us follow the same development strategy while doing science, we 
>>> write something quick and profile later. The fact that Julia can be fast 
>>> from the beginning is the best thing about it, no need for rewriting code.
>>>
>>> -Júlio
>>>
>>

[julia-users] Re: FYI: Second transpiler to Julia(?), from Ruby, and benchmarks

[Steven G. Johnson]

On Monday, August 29, 2016 at 11:57:32 AM UTC-4, Páll Haraldsson wrote:
>
> Interesting benchmarks here ("virtual_module" is transpiled, but "Julia 
> 0.4.6 not, only to compare, and [can be] a little slower than Python..):
>

An accurate transpiler from Ruby or Python to Julia is pretty much 
guaranteed not to give any performance improvements, because it will be 
limited by the semantics of the source language.(If it were possible to 
take advantage of Julia's performance in a transpiler, it would be possible 
to just write a direct-to-machine compiler for the original language.)   
And in practice it will probably be slower than the source language because 
Julia is not as heavily optimized for interpreting those semantics.

Re: [julia-users] code_native complex for simple snippet

[Yichao Yu]

On Tue, Aug 30, 2016 at 6:25 PM, mmh  wrote:

>
> 
>
>
>
FWIW, your "simple" function has 10 operations that generates ~40
instructions when fully inlined including the range check for the shifts,
prologue, epilogue, struct return etc, that seems like a reasonable number
for me.

>
>
>
> 
>
> The
> output for the Int32 and Int64 case
> 
>
> The
> following looks particularly bad for the Int32 case
> 
>
> 
>  movabsq $">>", %rax
> 
>  movl$6, %edx
> 
>  callq   *%rax
> 
>
>
>
> 
>
> 
>
>

[julia-users] code_native complex for simple snippet

[mmh]

The
 
output for the Int32 and Int64 case 

The
 
following looks particularly bad for the Int32 case  


 movabsq $">>", %rax 

  
 movl$6, %edx 

  
 callq   *%rax 

  

[julia-users] Re: Package development - best practices

[Kevin Liu]

Just make an alias of, say, ~/.julia/v0.4/mypackage (hidden) and paste the 
alias in the directory you want, e.g. Google Drive/mypackage_alias (not 
hidden)

Whatever changes you make to mypackage_alias will change mypackage, which 
can then be called on the REPL with 

This way you don't have to deal with hidden folders or having your only 
backup on Github. 

I couldn't find a clear alternative on Julialang's manual, which is 
understandable given their resource constraint and priorities. 

If my suggestion here is bullocks, please comment. 

On Friday, December 19, 2014 at 10:47:31 PM UTC-2, David van Leeuwen wrote:
>
> Hi, 
>
> On Wednesday, December 17, 2014 11:21:44 PM UTC+1, Seth wrote:
>>
>> I'm wondering whether folks are actually basing their repos in ~/.julia 
>> and doing their development and commits from there, or whether there's some 
>> other process that allows development to happen in a more standard location 
>> than a hidden directory off of ~ while still allowing use of Pkg. I'm 
>> probably overlooking something trivial.
>>
>> What's a recommended setup/process for creating packages using Pkg to 
>> manage them?
>>
>
> I'm not sure what the right answer would be, but I have been struggling 
> with it a lot.  This is my set-up for most of the stuff I do.  
>
>  - I develop packages in their own directories on the user filesystem. 
>  For me, that would be `.../julia//` .  
>  - from there, in `./src/` I have the various source files.  One, 
> specifically is called "nomodule.jl".  This contains a 
>- "require()"
>- "include()"
>- ...
>  - Each time I change something in the code, I do a `reload("nomodule.jl`) 
> from the REPL.  This replaces all functions, except for the type 
> definitions---they can't be updated
>  - When I need to change the types, I need to do a `ctrl-D; julia; 
> reload("nomodule.jl")` which can be a bit of a pain
>  - When I am happy with the functionality, I make this code into a module, 
> using a file ".jl", very similar to the "nomodule.jl" but with the 
> "require" replaced by "include"
>  - I sync this then with github
>  - If I need the functionality from another working directory, I do a 
> `Pkg.clone()` to load a sort-of-stable version into ~/.julia
>  - When I am really happy with the code, I do a pull request to 
> METADATA.jl for my package to be included in the standard set of Julia 
> packages. 
>
>  - If I need functionality on some other machines, I sync using git, 
> either through github or directly.
>
>  Cheers, 
>
> ---david
>

[julia-users] I want to publish a package into the Julia Ecosystem

[Jeffrey Sarnoff]

Please give me a way that requires no imagination (mourning Gene Wilder).

The github repository is here: https://github.com/JuliaArbTypes/ArbFloats.jl
I have read related: v0.4 does not help me, and generalities are unfamiliar 
to me.


julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing ArbFloats permanent tags: v0.1.0, v0.2.0, v0.3.0, v0.4.0, v0.
5.0, v0.6.0
ERROR: GitError(Code:EAUTH, Class:None, No errors)
 in macro expansion at ./libgit2/error.jl:99 [inlined]
 in #push#53(::Bool, ::Base.LibGit2.PushOptions, ::Function, 
::Base.LibGit2.GitRemote, ::Array{String,1}) at ./libgit2/remote.jl:84
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
::Base.LibGit2.GitRemote, ::Array{String,1}) at ./:0
 in #push#94(::String, ::String, ::Array{String,1}, ::Bool, 
::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Function, 
::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:185
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, 
::Base.LibGit2.GitRepo) at ./:0
 in 
(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo) 
at /home/jas/.julia/v0.5/PkgDev/src/entry.jl:114
 in with(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}}, 
::Base.LibGit2.GitRepo) at ./libgit2/types.jl:638
 in publish(::String, ::String) at /home/jas/.julia/v0.5/PkgDev/src/entry.jl
:97
 in publish() at /home/jas/.julia/v0.5/PkgDev/src/PkgDev.jl:70

[julia-users] Re: warning with @load in JLD

[Ethan Anderes]

Terrific. The second option works great (and will be my first sanctioned 
use of eval...been avoiding it like the plague after reading all the 
warnings about it on google groups).

Thanks!
Ethan

On Tuesday, August 30, 2016 at 12:37:44 PM UTC-7, Simon Kornblith wrote:
>
> The reason this particular form of @load now shows a warning is that it 
> can't be made to work properly in a function. If you can't know either the 
> file name or the variable names ahead of time, there are two things you 
> could do:
>
> - Use the load function (e.g. x = load("data_run$(run).jld")) rather than 
> the macro. This returns a Dict of all variables in the file, so you'd 
> access them as x["a"] etc.
> - Use @eval @load $("data_run$(run).jld")
>
> Simon 
>
> On Tuesday, August 30, 2016 at 3:05:58 PM UTC-4, Ethan Anderes wrote:
>>
>> I often use @load as a way to conveniently load a bunch of variables 
>> obtained from a simulation run.
>> For example, the simulation script might look like this
>>
>> using JLD
>> run   = 1 # <--- simulation parameter
>> a,b,c = 1,2,3
>> @save "data_run$(run).jld" a b c
>>
>> Then, to view these simulations later, I would use the load macro. In 
>> v0.4 my REPL session would look something like this:
>>
>> julia> using JLD
>>
>> julia> run = 1  # <--- specify which simulation run to load
>> 1
>>
>> julia> @load "data_run$(run).jld"
>> 3-element Array{Symbol,1}:
>>  :a
>>  :b
>>  :c
>>
>> julia> a
>> 1
>>
>> julia> b
>> 2
>>
>> julia> c
>> 3
>>
>> However, in v0.5.0-rc3+0 , I get the following warning.
>>
>> julia> using JLD
>>
>> julia> run = 1  # <--- specify which simulation run to load
>> 1
>>
>> julia> @load "data_run$(run).jld"
>> WARNING: @load-ing a file without specifying the variables to be loaded may 
>> produce
>> unexpected behavior unless the file is specified as a string literal. Future
>> versions of JLD will require that the file is specified as a string literal
>> in this case.
>> 3-element Array{Symbol,1}:
>>  :a
>>  :b
>>  :c
>>
>> julia> a
>> 1
>>
>> julia> b
>> 2
>>
>> julia> c
>> 3
>>
>> julia> Pkg.status("JLD")
>>  - JLD   0.6.3+ master
>>
>> I was going to file an issue with JLD but I figured I would post here 
>> first to see if I’m using an ill-advised workflow or missing some simple 
>> work around.
>>
>> Note: there are a few features of my situation which makes @load 
>> particularly convenient.
>>
>> - Each simulation produces a bunch of variables and some older runs have a 
>> smaller subset of variables than newer ones. This makes it clumsy to use the 
>> load function (rather than the macro) where one needs to specify the 
>> variable names ahead of time.
>> - I need to load in multiple `.jld` files, sometimes in a large plotting 
>> script, so printing out each file string in the REPL and pasting it into the 
>> REPL after `@load` is also unwieldy.
>>
>> ​
>>
>

[julia-users] Re: FYI: Second transpiler to Julia(?), from Ruby, and benchmarks

[Páll Haraldsson]

On Monday, August 29, 2016 at 6:42:08 PM UTC, Chris Rackauckas wrote:
>
> That's a good showing for Julia for the larger matrices? However, for 
> smaller matrices it's a large constant time. Is it including 
> startup/compilation time? Did they not "run it twice"?
>
> 1. I guess they must include startup time (and for JRuby..), not always 
unfair.., 2. So no.. they could for the benchmarks, while I'm not sure 
about it ideal for the transpiler (didn't look into it much).
 

> On Monday, August 29, 2016 at 8:57:32 AM UTC-7, Páll Haraldsson wrote:
>>
>>
>> I have no relation to this..
>>
>> https://github.com/remore/julializer
>>
>

"But there are very many differences between Ruby and Julia which I have no 
idea how to fix as of now. For example:

   - Julia supports only dec, bin and hex format but in Ruby the decimal 
   system is much powerful.(#to_s(num) and #to_i(num) are supported)
   - Julia doesn't have Class concept(there is workaround though) but Ruby 
   does
   - Julia does not have a "null" value 
   

 
   but Ruby does 
  - For example in Ruby [1,2,3].slice!(4) will return null but in julia 
  there is no nil therefore this raises causes error.
  - Current workaround for this is, do not write this way, instead you 
  need to check boundary by yourself.
   - And many more gaps to be solved 
  - e.g.) In Julia typemax() and typemin() returns Inf but in Ruby 
  Float::MAX, Float::MIN have specific value"
   

A. I like you Julia doesn't have "null" (not strictly true, at least for C 
API..) to not have Hoare's self-admitted billion dollar mistake. Not sure 
thought, how best to help that project..


B. About Classes and


https://en.wikipedia.org/wiki/Composition_over_inheritance


that is I guess best, but maybe not to helpful for that project.. Should 
that be enough, to compile to that, or any other ideas?


C. I'm sure Julia has as good decimal support as possible already, with two 
different packages. I'm not sure what's in Ruby (so can't comment on that 
code), I guess the maker of the project is not aware, only of what is in 
Base.


 

>
>>
>> [may not work to transpile Ruby on Rails to Julia - yet, there is an old 
>> package RoR that allows it to work with Julia though.]
>>
>
That is https://github.com/Ken-B/RoR_julia_eg

that uses ZMQ.jl (better for IPC)?



>>
>> Interesting benchmarks here ("virtual_module" is transpiled, but "Julia 
>> 0.4.6 not, only to compare, and [can be] a little slower than Python..):
>>
>> https://github.com/remore/virtual_module
>>
>>
>> Saw at:
>>
>>
>> https://www.reddit.com/r/Julia/comments/5049pq/a_fresh_approach_to_numerical_computing_with_ruby/
>>
>> [not really about to discuss here much. Fortran to Julia was first, 
>> though.]
>>
>> -- 
>> Palli.
>>
>>
>>

[julia-users] Re: warning with @load in JLD

[Simon Kornblith]

The reason this particular form of @load now shows a warning is that it 
can't be made to work properly in a function. If you can't know either the 
file name or the variable names ahead of time, there are two things you 
could do:

- Use the load function (e.g. x = load("data_run$(run).jld")) rather than 
the macro. This returns a Dict of all variables in the file, so you'd 
access them as x["a"] etc.
- Use @eval @load $("data_run$(run).jld")

Simon 

On Tuesday, August 30, 2016 at 3:05:58 PM UTC-4, Ethan Anderes wrote:
>
> I often use @load as a way to conveniently load a bunch of variables 
> obtained from a simulation run.
> For example, the simulation script might look like this
>
> using JLD
> run   = 1 # <--- simulation parameter
> a,b,c = 1,2,3
> @save "data_run$(run).jld" a b c
>
> Then, to view these simulations later, I would use the load macro. In v0.4 
> my REPL session would look something like this:
>
> julia> using JLD
>
> julia> run = 1  # <--- specify which simulation run to load
> 1
>
> julia> @load "data_run$(run).jld"
> 3-element Array{Symbol,1}:
>  :a
>  :b
>  :c
>
> julia> a
> 1
>
> julia> b
> 2
>
> julia> c
> 3
>
> However, in v0.5.0-rc3+0 , I get the following warning.
>
> julia> using JLD
>
> julia> run = 1  # <--- specify which simulation run to load
> 1
>
> julia> @load "data_run$(run).jld"
> WARNING: @load-ing a file without specifying the variables to be loaded may 
> produce
> unexpected behavior unless the file is specified as a string literal. Future
> versions of JLD will require that the file is specified as a string literal
> in this case.
> 3-element Array{Symbol,1}:
>  :a
>  :b
>  :c
>
> julia> a
> 1
>
> julia> b
> 2
>
> julia> c
> 3
>
> julia> Pkg.status("JLD")
>  - JLD   0.6.3+ master
>
> I was going to file an issue with JLD but I figured I would post here 
> first to see if I’m using an ill-advised workflow or missing some simple 
> work around.
>
> Note: there are a few features of my situation which makes @load 
> particularly convenient.
>
> - Each simulation produces a bunch of variables and some older runs have a 
> smaller subset of variables than newer ones. This makes it clumsy to use the 
> load function (rather than the macro) where one needs to specify the variable 
> names ahead of time.
> - I need to load in multiple `.jld` files, sometimes in a large plotting 
> script, so printing out each file string in the REPL and pasting it into the 
> REPL after `@load` is also unwieldy.
>
> ​
>

[julia-users] warning with @load in JLD

[Ethan Anderes]

I often use @load as a way to conveniently load a bunch of variables 
obtained from a simulation run.
For example, the simulation script might look like this

using JLD
run   = 1 # <--- simulation parameter
a,b,c = 1,2,3
@save "data_run$(run).jld" a b c

Then, to view these simulations later, I would use the load macro. In v0.4 
my REPL session would look something like this:

julia> using JLD

julia> run = 1  # <--- specify which simulation run to load
1

julia> @load "data_run$(run).jld"
3-element Array{Symbol,1}:
 :a
 :b
 :c

julia> a
1

julia> b
2

julia> c
3

However, in v0.5.0-rc3+0 , I get the following warning.

julia> using JLD

julia> run = 1  # <--- specify which simulation run to load
1

julia> @load "data_run$(run).jld"
WARNING: @load-ing a file without specifying the variables to be loaded may 
produce
unexpected behavior unless the file is specified as a string literal. Future
versions of JLD will require that the file is specified as a string literal
in this case.
3-element Array{Symbol,1}:
 :a
 :b
 :c

julia> a
1

julia> b
2

julia> c
3

julia> Pkg.status("JLD")
 - JLD   0.6.3+ master

I was going to file an issue with JLD but I figured I would post here first 
to see if I’m using an ill-advised workflow or missing some simple work 
around.

Note: there are a few features of my situation which makes @load 
particularly convenient.

- Each simulation produces a bunch of variables and some older runs have a 
smaller subset of variables than newer ones. This makes it clumsy to use the 
load function (rather than the macro) where one needs to specify the variable 
names ahead of time.
- I need to load in multiple `.jld` files, sometimes in a large plotting 
script, so printing out each file string in the REPL and pasting it into the 
REPL after `@load` is also unwieldy.

​

Re: [julia-users] Re: Return type of eye()

[Christoph Ortner]

> I agree that there can be a teaching problem with Julia. A lot of things 
happen like fast magic to "mathematicians not trained in CS". 

yes, this is the issue. FWIW in other respects it is nice to use Julia for 
teaching, e.g., the fact that there are no classes is great!


> I don't think the right way to go is to fill Base with methods that 
shouldn't be used in most cases: someone could make their own "helper 
libraries" if they really think it's an issue

This is always a possibility but external packages quickly become 
incompatible with the language. It is just another little barrier.




On Tuesday, 30 August 2016 18:26:22 UTC+1, Chris Rackauckas wrote:
>
> Even then, creating the 5x5 dense matrix to then copy it into A[1:5,1:5] 
> is not what you'd want to do. Ideally would just have eye(5) return 
> something like I which has a size, and just sets A[i,j]=1 if i=j 0 
> otherwise, with checks that it's the right size. Actually, the current I 
> would do it if you defined a dispatch on setindex!. You still don't need 
> the dense matrix there...
>
> Tim Holy had the only example where you need the dense matrix, and it's 
> when you want to make a dense matrix to start with and them modify the off 
> diagonals of that same matrix. Even then, collect(eye(5)) where eye 
> returned some smart type instead of a matrix wouldn't be bad syntax to do 
> so, and it would be consistent with the rest of the library.
>
> I agree that there can be a teaching problem with Julia. A lot of things 
> happen like fast magic to "mathematicians not trained in CS". I don't think 
> the right way to go is to fill Base with methods that shouldn't be used in 
> most cases: someone could make their own "helper libraries" if they really 
> think it's an issue (another example is "EasyFloats" which are not type 
> stable, so they will just turn into complex numbers when needed like in 
> MATLAB. It might help new users from MATLAB see less errors, but I'd never 
> want to see it in Base.) Instead I think it comes down to documentation and 
> extensive tutorials to fix the problem (and filling up StackOverflow with 
> answers).
>
> On Tuesday, August 30, 2016 at 10:15:08 AM UTC-7, Christoph Ortner wrote:
>>
>>
>> If this is your only use-case of I, then you don't need it anyways. Just 
>> write 1.0 * A instead; same effect, independent of what type of array A is.
>>
>> But what if I use I in a different way? Suppose I want to A[1:5,1:5] = 
>> eye(5); I can't do that with I. Of course we could give it another type 
>> parameters, and then do the whole collect thing again. But then we are back 
>> to what I said before above about needless distractions.
>>
>> I don't understand why there is such a resistance to providing both the 
>> explicit arrays and the lazy functionalities in Julia.
>>
>> Christoph
>>
>>
>> On Tuesday, 30 August 2016 17:32:37 UTC+1, Júlio Hoffimann wrote:
>>>
>>> I don't think there is anything like pushing the language to computer 
>>> scientists, it's the exact opposite, making it seamlessly fast without 
>>> forcing the user to manipulate types. Again, you write B = I*A and get B = 
>>> copy(A) performance. That is the original proposal.
>>>
>>> Most of us follow the same development strategy while doing science, we 
>>> write something quick and profile later. The fact that Julia can be fast 
>>> from the beginning is the best thing about it, no need for rewriting code.
>>>
>>> -Júlio
>>>
>>

Re: [julia-users] Re: Return type of eye()

[Chris Rackauckas]

Even then, creating the 5x5 dense matrix to then copy it into A[1:5,1:5] is 
not what you'd want to do. Ideally would just have eye(5) return something 
like I which has a size, and just sets A[i,j]=1 if i=j 0 otherwise, with 
checks that it's the right size. Actually, the current I would do it if you 
defined a dispatch on setindex!. You still don't need the dense matrix 
there...

Tim Holy had the only example where you need the dense matrix, and it's 
when you want to make a dense matrix to start with and them modify the off 
diagonals of that same matrix. Even then, collect(eye(5)) where eye 
returned some smart type instead of a matrix wouldn't be bad syntax to do 
so, and it would be consistent with the rest of the library.

I agree that there can be a teaching problem with Julia. A lot of things 
happen like fast magic to "mathematicians not trained in CS". I don't think 
the right way to go is to fill Base with methods that shouldn't be used in 
most cases: someone could make their own "helper libraries" if they really 
think it's an issue (another example is "EasyFloats" which are not type 
stable, so they will just turn into complex numbers when needed like in 
MATLAB. It might help new users from MATLAB see less errors, but I'd never 
want to see it in Base.) Instead I think it comes down to documentation and 
extensive tutorials to fix the problem (and filling up StackOverflow with 
answers).

On Tuesday, August 30, 2016 at 10:15:08 AM UTC-7, Christoph Ortner wrote:
>
>
> If this is your only use-case of I, then you don't need it anyways. Just 
> write 1.0 * A instead; same effect, independent of what type of array A is.
>
> But what if I use I in a different way? Suppose I want to A[1:5,1:5] = 
> eye(5); I can't do that with I. Of course we could give it another type 
> parameters, and then do the whole collect thing again. But then we are back 
> to what I said before above about needless distractions.
>
> I don't understand why there is such a resistance to providing both the 
> explicit arrays and the lazy functionalities in Julia.
>
> Christoph
>
>
> On Tuesday, 30 August 2016 17:32:37 UTC+1, Júlio Hoffimann wrote:
>>
>> I don't think there is anything like pushing the language to computer 
>> scientists, it's the exact opposite, making it seamlessly fast without 
>> forcing the user to manipulate types. Again, you write B = I*A and get B = 
>> copy(A) performance. That is the original proposal.
>>
>> Most of us follow the same development strategy while doing science, we 
>> write something quick and profile later. The fact that Julia can be fast 
>> from the beginning is the best thing about it, no need for rewriting code.
>>
>> -Júlio
>>
>

Re: [julia-users] Re: Return type of eye()

[Christoph Ortner]

If this is your only use-case of I, then you don't need it anyways. Just 
write 1.0 * A instead; same effect, independent of what type of array A is.

But what if I use I in a different way? Suppose I want to A[1:5,1:5] = 
eye(5); I can't do that with I. Of course we could give it another type 
parameters, and then do the whole collect thing again. But then we are back 
to what I said before above about needless distractions.

I don't understand why there is such a resistance to providing both the 
explicit arrays and the lazy functionalities in Julia.

Christoph


On Tuesday, 30 August 2016 17:32:37 UTC+1, Júlio Hoffimann wrote:
>
> I don't think there is anything like pushing the language to computer 
> scientists, it's the exact opposite, making it seamlessly fast without 
> forcing the user to manipulate types. Again, you write B = I*A and get B = 
> copy(A) performance. That is the original proposal.
>
> Most of us follow the same development strategy while doing science, we 
> write something quick and profile later. The fact that Julia can be fast 
> from the beginning is the best thing about it, no need for rewriting code.
>
> -Júlio
>

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

I don't think there is anything like pushing the language to computer
scientists, it's the exact opposite, making it seamlessly fast without
forcing the user to manipulate types. Again, you write B = I*A and get B =
copy(A) performance. That is the original proposal.

Most of us follow the same development strategy while doing science, we
write something quick and profile later. The fact that Julia can be fast
from the beginning is the best thing about it, no need for rewriting code.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Christoph Ortner]

I agree with Sheehan that this affect a number of functions in Base, not 
just eye - that was the point I was trying to make, sorry I wasn't clear.

I raised this in a discussion in a formal issue somewhere, which I can't 
find now. Somebody (Steven Johnson?) argued that `zeros` and `ones` are 
different because they are used for allocating arrays.

With every discussion like this one (previously linspace) targeted at 
"cleaning up" the language and disallowing "bad practices" I always feel 
that Julia is becoming more and more a language for Computer Scientists 
while casual programmers are pushed away. In fact this doesn't just affect 
casual programmers: for almost every new project I first cook up a little 
toy-model where I really just brainstorm - often I write bad code, but this 
is ok. I like that fact that I can do that in Julia (for now) without 
getting punished.  In those little toy models it is usual irrelevant that 
linspace and even eye return arrays instead of lazy data structures - in 
fact it allows me to think less about what are the objects I am 
manipulating. Every small distraction like that breaks the flow of 
thoughts.  I can always go back later and refactor my code (or usually 
rewrite it) with good structure and performance in mind. 

Christoph

On Tuesday, 30 August 2016 06:39:40 UTC+1, Sheehan Olver wrote:
>
> But the core issue isn't 'eye' specific, it's "what should the default 
> type for functions that create matrices be?"Christoph's comments do not 
> deviate from this question.
>
> The answer to this question affects 'rand', 'zeros', 'ones', 'linspace', 
> etc. just as much as eye.  ArrayFire means that this might want to be 
> created on the GPU which should be taken into account: the syntax 
> 'AFArray(eye(10))' cannot work unless eye(10) returns a special type.
>
> On Tuesday, August 30, 2016 at 2:19:38 PM UTC+10, Júlio Hoffimann wrote:
>>
>> Sorry for the combative tone Christoph. I thought it was necessary in 
>> order to not deviate too much from the core issue. Thank you for your 
>> participation and for raising your personal opinions about the topic.
>>
>> -Júlio
>>
>

Re: [julia-users] Set Data Structure

[Kevin Squire]

Note that you can also write this as

julia> s = Set([2,3,1])
Set([2,3,1])

julia> 2 in s
true

julia> s in 2
false

Which might make it easier to understand why the second one fails.

Cheers,
   Kevin


On Mon, Aug 29, 2016 at 3:00 PM, Erik Schnetter  wrote:

> On Mon, Aug 29, 2016 at 3:59 PM, Jared Crean  wrote:
>
>> Here is an oddity:
>>
>> julia> s
>> Set([2,3,1])
>>
>> julia> in(s, 2)
>> false
>>
>> julia> in(2, s)
>> true
>>
>> I would have though the first use of in would be an error because asking
>> if a set is contained in a number is not defined.  Is there some other
>> interpretation of the operation?
>>
>
> In Julia, every number is a collection that contains just this number.
> Thus you are essentially asking whether the set `s` is equal to this
> number, which it is not.
>
> -erik
>
> On Monday, August 29, 2016 at 3:27:30 PM UTC-4, Jared Crean wrote:
>>>
>>> Ah, yes.  That's it.
>>>
>>>   Thanks,
>>> Jared Crean
>>>
>>> On Monday, August 29, 2016 at 3:11:02 PM UTC-4, Erik Schnetter wrote:

 Jared

 Are you looking for the function `in`?

 -erik

 On Mon, Aug 29, 2016 at 3:06 PM, Jared Crean  wrote:

> I'm looking for a data structure that allows O(1) querying if a value
> is contained in the data structure, and reasonably fast construction of 
> the
> data structure given that the initial size is unknown (although this
> criteria is not that strict).  I was looking at the Set in base, but I
> can't find a way to test if a Set contains contains a particular value.  I
> also don't know about the efficiency of appending to a Set.  Any
> suggestions or information about Set would be appreciated.
>
> Jared Crean
>



 --
 Erik Schnetter  http://www.perimeterinstitute.
 ca/personal/eschnetter/

>>>
>
>
> --
> Erik Schnetter  http://www.perimeterinstitute.
> ca/personal/eschnetter/
>

[julia-users] Re: RandIP A random IP generator for Large scale network mapping.

[Páll Haraldsson]

On Tuesday, August 30, 2016 at 5:26:29 AM UTC, Jacob Yates wrote:
>
> I've been working on porting a script I wrote in python to julia and have 
> been having some issues with the script freezing.
>
> So pretty much all this script does is generate a random IP address and 
> checks to see if its valid(the Python version will give http error codes) 
> then logs the results for further analysis.
>
> function gen_ip()
> ip = Any[]
> for i in rand(1:255, 4)
> push!(ip, i)
> end
> global ipaddr = join(ip, ".")
> end
>
 
[..]

println("Bactrace: ", backtrace())
>

Note, there is a type for IP addresses, done like: ip"127.0.0.1" (should 
also work for IPv6) or:

gen_ip() = IPv4(rand(0:256^4-1)) #not sure why you excluded 0 in 1:255 
(might want to exclude some IPs but not as much as you did?), or used 
global.

http://docs.julialang.org/en/release-0.4/manual/networking-and-streams/

Generally global is bad form, and I'm not sure, but it might have something 
to do with @async not working, as I guess it's not "thread-safe" or 
related..

-- 
Palli.

 
 

Re: [julia-users] Re: Adding items into a tuple

[Tim Holy]

On Tuesday, August 30, 2016 3:57:36 PM CDT Yichao Yu wrote:
> And even then, this completely looses the advantage of using tuple
> (inferrable size and element types) so you shouldn't do this in general
> unless you are going to do a lot of work with the tiny tuple afterwards.

Right. If you want to grow a tuple, you should use "lispy recursion" so the 
compiler can reason about the size, which is part of the type of the tuple. 
(`for` loops are completely off the table for this kind of programming.) 
Here's an example that builds a tuple of N `true`s (i.e., functionally 
equivalent to `ntuple(d->true, N)`):

buildtrues{N}(::Type{Val{N}}) = _buildtrues((), Val{N}) # initialization

_buildtrues{N}(out::NTuple{N}, ::Type{Val{N}}) = out   # termination

@inline _buildtrues{N}(out, ::Type{Val{N}}) =
_buildtrues((out..., true), Val{N})   # the "inner loop"

Key features include the `@inline` and the fact that `N` is available to the 
type system. For a particular inferrable`N<15`, the compiler will just figure 
out the end result and return that; the "apparent" recursion runs at compile 
time, not runtime.

Note that if your condition isn't evaluatable at compile time, and especially 
if it changes from one "iteration" to the next, then you're frankly much 
better off using Arrays rather than tuples. Don't attempt to fake compile-time 
evaluation using `Val` unless the condition is already embedded in the type 
system (e.g., coming from an `::Array{T,N}`) or it's the same `Val` being used 
bazillions of times. See http://docs.julialang.org/en/latest/manual/
performance-tips/#types-with-values-as-parameters and the section after that.

This is not just a theoretical concern: while tuples have enormous advantages 
in certain settings, trying to fake this with tuples/Val and getting it wrong 
could easily result in a 30-100 fold performance *penalty* compared to using 
Arrays.

Best,
--Tim

Re: [julia-users] Re: Adding items into a tuple

[Alexei Serdiuk]

Thank you very much, it works!

I'll explain you why I want to use tuples.
I have a range of products 1:N. Every product can be made of several 
mutually exclusive bases. I need to make a lot of loop of this kind

for k in Products, b in Bases
  if accordance[k,b]=true
  ...
end; end



I want to create an array of tulpes, so that I could decrease my loops to

for k in Products, b in Bases[k]

where Bases[1] is the tuple of bases for product[1]

I could use an array, but one product can correspond to different number of 
bases.

That's why I decided to use tuples.

Thanks again!

вторник, 30 августа 2016 г., 10:58:00 UTC+3 пользователь Yichao Yu написал:
>
>
>
> On Tue, Aug 30, 2016 at 3:55 PM, Yichao Yu  > wrote:
>
>>
>>
>> On Tue, Aug 30, 2016 at 3:39 PM, Kristoffer Carlsson > > wrote:
>>
>>> t = ()
>>> condition = true
>>> for i = 1:N
>>> if condition==true
>>> t = (t..., i)
>>>
>>
>> Note that you shouldn't do this unless you only have very few elements.
>>
>
> And even then, this completely looses the advantage of using tuple 
> (inferrable size and element types) so you shouldn't do this in general 
> unless you are going to do a lot of work with the tiny tuple afterwards.
>  
>
>>  
>>
>>> end
>>> end
>>>
>>>
>>> This does not modify the tuple but replaces it with a new longer one. 
>>>
>>> On Tuesday, August 30, 2016 at 7:26:29 AM UTC+2, Alexei Serdiuk wrote:

 Hi,

 I need to choose items from a range (1:N) according to some condition 
 and to push them into a tuple.

 I understand how to add items into an array:
 t=Array()
 for i = 1:N
  if condition==true
   push!(t, i)
 end; end

 Is there a way to push them into a tuple?

 Thanks.

>>>
>>
>

Re: [julia-users] Running Julia in Ubuntu

[Tim Holy]

I'm rather surprised that building from source gave you version 0.4.7; I don't 
think that makes sense (it should have been master). Are you sure you know 
what you're running? (Try `.julia` from the directory in which you built 
julia, rather than just `julia`.)

Best,
--Tim

On Monday, August 29, 2016 3:30:34 PM CDT Angshuman Goswami wrote:
> I was running Julia to run my MPC code. I needed to upgrade and hence i
> deleted the folder i cloned from git hub. Now I have two problems:
> 
> 1) Installing julia by sudo get-apt install julia, I get the following
> message:
> 
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package julia is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> 
> E: Package 'julia' has no installation candidate
> 
> 2) When I cloned the github link by  git clone
> https://github.com/JuliaLang/julia.git
> 
> I tried make -j N
> 
> it didn't work
> 
> 3) I then used
> 
> git pull && make
> 
> Now Julia was updated to 0.4.7
> And now I thought it will finally work.
> But now when I do i) using PyCall or  ii) using RobotOS
> I get the following error:
> julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*,
> bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.
> 
> signal (6): Aborted
> ERROR: LoadError: Failed to precompile PyCall to
> /home/odroid/.julia/lib/v0.4/PyCall.ji while loading
> /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in expression starting on
> line 3
> 
> M stuck

[julia-users] Re: Running Julia in Ubuntu

[Kaj Wiik]

I have been using the third route very successfully:

Download the binary from e.g.
https://julialang.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-linux-x86_64.tar.gz
or
https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc3-linux-x86_64.tar.gz

(http://julialang.org/downloads/)

cd /opt

sudo tar xzvf tarball.tgz
A directory like julia-2e358ce975 will be created.

Then make a symlink
sudo ln -s /opt/julia-2e358ce975/bin/julia /usr/local/bin

That's it, very easy.

Note that now you can support multiple versions by making symlinks e.g. 
julia-v0.5, that's not possible (or very difficult) with the deb packages.

Cheers,
Kaj


On Tuesday, August 30, 2016 at 8:26:29 AM UTC+3, Angshuman Goswami wrote:
>
> I was running Julia to run my MPC code. I needed to upgrade and hence i 
> deleted the folder i cloned from git hub. Now I have two problems:
>
> 1) Installing julia by sudo get-apt install julia, I get the following 
> message:
>
> Reading package lists... Done
> Building dependency tree   
> Reading state information... Done
> Package julia is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
>
> E: Package 'julia' has no installation candidate
>
> 2) When I cloned the github link by  git clone 
> https://github.com/JuliaLang/julia.git
>
> I tried make -j N
>
> it didn't work
>
> 3) I then used 
>
> git pull && make
>
> Now Julia was updated to 0.4.7 
> And now I thought it will finally work.
> But now when I do i) using PyCall or  ii) using RobotOS
> I get the following error:
> julia: codegen.cpp:3155: llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, 
> bool, bool): Assertion `ctx->gensym_assigned.at(idx)' failed.
>
> signal (6): Aborted
> ERROR: LoadError: Failed to precompile PyCall to 
> /home/odroid/.julia/lib/v0.4/PyCall.ji
> while loading /home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in expression 
> starting on line 3
>
> M stuck
>
>

Re: [julia-users] Re: Adding items into a tuple

[Yichao Yu]

On Tue, Aug 30, 2016 at 3:55 PM, Yichao Yu  wrote:

>
>
> On Tue, Aug 30, 2016 at 3:39 PM, Kristoffer Carlsson <
> kcarlsso...@gmail.com> wrote:
>
>> t = ()
>> condition = true
>> for i = 1:N
>> if condition==true
>> t = (t..., i)
>>
>
> Note that you shouldn't do this unless you only have very few elements.
>

And even then, this completely looses the advantage of using tuple
(inferrable size and element types) so you shouldn't do this in general
unless you are going to do a lot of work with the tiny tuple afterwards.


>
>
>> end
>> end
>>
>>
>> This does not modify the tuple but replaces it with a new longer one.
>>
>> On Tuesday, August 30, 2016 at 7:26:29 AM UTC+2, Alexei Serdiuk wrote:
>>>
>>> Hi,
>>>
>>> I need to choose items from a range (1:N) according to some condition
>>> and to push them into a tuple.
>>>
>>> I understand how to add items into an array:
>>> t=Array()
>>> for i = 1:N
>>>  if condition==true
>>>   push!(t, i)
>>> end; end
>>>
>>> Is there a way to push them into a tuple?
>>>
>>> Thanks.
>>>
>>
>

Re: [julia-users] Re: Adding items into a tuple

[Yichao Yu]

On Tue, Aug 30, 2016 at 3:39 PM, Kristoffer Carlsson 
wrote:

> t = ()
> condition = true
> for i = 1:N
> if condition==true
> t = (t..., i)
>

Note that you shouldn't do this unless you only have very few elements.


> end
> end
>
>
> This does not modify the tuple but replaces it with a new longer one.
>
> On Tuesday, August 30, 2016 at 7:26:29 AM UTC+2, Alexei Serdiuk wrote:
>>
>> Hi,
>>
>> I need to choose items from a range (1:N) according to some condition and
>> to push them into a tuple.
>>
>> I understand how to add items into an array:
>> t=Array()
>> for i = 1:N
>>  if condition==true
>>   push!(t, i)
>> end; end
>>
>> Is there a way to push them into a tuple?
>>
>> Thanks.
>>
>

[julia-users] Re: Adding items into a tuple

[Kristoffer Carlsson]

t = ()
condition = true
for i = 1:N
if condition==true
t = (t..., i)
end
end


This does not modify the tuple but replaces it with a new longer one. 

On Tuesday, August 30, 2016 at 7:26:29 AM UTC+2, Alexei Serdiuk wrote:
>
> Hi,
>
> I need to choose items from a range (1:N) according to some condition and 
> to push them into a tuple.
>
> I understand how to add items into an array:
> t=Array()
> for i = 1:N
>  if condition==true
>   push!(t, i)
> end; end
>
> Is there a way to push them into a tuple?
>
> Thanks.
>

Re: [julia-users] Running Julia in Ubuntu

[Henri Girard]

WHICH UBUNTU VERSION ?


Le 30/08/2016 à 00:30, Angshuman Goswami a écrit :
I was running Julia to run my MPC code. I needed to upgrade and hence 
i deleted the folder i cloned from git hub. Now I have two problems:


1) Installing julia by sudo get-apt install julia, I get the following 
message:


Reading package lists... Done
Building dependency tree
Reading state information... Done
Package julia is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'julia' has no installation candidate

2) When I cloned the github link by  git clone 
https://github.com/JuliaLang/julia.git


I tried make -j N

it didn't work

3) I then used
|git pull && make Now Julia was updated to 0.4.7 And now I thought it 
will finally work. But now when I do i) using PyCall or ii) using 
RobotOS I get the following error: julia: codegen.cpp:3155: 
llvm::Value* emit_expr(jl_value_t*, jl_codectx_t*, bool, bool): 
Assertion `ctx->gensym_assigned.at(idx)' failed. signal (6): Aborted 
ERROR: LoadError: Failed to precompile PyCall to 
/home/odroid/.julia/lib/v0.4/PyCall.ji while loading 
/home/odroid/.julia/v0.4/RobotOS/src/RobotOS.jl, in expression 
starting on line 3 M stuck |