[julia-users] Re: a floating point test for other's environments

[Jeffrey Sarnoff]

Thank you for running it.  My guess is that, for your environment, this is 
false:  VERSION >= v"0.4.0-dev+2823"
I am putting that test in now. If that is true -- pls let me know.

On Wednesday, September 9, 2015 at 2:17:02 AM UTC-4, Arch Call wrote:
>
> I am getting fma not defined!  It is probably related to line 22.
>
> On Wednesday, September 9, 2015 at 1:04:34 AM UTC-4, Jeffrey Sarnoff wrote:
>>
>> I wrote this  FloatTest 
>> 
>>  to 
>> find out if three things that should be true of numerics with Julia hold 
>> for the wide variety of working environments in use.
>> It is 40 lines -- just click 'raw', copy it and paste it into your REPL. 
>>  It is done immediately.
>> If a test fails, please note that with basic system infoas an 'issue' at 
>> the github site.
>>
>>
>>
>>
>>

Re: [julia-users] a floating point test for other's environments

[René Donner]

what is your versioninfo()?

Am 09.09.2015 um 08:17 schrieb Arch Call :

> I am getting fma not defined!  It is probably related to line 22.
> 
> On Wednesday, September 9, 2015 at 1:04:34 AM UTC-4, Jeffrey Sarnoff wrote:
> I wrote this  FloatTest to find out if three things that should be true of 
> numerics with Julia hold for the wide variety of working environments in use.
> It is 40 lines -- just click 'raw', copy it and paste it into your REPL.  It 
> is done immediately.
> If a test fails, please note that with basic system infoas an 'issue' at the 
> github site.
> 
> 
> 
> 

[julia-users] Re: a floating point test for other's environments

[Arch Call]

I am getting fma not defined!  It is probably related to line 22.

On Wednesday, September 9, 2015 at 1:04:34 AM UTC-4, Jeffrey Sarnoff wrote:
>
> I wrote this  FloatTest 
> 
>  to 
> find out if three things that should be true of numerics with Julia hold 
> for the wide variety of working environments in use.
> It is 40 lines -- just click 'raw', copy it and paste it into your REPL. 
>  It is done immediately.
> If a test fails, please note that with basic system infoas an 'issue' at 
> the github site.
>
>
>
>
>

[julia-users] a floating point test for other's environments

[Jeffrey Sarnoff]

I wrote this  FloatTest  to find 
out if three things that should be true of numerics with Julia hold for the 
wide variety of working environments in use.
It is 40 lines -- just click 'raw', copy it and paste it into your REPL. 
 It is done immediately.
If a test fails, please note that with basic system infoas an 'issue' at 
the github site.

[julia-users] Re: strange performance with boolean operators and sparse matrices

[Seth]

Ref: https://github.com/JuliaLang/julia/issues/13024

On Tuesday, September 8, 2015 at 5:17:57 PM UTC-7, Seth wrote:
>
> Thanks, James. Is this worthy of a github issue?
>
> On Tuesday, September 8, 2015 at 5:00:53 PM UTC-7, James Fairbanks wrote:
>>
>> #= This is the function from Base that is being used when you call & on 
 two sparse matrices 
 function ($f){S,T}(A::AbstractArray{S}, B::AbstractArray{T})
 F = similar(A, promote_op($F,S,T), promote_shape(size(A),size(B)))
 for i in eachindex(A,B)
 @inbounds F[i] = ($f)(A[i], B[i])
 end
 return F
 end

 # look at what eachindex does for SparseMatrixCSC
 a = sprand(10,10,0.3)
 @show collect(eachindex(a))
 100-element Array{CartesianIndex{2},1}:
  CartesianIndex{2}((1,1))  
  CartesianIndex{2}((2,1))  
  CartesianIndex{2}((3,1))  
  CartesianIndex{2}((4,1))  
  CartesianIndex{2}((5,1))  
  CartesianIndex{2}((6,1))  
  CartesianIndex{2}((7,1))  
  CartesianIndex{2}((8,1))  
  CartesianIndex{2}((9,1))  
  CartesianIndex{2}((10,1)) 
  CartesianIndex{2}((1,2))  
  CartesianIndex{2}((2,2))  
  CartesianIndex{2}((3,2))  
  CartesianIndex{2}((4,2))   
  ⋮ 
  CartesianIndex{2}((6,8))  
  CartesianIndex{2}((7,8))  
  CartesianIndex{2}((8,8))  
  CartesianIndex{2}((9,8))  
  CartesianIndex{2}((10,8)) 
  CartesianIndex{2}((1,9))  
  CartesianIndex{2}((2,9))  
  CartesianIndex{2}((3,9))  
  CartesianIndex{2}((4,9))  
  CartesianIndex{2}((5,9))  
  CartesianIndex{2}((6,9))  
  CartesianIndex{2}((7,9))  
  CartesianIndex{2}((8,9))  
  CartesianIndex{2}((9,9))  
  CartesianIndex{2}((10,9)) 
  CartesianIndex{2}((1,10)) 
  CartesianIndex{2}((2,10)) 
  CartesianIndex{2}((3,10)) 
  CartesianIndex{2}((4,10)) 
  CartesianIndex{2}((5,10)) 
  CartesianIndex{2}((6,10)) 
  CartesianIndex{2}((7,10)) 
  CartesianIndex{2}((8,10)) 
  CartesianIndex{2}((9,10)) 
  CartesianIndex{2}((10,10))

  The problem is that the generic AbstractArray implementation of & and 
>> some other operators is iterating over all the n x m entries of A. 
>>
>

[julia-users] Re: strange performance with boolean operators and sparse matrices

[Seth]

Thanks, James. Is this worthy of a github issue?

On Tuesday, September 8, 2015 at 5:00:53 PM UTC-7, James Fairbanks wrote:
>
> #= This is the function from Base that is being used when you call & on 
>>> two sparse matrices 
>>> function ($f){S,T}(A::AbstractArray{S}, B::AbstractArray{T})
>>> F = similar(A, promote_op($F,S,T), promote_shape(size(A),size(B)))
>>> for i in eachindex(A,B)
>>> @inbounds F[i] = ($f)(A[i], B[i])
>>> end
>>> return F
>>> end
>>>
>>> # look at what eachindex does for SparseMatrixCSC
>>> a = sprand(10,10,0.3)
>>> @show collect(eachindex(a))
>>> 100-element Array{CartesianIndex{2},1}:
>>>  CartesianIndex{2}((1,1))  
>>>  CartesianIndex{2}((2,1))  
>>>  CartesianIndex{2}((3,1))  
>>>  CartesianIndex{2}((4,1))  
>>>  CartesianIndex{2}((5,1))  
>>>  CartesianIndex{2}((6,1))  
>>>  CartesianIndex{2}((7,1))  
>>>  CartesianIndex{2}((8,1))  
>>>  CartesianIndex{2}((9,1))  
>>>  CartesianIndex{2}((10,1)) 
>>>  CartesianIndex{2}((1,2))  
>>>  CartesianIndex{2}((2,2))  
>>>  CartesianIndex{2}((3,2))  
>>>  CartesianIndex{2}((4,2))   
>>>  ⋮ 
>>>  CartesianIndex{2}((6,8))  
>>>  CartesianIndex{2}((7,8))  
>>>  CartesianIndex{2}((8,8))  
>>>  CartesianIndex{2}((9,8))  
>>>  CartesianIndex{2}((10,8)) 
>>>  CartesianIndex{2}((1,9))  
>>>  CartesianIndex{2}((2,9))  
>>>  CartesianIndex{2}((3,9))  
>>>  CartesianIndex{2}((4,9))  
>>>  CartesianIndex{2}((5,9))  
>>>  CartesianIndex{2}((6,9))  
>>>  CartesianIndex{2}((7,9))  
>>>  CartesianIndex{2}((8,9))  
>>>  CartesianIndex{2}((9,9))  
>>>  CartesianIndex{2}((10,9)) 
>>>  CartesianIndex{2}((1,10)) 
>>>  CartesianIndex{2}((2,10)) 
>>>  CartesianIndex{2}((3,10)) 
>>>  CartesianIndex{2}((4,10)) 
>>>  CartesianIndex{2}((5,10)) 
>>>  CartesianIndex{2}((6,10)) 
>>>  CartesianIndex{2}((7,10)) 
>>>  CartesianIndex{2}((8,10)) 
>>>  CartesianIndex{2}((9,10)) 
>>>  CartesianIndex{2}((10,10))
>>>
>>>  The problem is that the generic AbstractArray implementation of & and 
> some other operators is iterating over all the n x m entries of A. 
>

Re: [julia-users] Re: Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Ian Butterworth]

Ok. I'm running 0.4.0-dev on juliabox

I managed to make it work by removing the last 3 characters from the input 
string, and using single format characters.  
Also caught a bug that occurs when the hour in the string = 12, and it's 
PM. I believe that hour was incorrectly being shifted.. but then again, 
that was after my tweak of your example.

using Base.Dates 
ds = "2015/9/8 10:05:21 AM"
firstread = DateTime(ds[1:end-3], Dates.DateFormat("y/m/d H:M:S")) 
datetimetimeout = firstread + ((contains(ds, "PM") && hour(firstread) !=12) 
? Hour(12) : Hour(0)) 

Thanks for testing btw

[julia-users] Re: strange performance with boolean operators and sparse matrices

[James Fairbanks]

>
> #= This is the function from Base that is being used when you call & on 
>> two sparse matrices 
>> function ($f){S,T}(A::AbstractArray{S}, B::AbstractArray{T})
>> F = similar(A, promote_op($F,S,T), promote_shape(size(A),size(B)))
>> for i in eachindex(A,B)
>> @inbounds F[i] = ($f)(A[i], B[i])
>> end
>> return F
>> end
>>
>> # look at what eachindex does for SparseMatrixCSC
>> a = sprand(10,10,0.3)
>> @show collect(eachindex(a))
>> 100-element Array{CartesianIndex{2},1}:
>>  CartesianIndex{2}((1,1))  
>>  CartesianIndex{2}((2,1))  
>>  CartesianIndex{2}((3,1))  
>>  CartesianIndex{2}((4,1))  
>>  CartesianIndex{2}((5,1))  
>>  CartesianIndex{2}((6,1))  
>>  CartesianIndex{2}((7,1))  
>>  CartesianIndex{2}((8,1))  
>>  CartesianIndex{2}((9,1))  
>>  CartesianIndex{2}((10,1)) 
>>  CartesianIndex{2}((1,2))  
>>  CartesianIndex{2}((2,2))  
>>  CartesianIndex{2}((3,2))  
>>  CartesianIndex{2}((4,2))   
>>  ⋮ 
>>  CartesianIndex{2}((6,8))  
>>  CartesianIndex{2}((7,8))  
>>  CartesianIndex{2}((8,8))  
>>  CartesianIndex{2}((9,8))  
>>  CartesianIndex{2}((10,8)) 
>>  CartesianIndex{2}((1,9))  
>>  CartesianIndex{2}((2,9))  
>>  CartesianIndex{2}((3,9))  
>>  CartesianIndex{2}((4,9))  
>>  CartesianIndex{2}((5,9))  
>>  CartesianIndex{2}((6,9))  
>>  CartesianIndex{2}((7,9))  
>>  CartesianIndex{2}((8,9))  
>>  CartesianIndex{2}((9,9))  
>>  CartesianIndex{2}((10,9)) 
>>  CartesianIndex{2}((1,10)) 
>>  CartesianIndex{2}((2,10)) 
>>  CartesianIndex{2}((3,10)) 
>>  CartesianIndex{2}((4,10)) 
>>  CartesianIndex{2}((5,10)) 
>>  CartesianIndex{2}((6,10)) 
>>  CartesianIndex{2}((7,10)) 
>>  CartesianIndex{2}((8,10)) 
>>  CartesianIndex{2}((9,10)) 
>>  CartesianIndex{2}((10,10))
>>
>>  The problem is that the generic AbstractArray implementation of & and 
some other operators is iterating over all the n x m entries of A. 

Re: [julia-users] Re: Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Jacob Quinn]

I'm not seeing the error on the latest master

   _
   _   _ _(_)_ |  A fresh approach to technical computing
  (_) | (_) (_)|  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-pre+7400 (2015-09-08 22:42 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit bffe239* (0 days old master)
|__/   |  x86_64-apple-darwin14.4.0

julia> using Base.Dates

julia> ds = "2015-08-12 12:01:23 PM"
"2015-08-12 12:01:23 PM"

julia> DateTime(ds, DateFormat("-mm-dd HH:MM:SS")) - (contains(ds,
"AM") ? Hour(12) : Hour(0))
2015-08-12T12:01:23

julia>

On Tue, Sep 8, 2015 at 5:23 PM, Ian Butterworth 
wrote:

> Thanks for the tip, although I'm getting an error with your code in
> 0.4.0-dev
> Any idea why?
>
> using Base.Dates
> ds = "2015-08-12 12:01:23 PM"
> DateTime(ds, DateFormat("-mm-dd HH:MM:SS")) - (contains(ds, "AM") ?
> Hour(12) : Hour(0))
>
>
> LoadError: ArgumentError: Non-digit character encountered
> while loading In[51], in expression starting on line 3
>
>  in getslot at dates/io.jl:110
>  in parse at dates/io.jl:122
>  in call at dates/io.jl:164
>
>

Re: [julia-users] Re: Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Ian Butterworth]

Thanks for the tip, although I'm getting an error with your code in 
0.4.0-dev
Any idea why?

using Base.Dates 
ds = "2015-08-12 12:01:23 PM" 
DateTime(ds, DateFormat("-mm-dd HH:MM:SS")) - (contains(ds, "AM") ? Hour
(12) : Hour(0))  


LoadError: ArgumentError: Non-digit character encountered
while loading In[51], in expression starting on line 3

 in getslot at dates/io.jl:110
 in parse at dates/io.jl:122
 in call at dates/io.jl:164

[julia-users] Re: Good, short set of slides introducing Julia

[cdm]

maybe helpful, maybe not ...

some links from the main Julia page:

   http://nbviewer.ipython.org/url/julialang.org/benchmarks.ipynb

   http://julialang.org/benchmarks.csv



obviously, the most "famous" collection
of the benchmark data from the
"High-Performance JIT Compiler"

section of the main Julia page

   http://julialang.org/

has seen plenty of slide time ...



this deck is a bit dated, but maybe fine:

   http://www.slideshare.net/OReillyStrata/strata-santa-clara-february-2013


good luck !!!

[julia-users] Re: strange performance with boolean operators and sparse matrices

[Seth]

(and I realize that my code is really just resultmx[r,c] = !b[r,c] but I 
wanted to focus on the timing of equivalent boolean operators.)

On Tuesday, September 8, 2015 at 3:21:35 PM UTC-7, Seth wrote:
>
> Hi all,
>
> I ran into some puzzling performance today with sparse matrices. I defined
>
> _column(a::AbstractSparseArray, i::Integer) = sub(a.rowval, a.colptr[i]:a.
> colptr[i+1]-1)
>
> # material nonimplication
> ⊅(p::Bool, q::Bool) = p & !q 
>
> function ⊅(a::SparseMatrixCSC, b::SparseMatrixCSC) 
> (m,n) = size(a) 
> resultmx = spzeros(Bool,m,n) 
> for c = 1:n 
> for r in _column(a,n) 
> # info("row $r, col $c") 
> resultmx[r,c] = ⊅(a[r,c], b[r,c]) 
> end 
> end 
> return resultmx 
> end
>
> a = sprandbool(100,100,0.0001) 
> b = sprandbool(100,100,0.0001)
>
>
> and ran it in comparison with the & operator:
>
> julia> @time z = a ⊅ b;
>  15.272250 seconds (1.10 M allocations: 73.870 MB)
>
> julia> @time z = a & b;   # this is still going ~10 minutes later.
>
>
> It seems strange that my home-grown function is orders of magnitude more 
> efficient than a built-in boolean primitive. Am I missing something?
>

[julia-users] strange performance with boolean operators and sparse matrices

[Seth]

Hi all,

I ran into some puzzling performance today with sparse matrices. I defined

_column(a::AbstractSparseArray, i::Integer) = sub(a.rowval, a.colptr[i]:a.
colptr[i+1]-1)

# material nonimplication
⊅(p::Bool, q::Bool) = p & !q 

function ⊅(a::SparseMatrixCSC, b::SparseMatrixCSC) 
(m,n) = size(a) 
resultmx = spzeros(Bool,m,n) 
for c = 1:n 
for r in _column(a,n) 
# info("row $r, col $c") 
resultmx[r,c] = ⊅(a[r,c], b[r,c]) 
end 
end 
return resultmx 
end

a = sprandbool(100,100,0.0001) 
b = sprandbool(100,100,0.0001)


and ran it in comparison with the & operator:

julia> @time z = a ⊅ b;
 15.272250 seconds (1.10 M allocations: 73.870 MB)

julia> @time z = a & b;   # this is still going ~10 minutes later.


It seems strange that my home-grown function is orders of magnitude more 
efficient than a built-in boolean primitive. Am I missing something?

Re: [julia-users] Re: Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Pontus Stenetorp]

On 8 September 2015 at 22:31, Ian Butterworth  wrote:
>
> On Tuesday, 8 September 2015 17:27:25 UTC-4, Ian Butterworth wrote:
>>
>> I current use Calendar.jl to convert a datetime that contains AM/PM to the
>> datetime type like this:
>> datetime_string = "2015-08-12 12:01:23 PM"
>> timeout = unix2datetime(Calendar.parse(datetime_string,
>> timein).millis/1000)
>>
>> I'd like to do it without calendar.jl if possible.
>>
>> Any ideas?
>
> Sorry, error in example. Should read:
>
> datetime_string = "2015-08-12 12:01:23 PM"
> timeout = unix2datetime(Calendar.parse("-MM-dd hh:mm:ss aa",
> datetime_string).millis/1000)

As far as I am aware, there is no way to parse the AM and PM periods
with the functionality that we have in Base [1].  Currently I have
this ugly hack to handle data from a third-party database where for
some reason someone opted to store points in time in a 12-hour format.

using Base.Dates
ds = "2015-08-12 12:01:23 PM"
DateTime(ds, DateFormat("-mm-dd HH:MM:SS")) - (
contains(ds, "AM") ? Hour(12) : Hour(0))

Ideally AM/PM period handling should be added to Base and it would be
a nice and fairly atomic contribution if someone was to try to make it
happen.

Pontus

[1]: 
https://github.com/JuliaLang/julia/blob/c2527882686e15f949b7f460ba71d5bdeeb362c0/base/dates/io.jl#L71-L81

[julia-users] Re: Good, short set of slides introducing Julia

[Jeffrey Sarnoff]

I had thought they might have some specific slides that would be helpful. 
 In my experience, whatever the "common task," the audience and the 
intended result shape the contents of (some of) the presentation.  A quick 
overview of major ways that Julia advances the both the efficacy and the 
art of scientific programming is a presentation others will want to see.  
You may have looked at http://julialang.org/teaching/ and 
http://julialang.org/learning/ (tutorials) 00 it  is worth a second look 
with an eye toward what might make a good slide or two.
For your purpose these two pages are important performance-tips 
 and 
fast-numerics . There may 
be something already done that would work for you -- or not (dunno).  If 
you are comfortable with some manner of producing slides, give it a go.  I 
have found that organizing and integrating mulitsourced information leads 
to a more authoritative presentation.

If you go forward and want some feedback on the way -- let me know.



On Tuesday, September 8, 2015 at 5:28:04 PM UTC-4, andrew cooke wrote:
>
> i'd actually already looked at those (should have said, sorry) - they're 
> longer than what i was needing, but i could pick a few slides.  i just 
> wondered if what i was doing (basically, selling to scientific programmers 
> in < 10 mins) was a common task.
>
> andrew
>
> On Tuesday, 8 September 2015 18:18:25 UTC-3, Jeffrey Sarnoff wrote:
>>
>> Here is one place to look svaksha's list of slide presentations 
>> .
>>
>> For those who may not be familiar with details of copyright 
>> (a) written material is copyrighted even when there is no formal 
>> copyright statement
>> (b) it is the responsibility of the person who wants to use the 
>> material to get the author's permission
>> (sometimes that permission may accompany the material, e.g. 
>> "this material is placed in the public domain", or with some other 
>> permissive language)
>> (c) when using another person's written work, cite the person and 
>> identify the work so others can find it if they want
>>
>> The Julia community is more helpful and supportive than most; so, go 
>> ahead and ask, and let the person know you would cite their work.
>>
>>
>> On Tuesday, September 8, 2015 at 4:58:33 PM UTC-4, andrew cooke wrote:
>>>
>>>
>>> I need to give a presentation at work and was wondering is slides 
>>> already exist that:
>>>
>>>   * show how fast it is in benchmarks
>>>
>>>   * show that it's similar to matlab (matrix stuff)
>>>
>>>   * show that you can write fast inner loops
>>>
>>>  For bonus points:
>>>
>>>   * show how you can add other numerical types at no "cost"
>>>
>>>   * show how mutiple dispatch can be useful
>>>
>>>   * show how someone used to OO in, say, python, won't feel too lost
>>>
>>> Preferably just one slide per point.  Very short.
>>>
>>> Thanks,
>>> Andrew
>>>
>>>

[julia-users] Re: cleaning up objects in parallel processes?

[Seth]

Following up - how do I even begin to determine what's eating up memory on 
remote processes? Is there something out there I can use to get a report?

On Saturday, September 5, 2015 at 5:53:54 PM UTC-7, Seth wrote:
>
> I've finally made some progress in parallelizing my code. However, at the 
> end of the run, I have my answer in my main process (the REPL) and each 
> worker process has about 1GB of memory held. Is there a way to tell the 
> worker processes to free that memory? @everywhere gc() didn't seem to do 
> it, and I don't really know what it's from since the only thing that was 
> done on the worker processes was
>
> @sync @parallel for s in i
> state = dijkstra_shortest_paths_sparse(spmx, s, distmx, true)
> if endpoints
> _parallel_accumulate_endpoints!(betweenness, state, s)
> else
> _parallel_accumulate_basic!(betweenness, state, s)
> end
> end
>
>
>
> Every large structure I'm passing to the remote workers is some form of 
> shared array (spmx, distmx, betweenness). (The answer I need is in the 
> betweenness shared array.)
>
> Any ideas? Thank you.
>

[julia-users] Re: Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Ian Butterworth]

Sorry, error in example. Should read:

datetime_string = "2015-08-12 12:01:23 PM"
timeout = unix2datetime(Calendar.parse("-MM-dd hh:mm:ss aa", 
datetime_string).millis/1000)



On Tuesday, 8 September 2015 17:27:25 UTC-4, Ian Butterworth wrote:
>
> I current use Calendar.jl to convert a datetime that contains AM/PM to the 
> datetime type like this:
> datetime_string = "2015-08-12 12:01:23 PM"
> timeout = unix2datetime(Calendar.parse(datetime_string, timein).millis/
> 1000)
>
> I'd like to do it without calendar.jl if possible.
>
> Any ideas?
>

[julia-users] Re: Good, short set of slides introducing Julia

[andrew cooke]

i'd actually already looked at those (should have said, sorry) - they're 
longer than what i was needing, but i could pick a few slides.  i just 
wondered if what i was doing (basically, selling to scientific programmers 
in < 10 mins) was a common task.

andrew

On Tuesday, 8 September 2015 18:18:25 UTC-3, Jeffrey Sarnoff wrote:
>
> Here is one place to look svaksha's list of slide presentations 
> .
>
> For those who may not be familiar with details of copyright 
> (a) written material is copyrighted even when there is no formal 
> copyright statement
> (b) it is the responsibility of the person who wants to use the 
> material to get the author's permission
> (sometimes that permission may accompany the material, e.g. 
> "this material is placed in the public domain", or with some other 
> permissive language)
> (c) when using another person's written work, cite the person and 
> identify the work so others can find it if they want
>
> The Julia community is more helpful and supportive than most; so, go ahead 
> and ask, and let the person know you would cite their work.
>
>
> On Tuesday, September 8, 2015 at 4:58:33 PM UTC-4, andrew cooke wrote:
>>
>>
>> I need to give a presentation at work and was wondering is slides already 
>> exist that:
>>
>>   * show how fast it is in benchmarks
>>
>>   * show that it's similar to matlab (matrix stuff)
>>
>>   * show that you can write fast inner loops
>>
>>  For bonus points:
>>
>>   * show how you can add other numerical types at no "cost"
>>
>>   * show how mutiple dispatch can be useful
>>
>>   * show how someone used to OO in, say, python, won't feel too lost
>>
>> Preferably just one slide per point.  Very short.
>>
>> Thanks,
>> Andrew
>>
>>

[julia-users] Convert datetime string that has AM/PM to datetime, without using Calendar.jl

[Ian Butterworth]

I current use Calendar.jl to convert a datetime that contains AM/PM to the 
datetime type like this:
datetime_string = "2015-08-12 12:01:23 PM"
timeout = unix2datetime(Calendar.parse(datetime_string, timein).millis/1000)

I'd like to do it without calendar.jl if possible.

Any ideas?

[julia-users] Re: Good, short set of slides introducing Julia

[Jeffrey Sarnoff]

Here is one place to look svaksha's list of slide presentations 
.

For those who may not be familiar with details of copyright 
(a) written material is copyrighted even when there is no formal 
copyright statement
(b) it is the responsibility of the person who wants to use the 
material to get the author's permission
(sometimes that permission may accompany the material, e.g. 
"this material is placed in the public domain", or with some other 
permissive language)
(c) when using another person's written work, cite the person and 
identify the work so others can find it if they want

The Julia community is more helpful and supportive than most; so, go ahead 
and ask, and let the person know you would cite their work.


On Tuesday, September 8, 2015 at 4:58:33 PM UTC-4, andrew cooke wrote:
>
>
> I need to give a presentation at work and was wondering is slides already 
> exist that:
>
>   * show how fast it is in benchmarks
>
>   * show that it's similar to matlab (matrix stuff)
>
>   * show that you can write fast inner loops
>
>  For bonus points:
>
>   * show how you can add other numerical types at no "cost"
>
>   * show how mutiple dispatch can be useful
>
>   * show how someone used to OO in, say, python, won't feel too lost
>
> Preferably just one slide per point.  Very short.
>
> Thanks,
> Andrew
>
>

[julia-users] Good, short set of slides introducing Julia

[andrew cooke]

I need to give a presentation at work and was wondering is slides already 
exist that:

  * show how fast it is in benchmarks

  * show that it's similar to matlab (matrix stuff)

  * show that you can write fast inner loops

 For bonus points:

  * show how you can add other numerical types at no "cost"

  * show how mutiple dispatch can be useful

  * show how someone used to OO in, say, python, won't feel too lost

Preferably just one slide per point.  Very short.

Thanks,
Andrew

[julia-users] Re: Could someone explain ordering on tuples?

[Seth]

Gotcha - thanks for confirming. I guess I'll roll my own function to 
compare matrix sizes.

On Tuesday, September 8, 2015 at 12:44:12 PM UTC-7, Kristoffer Carlsson 
wrote:
>
> It is based on the first element, if these are equal, compare the next etc 
> until you find an element that does not match. This is consistent with how 
> Python compares tuples. It is also consistent with how C++ sorts Pairs.
>
> On Tuesday, September 8, 2015 at 9:30:32 PM UTC+2, Seth wrote:
>>
>> I *think *it's based on the first element:
>>
>> julia> (2,2) < (3,3)   # this makes intuitive sense
>> true
>>
>> julia> (2,2) < (1,3)   # this makes intuitive sense
>> false
>>
>> julia> (2,2) < (3,1)   # this is somewhat confusing
>> true
>>
>> but it might be nice to have pairwise comparisons, so that, for example, 
>> one can tell whether a matrix can "fit" inside another matrix across all 
>> dimensions via comparison of each matrix's size (in this case, the last 
>> result would be false).
>>
>> I'm sure there's a reason the ordering is the way it is, though - does 
>> anyone have any insight into what applications rely on this sort of 
>> behavior? I'm probably overlooking something very basic since I'm focused 
>> on a particular issue right now.
>>
>> Thanks for any insight.
>>
>

[julia-users] Re: Could someone explain ordering on tuples?

[Kristoffer Carlsson]

It is based on the first element, if these are equal, compare the next etc 
until you find an element that does not match. This is consistent with how 
Python compares tuples. It is also consistent with how C++ sorts Pairs.

On Tuesday, September 8, 2015 at 9:30:32 PM UTC+2, Seth wrote:
>
> I *think *it's based on the first element:
>
> julia> (2,2) < (3,3)   # this makes intuitive sense
> true
>
> julia> (2,2) < (1,3)   # this makes intuitive sense
> false
>
> julia> (2,2) < (3,1)   # this is somewhat confusing
> true
>
> but it might be nice to have pairwise comparisons, so that, for example, 
> one can tell whether a matrix can "fit" inside another matrix across all 
> dimensions via comparison of each matrix's size (in this case, the last 
> result would be false).
>
> I'm sure there's a reason the ordering is the way it is, though - does 
> anyone have any insight into what applications rely on this sort of 
> behavior? I'm probably overlooking something very basic since I'm focused 
> on a particular issue right now.
>
> Thanks for any insight.
>

[julia-users] Could someone explain ordering on tuples?

[Seth]

I *think *it's based on the first element:

julia> (2,2) < (3,3)   # this makes intuitive sense
true

julia> (2,2) < (1,3)   # this makes intuitive sense
false

julia> (2,2) < (3,1)   # this is somewhat confusing
true

but it might be nice to have pairwise comparisons, so that, for example, 
one can tell whether a matrix can "fit" inside another matrix across all 
dimensions via comparison of each matrix's size (in this case, the last 
result would be false).

I'm sure there's a reason the ordering is the way it is, though - does 
anyone have any insight into what applications rely on this sort of 
behavior? I'm probably overlooking something very basic since I'm focused 
on a particular issue right now.

Thanks for any insight.

[julia-users] Re: [ANN] ForwardDiff.jl v0.1.0 Released

[Jarrett Revels]

>
> For the latter, you would need to be able to take linear combinations of 
> epsilons. Is that currently possible?
>

If I correctly understand what you're saying, then yes. See the Types 
section of the notebook I previously linked 
 (the 
types described will be documented in more detail soon).

On Sunday, September 6, 2015 at 7:14:09 PM UTC-4, Eric Forgy wrote:
>
> I like this. I think AD can be extended in a fairly straightforward manner 
> to stochastic differentials, e.g. Ito formula. Has anybody looked into 
> this? That could be interesting for finance applications.
>
> This could also be interesting for use in other differential algebras. In 
> particular, extending it to higher degree differential forms could be nice.
>
> For the latter, you would need to be able to take linear combinations of 
> epsilons. Is that currently possible?
>
> For example,
>
> e = e1 + e2
>
> e^2 = e1*e2 + e2*e1 = 0
>
> => 
>
> e1*e2 = -e2*e1
>
> That would be cool.
>
>

Re: [julia-users] Re: The new Dict syntax in 0.4 is very verbose

[Michael Turok]

That is elegantif I only could follow how it works

On Thursday, September 3, 2015 at 4:40:54 PM UTC-4, Mike Innes wrote:
>
> FWIW I mocked up a json syntax macro:
>
> using MacroTools, Lazy
>
> import MacroTools: prewalk
>
> function prockey(key)
>   @capture(key, (a_:b_) | (a_=>b_)) || error("Invalid json key $key")
>   isa(a, Symbol) && (a = Expr(:quote, a))
>   :($a=>$b)
> end
>
> function procmap(d)
>   @capture(d, {xs__}) || return d
>   :(Dict{Any, Any}($(map(prockey, xs)...)))
> end
>
> macro json(ex)
>   @>> ex prewalk(procmap) esc
> end
>
> Michael's original example becomes:
>
> data = @json {
> displayrows: 20,
> cols: [
> { col: "l1" },
> { col: "l2" },
> { col: "l3" },
> { col: "num", display: true },
> { col: "sum", display: true, conf: { style: 1, func: { 
> method: "sum", col: "num"  } } }
> ]
>   # ...
> }
>
> You might argue that it's actually nicer than the original.
>
> On Thu, 3 Sep 2015 at 21:22 Scott Jones  > wrote:
>
>> Another use is marking off the keyword arguments or parameters.
>>
>>
>> On Thursday, September 3, 2015 at 3:11:34 PM UTC-4, Jonathan Malmaud 
>> wrote:
>>>
>>> What are the other uses of ; in Julia? I can only think of suppressing 
>>> output on the REPL and separating expressions on a single line - neither 
>>> seems inconsistent or really related at all to the use within []. 
>>>
>>> On Thursday, September 3, 2015 at 3:06:22 PM UTC-4, Scott Jones wrote:



 On Thursday, September 3, 2015 at 1:47:07 PM UTC-4, Sean Marshallsay 
 wrote:
>
> [1:10;] is simply a consequence of matrix literal syntax (like [1:10; 
> 11:20]) and gets translated into vcat(1:10). It might be a bit confusing 
> but there's no point in making it a special case
>

 Yes, I understand that, and that too is not consistent with the use of 
 ; outside of [ ].
 spaces, colon, semicolon, and commas are all treated specially instead 
 of [ ], which can be rather confusing.
 Some of that special behavior is being deprecated, but some remains.
  

>>>

[julia-users] Re: cleaning up objects in parallel processes?

[Seth]

Yes, but it's small - it's a type with a couple of vectors that won't 
exceed the number of vertices in the graph.

On Monday, September 7, 2015 at 2:04:58 AM UTC-7, Nils Gudat wrote:
>
> Aren't you locally creating state on each of the worker processes?
>

[julia-users] ANN: SoftConfidenceWeighted.jl.

[Takeshi ISHITA]

New release of Softconfidenceweighted.jl 

I just implemented Exact Soft Confidence Weighted 
(SCW) in Julia, and published as a 
package Softconfidenceweighted.jl 
 (still not 
registered, but will be soon). 

SoftConfidenceWeighted.jl performs online binary linear classification with 
these advantages:

   - 
   
   Large margin training 
   - 
   
   Confidence weighting 
   - 
   
   Capability to handle non-separable data 
   - 
   
   Adaptive margin 
   
Since SCW is an online learning algorithm, this also has the 
characteristics such as rapid execution with low memory usage. 
Performance comparison with LinearSVC in scikit-learn 

All code used in this experiment is here 
.
 


I just compared the performance of LinearSVC in scikit-learn and SCW in 
this package, using the dataset generated by generate_dataset.py in my 
repository. 
Result 

LinearSVC 


$python3 linearsvc.py  

  136 function calls in 52.177 seconds  

  Accuracy: 0.9203

SCW 


$julia profile.jl

  5.785278 seconds (787.71 k allocations: 501.247 MB, 0.32% gc time)  

  Accuracy: 0.90405

LinearSVC is faster than SCW on small datasets, but the execution of SCW 
becomes much faster compare to LinearSVC as the number of samples 
increases. 
Environment 

Julia: Version 0.4.0-dev

OS: GNU/Linux

CPU: Intel Core i5


Feedback and contributions are appreciated.

--Ishita
​

Re: [julia-users] Adding backslashes to a string fails

[amiksvi]

Oh God, you're right, I don't know if I'm more ashamed or mad to have lost 
so much time on that, thank you very much anyway!

Le mardi 8 septembre 2015 16:11:44 UTC+2, Joshua Ballanco a écrit :
>
> I think you’re just seeing the REPL printing the String object, which 
> represents backslashes as escaped. Printing the result should clarify 
> things:
>

[julia-users] Re: automatic conversion Pair to Tuple -- failure of type inference?

[Simon Kornblith]

There is no conversion from Pair to Tuple. The construction:

(a, b, c) = d

works for any iterable collection d. The same holds for the for loop 
construction.

There used to be a type inference issue, but I fixed it in 
https://github.com/JuliaLang/julia/pull/12493. The output of code_warntype 
is a little wonky but you can see it knows the types of k and v here:

  s1 = (Base.box)(Base.Int,(Base.add_int)(s1::Int64,k::Int64)::Any)::
Int64 # none, line 7:
  s2 = (Base.box)(Base.Float64,(Base.add_float)(s2::Float64,v::Float64
)::Any)::Float64

The LLVM IR also indicates it's being optimized correctly.

Simon

On Tuesday, September 8, 2015 at 10:10:21 AM UTC-4, vav...@uwaterloo.ca 
wrote:
>
> The following code loops over a Dict:
>
> function test_paircvt1()
> d = Dict(1=>5.5, 3=>-2.2)
> s1 = 0
> s2 = 0.0
> for (k,v) in d
> s1 += k
> s2 += v
> end
> s1, s2
> end
>   
> Two issues to ask about:
>
> (1) Since eltype(d)==Pair{Int,Float64}, Julia is automatically converting 
> Pair{Int,Float64} to Tuple{Int,Float64}.  How is this done?  I looked for 
> the appropriate convert() method but couldn't find it. 
>
> (2)  I ran @code_warntype on the above segment and found something 
> worrisome: the compiler apparently thinks that both k and v are of type 
> Union{Int,Float64}.  Why can't it infer the correct types?  Is there a 
> performance loss from the compiler's inability to know the correct types?
>
> The reason I ask is that I have developed SortedDict for the 
> DataStructures.jl package, and currently eltype{SortedDict{K,V}) is defined 
> to be Tuple{K,V} rather than Pair{K,V} so it is incompatible with Dict. 
>  I'm trying to understand the relevant issues in order to fix this.  I am 
> running Julia 0.4, 15-day-old master.
>
> Thanks,
> Steve Vavasis
>
>

Re: [julia-users] Adding backslashes to a string fails

[Joshua Ballanco]

I think you’re just seeing the REPL printing the String object, which 
represents backslashes as escaped. Printing the result should clarify things:

julia> replace("my_beautiful_string", "_", "\\_")
"my\\_beautiful\\_string"

julia> println(replace("my_beautiful_string", "_", "\\_"))
my\_beautiful\_string



On September 8, 2015 at 17:07:56, amik...@gmail.com (amik...@gmail.com) wrote:

Hi all,

I'm trying to do a simple thing, adding a backslash before every underscore in 
a string, ex: my_beautiful_string should become my\_beautiful\_string.
To do that, I tried:

julia> replace("my_beautiful_string", "_", "\_")
"my_beautiful_string"

julia> replace("my_beautiful_string", "_", "\\_")
"my\\_beautiful\\_string"

I looked it up but I didn't find an answer, any suggestion?

[julia-users] automatic conversion Pair to Tuple -- failure of type inference?

[vavasis]

The following code loops over a Dict:

function test_paircvt1()
d = Dict(1=>5.5, 3=>-2.2)
s1 = 0
s2 = 0.0
for (k,v) in d
s1 += k
s2 += v
end
s1, s2
end
  
Two issues to ask about:

(1) Since eltype(d)==Pair{Int,Float64}, Julia is automatically converting 
Pair{Int,Float64} to Tuple{Int,Float64}.  How is this done?  I looked for 
the appropriate convert() method but couldn't find it. 

(2)  I ran @code_warntype on the above segment and found something 
worrisome: the compiler apparently thinks that both k and v are of type 
Union{Int,Float64}.  Why can't it infer the correct types?  Is there a 
performance loss from the compiler's inability to know the correct types?

The reason I ask is that I have developed SortedDict for the 
DataStructures.jl package, and currently eltype{SortedDict{K,V}) is defined 
to be Tuple{K,V} rather than Pair{K,V} so it is incompatible with Dict. 
 I'm trying to understand the relevant issues in order to fix this.  I am 
running Julia 0.4, 15-day-old master.

Thanks,
Steve Vavasis

[julia-users] Adding backslashes to a string fails

[amiksvi]

Hi all,

I'm trying to do a simple thing, adding a backslash before every underscore 
in a string, ex: my_beautiful_string should become my\_beautiful\_string.
To do that, I tried:

julia> replace("my_beautiful_string", "_", "\_")
"my_beautiful_string"

julia> replace("my_beautiful_string", "_", "\\_")
"my\\_beautiful\\_string"

I looked it up but I didn't find an answer, any suggestion?

Re: [julia-users] Re: rownames in data matrices

[Michael Krabbe Borregaard]

Interesting to follow that discussion, thanks. I can see the philosophical
arguments not to, though I think rownames are intuitive and nice.

On Tue, Sep 8, 2015 at 2:37 PM, Cedric St-Jean 
wrote:

> DataFrame behavior has been discussed many times, eg.
> https://groups.google.com/d/msg/julia-users/8UFnEIfIW0k/QNEustV9BQAJ.
> Short answer: having row names is considered, but a bit of a philosophical
> difference, so it's not guaranteed to happen at some point.
>
> On Tuesday, September 8, 2015 at 7:59:24 AM UTC-4, Michael Borregaard
> wrote:
>>
>> Thanks, it looks like that package will do the trick!
>>
>> On Tue, Sep 8, 2015 at 1:33 PM, Tamas Papp  wrote:
>>
>>> AFAIK https://github.com/davidavdav/NamedArrays.jl already does this and
>>> is maintained actively.
>>>
>>> Best,
>>>
>>> Tamas
>>>
>>> On Tue, Sep 08 2015, Andreas Lobinger  wrote:
>>>
>>> > Hello colleague,
>>> >
>>> > in the first order i think this could be emulated by a dictionary
>>> mapping
>>> > the row name to an index into a matrix or DataFrame.
>>> > Afaics calling this a  'misfeature' comes from trying to make a matrix
>>> > datatype that has row names by default and many people with
>>> > numerics/engineering background reserve the name matrix for the
>>> simplest
>>> > possible form: rectangular array with single number entries and
>>> integer row
>>> > and column indexing.
>>> >
>>> > So what you look for: a rectangular collection accessible with both
>>> row and
>>> > column index as names is something new and should have different name.
>>> You
>>> > could browse the dataFrames development and see if there are enough
>>> hooks
>>> > to extend this.
>>> >
>>> > Bringing this into julia as package (written in julia) should not be
>>> that
>>> > complicated if defined clearly (but still, someone is needed to
>>> implement).
>>>
>>>
>>

Re: [julia-users] Re: rownames in data matrices

[Cedric St-Jean]

DataFrame behavior has been discussed many times, eg. 
https://groups.google.com/d/msg/julia-users/8UFnEIfIW0k/QNEustV9BQAJ. Short 
answer: having row names is considered, but a bit of a philosophical 
difference, so it's not guaranteed to happen at some point.

On Tuesday, September 8, 2015 at 7:59:24 AM UTC-4, Michael Borregaard wrote:
>
> Thanks, it looks like that package will do the trick!
>
> On Tue, Sep 8, 2015 at 1:33 PM, Tamas Papp 
> > wrote:
>
>> AFAIK https://github.com/davidavdav/NamedArrays.jl already does this and
>> is maintained actively.
>>
>> Best,
>>
>> Tamas
>>
>> On Tue, Sep 08 2015, Andreas Lobinger > 
>> wrote:
>>
>> > Hello colleague,
>> >
>> > in the first order i think this could be emulated by a dictionary 
>> mapping
>> > the row name to an index into a matrix or DataFrame.
>> > Afaics calling this a  'misfeature' comes from trying to make a matrix
>> > datatype that has row names by default and many people with
>> > numerics/engineering background reserve the name matrix for the simplest
>> > possible form: rectangular array with single number entries and integer 
>> row
>> > and column indexing.
>> >
>> > So what you look for: a rectangular collection accessible with both row 
>> and
>> > column index as names is something new and should have different name. 
>> You
>> > could browse the dataFrames development and see if there are enough 
>> hooks
>> > to extend this.
>> >
>> > Bringing this into julia as package (written in julia) should not be 
>> that
>> > complicated if defined clearly (but still, someone is needed to 
>> implement).
>>
>>
>

[julia-users] API deployments on JuliaBox

[Viral Shah]

Folks,

We are excited to bring to you a new feature on JuliaBox. This is alpha, and 
mainly being announced for you to try out and see if there is further interest.

In your settings page, you should see a new section on APIs. Basically, you can 
paste a small snippet of code in there and define an API endpoint. Every time 
anyone hits that API then, your code will run in its own docker container. The 
containers are created on demand (so the first use is slow), and then things 
will be smooth afterwards until there is a period of inactivity when the 
container shuts down.

I personally love it, and Tanmay has made this possible. The settings page also 
has a link to some examples you can try out. We need to do a lot more polishing 
to make this a regular feature, but figured we’ll put it out here for comments 
and to gauge interest.

-viral

Re: [julia-users] Re: rownames in data matrices

[Michael Krabbe Borregaard]

Thanks, it looks like that package will do the trick!

On Tue, Sep 8, 2015 at 1:33 PM, Tamas Papp  wrote:

> AFAIK https://github.com/davidavdav/NamedArrays.jl already does this and
> is maintained actively.
>
> Best,
>
> Tamas
>
> On Tue, Sep 08 2015, Andreas Lobinger  wrote:
>
> > Hello colleague,
> >
> > in the first order i think this could be emulated by a dictionary mapping
> > the row name to an index into a matrix or DataFrame.
> > Afaics calling this a  'misfeature' comes from trying to make a matrix
> > datatype that has row names by default and many people with
> > numerics/engineering background reserve the name matrix for the simplest
> > possible form: rectangular array with single number entries and integer
> row
> > and column indexing.
> >
> > So what you look for: a rectangular collection accessible with both row
> and
> > column index as names is something new and should have different name.
> You
> > could browse the dataFrames development and see if there are enough hooks
> > to extend this.
> >
> > Bringing this into julia as package (written in julia) should not be that
> > complicated if defined clearly (but still, someone is needed to
> implement).
>
>

Re: [julia-users] Re: rownames in data matrices

[Tamas Papp]

AFAIK https://github.com/davidavdav/NamedArrays.jl already does this and
is maintained actively.

Best,

Tamas

On Tue, Sep 08 2015, Andreas Lobinger  wrote:

> Hello colleague,
>
> in the first order i think this could be emulated by a dictionary mapping 
> the row name to an index into a matrix or DataFrame. 
> Afaics calling this a  'misfeature' comes from trying to make a matrix 
> datatype that has row names by default and many people with 
> numerics/engineering background reserve the name matrix for the simplest 
> possible form: rectangular array with single number entries and integer row 
> and column indexing.
>
> So what you look for: a rectangular collection accessible with both row and 
> column index as names is something new and should have different name. You 
> could browse the dataFrames development and see if there are enough hooks 
> to extend this. 
>
> Bringing this into julia as package (written in julia) should not be that 
> complicated if defined clearly (but still, someone is needed to implement).

Re: [julia-users] Compressing .jld files

[Páll Haraldsson]

On Thursday, November 13, 2014 at 4:25:30 AM UTC, Steven G. Johnson wrote:
>
> Blosc compression of HDF5 files in Julia is implemented at 
> https://github.com/timholy/HDF5.jl/issues/174 
> 
>

Strangely clicking this link gave me issue #173 (and I thought GitHub might 
have some kind of redirect..). Issue 174 is actually separate and merged.

Then I see this one still open:
Enable Blosc compression of JLD files by default?
https://github.com/timholy/HDF5.jl/issues/178

And:
Files created with blosc compression can't be read by Python and R
https://github.com/timholy/HDF5.jl/issues/254

That might not be a problem at least for .jld files.. Should those, at 
least, compress by default?
 

>
>
> Cursory testing seems to indicate that it is within a factor of 2 of 
> uncompressed HDF5 (as opposed to the orders-of-magnitude penalty of zlib), 
> and achieves pretty good compression (actually better than zlib), although 
> I've only tried it out on trivial data.
>
> It would be great to have people bang on it some more with real data.
>

[julia-users] Re: rownames in data matrices

[Andreas Lobinger]

Hello colleague,

in the first order i think this could be emulated by a dictionary mapping 
the row name to an index into a matrix or DataFrame. 
Afaics calling this a  'misfeature' comes from trying to make a matrix 
datatype that has row names by default and many people with 
numerics/engineering background reserve the name matrix for the simplest 
possible form: rectangular array with single number entries and integer row 
and column indexing.

So what you look for: a rectangular collection accessible with both row and 
column index as names is something new and should have different name. You 
could browse the dataFrames development and see if there are enough hooks 
to extend this. 

Bringing this into julia as package (written in julia) should not be that 
complicated if defined clearly (but still, someone is needed to implement).

[julia-users] Re: PyPlot ERROR: LAPACKException

[Tony Kelman]

I don't know enough about Gentoo to be sure, but I suspect the "lib64" 
there is referring to libblas being compiled as a 64-bit binary. Using 64 
bit integers in libblas is a bit unconventional and I'd be surprised if 
Gentoo were doing that by default. The relevant part of the ebuild is in 
the src_configure() step where they populate Make.user. If you 
add USE_BLAS64=0 to the list of flags there I suspect it'll help.


On Tuesday, September 8, 2015 at 3:39:14 AM UTC-7, Phil_n wrote:
>
> The only USE flag used, when installing Julia is emacs, from what I can 
> see.
>
> # equery uses julia
> [ Legend : U - final flag setting for installation]
> [: I - package is installed with flag ]
> [ Colors : set, unset ]
>  * Found these USE flags for dev-lang/julia-0.3.0:
>  U I
>  - - emacs : Add support for GNU Emacs
>
> I believe that BLAS is built with 64 bit support (but I am not entirely 
> sure):
>
> # eselect blas show
> lib64: reference
>
> The complete code of the e-build is:
>
>
> # Copyright 1999-2014 Gentoo Foundation
> # Distributed under the terms of the GNU General Public License v2
> # $Id$
>
> EAPI=5
>
> RESTRICT="test"
>
> inherit elisp-common eutils multilib pax-utils toolchain-funcs
>
> PDSFMT=dSFMT-src-2.2
>
> DESCRIPTION="High-performance programming language for technical computing"
> HOMEPAGE="http://julialang.org/";
> SRC_URI="
> https://github.com/JuliaLang/${PN}/archive/v${PV}.tar.gz -> 
> ${P}.tar.gz
> http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/${PDSFMT}.tar.gz
> https://dev.gentoo.org/~patrick/libuv-${P}.tar.bz2
> https://dev.gentoo.org/~patrick/rmath-0_p20140821.tar.bz2
> 
> http://www.public-software-group.org/pub/projects/utf8proc/v1.1.6/utf8proc-v1.1.6.tar.gz
> "
>
> LICENSE="MIT"
> SLOT="0"
> KEYWORDS="~amd64 ~x86 ~amd64-linux ~x86-linux"
> IUSE="emacs"
>
> RDEPEND="
> dev-lang/R:0=
> dev-libs/double-conversion:0=
> dev-libs/gmp:0=
> dev-libs/libpcre:3=
> dev-libs/mpfr:0=
> dev-libs/openspecfun
> sci-libs/arpack:0=
> sci-libs/camd:0=
> sci-libs/cholmod:0=
> sci-libs/fftw:3.0=
> sci-libs/openlibm:0=
> sci-libs/spqr:0=
> sci-libs/umfpack:0=
> sci-mathematics/glpk:0=
> =sys-devel/llvm-3.4*
> >=sys-libs/libunwind-1.1:7=
> sys-libs/readline:0=
> sys-libs/zlib:0=
> virtual/blas
> virtual/lapack
> emacs? ( app-emacs/ess )"
>
> DEPEND="${RDEPEND}
> dev-util/patchelf
> virtual/pkgconfig"
>
> src_prepare() {
> ln -s "${DISTDIR}"/${PDSFMT}.tar.gz deps/dsfmt-2.2.tar.gz || die
> ln -s "${DISTDIR}"/utf8proc-v1.1.6.tar.gz deps/utf8proc-v1.1.6.tar.gz 
> || die
> cp  -ar "${WORKDIR}"/Rmath deps/ || die
> rmdir deps/libuv && ln -s "${WORKDIR}"/libuv deps/libuv
> # no fetching in ebuild
> # /usr/include/suitesparse is for debian only
> # respect prefix, flags
> sed -i \
> -e 's|$(JLDOWNLOAD)|${EPREFIX}/bin/true|' \
> -e 's|git submodule|${EPREFIX}/bin/true|g' \
> -e 's|^\(SUITESPARSE_INC\s*=\).*||g' \
> -e "s|-O3|${CFLAGS}|g" \
> -e 's|/usr/bin/||g' \
> -e "s|/usr/include|${EPREFIX%/}/usr/include|g" \
> deps/Makefile || die
>
> sed -i \
> -e "s|\(JULIA_EXECUTABLE = \)\(\$(JULIAHOME)/julia\)|\1 
> LD_LIBRARY_PATH=\$(BUILD)/$(get_libdir) \2|" \
> -e "s|-O3|${CFLAGS}|g" \
> -e "s|LIBDIR = lib|LIBDIR = $(get_libdir)|" \
> -e "s|/usr/lib|${EPREFIX}/usr/$(get_libdir)|" \
> -e "s|/usr/include|${EPREFIX}/usr/include|" \
> -e "s|\$(BUILD)/lib|\$(BUILD)/$(get_libdir)|" \
> -e "s|^JULIA_COMMIT = .*|JULIA_COMMIT = v${PV}|" \
> Make.inc || die
>
> sed -i \
> -e "s|,lib)|,$(get_libdir))|g" \
> -e "s|\$(BUILD)/lib|\$(BUILD)/$(get_libdir)|g" \
> Makefile || die
>
> sed -i \
> -e "s|ar -rcs|$(tc-getAR) -rcs|g" \
> src/Makefile || die
>
> }
>
> src_configure() {
> # libuv is an incompatible fork from upstream, so don't use system one
> cat <<-EOF > Make.user
> USE_LLVM_SHLIB=1
> USE_SYSTEM_ARPACK=1
> USE_SYSTEM_BLAS=1
> USE_SYSTEM_FFTW=1
> USE_SYSTEM_GMP=1
> USE_SYSTEM_GRISU=1
> USE_SYSTEM_LAPACK=1
> USE_SYSTEM_LIBM=1
> USE_SYSTEM_LIBUNWIND=1
> USE_SYSTEM_LIBUV=0
> USE_SYSTEM_LLVM=1
> USE_SYSTEM_MPFR=1
> USE_SYSTEM_OPENLIBM=1
> USE_SYSTEM_OPENSPECFUN=1
> USE_SYSTEM_PCRE=1
> USE_SYSTEM_READLINE=1
> USE_SYSTEM_RMATH=1
> USE_SYSTEM_SUITESPARSE=1
> USE_SYSTEM_ZLIB=1
> VERBOSE=1
> EOF
> }
>
> src_compile() {
> # Not parallel-safe, #514882
> emake -j1 cleanall
> if [[ $(get_libdir) != lib ]]; then
> mkdir -p usr/$(get_libdir) || die
> ln -s $(get_libdir) usr/lib || die
> fi
> emake -j1 julia-release prefix="/usr" DESTDIR="${D}"
> pax-mark m $(file usr/bin/

[julia-users] Re: PyPlot ERROR: LAPACKException

[Phil_n]

The only USE flag used, when installing Julia is emacs, from what I can see.

# equery uses julia
[ Legend : U - final flag setting for installation]
[: I - package is installed with flag ]
[ Colors : set, unset ]
 * Found these USE flags for dev-lang/julia-0.3.0:
 U I
 - - emacs : Add support for GNU Emacs

I believe that BLAS is built with 64 bit support (but I am not entirely 
sure):

# eselect blas show
lib64: reference

The complete code of the e-build is:


# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$

EAPI=5

RESTRICT="test"

inherit elisp-common eutils multilib pax-utils toolchain-funcs

PDSFMT=dSFMT-src-2.2

DESCRIPTION="High-performance programming language for technical computing"
HOMEPAGE="http://julialang.org/";
SRC_URI="
https://github.com/JuliaLang/${PN}/archive/v${PV}.tar.gz -> ${P}.tar.gz
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/${PDSFMT}.tar.gz
https://dev.gentoo.org/~patrick/libuv-${P}.tar.bz2
https://dev.gentoo.org/~patrick/rmath-0_p20140821.tar.bz2

http://www.public-software-group.org/pub/projects/utf8proc/v1.1.6/utf8proc-v1.1.6.tar.gz
"

LICENSE="MIT"
SLOT="0"
KEYWORDS="~amd64 ~x86 ~amd64-linux ~x86-linux"
IUSE="emacs"

RDEPEND="
dev-lang/R:0=
dev-libs/double-conversion:0=
dev-libs/gmp:0=
dev-libs/libpcre:3=
dev-libs/mpfr:0=
dev-libs/openspecfun
sci-libs/arpack:0=
sci-libs/camd:0=
sci-libs/cholmod:0=
sci-libs/fftw:3.0=
sci-libs/openlibm:0=
sci-libs/spqr:0=
sci-libs/umfpack:0=
sci-mathematics/glpk:0=
=sys-devel/llvm-3.4*
>=sys-libs/libunwind-1.1:7=
sys-libs/readline:0=
sys-libs/zlib:0=
virtual/blas
virtual/lapack
emacs? ( app-emacs/ess )"

DEPEND="${RDEPEND}
dev-util/patchelf
virtual/pkgconfig"

src_prepare() {
ln -s "${DISTDIR}"/${PDSFMT}.tar.gz deps/dsfmt-2.2.tar.gz || die
ln -s "${DISTDIR}"/utf8proc-v1.1.6.tar.gz deps/utf8proc-v1.1.6.tar.gz 
|| die
cp  -ar "${WORKDIR}"/Rmath deps/ || die
rmdir deps/libuv && ln -s "${WORKDIR}"/libuv deps/libuv
# no fetching in ebuild
# /usr/include/suitesparse is for debian only
# respect prefix, flags
sed -i \
-e 's|$(JLDOWNLOAD)|${EPREFIX}/bin/true|' \
-e 's|git submodule|${EPREFIX}/bin/true|g' \
-e 's|^\(SUITESPARSE_INC\s*=\).*||g' \
-e "s|-O3|${CFLAGS}|g" \
-e 's|/usr/bin/||g' \
-e "s|/usr/include|${EPREFIX%/}/usr/include|g" \
deps/Makefile || die

sed -i \
-e "s|\(JULIA_EXECUTABLE = \)\(\$(JULIAHOME)/julia\)|\1 
LD_LIBRARY_PATH=\$(BUILD)/$(get_libdir) \2|" \
-e "s|-O3|${CFLAGS}|g" \
-e "s|LIBDIR = lib|LIBDIR = $(get_libdir)|" \
-e "s|/usr/lib|${EPREFIX}/usr/$(get_libdir)|" \
-e "s|/usr/include|${EPREFIX}/usr/include|" \
-e "s|\$(BUILD)/lib|\$(BUILD)/$(get_libdir)|" \
-e "s|^JULIA_COMMIT = .*|JULIA_COMMIT = v${PV}|" \
Make.inc || die

sed -i \
-e "s|,lib)|,$(get_libdir))|g" \
-e "s|\$(BUILD)/lib|\$(BUILD)/$(get_libdir)|g" \
Makefile || die

sed -i \
-e "s|ar -rcs|$(tc-getAR) -rcs|g" \
src/Makefile || die

}

src_configure() {
# libuv is an incompatible fork from upstream, so don't use system one
cat <<-EOF > Make.user
USE_LLVM_SHLIB=1
USE_SYSTEM_ARPACK=1
USE_SYSTEM_BLAS=1
USE_SYSTEM_FFTW=1
USE_SYSTEM_GMP=1
USE_SYSTEM_GRISU=1
USE_SYSTEM_LAPACK=1
USE_SYSTEM_LIBM=1
USE_SYSTEM_LIBUNWIND=1
USE_SYSTEM_LIBUV=0
USE_SYSTEM_LLVM=1
USE_SYSTEM_MPFR=1
USE_SYSTEM_OPENLIBM=1
USE_SYSTEM_OPENSPECFUN=1
USE_SYSTEM_PCRE=1
USE_SYSTEM_READLINE=1
USE_SYSTEM_RMATH=1
USE_SYSTEM_SUITESPARSE=1
USE_SYSTEM_ZLIB=1
VERBOSE=1
EOF
}

src_compile() {
# Not parallel-safe, #514882
emake -j1 cleanall
if [[ $(get_libdir) != lib ]]; then
mkdir -p usr/$(get_libdir) || die
ln -s $(get_libdir) usr/lib || die
fi
emake -j1 julia-release prefix="/usr" DESTDIR="${D}"
pax-mark m $(file usr/bin/julia-* | awk -F : '/ELF/ {print $1}')
emake
use emacs && elisp-compile contrib/julia-mode.el
}

src_test() {
emake test
}

src_install() {
emake install prefix="/usr" DESTDIR="${D}"
cat > 99julia <<-EOF
LDPATH=${EROOT%/}/usr/$(get_libdir)/julia
EOF
doenvd 99julia

if use emacs; then
elisp-install "${PN}" contrib/julia-mode.el
elisp-site-file-install "${FILESDIR}"/63julia-gentoo.el
fi
dodoc README.md
}

pkg_postinst() {
use emacs && elisp-site-regen
}

pkg_postrm() {
use emacs && elisp-site-regen
}


Here 

On Monday, September 7, 2015 at 10:51:41 AM UTC+2, Tony Kelman wrote:
>
> 0.3.0 is a year out of date, you should update to 0.3.11. Depending which 
> BLAS implementation is being used there,

[julia-users] Re: overwriting an object - deep copy without copy needed

[Allard van Mossel]

Hi Alex,

Great to see more people using Julia for scientific simulations!

Making copies of your offspring is indeed likely to be detrimental to your 
model's performance--especially if (presumably) you wish to repeat the 
simulation many times to obtain accurate estimates of whatever behavior you 
are interested in. However, I don't think that simply changing pointer from 
population individuals to offspring individuals will allow you to obtain 
the best performance since this is likely to lead to issues when you start 
pre-allocating memory to the population arrays that contain your 
individuals. Especially when you repeat the simulation many times over, 
pre-allocating will likely lead to huge performance gains.

Assuming your population size is fixed, I would guess something like the 
following could greatly speed up your simulation while only requiring minor 
adjustments to your program:

function  siminsects(repetitions, populationsize)
  # pre-allocate population arrays
  world = Array(TPop, populationsize)
  newpop = Array(TPop, populationsize)

  # run the simulation in a way that modifies population arrays instead of 
creating new ones for each run
  for i_rep in 1:repetitions
simonce!(world, newpop)
  end
end

function simonce!(world, newpop, populationsize)
  # now create the world population array by modifying each of the 
individuals in it to avoid memory allocation
  initworld!(world, populationsize)

  # then run the rest of the simulation, while 'creating' the offspring 
population arrays in the same way as the population array

  # when the offspring population has to replace the world population, do 
this by modifying as well
  replacepopulation!(world, newpop, populationsize)
end

function initworld!(world, populationsize)
  for i_individual in 1:populationsize
world[i_individual] = TInd(whatever_float64_you_want, 
whatever_bool_you_want)
  end
end

function replacepopulation!(world, newpop, populationsize)
  for i_individual in 1:population size
# in my experience, assigning the various attributes of the types 
directly is usually the fastest way to do this
world[i_individual].ld = newpop[i_individual].ld
world[i_individual].disp = newpop[i_individual].disp
  end
end 


I hope this helps.

Best,

Allard


Op maandag 7 september 2015 19:08:50 UTC+2 schreef alexande...@gmail.com:
>
> Hello everybody,
>
> I'm comparably new to Julia, but not completely new to programming. Yet, 
> I'm a biologist by training, so please excuse potentially dumb questions in 
> advance :)
>
> I am working in evolutionary ecology, programming individual-based 
> simulations. I have now transferred a (very) simple program that simulates 
> insect populations into Julia and am so far happy with its performance and 
> style (I really fell in love with Julia). Yet, I do have a performance 
> problem when it comes to copying a complex object. First of all my basic 
> type structure:
>
> type TInd # an individual
>   ld::Float64
>   disp::Bool
> end
>
> type TPop # a single population
> ind::Array{TInd,1}
> end
>
> world = TPop[] # just to mention, this is NOT a global variable, but in 
> my main simulation function to create multiple populations
>
> You see that I have a set (world) of populations (TPop), each being 
> defined as arrays of individuals. During reproduction, I create a second 
> Array of individuals, that stores the (mutating) offspring. So far so good. 
> Yet, since I assume discrete generations, after each individual in a 
> population has reproduced, the parental population is to be replaced by the 
> offspring. I have implemented that like this:
>
> newpop = TInd[]
>
> # ... then the new population gets filled with offspring ...
>
> world[p].ind = deepcopy(newpop)
>
> Of course, this solution is working, but it is really slow. And since I do 
> actually not need a copy of the newpop, but just want it to overwrite the 
> original population, I guessed there might probably be a faster and more 
> elegant solution (without complex workarounds, just somehow adjusting the 
> pointers!?)? From what I've seen the model will probably run faster than in 
> C++ as soon as I find the answer :)))
>
> I appreciate any help, thanks a lot in advance!
>
> All the best,
> Alex
>

Re: [julia-users] Adding remote workers on windows

['Greg Plowman' via julia-users]

>
> I meant the remote machine/network may be firewalled to only accept 
> incoming ssh, http and other known ports.

  
OK sorry. By now you can probably guess I don't really understand 
networking.

Anyway I turned off the remote firewall entirely, and addprocs() 
successfully added remote worker, Yippee!
Then I re-enabled the remote firewall and confirmed ssh server was enabled 
through the firewall. But now addprocs() failed with wait error as before.
Definitely something to do with firewall blocking but seemingly not ssh 
server. What to do?
After much playing around, and for some unknown but inspired reason (well, 
for me anyway),  I decided to add Julia to allowed "apps" and Presto it 
worked!
Not really sure why Julia wasn't in already in allowed list. I didn't need 
do this on another network where it just worked out of the box.
 
In any case, for completeness and so it might help anyone else, here's what 
I did:

Control Panel
Windows Firewall
Allow an app or feature through Windows Firewall
(since Julia was not already in list)
Allow another app
Select Julia from list of applications, and click Add.

 
 
On Tuesday, September 8, 2015 at 1:27:44 PM UTC+10, Amit Murthy wrote:

> I meant the remote machine/network may be firewalled to only accept 
> incoming ssh, http and other known ports.
>  
> On Tue, Sep 8, 2015 at 5:49 AM, greg_plowman via julia-users <
> julia...@googlegroups.com > wrote:
>  
>
>> Is port 9009 open on the remote machine? You could try with "tunnel=true" 
>>> if it is not open.
>>
>>  
>> I think so.
>> After running addprocs() and before the wait error, netstat on the 
>> remote machine outputs the following:
>>  
>> C:\Users\Greg>netstat -an
>> Active Connections
>>   Proto  Local Address  Foreign AddressState
>>   TCP0.0.0.0:22 0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:1350.0.0.0:0  LISTENING
>>   TCP0.0.0.0:4450.0.0.0:0  LISTENING
>>   TCP0.0.0.0:5540.0.0.0:0  LISTENING
>>   TCP0.0.0.0:2869   0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:3389   0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:5357   0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:8092   0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:9009   0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:10243  0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:26143  0.0.0.0:0  LISTENING
>>   TCP0.0.0.0:47984  0.0.0.0:0  LISTENING
>> ...
>>  
>> When the remote session terminates, the 9009 entry is missing from 
>> netstat output.
>>  
>> On Monday, September 7, 2015 at 9:24:38 PM UTC+10, Amit Murthy wrote:
>>
>>> Is port 9009 open on the remote machine? You could try with 
>>> "tunnel=true" if it is not open.
>>>  
>>> On Mon, Sep 7, 2015 at 4:32 PM, Greg Plowman  
>>> wrote:
>>>  
>>>
 Hi, 
  
 I'm trying to use addprocs() to add remote workers on another windows 
 machine.
  
 I'm using a ssh server for windows (Bitvise) with a modified Cluster 
 Manager, and have successfully used this method in another environment.
 So I know that it works, although one difference is Window 7 (works) vs 
 Windows 8.1 (does not work), but I don't think this should be problem.
  
 Now, I don't expect anyone to troubleshoot my particular setup / 
 environment / customisation.
 Rather I was hoping for some high level help with further diagnosis.
  
 I can confirm that the windows command to launch the remote worker is 
 executed, and the remote machine receives a connection and then successful 
 login.
 The remote ssh server shows a successful connection and login, and 
 windows Task Manager shows a Julia process has started.  
 Then the following error occurs on the local machine, after which the 
 remote session is terminated.
  
 Error evaluating c:\Users\Greg\Julia6\src\Launcher.jl:
 connect: connection timed out (ETIMEDOUT)
  in wait at task.jl:284
  in wait at task.jl:194
  in stream_wait at stream.jl:263
  in wait_connected at stream.jl:301
  in Worker at multi.jl:113
  in create_worker at multi.jl:1064
  in start_cluster_workers at multi.jl:1028
  
  
 I guess my first question is which side (local or remote) is failing.
 It seems to me that the local Julia process is waiting for some 
 confirmation of connection? Does that sound right?
 If so, are there any suggestions on how to further diagnose problem.
  
 When the ssh command to start a remote Julia worker is executed 
 from the windows command line, I get the following:
 julia_worker:9009#192.168.1.107
  
 Then after about 60s:
 Master process (id 1) could not connect within 60.0 seconds.
 exiting.
  
 Presumably this is the expected behaviour, si

[julia-users] Re: overwriting an object - deep copy without copy needed

[Allard van Mossel]

Hi Alex,

Great to see more people using Julia for scientific simulations!

Making copies of your offspring is indeed likely to be detrimental to your 
model's performance--especially if (presumably) you wish to repeat the 
simulation many times to obtain accurate estimates of whatever behavior you 
are interested in. However, I don't think that simply changing the pointer 
will allow you to obtain the best performance since this is likely to lead 
to issues when you start pre-allocating memory to the population arrays 
that contain your individuals. Especially when you repeat the simulation 
many times over, pre-allocating will likely lead to huge performance gains. 
Assuming your population size is fixed, I would suggest something like the 
following:

function siminsects(repetitions, populationsize, offspringsize)
  world = Array(TPop, populationsize)
  newpop = Array(Tpop, offspringsize)
  for i_rep in 1:repetitions

  end
end

function runonce()
  
end



Op maandag 7 september 2015 19:08:50 UTC+2 schreef alexande...@gmail.com:
>
> Hello everybody,
>
> I'm comparably new to Julia, but not completely new to programming. Yet, 
> I'm a biologist by training, so please excuse potentially dumb questions in 
> advance :)
>
> I am working in evolutionary ecology, programming individual-based 
> simulations. I have now transferred a (very) simple program that simulates 
> insect populations into Julia and am so far happy with its performance and 
> style (I really fell in love with Julia). Yet, I do have a performance 
> problem when it comes to copying a complex object. First of all my basic 
> type structure:
>
> type TInd # an individual
>   ld::Float64
>   disp::Bool
> end
>
> type TPop # a single population
> ind::Array{TInd,1}
> end
>
> world = TPop[] # just to mention, this is NOT a global variable, but in 
> my main simulation function to create multiple populations
>
> You see that I have a set (world) of populations (TPop), each being 
> defined as arrays of individuals. During reproduction, I create a second 
> Array of individuals, that stores the (mutating) offspring. So far so good. 
> Yet, since I assume discrete generations, after each individual in a 
> population has reproduced, the parental population is to be replaced by the 
> offspring. I have implemented that like this:
>
> newpop = TInd[]
>
> # ... then the new population gets filled with offspring ...
>
> world[p].ind = deepcopy(newpop)
>
> Of course, this solution is working, but it is really slow. And since I do 
> actually not need a copy of the newpop, but just want it to overwrite the 
> original population, I guessed there might probably be a faster and more 
> elegant solution (without complex workarounds, just somehow adjusting the 
> pointers!?)? From what I've seen the model will probably run faster than in 
> C++ as soon as I find the answer :)))
>
> I appreciate any help, thanks a lot in advance!
>
> All the best,
> Alex
>

[julia-users] rownames in data matrices

[Michael Borregaard]

Hi, I am learning julia, and thought I would practice by migrating some of 
my code from R. I run into the problem that the data structures in julia 
does not seem to offer row names? I looked here in the forum and found a 2 
year old discussion, that seems to end of the, IMHO slightly imprecise 
opinion that rownames are a misfeature 
(https://groups.google.com/forum/#!searchin/julia-users/rownames$20matrix/julia-users/OFbnNLPdWOc/qTnhlm33YzMJ).
 
Of course, in DataFrame it is always possible to add another column with 
names, and use this by convention, though it does preclude the nice 
behaviour of automatically extracting a Named Array with appropriate names 
when extracting a column.

Worse is the case for data matrices, that do not support multiple types. 
Take for instance an ecological community matrix, that has species as 
columns, sites as rows, and is filled with integers counting the abundance 
of individuals. Subsetting a column gives the occupancy of a species, 
subsetting a row gives the species community in a site. Having row and 
column names, and being able to index into the array on names, is a really 
important feature!

How would this be implemented in julia? Is there still a conviction that 
row names are a misfeature, and why?

Thanks!