RE: [julia-users] Re: performance of parallel code

2016-06-20 Thread Boylan, Ross
I can do it using pmap, but it doesn't run any faster (still 3 workers):
julia> @time h10 = Calculus.hessian(myf, RB.troubleParamsSim1.raw)
 186.704029 seconds (410.30 k allocations: 29.503 MB, 0.01% gc time)

That was using this code:
"""
Return a function that takes one argument, the optimization parameters.
It evaluates the -LL in parallel
"""
function makeParallelLL()
np = nworkers()
cuts = chunks(obs[:id], np)
function myLL(x)
args = [(cuts[i, 1], cuts[i,2], x) for i = 1:np]
ll= pmap(x->cutLL(x...), args)
-sum(ll)
end
end


From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Christopher Fisher [fishe...@miamioh.edu]
Sent: Monday, June 20, 2016 4:45 AM
To: julia-users
Subject: [julia-users] Re: performance of parallel code

Would using pmap() be suitable for your application? Usually when I use that I 
get slightly more than a 2X speed up with 4 cores.


On Monday, June 20, 2016 at 2:27:41 AM UTC-4, Boylan, Ross wrote:
I think I've taken steps to minimize parallel overhead by providing only one 
function call per process and passing really minimal arguments to the 
functions.  But the gains in speed don't seem commensurate with the number of 
processors.  I know that pure linear speedup is too much to hope for, but I 
suspect I'm doing something wrong--for example that large data is getting 
passed around despite my efforts.

All my code is defined inside a module, though I exercise it from the main REPL.

Single processor (times are representative of multiple samples and exclude 
burn-in runs):
julia> @time h=Calculus.hessian(RB.mylike, RB.troubleParamsSim1.raw)
 206.422562 seconds (2.43 G allocations: 83.867 GB, 10.14% gc time)

#with 3 workers
julia> myf = RB.makeParallelLL()  # code below
julia> @time h10 = Calculus.hessian(myf, RB.troubleParamsSim1.raw)
 182.567647 seconds (1.48 M allocations: 111.622 MB, 0.02% gc time)

#with 7 workers
julia> @time h10 = Calculus.hessian(myf, RB.troubleParamsSim1.raw)
  82.453033 seconds (3.43 M allocations: 259.838 MB, 0.08% gc time)

Any suggestions?  This on an 8 CPU VM; the underlying hardware has >>8 
processors.

Here's some of the code:

mylike(v) = -logitlite(RB.obs, v, 7)

"""
return a matrix whose columns are the start and the end of range of 
observations, inclusive.
these chunks are roughly equal sizes, and no id is  split between chunks.
n  target number of chunks, and thus of rows returned
id id for each row; generally one id may appear in several rows.
   all rows with the same id must be contiguous.
"""
function chunks(id, n::Int)
nTot = length(id)
s = nTot/n
cuts = Vector{Int}(n)
cuts[1] = 1
for i = 1:(n-1)
cut::Int = floor(s*i)
while id[cut] == id[cut-1]
cut += 1
end
cuts[i+1] = cut
end
stops = vcat(cuts[2:n]-1, nTot)
hcat(cuts, stops)
end

"""
Parallel helper: evaluate likelihood over limited range
i0..i1, inclusive, is the range.
This is an effort to avoid passing obs between processes.
"""
function cutLL(i0::Int, i1::Int, x)
logitlite(obs[i0:i1,:], x, 7)  #obs is a 6413x8 data.frame.  Maybe this is 
doing an extra copy?
end

"""
Return a function that takes one argument, the optimization parameters.
It evaluates the -LL in parallel
"""
function makeParallelLL()
np = nworkers()
cuts = chunks(obs[:id], np)
function myLL(x)
   ll= @parallel (+) for i=1:np
cutLL(cuts[i,1], cuts[i,2], x)
end
-ll
end
end

It might be relevant that logitlite uses a by() to process groups of 
observations.

So I think the only thing I'm passing into the  workers is a function call, 
cutLL, 2 integers, and a 10-element Vector{Float64}, and the only thing going 
back is 1 Float64.

Or perhaps fact that I'm using an inner function def (myLL in the last function 
definition) is doing something like causing transmission of all variables in 
lexical scope?


RE: [julia-users] errors using packages in parallel

2016-06-05 Thread Boylan, Ross
I've been encountering errors with packages in parallel, even though everything 
is happening on one machine (julia -p 4 or -p 3 under ESS).  This isn't 
necessarily the same issue, but it seems suggestive it's arising at the same 
time.

Originally I think the packages/cache were a bit out of date, and when I ran in 
parallel things went wrong and rebuilt everywhere.  If they were all rebuilding 
in the same spot, as they seemed to, it's easy to imagine that would cause 
problems.  I also had a single CPU copy of julia running at the same time 
(idle) throughout.

After that I started up a single cpu version, did a Pkg.update(), and ran my 
code.  Everything seemed OK.

However, I then ran a parallel julia, and continued to get errors.  The 
included code is all in a module and connect to a Postgresql DB to get some 
data.

Here are excerpts from the sessions:

   | | | | | | |/ _' |  |
   | | |_| | | | (_| |  |  Version 0.4.6-pre+28 (2016-04-22 00:59 UTC)
  _/ |\__'_|_|_|\__'_|  |  Commit 022917e (44 days old release-0.4)
 |__/   |  x86_64-linux-gnu

 WARNING: Terminal not fully functional
 julia>
 julia> nprocs()
 5

 julia> now()
 2016-06-05T00:04:52  # Pacific time US
julia> @everywhere include("trouble.jl")
 INFO: Recompiling stale cache file /home/ross/.julia/lib/v0.4/Debug.ji for 
module Debug.
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/PDMats.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/PDMats.ji
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/PDMats.ji
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/PDMats.ji
 WARNING: Module PDMats uuid did not match cache file
   This is likely because module PDMats does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/Distributions.ji
 WARNING: Module PDMats uuid did not match cache file
   This is likely because module PDMats does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/Distributions.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/StatsFuns.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/StatsFuns.ji
 WARNING: Module PDMats uuid did not match cache file
   This is likely because module PDMats does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/Distributions.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/StatsFuns.ji
 WARNING: Module PDMats uuid did not match cache file
   This is likely because module PDMats does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/Distributions.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 
/home/ross/.julia/lib/v0.4/StatsFuns.ji
 WARNING: Module Compat uuid did not match cache file
   This is likely because module Compat does not support  precompilation but is 
imported by a module that does.
 WARNING: deserialization checks failed while attempting to load cache from 

RE: [julia-users] Re: Calculus.hessian result oddities

2016-05-31 Thread Boylan, Ross
I believe Calculus.hessian does finite differences.  I had not expected it to 
computer both h[i, j] and h[j, i], since they should be the same.  That is, 
AFAIK they are the same mathematically; numerically some differences dependent 
on the order of operations seem inevitable.

The double calculation has the potential advantage of alerting you if the 
numerical issues are significant, but returning a matrix that lacks the 
expected symmetry seems a pretty high price.
Ross

From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Kristoffer Carlsson [kcarlsso...@gmail.com]
Sent: Tuesday, May 31, 2016 1:26 AM
To: julia-users
Subject: [julia-users] Re: Calculus.hessian result oddities

Doesn't Calculu's Hessian just do finite difference? There will naturally be 
floating point errors.

isposdef returns false if the matrix is not hermitian, under the hood, potrf 
from Lapack is used which work with hermitian matrices.

On Tuesday, May 31, 2016 at 7:04:04 AM UTC+2, Boylan, Ross wrote:
The hessian produced by Calculus.hessian has some properties that seem odd to 
me.  First, the resulting matrix is not symmetric (though it's really close).  
Second, it is not positive definite even though the eigenvalues are all 
positive.  This might partly be the result of the first problem.

Comments?

 julia> @time h6 = Calculus.hessian(RB.mylike, sol6[2])
 206.988247 seconds (2.43 G allocations: 83.830 GB, 15.48% gc time)
 10x10 Array{Float64,2}:

 julia> issym(h6)
 false
 julia> isposdef(h6)
 false
 julia> eigvals(h6)
 10-element Array{Float64,1}:
  3.27548e5
  41407.1
   1873.44
809.515
385.469
336.118
269.82
 64.4228
117.009
122.845
 julia> [det(h6[1:i, 1:i]) for i in 1:10]  # all submatrix determinants are 
positive
 10-element Array{Any,1}:
  583.356
2.93666e5
3.76334e7
4.26776e9
3.17529e12
6.53405e14
1.12477e17
8.54011e21
3.75658e26
6.65866e29

 julia> showall(h6)
 [583.356378500377 -215.9519377704782 42.707753088841706 -12.046905619427504 
100.26418659571075 30.907721733245413 5.1999446295105445 -3522.0797654066855 
4410.401341876791 29.123944464570087
  -215.95193777047822 583.3512842422355 -12.0619061856546 42.69860798380086 
-82.01778794681003 -20.00373356894301 -8.643894790795585 1484.6195212852567 
-3316.905502014349 -2.7622075180025965
  42.7077530888417 -12.061906185654598 131.30467049554812 -44.14799847519552 
16.748039679197223 -10.98960509547607 43.02830286169418 234.68906012694512 
-1258.6622801992237 9.86994636336174
  -12.046905619427504 42.69860798380086 -44.147998475195514 131.35743705036847 
-17.498616283805696 4.724998443749094 -31.481387268916205 -95.390157895702 
209.5245372890838 12.903636963625035
  100.26418659571075 -82.01778794681003 16.748039679197223 -17.498616283805696 
766.6651018177663 -20.766430168130857 -2.164074090378522 610.206909770971 
-6927.858446549729 -200.4409796490271
  30.907721733245413 -20.00373356894301 -10.98960509547607 4.724998443749093 
-20.766430168130857 209.8283757716871 -60.222027409043555 280.3888402431943 
-1638.62573982097 -158.60599205247166
  5.199944629510544 -8.643894790795585 43.02830286169418 -31.4813872689162 
-2.164074090378524 -60.222027409043555 204.34068072875016 -3.631354965653126 
314.50383342119943 30.12788292183477
  -3522.079765406686 1484.6195212852565 234.6890601269451 -95.39015789570199 
610.2069097709712 280.3888402431943 -3.631354965653125 102596.98982032725 
-117630.58331133435 267.23733374838395
  4410.401341876791 -3316.9055020143487 -1258.6622801992235 209.5245372890838 
-6927.858446549729 -1638.62573982097 314.5038334211995 -117630.58331133435 
265600.50878023007 5508.687364627178
  29.123944464570084 -2.762207518002596 9.86994636336174 12.903636963625033 
-200.4409796490271 -158.6059920524717 30.12788292183477 267.23733374838395 
5508.687364627178 2125.8954716959997]

The largest absolute difference between symmetrical entries is < 5e-13.

Ross Boylan


[julia-users] Calculus.hessian result oddities

2016-05-30 Thread Boylan, Ross
The hessian produced by Calculus.hessian has some properties that seem odd to 
me.  First, the resulting matrix is not symmetric (though it's really close).  
Second, it is not positive definite even though the eigenvalues are all 
positive.  This might partly be the result of the first problem.

Comments?

 julia> @time h6 = Calculus.hessian(RB.mylike, sol6[2])
 206.988247 seconds (2.43 G allocations: 83.830 GB, 15.48% gc time)
 10x10 Array{Float64,2}:

 julia> issym(h6)
 false
 julia> isposdef(h6)
 false
 julia> eigvals(h6)
 10-element Array{Float64,1}:
  3.27548e5
  41407.1
   1873.44
809.515
385.469
336.118
269.82
 64.4228
117.009
122.845
 julia> [det(h6[1:i, 1:i]) for i in 1:10]  # all submatrix determinants are 
positive
 10-element Array{Any,1}:
  583.356
2.93666e5
3.76334e7
4.26776e9
3.17529e12
6.53405e14
1.12477e17
8.54011e21
3.75658e26
6.65866e29

 julia> showall(h6)
 [583.356378500377 -215.9519377704782 42.707753088841706 -12.046905619427504 
100.26418659571075 30.907721733245413 5.1999446295105445 -3522.0797654066855 
4410.401341876791 29.123944464570087
  -215.95193777047822 583.3512842422355 -12.0619061856546 42.69860798380086 
-82.01778794681003 -20.00373356894301 -8.643894790795585 1484.6195212852567 
-3316.905502014349 -2.7622075180025965
  42.7077530888417 -12.061906185654598 131.30467049554812 -44.14799847519552 
16.748039679197223 -10.98960509547607 43.02830286169418 234.68906012694512 
-1258.6622801992237 9.86994636336174
  -12.046905619427504 42.69860798380086 -44.147998475195514 131.35743705036847 
-17.498616283805696 4.724998443749094 -31.481387268916205 -95.390157895702 
209.5245372890838 12.903636963625035
  100.26418659571075 -82.01778794681003 16.748039679197223 -17.498616283805696 
766.6651018177663 -20.766430168130857 -2.164074090378522 610.206909770971 
-6927.858446549729 -200.4409796490271
  30.907721733245413 -20.00373356894301 -10.98960509547607 4.724998443749093 
-20.766430168130857 209.8283757716871 -60.222027409043555 280.3888402431943 
-1638.62573982097 -158.60599205247166
  5.199944629510544 -8.643894790795585 43.02830286169418 -31.4813872689162 
-2.164074090378524 -60.222027409043555 204.34068072875016 -3.631354965653126 
314.50383342119943 30.12788292183477
  -3522.079765406686 1484.6195212852565 234.6890601269451 -95.39015789570199 
610.2069097709712 280.3888402431943 -3.631354965653125 102596.98982032725 
-117630.58331133435 267.23733374838395
  4410.401341876791 -3316.9055020143487 -1258.6622801992235 209.5245372890838 
-6927.858446549729 -1638.62573982097 314.5038334211995 -117630.58331133435 
265600.50878023007 5508.687364627178
  29.123944464570084 -2.762207518002596 9.86994636336174 12.903636963625033 
-200.4409796490271 -158.6059920524717 30.12788292183477 267.23733374838395 
5508.687364627178 2125.8954716959997]

The largest absolute difference between symmetrical entries is < 5e-13.

Ross Boylan

RE: [julia-users] Why isn't my constructor recognized?

2016-05-28 Thread Boylan, Ross
Thank you; that did the trick (with `Coefficients{T}(vals::Vector{T}) = 
Coefficients{T}(vals)` defined outside the type).
The fact that defining Coefficients{T} does not necessarily define Coefficients 
seems like an awfully subtle point, though it is mentioned in 
http://docs.julialang.org/en/release-0.4/manual/constructors/:

Some features of parametric constructor definitions at work here deserve 
comment. First, inner constructor declarations always define methods of 
Point{T} rather than methods of the general Point constructor function. Since 
Point is not a concrete type, it makes no sense for it to even have inner 
constructor methods at all. Thus, the inner method declaration Point(x,y) = 
new(x,y) provides an inner constructor method for each value of T. It is this 
method declaration that defines the behavior of constructor calls with explicit 
type parameters like Point{Int64}(1,2) and Point{Float64}(1.0,2.0). The outer 
constructor declaration, on the other hand, defines a method for the general 
Point constructor which only applies to pairs of values of the same real type. 


It's even trickier because the definition that makes Coefficients without the 
type work looks exactly like (on the left) the declaration which does not have 
the desired effect, Coefficients{T}.

Does the language have to work this way?
Ross

From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Yichao Yu [yyc1...@gmail.com]
Sent: Friday, May 27, 2016 6:02 PM
To: Julia Users
Subject: Re: [julia-users] Why isn't my constructor recognized?

On Fri, May 27, 2016 at 7:53 PM, Boylan, Ross <ross.boy...@ucsf.edu> wrote:
> A type is defined as
> type Coefficients{T}
> raw::Vector{T}
> σ_0::T
> σ_T::T
> ρ::T
> ν::T
> Σ::Matrix{T}
> # because of this c'tor the type must be mutable
> function Coefficients(vals::Vector{T})

This defines a method for `Coefficients{T}` and needs to be called
with `Coefficients{T}(...)` and not `Coefficients(...)`
Defining a inner constructor turns off the automatically defined
construction (method for `Coefficients`) that automatically deduce the
type parameter so you need to manually define that constructor with
`Coefficients{T}(vals::Vector{T}) = Coefficients{T}(vals)` (note that
the first `{T}` is a parameter for this method of `Coefficients`, the
last `{T}` uses this variable to instantiate `Coefficients` and then
call the `Coefficients{T}` object instead.)



[julia-users] Why isn't my constructor recognized?

2016-05-27 Thread Boylan, Ross
A type is defined as 
type Coefficients{T}
raw::Vector{T}
σ_0::T
σ_T::T
ρ::T
ν::T
Σ::Matrix{T}
# because of this c'tor the type must be mutable 
function Coefficients(vals::Vector{T})
if length(vals) ≠ 10
error("Must have exactly 10 parameters")
end
self = new(vals)
self.σ_0 = exp(lnσ_0(self))
self.σ_T = exp(lnσ_T(self))
self.ρ = tanh(atanhρ(self))
self.ν = exp(lnν(self))
# chol doesn't work on SymTridiagonal  
self.Σ = Matrix(SymTridiagonal([ self.σ_0, self.σ_T].^2, [self.ρ * 
self.σ_0 * self.σ_T]))
self
end
end

But
julia> @time g = Calculus.gradient(RB.mylike, RB.troubleParamsInit.raw)
 ERROR: MethodError: `convert` has no method matching 
convert(::Type{RB.Coefficients{T}}, ::Array{Float64,1})
 This may have arisen from a call to the constructor RB.Coefficients{T}(...),
 since type constructors fall back to convert methods.
 Closest candidates are:
   call{T}(::Type{T}, ::Any)
   convert{T}(::Type{T}, !Matched::T)
  in logitlite at /home/ross/PCORI/trouble.jl:316
  in mylike at /home/ross/PCORI/trouble.jl:345
  in finite_difference! at 
/home/ross/.julia/v0.4/Calculus/src/finite_difference.jl:126
  in gradient at /home/ross/.julia/v0.4/Calculus/src/derivative.jl:14

The triggering line was
c = Coefficients(params)

Since Array{Float64,1} = Vector{Float64} I am puzzled that the inner 
constructor isn't called.
I also tried with  the inner constructor defined as 
  function Coefficients{T}(vals::Vector{T})

When I call this from my own code things work  OK.

Can anyone help me identify the problem, or even the solution :)

Thanks.
Ross Boylan

[julia-users] old objects and new code

2016-05-12 Thread Boylan, Ross
A file has code surrounded by a module declaration.  It includes definitions of 
some types and some functions.
In the REPL (ESS) I include the file and execute a function that returns me an 
object x of type MyModule.Foo.
Then I change some of the code and include the module again.  The type Foo is 
unchanged.

It seems to me that when I execute one of the module functions on x the 
original version of the function, not the revised version, gets called.

If I recreate x then I get the current function definitions aka what I wanted.

First, is my understanding of what's going on correct? 

Second, any recommendations for dealing with this?  In this case regenerating x 
was not a big deal, but I can imagine situations that weren't so trivial.  If 
there were lots of objects involved, one could keep a script that creates them 
and rerun it.

Thanks
Ross Boylan

RE: [julia-users] Re: newbie questions (was undefined reference error)

2016-05-10 Thread Boylan, Ross
But if I make an array of immutables I won't be able to change them afterwards. 
 Though I can replace them.

With the old code except for

immutable Stuff
a::Int
b::Int
end

I now have
 julia> v=Array(TT.Stuff, 2)
 2-element Array{TT.Stuff,1}:
  TT.Stuff(140124021836848,140124000460928)
  TT.Stuff(140124048730192,0)

 julia> v[1].a=44  # what I'd like to do
 ERROR: type Stuff is immutable

 julia> v[1]=TT.Stuff(3, 4)  # work-around
 TT.Stuff(3,4)

!julia> v
 2-element Array{TT.Stuff,1}:
  TT.Stuff(3,4)
  TT.Stuff(140124048730192,0)

With immutable, is everything in the array laid out contiguous in memory?  It 
seems sort of odd that the type is immutable yet I can overwrite data of that 
type.  I suppose that would only be prohibited if the array were immutable.

Thanks for the tip about using Type to get the zero to work.
Ross


From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Cedric St-Jean [cedric.stj...@gmail.com]
Sent: Tuesday, May 10, 2016 1:20 PM
To: julia-users
Subject: [julia-users] Re: newbie questions (was undefined reference error)

"normal" types are by definition heap-allocated, and are always manipulated 
them through pointers. What you want is 
immutables<http://docs.julialang.org/en/release-0.4/manual/types/#immutable-composite-types>

immutable Stuff
a::Int
b::Int
end

# Also, for zeros to work,

function zero(x::Type{Stuff})
Stuff(0, 0)
end


On Tuesday, May 10, 2016 at 4:04:37 PM UTC-4, Boylan, Ross wrote:
I'm puzzled that a type  consisting  only of 2 integers doesn't qualify as 
"bitstype".  Further experiment shows that the array seems to be an array of 
references, I don't know how to implement zero, and generally that I'm a bit 
lost :)  My goal is to get a densely packed array of data.  I assume that will 
use less memory and generate faster code; if not, maybe I should change my goal.

BTW, my real use has a type more heterogeneous than 2 Int's, so a solution that 
uses a 2D array doesn't really generalize appropriately for me.

module TT
import Base.zero

type Stuff
a::Int
b::Int
end

function zero(x::Stuff)
Stuff(0, 0)
end

end

julia> v=Array(TT.Stuff, 3)  #as before
 3-element Array{TT.Stuff,1}:
  #undef
  #undef
  #undef

 julia> s=TT.Stuff(3, 5)
 TT.Stuff(3,5)

 julia> v=fill(s, 2)
 2-element Array{TT.Stuff,1}:
  TT.Stuff(3,5)
  TT.Stuff(3,5)

 julia> s.a=900
 900

 julia> v  ###OOPS: every array element points to the same instance
 2-element Array{TT.Stuff,1}:
  TT.Stuff(900,5)
  TT.Stuff(900,5)

 julia> zero(TT.Stuff)  # This is probably what needs to work for zeros() to 
work
 ERROR: MethodError: `zero` has no method matching zero(::Type{TT.Stuff})

!julia> zero(TT.Stuff(1, 1))  # this at least calls the right c'tor
 TT.Stuff(0,0)

Ross

From: julia...@googlegroups.com 
[julia...@googlegroups.com] on behalf of Lutfullah Tomak 
[tomak...@gmail.com]
Sent: Tuesday, May 10, 2016 12:04 AM
To: julia-users
Subject: [julia-users] undefined reference error

You need to initialize array entries if you don't have eltype as bitstype. 
Here, undefined reference means you had not initialize the entry. And, full 
type assignment works because it initializes the entry.


[julia-users] newbie questions (was undefined reference error)

2016-05-10 Thread Boylan, Ross
I'm puzzled that a type  consisting  only of 2 integers doesn't qualify as 
"bitstype".  Further experiment shows that the array seems to be an array of 
references, I don't know how to implement zero, and generally that I'm a bit 
lost :)  My goal is to get a densely packed array of data.  I assume that will 
use less memory and generate faster code; if not, maybe I should change my goal.

BTW, my real use has a type more heterogeneous than 2 Int's, so a solution that 
uses a 2D array doesn't really generalize appropriately for me.

module TT
import Base.zero

type Stuff
a::Int
b::Int
end

function zero(x::Stuff)
Stuff(0, 0)
end

end

julia> v=Array(TT.Stuff, 3)  #as before
 3-element Array{TT.Stuff,1}:
  #undef
  #undef
  #undef

 julia> s=TT.Stuff(3, 5)
 TT.Stuff(3,5)

 julia> v=fill(s, 2)
 2-element Array{TT.Stuff,1}:
  TT.Stuff(3,5)
  TT.Stuff(3,5)

 julia> s.a=900
 900

 julia> v  ###OOPS: every array element points to the same instance
 2-element Array{TT.Stuff,1}:
  TT.Stuff(900,5)
  TT.Stuff(900,5)

 julia> zero(TT.Stuff)  # This is probably what needs to work for zeros() to 
work
 ERROR: MethodError: `zero` has no method matching zero(::Type{TT.Stuff})

!julia> zero(TT.Stuff(1, 1))  # this at least calls the right c'tor
 TT.Stuff(0,0)


Ross

From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Lutfullah Tomak [tomaklu...@gmail.com]
Sent: Tuesday, May 10, 2016 12:04 AM
To: julia-users
Subject: [julia-users] undefined reference error

You need to initialize array entries if you don't have eltype as bitstype. 
Here, undefined reference means you had not initialize the entry. And, full 
type assignment works because it initializes the entry.


[julia-users] undefined reference error

2016-05-10 Thread Boylan, Ross
julia 0.4:

module TT
type Stuff
a::Int
b::Int
end
v = Array(Stuff, 3)
end

Produces
 julia> TT.v[1].a = 5
 ERROR: UndefRefError: access to undefined reference  problem
  in getindex at array.jl:282

!julia> TT.v
 3-element Array{TT.Stuff,1}:
  #undef
  #undef
  #undef

Could anyone help me to understand what's going on, and what I can do about it? 
 I know Array() produces uninitialized elements, but I don't understand why I 
can't write values into them.

In contrast, replacing the entire object works:
julia> TT.v[1] = TT.Stuff(4, 78)
 TT.Stuff(4,78)

!julia> TT.v[1]
 TT.Stuff(4,78)

Thanks.
Ross Boylan


RE: [julia-users] Can't add PostgreSQL

2016-05-06 Thread Boylan, Ross
For the PostgreSQL package, when I do fetchdf against the result of execute, 
instead of the stmt, it does work.  Another problem solved!

The DataFrame returned by fetchdf seems to have lost the type information.  Am 
I understanding that right?  It seems like undesirable behavior.
---
stmt = prepare(conn, "SELECT isim::int, id::int, x::int, t::double precision, 
y::int, regular::int, censor::int, wait::double precision from obs2619 where 
isim=1")
result = execute(stmt)
obs = fetchdf(result)

produces

julia> eltypes(obs)
 8-element Array{Type{T},1}:
  Any
  Any
  Any
  Any
  Any
  Any
  Any
  Any

Ross


From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Cameron McBride [cameron.mcbr...@gmail.com]
Sent: Wednesday, May 04, 2016 3:17 AM
To: julia-users
Subject: Re: [julia-users] Can't add PostgreSQL

I've been using PostgreSQL and DBI just fine for a few months now.

The implementation has a few edges that need polishing and work, but the basics 
work just fine for the julia 0.4.x branch.

The problem you ran into with fetchdf() not being implemented is that you tried 
to run it on the stmt (before running execute). This is one of the rough edges 
that not all convenience functions are implemented. In short, I just confirmed 
the following example runs on julia 0.4.5 and the latest DBI.jl / PostgreSQL.jl:


# slightly modified example from https://github.com/JuliaDB/PostgreSQL.jl

using DBI
using PostgreSQL

conn = connect(Postgres, "localhost", "username", "password", "dbname", 5432)
stmt = prepare(conn, "SELECT 1::bigint, 2.0::double precision, 'foo'::character 
varying, " *
 "'foo'::character(10);")
result = execute(stmt) # this was missing in your use!
df = fetchdf(result)
finish(stmt)

disconnect(conn)

Perhaps that helps clarify? Happy to help further, here or offline.

Cameron


RE: [julia-users] Can't add PostgreSQL

2016-05-06 Thread Boylan, Ross
ODBC is now working for me with the documented interface (after updates).  
Thanks!  I do get a lot of chatter as it loads, mostly from the DecFP package.

The documentation doesn't mention how to shutdown.  I used disconnect!(dsn) 
after peeking at the code.  Does it clean up the connection if you just exit 
the julia session without disconnect?

Also, in this sequence
1. datatable = ODBC.query(dsn, "show databases")

# convert result to a DataFrame for additional data manipulation functionality
2. df = DataFrame(datatable)
3. do stuff with df
at what point is it safe to disconnect?

Is the odbc interface in general, or particularly with postgresql, thread safe?

On a more general topic
I got lots of notices my packages were out of date, even immediately after I 
installed them.  What's that about?  E.g.,
 julia> Pkg.add("Cairo")
 INFO: Nothing to be done
 INFO: METADATA is out-of-date — you may not have the latest version of Cairo
 INFO: Use 'Pkg.update()' to get the latest versions of your packages

 julia> Pkg.update()
 INFO: Updating METADATA...
 INFO: Updating DBI...
 INFO: Updating PostgreSQL...
 INFO: Updating Postgres...
 INFO: Computing changes...
 INFO: No packages to install, update or remove

Ross



From: karbar...@gmail.com [karbar...@gmail.com] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com]

Sent: Wednesday, May 04, 2016 7:47 AM

To: julia-users@googlegroups.com

Subject: Re: [julia-users] Can't add PostgreSQL






Hey Ross,



It actually sounds like you're still on the older release of ODBC (you can 
check by running Pkg.installed()). To get the latest version (that matches the 
current documentation), you'll have to run:



Pkg.update()



Then restart Julia to make sure you have the latest code.



-Jacob






On Tue, May 3, 2016 at 9:58 PM, Boylan, Ross 
<ross.boy...@ucsf.edu> wrote:



I got it working before your update.  Yay!  Thank you!



There were a couple of issues, mostly documentation, leaving aside getting ODBC 
setup outside of julia.



The docs (the help at 
https://github.com/JuliaDB/ODBC.jl) say to use DSN to get things going.  As far 
as I can tell, it doesn't exist.  I used connect.



The docs mention a connection string, but provide no pointers to its content.  
It would be nice to have one; the odbc packages I installed were almost 
documentation free.



The docs imply the results of a query need to be turned into a DataFrame, but 
it seems to come out as one.



The source code suggested one could get a dataframe with the output argument.  
I'm not really sure what that's supposed to be or do, but when I tried

julia> raw = ODBC.query("SELECT isim, id, x, t, y, regular, censor, wait from 
obs2619 where isim=1", conn, output=df )
   


 ERROR: TypeError: typeassert: expected Union{AbstractString,DataType}, got 
Function


I had not set df to anything before the call, AFAIK.



Ross



From:

karbar...@gmail.com [karbar...@gmail.com] on behalf
 of Jacob Quinn [quinn.jac...@gmail.com]

Sent: Tuesday, May 03, 2016 7:57 PM

To: 
julia-users@googlegroups.com

Subject: Re: [julia-users] Can't add PostgreSQL








Hey Ross,



Just confirmed that connecting to Postgres does indeed work; haven't seen any 
issues yet. I'm using the PostgreSQL ODBC Driver from their website with 
unixODBC on a mac. I'm using a connection string I built by following the 
guidelines here: 
http://www.connectionstrings.com/postgresql-odbc-driver-psqlodbc/



Here's a little sample code (I have more testing all the various data types in 
Postgres, but I won't post the full snippet here):




using ODBC



dsn = 
ODBC.DSN("Driver={PostgreSQL};Server=[server];Port=5432;Database=testdb;Uid=username;Pwd=password")



# Check some basic queries
dbs = ODBC.query(dsn, "SELECT datname FROM pg_database WHERE datistemplate = 
false;")
data = ODBC.query(dsn, "SELECT table_schema,table_name FROM 
information_schema.tables ORDER BY table_schema,table_name;")







The latest ODBC release was just merged in METADATA.jl, so if you do 
Pkg.add("ODBC") you should get the latest release/code. If you already had it 
installed, just run Pkg.update() to get the latest release.



Good luck!



-Jacob



On Tue, May 3, 2016 at 7:11 PM, Boylan, Ross 
<ross.boy...@ucsf.edu> wrote:


Attempting to post comments inline, even though my mailer is allergic.  Look 
for >>





From: 
karbar...@gmail.com [karbar...@gmail.com] on behalf
 of Jacob Quinn [quinn.jac...@gmail.com]

Sent: Tuesday, May 03, 2016 5:47 PM



I'm not sure I understand: I don't see DBI.jl nor PostgreSQL.jl at

pkg.julialang.org, which tells me they
 aren't officially registered as packages. The difficulties you're seeing then 
aren't that surprising because they haven't felt comfortable enough to 
officially register.



>> Yo

RE: [julia-users] Can't add PostgreSQL

2016-05-03 Thread Boylan, Ross
I got it working before your update.  Yay!  Thank you!

There were a couple of issues, mostly documentation, leaving aside getting ODBC 
setup outside of julia.

The docs (the help at https://github.com/JuliaDB/ODBC.jl) say to use DSN to get 
things going.  As far as I can tell, it doesn't exist.  I used connect.

The docs mention a connection string, but provide no pointers to its content.  
It would be nice to have one; the odbc packages I installed were almost 
documentation free.

The docs imply the results of a query need to be turned into a DataFrame, but 
it seems to come out as one.

The source code suggested one could get a dataframe with the output argument.  
I'm not really sure what that's supposed to be or do, but when I tried
julia> raw = ODBC.query("SELECT isim, id, x, t, y, regular, censor, wait from 
obs2619 where isim=1", conn, output=df )
 ERROR: TypeError: typeassert: expected Union{AbstractString,DataType}, got 
Function
I had not set df to anything before the call, AFAIK.

Ross

From: karbar...@gmail.com [karbar...@gmail.com] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com]
Sent: Tuesday, May 03, 2016 7:57 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Can't add PostgreSQL

Hey Ross,

Just confirmed that connecting to Postgres does indeed work; haven't seen any 
issues yet. I'm using the PostgreSQL ODBC Driver from their website with 
unixODBC on a mac. I'm using a connection string I built by following the 
guidelines here: 
http://www.connectionstrings.com/postgresql-odbc-driver-psqlodbc/

Here's a little sample code (I have more testing all the various data types in 
Postgres, but I won't post the full snippet here):

using ODBC

dsn = 
ODBC.DSN("Driver={PostgreSQL};Server=[server];Port=5432;Database=testdb;Uid=username;Pwd=password")

# Check some basic queries
dbs = ODBC.query(dsn, "SELECT datname FROM pg_database WHERE datistemplate = 
false;")
data = ODBC.query(dsn, "SELECT table_schema,table_name FROM 
information_schema.tables ORDER BY table_schema,table_name;")


The latest ODBC release was just merged in METADATA.jl, so if you do 
Pkg.add("ODBC") you should get the latest release/code. If you already had it 
installed, just run Pkg.update() to get the latest release.

Good luck!

-Jacob

On Tue, May 3, 2016 at 7:11 PM, Boylan, Ross 
<ross.boy...@ucsf.edu<mailto:ross.boy...@ucsf.edu>> wrote:
Attempting to post comments inline, even though my mailer is allergic.  Look 
for >>


From: karbar...@gmail.com<mailto:karbar...@gmail.com> 
[karbar...@gmail.com<mailto:karbar...@gmail.com>] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com<mailto:quinn.jac...@gmail.com>]
Sent: Tuesday, May 03, 2016 5:47 PM

I'm not sure I understand: I don't see DBI.jl nor PostgreSQL.jl at
pkg.julialang.org<http://pkg.julialang.org>, which tells me they aren't 
officially registered as packages. The difficulties you're seeing then aren't 
that surprising because they haven't felt comfortable enough to officially 
register.

>> You're right.  I got to them through https://github.com/JuliaDB

ODBC actually works fine on Linux/OSX through the unixODBC library. It can be 
installed through a variety of means, but works quite well from what I've seen. 
Postgres is actually next on my list to test with ODBC (so far the testing has 
focused
 on Teradata, SQL Server, and MySQL), so if I find time in the next few days, 
I'll try to put together an example and share it here.
>>  The status page indicates errors with julia 0.4 and Linux.  Does it matter? 
>>  Clicking on the build logs it appears the most recent build for 0.4, 124.4 
>> (https://travis-ci.org/JuliaDB/ODBC.jl) succeeded, so I'm not sure why the 
>> problem is showing (also the previous error looks like a problem with the 
>> configuration of the build machine rather than a problem with the package).

>> No joy with the alternate Postgresl package; 
>> https://github.com/NCarson/Postgres.jl/issues/3 has the details.

>> So I guess I'll give ODBC a whirl.  Thanks  for your help.



RE: [julia-users] Can't add PostgreSQL

2016-05-03 Thread Boylan, Ross
Attempting to post comments inline, even though my mailer is allergic.  Look 
for >>


From: karbar...@gmail.com [karbar...@gmail.com] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com]
Sent: Tuesday, May 03, 2016 5:47 PM

I'm not sure I understand: I don't see DBI.jl nor PostgreSQL.jl at 
pkg.julialang.org, which tells me they aren't officially registered as 
packages. The difficulties you're seeing then aren't that surprising because 
they haven't felt comfortable enough to officially register.

>> You're right.  I got to them through https://github.com/JuliaDB

ODBC actually works fine on Linux/OSX through the unixODBC library. It can be 
installed through a variety of means, but works quite well from what I've seen. 
Postgres is actually next on my list to test with ODBC (so far the testing has 
focused
 on Teradata, SQL Server, and MySQL), so if I find time in the next few days, 
I'll try to put together an example and share it here.
>>  The status page indicates errors with julia 0.4 and Linux.  Does it matter? 
>>  Clicking on the build logs it appears the most recent build for 0.4, 124.4 
>> (https://travis-ci.org/JuliaDB/ODBC.jl) succeeded, so I'm not sure why the 
>> problem is showing (also the previous error looks like a problem with the 
>> configuration of the build machine rather than a problem with the package).

>> No joy with the alternate Postgresl package; 
>> https://github.com/NCarson/Postgres.jl/issues/3 has the details.

>> So I guess I'll give ODBC a whirl.  Thanks  for your help.


RE: [julia-users] Can't add PostgreSQL

2016-05-03 Thread Boylan, Ross
That's odd.  After cloning the Postgres package, and then executing the script 
from the command line, it gets further:
$ julia trouble.jl
ERROR: LoadError: DBI API not fully implemented
 in fetchdf at /home/ross/.julia/v0.4/DBI/src/DBI.jl:97
 in include at ./boot.jl:261
 in include_from_node1 at ./loading.jl:320
 in process_options at ./client.jl:280
 in _start at ./client.jl:378
while loading /home/ross/PCORI/trouble.jl, in expression starting on line 6

Here are the 3 additional lines:
stmt = prepare(conn, "SELECT isim::int, id::int, x::int, t::double precision, 
y::int, regular::int, censor::int, wait::double precision from obs2619 where 
isim=1")
execute(stmt)
obs = fetchdf(stmt)

So the fetchdf fails because it's not implemented.
____
From: Boylan, Ross
Sent: Tuesday, May 03, 2016 4:06 PM
To: julia-users@googlegroups.com
Subject: RE: [julia-users] Can't add PostgreSQL

Thank you, Jacob, for the pointer.  I'm further along but still not there.  I 
did the clone and then build.  I'm not sure if the build was necessary, but it 
took awhile.

This process did not install DBI although it's listed as a requirement, and  
the installer gave no warning about it.  When that didn't work I cloned DBI.

I'm surprised that packages shown on http://pkg.julialang.org/ are not 
necessarily in https://github.com/JuliaLang/METADATA.jl.  Neither DBI nor 
PostgreSQL are there.  Does it mean anything if a  package is not in METADATA?

After all that I got
 julia> Pkg.clone("https://github.com/JuliaDB/DBI.jl.git;)
 INFO: Cloning DBI from https://github.com/JuliaDB/DBI.jl.git
 INFO: Computing changes...

!julia> include("/home/ross/PCORI/trouble.jl")

 INFO: Precompiling module DataArrays...
 INFO: Precompiling module DataFrames...
 ERROR: LoadError: UndefVarError: Postgres not defined
  in include at ./boot.jl:261
  in include_from_node1 at ./loading.jl:320
 while loading /home/ross/PCORI/trouble.jl, in expression starting on line 3

trouble.jl begins

using DBI
using PostgreSQL
conn = connect(Postgres, "localhost", "user", "word", "table")

Ideas?

Isn't ODBC just for MS Windows?

There's also https://github.com/NCarson/Postgres.jl; an announcement said it 
didn't implement the full DBI spec.

Ross


From: karbar...@gmail.com [karbar...@gmail.com] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com]
Sent: Tuesday, May 03, 2016 3:23 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Can't add PostgreSQL

I don't think the PostgreSQL.jl package was ever officially registered. You 
could try Pkg.clone("https://github.com/JuliaDB/PostgreSQL.jl;) to manually 
download/install it; you may need to do Pkg.build("PostgreSQL") as well after 
cloning. Alternatively, you might try the latest master of 
https://github.com/JuliaDB/ODBC.jl for interacting with databases.

-Jacob

On Tue, May 3, 2016 at 4:19 PM, Boylan, Ross 
<ross.boy...@ucsf.edu<mailto:ross.boy...@ucsf.edu>> wrote:
Am I misunderstanding how things work?  Is the package only available for julia 
0.3?
I have a julia obtained via git a few days ago, on the release-0.4 branch and 
built locally.
Seeing https://github.com/JuliaDB/PostgreSQL.jl I tried, from within an ESS 
session,
julia> Pkg.add("PostgreSQL")
 ERROR: unknown package PostgreSQL
  in error at ./error.jl:21
  [inlined code] from pkg/entry.jl:49
  in anonymous at task.jl:447
  in sync_end at ./task.jl:413
  [inlined code] from task.jl:422
  in add at pkg/entry.jl:64
  in add at pkg/entry.jl:73
  in anonymous at pkg/dir.jl:31
  in cd at file.jl:22
  in cd at pkg/dir.jl:31
  in add at pkg.jl:23

To see if this was a spelling mistake:
julia> Pkg.add("PostgresSQL")
 ERROR: unknown package PostgresSQL
...

I was able to add the DataFrames package and dependencies.

This is on a current Debian system, amd64 architecture.



RE: [julia-users] Can't add PostgreSQL

2016-05-03 Thread Boylan, Ross
Thank you, Jacob, for the pointer.  I'm further along but still not there.  I 
did the clone and then build.  I'm not sure if the build was necessary, but it 
took awhile.

This process did not install DBI although it's listed as a requirement, and  
the installer gave no warning about it.  When that didn't work I cloned DBI.

I'm surprised that packages shown on http://pkg.julialang.org/ are not 
necessarily in https://github.com/JuliaLang/METADATA.jl.  Neither DBI nor 
PostgreSQL are there.  Does it mean anything if a  package is not in METADATA?

After all that I got
 julia> Pkg.clone("https://github.com/JuliaDB/DBI.jl.git;)
 INFO: Cloning DBI from https://github.com/JuliaDB/DBI.jl.git
 INFO: Computing changes...

!julia> include("/home/ross/PCORI/trouble.jl")

 INFO: Precompiling module DataArrays...
 INFO: Precompiling module DataFrames...
 ERROR: LoadError: UndefVarError: Postgres not defined
  in include at ./boot.jl:261
  in include_from_node1 at ./loading.jl:320
 while loading /home/ross/PCORI/trouble.jl, in expression starting on line 3

trouble.jl begins

using DBI
using PostgreSQL
conn = connect(Postgres, "localhost", "user", "word", "table")

Ideas?

Isn't ODBC just for MS Windows?

There's also https://github.com/NCarson/Postgres.jl; an announcement said it 
didn't implement the full DBI spec.

Ross


From: karbar...@gmail.com [karbar...@gmail.com] on behalf of Jacob Quinn 
[quinn.jac...@gmail.com]
Sent: Tuesday, May 03, 2016 3:23 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Can't add PostgreSQL

I don't think the PostgreSQL.jl package was ever officially registered. You 
could try Pkg.clone("https://github.com/JuliaDB/PostgreSQL.jl;) to manually 
download/install it; you may need to do Pkg.build("PostgreSQL") as well after 
cloning. Alternatively, you might try the latest master of 
https://github.com/JuliaDB/ODBC.jl for interacting with databases.

-Jacob

On Tue, May 3, 2016 at 4:19 PM, Boylan, Ross 
<ross.boy...@ucsf.edu<mailto:ross.boy...@ucsf.edu>> wrote:
Am I misunderstanding how things work?  Is the package only available for julia 
0.3?
I have a julia obtained via git a few days ago, on the release-0.4 branch and 
built locally.
Seeing https://github.com/JuliaDB/PostgreSQL.jl I tried, from within an ESS 
session,
julia> Pkg.add("PostgreSQL")
 ERROR: unknown package PostgreSQL
  in error at ./error.jl:21
  [inlined code] from pkg/entry.jl:49
  in anonymous at task.jl:447
  in sync_end at ./task.jl:413
  [inlined code] from task.jl:422
  in add at pkg/entry.jl:64
  in add at pkg/entry.jl:73
  in anonymous at pkg/dir.jl:31
  in cd at file.jl:22
  in cd at pkg/dir.jl:31
  in add at pkg.jl:23

To see if this was a spelling mistake:
julia> Pkg.add("PostgresSQL")
 ERROR: unknown package PostgresSQL
...

I was able to add the DataFrames package and dependencies.

This is on a current Debian system, amd64 architecture.



[julia-users] Can't add PostgreSQL

2016-05-03 Thread Boylan, Ross
Am I misunderstanding how things work?  Is the package only available for julia 
0.3?
I have a julia obtained via git a few days ago, on the release-0.4 branch and 
built locally.
Seeing https://github.com/JuliaDB/PostgreSQL.jl I tried, from within an ESS 
session,
julia> Pkg.add("PostgreSQL")
 ERROR: unknown package PostgreSQL
  in error at ./error.jl:21
  [inlined code] from pkg/entry.jl:49
  in anonymous at task.jl:447
  in sync_end at ./task.jl:413
  [inlined code] from task.jl:422
  in add at pkg/entry.jl:64
  in add at pkg/entry.jl:73
  in anonymous at pkg/dir.jl:31
  in cd at file.jl:22
  in cd at pkg/dir.jl:31
  in add at pkg.jl:23

To see if this was a spelling mistake:
julia> Pkg.add("PostgresSQL")
 ERROR: unknown package PostgresSQL
...

I was able to add the DataFrames package and dependencies.

This is on a current Debian system, amd64 architecture.

RE: [julia-users] Roadmap

2014-12-10 Thread Boylan, Ross
Somewhere, not necessarily on the front page, some tips for people wondering 
where the project is heading would be good.  Not a list of plans, but orienting 
info like no one knows, and an explanation of how to get a sense of current 
(i.e., for 0.4 right now) and future issues from the bug tracker.

I found the tip I got awhile ago to search on a particular tag (0.4?) and sort 
by bug/issue activity pretty helpful.

BTW, is 0.4 still in a you don't want to go there state for users of julia?

Ross

From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
David Anthoff [anth...@berkeley.edu]
Sent: Wednesday, December 10, 2014 5:38 PM
To: julia-users@googlegroups.com
Subject: RE: [julia-users] Roadmap

Thanks, this is really useful! I would very much cherish an email like this 
maybe after a release from some of the core team members. It just gives a nice 
insight into the current plans.

Cheers,
David

From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On 
Behalf Of John Myles White
Sent: Wednesday, December 10, 2014 4:56 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Roadmap

FWIW, my sense is that no one really knows what's going to happen between 0.4 
and 1.0. There are lots of projects that are seen as essential before 1.0, but 
many of those are tenatively on the 0.4 release targets (static compilation, 
array views, package documentation, etc.).

At JuliaCon, I realized that I was one of the longest standing users of Julia 
-- many people at JuliaCon had never tried Julia 0.1 and therefore don't 
remember how much the 0.2 release improved the language and redefined the way 
Julia code was written. I feel like 0.4 is going to be a similar release: a lot 
of the most egregious problems with the current version of Julia are going to 
be fixed. But once those problems are solved, it seems hard to believe that we 
won't start realizing that there are lots of parts of the language that could 
be cleaned up before 1.0. My sense is that Julia, like ggplot2, will start to 
be mature enough for almost all users well before 1.0 is released, but that1.0 
will still thankfully have the freedom to make any changes that are necessary 
before something gets declared as the finished product.

 -- John

On Dec 10, 2014, at 4:45 PM, David Anthoff 
anth...@berkeley.edumailto:anth...@berkeley.edu wrote:


I hear you, and I didn’t think much before sending my email. Couple of points:

I totally agree this should certainly not be on the homepage. I also agree that 
there is no need for a detailed schedule, deadlines or anything like that. I 
think the only thing that would be immensely helpful at least for me is just a 
very high level idea of what the core team is thinking about a roadmap/timing. 
Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now 
expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess 
is that the core team has an idea about those kinds of questions, and it would 
be great if you could share that kind of stuff from time to time. Maybe one 
idea here would be that the core team just sends out a brief email after a 
major release what the current thinking is about the next version and the road 
to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, 
but that in itself would also be valuable information for us users.

I am following the issue tracker and am subscribed to the email lists, and I 
don’t get any sense/picture about those kind of high level questions from those 
sources.

Cheers,
David

From: julia-users@googlegroups.commailto:julia-users@googlegroups.com 
[mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman
Sent: Wednesday, December 10, 2014 4:31 PM
To: julia-users@googlegroups.commailto:julia-users@googlegroups.com
Subject: Re: [julia-users] Re: home page content

-1 on trying to put plans, schedule, roadmap on the website. This week in 
Julia was a great contribution to the community but evidently took more effort 
than Matt had time to keep up with.

New features get developed as the PR's for them get worked on and finished. You 
can subscribe to just the subset of issues/PR's for things you (along with 
everyone else) are eagerly awaiting. Better yet, help with testing and code 
review if you can.

We have been doing a good job of monthly backport bugfix releases, we should be 
able to continue doing that. But 0.4 is still unstable and has several 
big-ticket items still open and being worked on (check the milestones on 
github). It's too early to try to make time estimates, if people are impatient 
and want a release sooner it's not going to be possible without punting on a 
number of targeted features and pushing them back to 0.5 or later.


On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote:
I think it would please everyone if you moved daily televised scrums.


On Wednesday, December 10, 2014