[julia-users] Using rank() and nullspace() with sparse matrix

[Alex Dowling]

When I use rank() on a sparse matrix, I get the following error:

ERROR: LoadError: MethodError: `svdvals!` has no method matching 
svdvals!(::SparseMatrixCSC{Float64,Int64})
 in rank at linalg/generic.jl:300

Small example:
A = sprand(5,5,0.6)
rank(A)

Is there an alternate way to calculate the rank of a sparse matrix? Same 
question for calculating the nullspace basis.

Thanks,
Alex

Re: [julia-users] Re: sparse rank, sparse nullspace, sparse linear algebra over rationals?

[Alex Dowling]

Hello Laurent,

Thanks. I've used svds in MATLAB before with some success. Ideally I'd like 
to have a pure Julia implementation. Do you know of any appropriate 
packages?

Alex

On Tuesday, April 12, 2016 at 4:41:06 PM UTC-5, Laurent Bartholdi wrote:
>
> @Stefan: sorry, I had meant that Julia doesn't yet have sparse 
> non-(real/complex) matrices. However, I see that that's wrong too: I can 
> create sparse matrices with Int64 entries. It would now make a lot of sense 
> to implement the algorithms in linbox so as to compute rank etc. of Int or 
> BigInt sparse matrices.
>
> @Alex: if you're only interested in the 10 lowest singular values of a 
> sparse matrix, then MATLAB is quite good at it. Here's from the doc:
>
> S = svds(A,K,SIGMA) computes the K singular values closest to the 
> scalar shift SIGMA.  For example, S = svds(A,K,0) computes the K
> smallest singular values.
>
>
> On Tue, 12 Apr 2016 at 22:29 Stefan Karpinski <ste...@karpinski.org 
> > wrote:
>
>> Julia does have complex sparse matrices:
>>
>> julia> C = sparse(rand(1:10, 10), rand(1:10, 10), randn(10) + 
>> randn(10)im, 10, 10)
>> 10x10 sparse matrix with 9 Complex{Float64} entries:
>>   [7 ,  1]  =  1.1711-0.587334im
>>   [5 ,  3]  =  0.940755+1.00755im
>>   [1 ,  4]  =  1.51515-1.77335im
>>   [4 ,  4]  =  -0.815624-0.678038im
>>   [9 ,  5]  =  -0.598111-0.970266im
>>   [2 ,  6]  =  0.671339-1.07398im
>>   [7 ,  6]  =  -1.69705+1.08698im
>>   [10,  7]  =  -1.32233-1.88083im
>>   [7 , 10]  =  1.26453-0.399423im
>>
>> How much you can do with these depends on what kinds of special methods 
>> are implemented, but you can call eigs on them, which lets you figure out 
>> what the rank is:
>>
>> julia> map(norm,eigs(C)[1])
>> 6-element Array{Float64,1}:
>>  1.74612
>>  1.74612
>>  1.06065
>>  4.28017e-6
>>  4.28016e-6
>>  4.28016e-6
>>
>> In this case, for example, the matrix is rank 3.
>>
>> On Tue, Apr 12, 2016 at 3:29 PM, Laurent Bartholdi <laurent@gmail.com 
>> > wrote:
>>
>>> It's tricky, but sprank is definitely not competitive with finer 
>>> implementations. As you certainly know, rank calculations are very 
>>> unstable, so if your matrix has integer entries you should prefer an exact 
>>> method, or a calculation in a finite field. (Sadly, Julia does not contain, 
>>> yet, sparse matrices with non-real/complex entries). See 
>>> http://dx.doi.org.sci-hub.io/10.1145/2513109.2513116 for a recent 
>>> discussion of sparse matrix rank.
>>>
>>> The following routines compute null spaces. To obtain the rank, you 
>>> subtract the nullspace dimension from the row space dimension (i.e. number 
>>> of columns). If you want pure Julia, you could try
>>>
>>> function julia_null_space{T}(A::SparseMatrixCSC{T})
>>> SVD = svdfact(full(A), thin = false)
>>> any(1.e-8 .< SVD.S .< 1.e-4) && error("Values are dangerously small")
>>> indstart = sum(SVD.S .> 1.e-8) + 1
>>> nullspace = SVD.Vt[indstart:end,:]'
>>> return sparse(nullspace .* (abs(nullspace) .> 1.e-8))
>>> end
>>>
>>> If you have MATLAB, the following is faster:
>>>
>>> using MATLAB
>>> global m_session = MSession()
>>>
>>> function matlab_null_space{T}(A::SparseMatrixCSC{T})
>>> m, n = size(A)
>>> (m == 0 || n == 0) && return speye(T, n)
>>>
>>> put_variable(m_session,:A,convert(SparseMatrixCSC{Float64},A))
>>> eval_string(m_session,"N = spqr_null(A,struct('tol',1.e-8));")
>>> eval_string(m_session,"ns = spqr_null_mult(N,speye(size(N.X,2)),1);")
>>> eval_string(m_session,"ns = sparseclean(ns,1.e-8);")
>>> ns = get_mvariable(m_session,:ns)
>>> return jvariable(ns)
>>> end
>>>
>>> Finally, if your matrices are over Z, Q or a finite field, do give a 
>>> look to linbox (http://www.linalg.org/linbox-html/index.html).
>>>
>>> HTH, Laurent
>>>
>>> On Tue, 12 Apr 2016 at 20:49 Alex Dowling <alex...@gmail.com 
>>> > wrote:
>>>
>>>> Hello,
>>>>
>>>> I am also looking to compute the (approximate) rank of a sparse matrix. 
>>>> For my applications I'm consider dimensions of 10k - 100k by 10k - 100k, 
>>>> not millions by millions. I've previously done this in MATLAB using the 
>>>> sprank 
>>>> command <ht

Re: [julia-users] Re: sparse rank, sparse nullspace, sparse linear algebra over rationals?

[Alex Dowling]

Hello Laurent,

Thank you for the prompt reply. I noticed your pure Julia code converted to 
a full matrix. Any luck with the sparse SVD routines?

I thought more about my application, and I really want to know the smallest 
n < 10 singular values and associated vectors for the system. Tight 
tolerances are not required. I see that eigs() supports specifying the 
largest or smallest eigenvalues and the number, but SVD does not have a 
similar option. The sparse matricies I'm considering an not symmetric.

Alex

On Tuesday, April 12, 2016 at 2:29:56 PM UTC-5, Laurent Bartholdi wrote:
>
> It's tricky, but sprank is definitely not competitive with finer 
> implementations. As you certainly know, rank calculations are very 
> unstable, so if your matrix has integer entries you should prefer an exact 
> method, or a calculation in a finite field. (Sadly, Julia does not contain, 
> yet, sparse matrices with non-real/complex entries). See 
> http://dx.doi.org.sci-hub.io/10.1145/2513109.2513116 for a recent 
> discussion of sparse matrix rank.
>
> The following routines compute null spaces. To obtain the rank, you 
> subtract the nullspace dimension from the row space dimension (i.e. number 
> of columns). If you want pure Julia, you could try
>
> function julia_null_space{T}(A::SparseMatrixCSC{T})
> SVD = svdfact(full(A), thin = false)
> any(1.e-8 .< SVD.S .< 1.e-4) && error("Values are dangerously small")
> indstart = sum(SVD.S .> 1.e-8) + 1
> nullspace = SVD.Vt[indstart:end,:]'
> return sparse(nullspace .* (abs(nullspace) .> 1.e-8))
> end
>
> If you have MATLAB, the following is faster:
>
> using MATLAB
> global m_session = MSession()
>
> function matlab_null_space{T}(A::SparseMatrixCSC{T})
> m, n = size(A)
> (m == 0 || n == 0) && return speye(T, n)
>
> put_variable(m_session,:A,convert(SparseMatrixCSC{Float64},A))
> eval_string(m_session,"N = spqr_null(A,struct('tol',1.e-8));")
> eval_string(m_session,"ns = spqr_null_mult(N,speye(size(N.X,2)),1);")
> eval_string(m_session,"ns = sparseclean(ns,1.e-8);")
> ns = get_mvariable(m_session,:ns)
> return jvariable(ns)
> end
>
> Finally, if your matrices are over Z, Q or a finite field, do give a look 
> to linbox (http://www.linalg.org/linbox-html/index.html).
>
> HTH, Laurent
>
> On Tue, 12 Apr 2016 at 20:49 Alex Dowling <alex...@gmail.com > 
> wrote:
>
>> Hello,
>>
>> I am also looking to compute the (approximate) rank of a sparse matrix. 
>> For my applications I'm consider dimensions of 10k - 100k by 10k - 100k, 
>> not millions by millions. I've previously done this in MATLAB using the 
>> sprank 
>> command <http://www.mathworks.com/help/matlab/ref/sprank.html>. Does 
>> anyone have recommendations for similar functions in Julia?
>>
>> Thanks,
>> Alex
>>
>>
>> On Tuesday, November 17, 2015 at 3:08:29 AM UTC-6, ming...@irt-systemx.fr 
>> wrote:
>>>
>>> hello,
>>>
>>> getting the rank of a huge sparse matrix is mathematically difficult
>>>
>>>
>>> http://math.stackexchange.com/questions/554777/rank-computation-of-large-matrices
>>>
>>> bests,
>>> M.
>>>
>>> Le lundi 16 novembre 2015 22:12:04 UTC+1, Laurent Bartholdi a écrit :
>>>>
>>>> Hello world,
>>>> I'm new at julia, and trying it out as a replacement for matlab and 
>>>> other computer algebra systems. I'm a bit stuck with sparse matrices: I 
>>>> have largeish matrices (10^6 x 10^6) that are very sparse, and
>>>> want to know their rank and nullspace. (the matrices decompose by 
>>>> blocks, so the nullspace should be expressible as a sparse matrix).
>>>>
>>>> I tried with the latest (github) julia 0.5:
>>>>
>>>> julia> nullspace(sparse([1],[1],[1]))
>>>> ERROR: MethodError: `nullspace` has no method matching 
>>>> nullspace(::SparseMatrixCSC{Int64,Int64})
>>>>
>>>> julia> nullspace(full(sparse([1],[1],[1])))
>>>> 1x0 Array{Float64,2} # I'm a bit unhappy here, I was hoping to get a 
>>>> rational answer.
>>>>
>>>> julia> nullspace([1//1])
>>>> 1x0 Array{Float32,2} # yikes! I'm down to 32 bits floats now.
>>>>
>>>> julia> rank(sparse([1],[1],[1.0]))
>>>> ERROR: MethodError: `svdvals!` has no method matching 
>>>> svdvals!(::SparseMatrixCSC{Float64,Int64})
>>>>
>>>> julia> rank([1.0])
>>>> ERROR: MethodError: `rank` has no method matching 
>>>> rank(::Array{Float64,1})
>>>>
>>>> julia> rank([1 0;0 1])
>>>> 2 # finally something that works...
>>>>
>>>> Many thanks in advance! Laurent
>>>>
>>>> -- 
> Laurent Bartholdi
> DMA, École Normale Supérieure, 45 rue d'Ulm, 75005 Paris. +33 14432 2060
> Mathematisches Institut, Universität Göttingen, Bunsenstrasse 3-5, D-37073 
> Göttingen. +49 551 39 7826
>

[julia-users] Re: sparse rank, sparse nullspace, sparse linear algebra over rationals?

[Alex Dowling]

Hello,

I am also looking to compute the (approximate) rank of a sparse matrix. For 
my applications I'm consider dimensions of 10k - 100k by 10k - 100k, not 
millions by millions. I've previously done this in MATLAB using the sprank 
command . Does anyone 
have recommendations for similar functions in Julia?

Thanks,
Alex

On Tuesday, November 17, 2015 at 3:08:29 AM UTC-6, ming...@irt-systemx.fr 
wrote:
>
> hello,
>
> getting the rank of a huge sparse matrix is mathematically difficult
>
>
> http://math.stackexchange.com/questions/554777/rank-computation-of-large-matrices
>
> bests,
> M.
>
> Le lundi 16 novembre 2015 22:12:04 UTC+1, Laurent Bartholdi a écrit :
>>
>> Hello world,
>> I'm new at julia, and trying it out as a replacement for matlab and other 
>> computer algebra systems. I'm a bit stuck with sparse matrices: I have 
>> largeish matrices (10^6 x 10^6) that are very sparse, and
>> want to know their rank and nullspace. (the matrices decompose by blocks, 
>> so the nullspace should be expressible as a sparse matrix).
>>
>> I tried with the latest (github) julia 0.5:
>>
>> julia> nullspace(sparse([1],[1],[1]))
>> ERROR: MethodError: `nullspace` has no method matching 
>> nullspace(::SparseMatrixCSC{Int64,Int64})
>>
>> julia> nullspace(full(sparse([1],[1],[1])))
>> 1x0 Array{Float64,2} # I'm a bit unhappy here, I was hoping to get a 
>> rational answer.
>>
>> julia> nullspace([1//1])
>> 1x0 Array{Float32,2} # yikes! I'm down to 32 bits floats now.
>>
>> julia> rank(sparse([1],[1],[1.0]))
>> ERROR: MethodError: `svdvals!` has no method matching 
>> svdvals!(::SparseMatrixCSC{Float64,Int64})
>>
>> julia> rank([1.0])
>> ERROR: MethodError: `rank` has no method matching rank(::Array{Float64,1})
>>
>> julia> rank([1 0;0 1])
>> 2 # finally something that works...
>>
>> Many thanks in advance! Laurent
>>
>>

Re: [julia-users] Redirecting println output

[Alex Dowling]

I just tried script. That works.

On Thursday, April 7, 2016 at 9:43:43 AM UTC-5, Yichao Yu wrote:
>
> On Thu, Apr 7, 2016 at 9:45 AM, Alex Dowling <alex...@gmail.com 
> > wrote: 
> > Hello Yichao, 
> > 
> > Vanished. I expect println output between other output that is properly 
> > redirected. I should also mention println output is not properly 
> redirected 
> > (i.e., vanishes) using nohup. 
>
> So the output does not appear even when the whole script exit? 
> Also see my link about script below. 
>
> > 
> > Alex 
> > 
> > On Thursday, April 7, 2016 at 8:05:30 AM UTC-5, Yichao Yu wrote: 
> >> 
> >> On Thu, Apr 7, 2016 at 8:12 AM, Alex Dowling <alex...@gmail.com> 
> wrote: 
> >> > Hello, 
> >> > 
> >> > I have a Julia script that does some analysis and includes println() 
> >> > statements. When I use julia my_code.jl > output.txt 2>&1 from a 
> Linux 
> >> > command line, output from optimization solvers (IPOPT, Gurobi) are 
> >> > redirected to output.txt but output from println() just vanishes. 
> They 
> >> > aren't in output.txt or printed to the console. 
> >> 
> >> Are they vanished or delayed? The output is buffered. 
> >> 
> >> > 
> >> > One option I found (haven't tested) is using redirect_stout() and 
> >> > redirect_sterr(): https://github.com/JuliaLang/julia/issues/12050. 
> >> > Ideally 
> >> > I'd like to redirect println output without modifying the script. 
> >> 
> >> http://man7.org/linux/man-pages/man1/script.1.html 
> >> 
> >> > 
> >> > Thanks, 
> >> > Alex 
>

Re: [julia-users] Redirecting println output

[Alex Dowling]

Hello Yichao,

Vanished. I expect println output between other output that is properly 
redirected. I should also mention println output is not properly redirected 
(i.e., vanishes) using nohup.

Alex

On Thursday, April 7, 2016 at 8:05:30 AM UTC-5, Yichao Yu wrote:
>
> On Thu, Apr 7, 2016 at 8:12 AM, Alex Dowling <alex...@gmail.com 
> > wrote: 
> > Hello, 
> > 
> > I have a Julia script that does some analysis and includes println() 
> > statements. When I use julia my_code.jl > output.txt 2>&1 from a Linux 
> > command line, output from optimization solvers (IPOPT, Gurobi) are 
> > redirected to output.txt but output from println() just vanishes. They 
> > aren't in output.txt or printed to the console. 
>
> Are they vanished or delayed? The output is buffered. 
>
> > 
> > One option I found (haven't tested) is using redirect_stout() and 
> > redirect_sterr(): https://github.com/JuliaLang/julia/issues/12050. 
> Ideally 
> > I'd like to redirect println output without modifying the script. 
>
> http://man7.org/linux/man-pages/man1/script.1.html 
>
> > 
> > Thanks, 
> > Alex 
>

[julia-users] Redirecting println output

[Alex Dowling]

Hello,

I have a Julia script that does some analysis and includes println() 
statements. When I use julia my_code.jl > output.txt 2>&1 from a Linux 
command line, output from optimization solvers (IPOPT, Gurobi) are 
redirected to output.txt but output from println() just vanishes. They 
aren't in output.txt or printed to the console.

One option I found (haven't tested) is using redirect_stout() and 
redirect_sterr(): https://github.com/JuliaLang/julia/issues/12050. Ideally 
I'd like to redirect println output without modifying the script.

Thanks,
Alex

[julia-users] Re: PyPlot and art3D

[Alex Dowling]

Thanks!

On Wednesday, February 3, 2016 at 2:41:03 PM UTC-6, Steven G. Johnson wrote:
>
>
>
> On Tuesday, February 2, 2016 at 5:49:48 PM UTC-5, Alex Dowling wrote:
>>
>> Hello,
>>
>> I'm having trouble finding examples for using art3D through the PyPlot 
>> module in Julia. Ultimately I'd like to reproduce this example: 
>> http://stackoverflow.com/questions/4622057/plotting-3d-polygons-in-python-matplotlib
>>  
>> (Python, code below)
>>
>
> This works for me (in Julia 0.4):
>
> using PyPlot
> const Poly3DCollection = PyPlot.mplot3d[:art3d][:Poly3DCollection]
> fig = figure()
> ax = Axes3D(fig)
> x = [0,1,1,0]
> y = [0,0,1,1]
> z = [0,1,0,1]
> verts = (collect(zip(x,y,z)),)
> ax[:add_collection3d](Poly3DCollection(verts)) 
>
>

[julia-users] PyPlot and art3D

[Alex Dowling]

Hello,

I'm having trouble finding examples for using art3D through the PyPlot 
module in Julia. Ultimately I'd like to reproduce this 
example: 
http://stackoverflow.com/questions/4622057/plotting-3d-polygons-in-python-matplotlib
 
(Python, code below)

from mpl_toolkits.mplot3d import Axes3Dfrom mpl_toolkits.mplot3d.art3d import 
Poly3DCollectionimport matplotlib.pyplot as plt
fig = plt.figure()
ax = Axes3D(fig)
x = [0,1,1,0]
y = [0,0,1,1]
z = [0,1,0,1]
verts = [zip(x, y,z)]
ax.add_collection3d(Poly3DCollection(verts))
plt.show()


I can't figure out how to call Poly3DCollection correctly in Julia. Any 
suggestions?

Thank you in advance for the advice.

Alex

[julia-users] Re: Defining and using Python classes with PyCall

[Alex Dowling]

Thank you for the prompt response.

On Tuesday, January 5, 2016 at 10:18:51 AM UTC-6, Steven G. Johnson wrote:
>
> I haven't devised an easy way to define new Python classes from Julia via 
> PyCall.
>
> It's possible (see how I define new IO subclasses in 
> https://github.com/stevengj/PyCall.jl/blob/master/src/io.jl) but it is 
> rather low-level at the moment, close to how you would define new Python 
> classes from C.  The key function is PyCall.pyjlwrap_type.
>

[julia-users] Defining and using Python classes with PyCall

[Alex Dowling]

Hello Everyone,

I'd like to use two classes written in Python in Julia. More specifically, 
I'd like to get the follow code example in Python (taken verbatim from here 
)
 
running with PyCall, PyPlot and Julia.

import matplotlib.pyplot as pltimport matplotlib.text as mpl_text
class AnyObject(object):
def __init__(self, text, color):
self.my_text = text
self.my_color = color
class AnyObjectHandler(object):
def legend_artist(self, legend, orig_handle, fontsize, handlebox):
print orig_handle
x0, y0 = handlebox.xdescent, handlebox.ydescent
width, height = handlebox.width, handlebox.height
patch = mpl_text.Text(x=0, y=0, text=orig_handle.my_text, 
color=orig_handle.my_color, verticalalignment=u'baseline', 
horizontalalignment=u'left', 
multialignment=None, 
fontproperties=None, rotation=45, 
linespacing=None, 
rotation_mode=None)
handlebox.add_artist(patch)
return patch

obj_0 = AnyObject("A", "purple")
obj_1 = AnyObject("B", "green")

plt.legend([obj_0, obj_1], ['Model Name A', 'Model Name B'],
   handler_map={obj_0:AnyObjectHandler(), obj_1:AnyObjectHandler()})

plt.show()


I've been using Julia for about 1/2 a year, and its been over 1/2 a decade 
since I used Python seriously. I've looked at the PyCall documentation 
, but I'm still rather confused.

Thanks,
Alex

[julia-users] Re: Use unicode character in PyPlot

[Alex Dowling]

Andreas and Steve,

Thank you for the help. I'm now using text and figured out how to get 
matplotlib to use the correct font.

Naturally, I'd like to add the unicode symbols to the legend I'm generating 
with PyPlot. This stackoverflow post 
<http://stackoverflow.com/questions/27174425/how-to-add-a-string-as-the-artist-in-matplotlib-legend>
 suggests 
a legend handler goes exist for text objects, and I'll need to implement 
one. The post gives some Python code. How exactly would I go about 
implementing this custom handler in Julia? I suspect this is a more general 
PyCall question. If this group isn't an appropriate place for my question, 
any recommends on where I should ask it?

Thanks,
Alex

On Friday, January 1, 2016 at 11:17:11 AM UTC-6, Andreas Lobinger wrote:
>
> Hello c,
>
> On Friday, January 1, 2016 at 5:01:37 AM UTC+1, Alex Dowling wrote:
>>
>> I'd like to use a unicode character as a marker in plots created with 
>> PyPlot. Here is a minimal code example:
>>
>> x = [ 0 1 2]
>> y = x
>> scatter(x,y,marker="\U0001F4A7")
>>
>> which produces the following error:
>>
>>
>>  
>
>> My questions:
>>
>>1. Any suggestions regarding the error above?
>>
>>
> as far as i can see the mathplotlib documentation about scatter and 
> entries in marker (
> http://matplotlib.org/api/markers_api.html#module-matplotlib.markers) 
> there are only certain entries in marker allowed and no free text. If you 
> have a limited set of x,y you could use pyplot.text which handles local 
> coordinates and unicode text.
>  
>

[julia-users] Use unicode character in PyPlot

[Alex Dowling]

Hello,

I'd like to use a unicode character as a marker in plots created with 
PyPlot. Here is a minimal code example:

x = [ 0 1 2]
y = x
scatter(x,y,marker="\U0001F4A7")

which produces the following error:

ERROR: PyError (:PyObject_Call) 
UnicodeEncodeError('ascii', u'\U0001f4a7', 0, 1, 'ordinal not in 
range(128)')
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 3087, in 
scatter
linewidths=linewidths, verts=verts, **kwargs)
  File "/usr/lib/pymodules/python2.7/matplotlib/axes.py", line 6296, in 
scatter
marker_obj = mmarkers.MarkerStyle(marker)
  File "/usr/lib/pymodules/python2.7/matplotlib/markers.py", line 162, in 
__init__
self.set_marker(marker)
  File "/usr/lib/pymodules/python2.7/matplotlib/markers.py", line 236, in 
set_marker
raise ValueError('Unrecognized marker style {}'.format(marker))

 [inlined code] from /home/adowling2/.julia/v0.4/PyCall/src/exception.jl:81
 in pycall at /home/adowling2/.julia/v0.4/PyCall/src/PyCall.jl:79
 in scatter at /home/adowling2/.julia/v0.4/PyPlot/src/PyPlot.jl:460

Interestingly, title("\U0001F4A7") half works. It adds a title with a box 
in place of the unicode character. I suspect this is because the default 
font does not include the character.

My questions:

   1. Any suggestions regarding the error above?
   2. Any suggestions to change the font used for the title command?

Thanks,
Alex

Re: [julia-users] Does the Windows Binary include Python?

[Alex Dowling]

Thank you for the suggestions. I'm actually helping a colleague debug this, 
so I don't know the exact sequence of commands that were tried. I'm going 
to recommend just nuking the Julia v0.4 user directory, upgrading to the 
Julia v0.4 stable release and trying to install PyPlot again. If this 
doesn't work, I'll post again here.

On Thursday, October 8, 2015 at 10:39:06 AM UTC-5, Steven G. Johnson wrote:
>
>
>
> On Thursday, October 8, 2015 at 11:24:28 AM UTC-4, Steven G. Johnson wrote:
>>
>>
>>
>> On Thursday, October 8, 2015 at 10:42:38 AM UTC-4, Luthaf wrote:
>>>
>>> I guess that is a bug in PyCall, so you should do your bug report there 
>>> : https://github.com/stevengj/PyCall.jl/issues/new. Please ping me 
>>> (@Luthaf) on the issue, because this might be related with Conda.jl
>>>
>>
>> Conda.PYTHONDIR was defined in recent versions of Conda, so I'm guessing 
>> you have an out-of-date Conda.  That, combined with the fact that your 
>> PyCall package was not already built the first time you tried it, leads me 
>> to suspect that you did a Pkg.checkout("PyCall") at some point to get the 
>> latest (master) version.  By doing this, you turned off automatic 
>> dependency management, which is why you were able to update PyCall without 
>> updating Conda.
>>
>
> (Either that, or you're hitting 
> https://github.com/JuliaLang/julia/issues/13458 ... just start a new 
> Julia session and do Pkg.update() followed by Pkg.build("PyCall") ...) 
>

Re: [julia-users] Does the Windows Binary include Python?

[Alex Dowling]

Update:

This morning I tried installing Gadfly, which required installing a bunch 
of other packages and updating others. PyCall is now broken on my Linux 
system (running 0.4.0-rc3). It's the same "PYTHONDIR not defined error". 
The Linux system (Mint) has python installed. Restarting Julia, running 
Pkg.update() and then Pkg.build("PyCall") fixed the issue on the Linux 
machine. I haven't sorted out the problem with the Windows machine yet.

On Monday, October 12, 2015 at 9:59:27 AM UTC-5, Alex Dowling wrote:
>
> Thank you for the suggestions. I'm actually helping a colleague debug 
> this, so I don't know the exact sequence of commands that were tried. I'm 
> going to recommend just nuking the Julia v0.4 user directory, upgrading to 
> the Julia v0.4 stable release and trying to install PyPlot again. If this 
> doesn't work, I'll post again here.
>
> On Thursday, October 8, 2015 at 10:39:06 AM UTC-5, Steven G. Johnson wrote:
>>
>>
>>
>> On Thursday, October 8, 2015 at 11:24:28 AM UTC-4, Steven G. Johnson 
>> wrote:
>>>
>>>
>>>
>>> On Thursday, October 8, 2015 at 10:42:38 AM UTC-4, Luthaf wrote:
>>>>
>>>> I guess that is a bug in PyCall, so you should do your bug report there 
>>>> : https://github.com/stevengj/PyCall.jl/issues/new. Please ping me 
>>>> (@Luthaf) on the issue, because this might be related with Conda.jl
>>>>
>>>
>>> Conda.PYTHONDIR was defined in recent versions of Conda, so I'm guessing 
>>> you have an out-of-date Conda.  That, combined with the fact that your 
>>> PyCall package was not already built the first time you tried it, leads me 
>>> to suspect that you did a Pkg.checkout("PyCall") at some point to get the 
>>> latest (master) version.  By doing this, you turned off automatic 
>>> dependency management, which is why you were able to update PyCall without 
>>> updating Conda.
>>>
>>
>> (Either that, or you're hitting 
>> https://github.com/JuliaLang/julia/issues/13458 ... just start a new 
>> Julia session and do Pkg.update() followed by Pkg.build("PyCall") ...) 
>>
>

[julia-users] Does the Windows Binary include Python?

[Alex Dowling]

Hello,

I am helping a colleague debug some errors with the PyPlot in the Windows 
binary [Version 0.4.0-rc1 (2015-09-09 16:07 UTC)]. Here are the error 
messages:

julia> using Pyplot

INFO: Precompiling module Pyplot...

ERROR: LoadError: PyCall not properly installed. Please run 
Pkg.build("PyCall") in error at error.jl:21

while loading C:\Users\Tian\.julia\v0.4\PyCall\src\PyCall.jl, in expression 
starting on line 36

ERROR: LoadError: Failed to precompile PyCall to 
C:\Users\Tian\.julia\lib\v0.4\PyCall.ji in error at error.jl:21

while loading C:\Users\Tian\.julia\v0.4\Pyplot\src\Pyplot.jl, in expression 
starting on line 5

ERROR: Failed to precompile Pyplot to 
C:\Users\Tian\.julia\lib\v0.4\Pyplot.ji in error at error.jl:21

 


julia> Pkg.build("PyCall")

INFO: Building PyCall

INFO: No system-wide Python was found; got the following error:

ErrorException("failed process: Process(`python -c \"import 
distutils.sysconfig;

 print(distutils.sysconfig.get_config_var('VERSION'))\"`, 
ProcessExited(3221225781)) [3221225781]")

using the Python distribution in the Conda package

===[ ERROR: PyCall 
]

 

 

LoadError: UndefVarError: PYTHONDIR not defined

while loading C:\Users\Tian\.julia\v0.4\PyCall\deps\build.jl, in expression 
starting on line 17

 



 

 

[ BUILD ERRORS 
]

 

 

WARNING: PyCall had build errors.

 

 - packages with build errors remain installed in C:\Users\Tian\.julia\v0.4

 - build the package(s) and all dependencies with `Pkg.build("PyCall")`

 - build a single package by running its `deps/build.jl` script

 




Shouldn't the Windows binary package include Python, and thus a system wide 
install wouldn't be required? The documentation for PyCall 
(https://github.com/stevengj/PyCall.jl) 
suggests it should install Conda if it can't find Python automatically. If 
this is a bug, I'd appreciate advise on whether I should report it in the 
Julia or PyCall repositories.


Thanks,

Alex

[julia-users] Re: Optimal Control using Dynamic Programming

[Alex Dowling]

I'm also bias to recommend formulating a nonlinear programming problem and 
using JuMP with IPOPT. This approach has been successfully demonstrated in 
the chemical engineering controls/systems literature on some rather nasty 
large nonlinear systems.

Here is a sample problem for nonlinear control of a chemical reactor using 
the discretized NLP approach in GAMS:
http://cepac.cheme.cmu.edu/pasi2011/library/biegler/hicks.pdf

Re: [julia-users] Re: Counterintuitive slices of 2d array

[Alex Dowling]

I'd like to kick-up this thread again, and ask a related questions:

Let's say I have a 2d array, and take a slice of it. How do I then append 
the slice with an additional value? The slice is type Array{Float64,2}. 
Neither push! or append! work.

Thanks,
Alex

On Wednesday, April 16, 2014 at 5:25:36 PM UTC-5, Paweł Biernat wrote:
>
> Thanks for linking to the issue.  It seems that the array slicing is just 
> a tip of the iceberg, I had no idea about the discussion on the algebraic 
> interpretation of n x n arrays and respective operators.  I somewhat 
> understand the arguments behind the need of elementwise operators (although 
> they are still pretty arbitrary), but I find the current state of array 
> slicing really unintuitive.  I am new to Julia and I am constantly 
> impressed by its features but for my work a decent array slicing rules are 
> critical and lack of them is, to me, very annoying.
>
> W dniu środa, 16 kwietnia 2014 20:54:59 UTC+2 użytkownik Bob Nnamtrop 
> napisał:
>>
>> I completely agree with you and would love to see julia drop singleton 
>> dimensions indexed with scalar. There is an issue on this with lots of 
>> discussion, which was left at the point of waiting til someone implements 
>> some code to give it a try (see 
>> https://github.com/JuliaLang/julia/issues/5949). Maybe after 0.3 is 
>> released there will be some action on this (I am willing to try myself but 
>> it does seem like a hard 1st problem to deal with the julia internals).  
>> IMO, there is some underlying tension in julia that revolves around people 
>> who primarily use arrays as containers (who generally want this changed) 
>> and those who use one and two dimensional arrays as a linear algebra system 
>> (some of who do not want it implemented). I am firmly in the camp of using 
>> arrays primarily as containers and, in addition to this issue, find it very 
>> annoying that the elementwise operators are all are 2nd class citizens 
>> (i.e., you must use .*, ./, .+, .-, etc). The fact that the 1st class 
>> versions only work on a subset of arrays (i.e., those 1 and 2 dimensional) 
>> seems like a poor design choice to me. But this is different to the 
>> dripping dimensions indexed by scalars issue where there is still hope for 
>> a change.
>>
>> As a practical version of your function example in your second email is 
>> the plot command. You are plotting some slices from some higher dimensional 
>> arrays, e.g.
>>
>> plot(x[2,3,7,:,9], y[2,3,7,:,9])
>>
>> and you just want to get two one dimensional slices passed into plot with 
>> no fuss.
>>
>> Bob
>>
>>
>> On Wed, Apr 16, 2014 at 12:13 PM, Paweł Biernat  
>> wrote:
>>
>>> Adding a real life use case:
>>>
>>> x=[j+10*i for i=1:2, j=1:2]
>>>
>>> function f(x::Array{Int,1})
>>> x
>>> end
>>>
>>> and then
>>>
>>> julia> f(x[1:2,1])
>>>
>>> 2-element Array{Int64,1}:
>>>  11
>>>  21
>>>
>>>
>>> julia> f(x[1,1:2])
>>> ERROR: no method f(Array{Int64,2})
>>>
>>>
>>>
>>

[julia-users] LoadError and other warning while building Cbc dependencies

[Alex Dowling]

I am having a bit of trouble installing Cbc on a new Julia install. After a 
bunch of Google searches and trying a few things, I deleted the 
/home/username/.julia/v0.4 directory and tried Pkg.add("JuMP") followed by 
Pkg.add("Cbc"). The results are below. Any advice?

System information: 

CentOS Linux release 7.1.1503 (Core)


Julia console output:

   *_*

   *_**   _ **_**(_)**_** |  A fresh approach to technical 
computing*

  *(_)** | **(_)* *(_)**|  Documentation: http://docs.julialang.org*

*   _ _   _| |_  __ _   |  Type "?help" for help.*

*  | | | | | | |/ _` |  |*

*  | | |_| | | | (_| |  |  Version 0.4.0-rc1+66 (2015-09-16 08:37 UTC)*

* _/ |\__'_|_|_|\__'_|  |  Commit 88e55be (0 days old release-0.4)*

*|__/   |  x86_64-redhat-linux*


*julia> **Pkg.add("JuMP")*

*INFO: Initializing package repository /home/adowling2/.julia/v0.4*

*INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl*

*INFO: Installing ArrayViews v0.6.3*

*INFO: Installing Calculus v0.1.10*

*INFO: Installing Compat v0.7.0*

*INFO: Installing DataStructures v0.3.12*

*INFO: Installing Docile v0.5.18*

*INFO: Installing DualNumbers v0.1.3*

*INFO: Installing JuMP v0.10.1*

*INFO: Installing MathProgBase v0.3.17*

*INFO: Installing NaNMath v0.1.0*

*INFO: Installing ReverseDiffSparse v0.2.10*

*INFO: Package database updated*


*julia> **Pkg.add("Cbc")*

*INFO: Installing BinDeps v0.3.15*

*INFO: Installing Cbc v0.1.7*

*INFO: Installing HttpCommon v0.2.3*

*INFO: Installing SHA v0.1.1*

*INFO: Installing URIParser v0.1.0*

*INFO: Building Cbc*

*INFO: Recompiling stale cache file 
/home/adowling2/.julia/lib/v0.4/Compat.ji for module Compat.*

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

in anonymous at null:-1

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.Nothing is deprecated, use Void instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is deprecated, use AbstractString instead.

WARNING: Base.String is 

[julia-users] Re: LoadError and other warning while building Cbc dependencies

[Alex Dowling]

Thank you. The revert to RC1 tag fixed the issue. Should I create a bug 
report for this issue?

On Wednesday, September 16, 2015 at 4:49:13 PM UTC-5, Tony Kelman wrote:
>
> Something broke on release-0.4 and will need to be fixed before we tag 
> RC2. In the meantime you can check out the RC1 tag and use that.
>
>
> On Wednesday, September 16, 2015 at 10:46:48 AM UTC-7, Alex Dowling wrote:
>>
>> I am having a bit of trouble installing Cbc on a new Julia install. After 
>> a bunch of Google searches and trying a few things, I deleted the 
>> /home/username/.julia/v0.4 directory and tried Pkg.add("JuMP") followed by 
>> Pkg.add("Cbc"). The results are below. Any advice?
>>
>> System information: 
>>
>> CentOS Linux release 7.1.1503 (Core)
>>
>>
>> Julia console output:
>>
>>*_*
>>
>>*_**   _ **_**(_)**_** |  A fresh approach to technical 
>> computing*
>>
>>   *(_)** | **(_)* *(_)**|  Documentation: 
>> http://docs.julialang.org <http://docs.julialang.org>*
>>
>> *   _ _   _| |_  __ _   |  Type "?help" for help.*
>>
>> *  | | | | | | |/ _` |  |*
>>
>> *  | | |_| | | | (_| |  |  Version 0.4.0-rc1+66 (2015-09-16 08:37 UTC)*
>>
>> * _/ |\__'_|_|_|\__'_|  |  Commit 88e55be (0 days old release-0.4)*
>>
>> *|__/   |  x86_64-redhat-linux*
>>
>>
>> *julia> **Pkg.add("JuMP")*
>>
>> *INFO: Initializing package repository /home/adowling2/.julia/v0.4*
>>
>> *INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl 
>> <http://github.com/JuliaLang/METADATA.jl>*
>>
>> *INFO: Installing ArrayViews v0.6.3*
>>
>> *INFO: Installing Calculus v0.1.10*
>>
>> *INFO: Installing Compat v0.7.0*
>>
>> *INFO: Installing DataStructures v0.3.12*
>>
>> *INFO: Installing Docile v0.5.18*
>>
>> *INFO: Installing DualNumbers v0.1.3*
>>
>> *INFO: Installing JuMP v0.10.1*
>>
>> *INFO: Installing MathProgBase v0.3.17*
>>
>> *INFO: Installing NaNMath v0.1.0*
>>
>> *INFO: Installing ReverseDiffSparse v0.2.10*
>>
>> *INFO: Package database updated*
>>
>>
>> *julia> **Pkg.add("Cbc")*
>>
>> *INFO: Installing BinDeps v0.3.15*
>>
>> *INFO: Installing Cbc v0.1.7*
>>
>> *INFO: Installing HttpCommon v0.2.3*
>>
>> *INFO: Installing SHA v0.1.1*
>>
>> *INFO: Installing URIParser v0.1.0*
>>
>> *INFO: Building Cbc*
>>
>> *INFO: Recompiling stale cache file 
>> /home/adowling2/.julia/lib/v0.4/Compat.ji for module Compat.*
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>> WARNING: Base.String is deprecated, use AbstractString instead.
>>
>&