Re: [julia-users] ANN: Docile Lexicon update.

2015-02-01 Thread Michael Hatherly


Yes, the help system in 0.4 does search package documentation as well.

— Mike
​


On Sunday, 1 February 2015 03:32:26 UTC+2, i.cos...@me.com wrote:


 Ok, will Lexicon be baked into 0.4? If so, then the instructions in the 
 latest documentation makes sense:

 http://docs.julialang.org/en/latest/manual/documentation/


 Otherwise, there will need to be an extra line in the Accessing 
 Documentation section.

 Cheers

 On Sunday, 1 February 2015 10:09:34 UTC+11, Michael Hatherly wrote:

 Make sure you’ve installed Lexicon as Tom has mentioned. Also call using 
 Lexicon prior to trying to search for package docs at the REPL.

 — Mike
 ​


 On Saturday, 31 January 2015 22:57:36 UTC+2, i.cos...@me.com wrote:

 I installed Sims and ran `?sim` at REPL. I got the following back: 
 `dasslsim (generic function with 20 methods)`. I see that there is an 
 extensive docstring against sim in src/sim.jl. Why am I not seeing this at 
 REPL? I'm on julia 0.3.5

 On Thursday, 22 January 2015 03:54:14 UTC+11, Ivan Ogasawara wrote:

 great!
 El 21/01/2015 14:22, Michael Hatherly michael...@gmail.com 
 escribió:

 Hi all,

 I’m pleased to announce the latest update to the Docile 
 https://github.com/MichaelHatherly/Docile.jl and Lexicon 
 https://github.com/MichaelHatherly/Lexicon.jl documentation 
 packages.

 New features include:

- Docile now supports plain strings 
http://michaelhatherly.github.io/Docile.jl/syntax/#plain-strings, 
ie. without @doc, as docstrings. Compatibility with the Julia 0.4 
doc system is still present. 
- Thanks to Tom Short, Lexicon can now output nicely formatted 
markdown. This can then be used to create static documentation using 
programs such as MkDocs http://www.mkdocs.org/. See the 
documentation from the following packages for examples of the results: 
Sims https://tshort.github.io/Sims.jl/, Docile 
http://michaelhatherly.github.io/Docile.jl/, and Lexicon 
http://michaelhatherly.github.io/Lexicon.jl/. 

 Any bugs or feature requests can be opened in either the Docile or 
 Lexicon repos.

 Happy documenting!

 — Mike
 ​



[julia-users] Re: Robust Inner Products

2015-02-01 Thread Simon Byrne
If you wanted to implement such an algorithm, you would need to robust-ify 
the multiplication as well, using a two-product style algorithm: this 
paper goes into a lot of detail:
http://www.ti3.tu-harburg.de/paper/rump/OgRuOi05.pdf

Alternatively, you could use full double-double arithmetic: see
https://github.com/simonbyrne/DoubleDouble.jl

Simon

On Sunday, 1 February 2015 11:37:13 UTC, Christoph Ortner wrote:


 I was delighted to find that Julia has robust summation algorithms 
 implemented in Base.

 In my code I also need robust inner product. Is that implemented as well? 
 I couldn't find it, but it is easy to write for myself
 (trivially sum_kbn(x.*y), but this allocates extra memory)

 But are there other similar summation-type algorithms that could benefit, 
 and is there any interest in this?

 Christoph



[julia-users] Data structure with variable length outputs

2015-02-01 Thread Christopher Fisher
Hi all-

I am trying to figure out a graceful way to store output from a model 
simulation. Each simulated trial produces a single reaction time, a single 
choice and a vector of visual fixations generated by a Markov chain. The 
complicating factor is that the size of the vector of visual fixations 
varies on each simulated trial and I would like to generate multiple 
simulated trials. Is there a way to store these outputs in a data structure 
that will be easy to access in other functions?

A related issue is the dynamic nature of the vector size. I am currently 
using vcat to add new values to the vector. Is there a more efficient 
method?

Thank you in advance for your help.


[julia-users] create a multi data type structure in Julia

2015-02-01 Thread alex codoreanu
Hi all,

I'm a new Julia user and I'm starting to write a high-level multi data type 
fits file reader and writer akin to IDL's mwrfits/mrdfits. I got most of 
the dynamics figured out by I can't quite make a custom structure. For 
example, I would like something like:

element = {name::string, lambda::[float_array, 5000 elements], 
flux::[float_array, 5000 elements]}

structure = replicate(element, 500)

where structure inherits the members of element and can be called,

structure[1].name = the name of element 1
strucutre[1].lambda[1] = the first lambda element of the first structure


Thanks !!!


[julia-users] select! with range creating a memory allocation

2015-02-01 Thread julia . elison . 2
I realize this is a more developer issue, but I'm wondering if anyone else 
thinks it would be a good idea for select! to be changed to not return A[r] 
(which allocates memory).


I think it could return (A[r.start], A[r.stop]), sub(A, r) or even 
nothing.  As it stands it seems like a performance penalty that can be 
avoided since the rest of the function should operate on memory in place. 


In my own project I've made a local copy of select for this reason and 
taken out the return value.


Re: [julia-users] create a multi data type structure in Julia

2015-02-01 Thread Tim Holy
Have you seen
https://github.com/JuliaAstro/FITSIO.jl

Off-topic, but it would be awesome to have FITS integrated into Images.jl. I 
could help with the integration, but it is pretty well documented:
https://github.com/timholy/Images.jl/blob/master/doc/extendingIO.md
Further discussion could happen at 
https://github.com/JuliaAstro/FITSIO.jl/issues/18

--Tim

On Sunday, February 01, 2015 12:35:53 AM alex codoreanu wrote:
 Hi all,
 
 I'm a new Julia user and I'm starting to write a high-level multi data type
 fits file reader and writer akin to IDL's mwrfits/mrdfits. I got most of
 the dynamics figured out by I can't quite make a custom structure. For
 example, I would like something like:
 
 element = {name::string, lambda::[float_array, 5000 elements],
 flux::[float_array, 5000 elements]}
 
 structure = replicate(element, 500)
 
 where structure inherits the members of element and can be called,
 
 structure[1].name = the name of element 1
 strucutre[1].lambda[1] = the first lambda element of the first structure
 
 
 Thanks !!!



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Mike Innes
This is something I've been thinking about as well, and overloading `print`
etc. to work with markdown is definitely doable. I'm not sure how it would
work in IJulia, but it would definitely be ok in the terminal.

Use md unless you're writing inline docs (you might have to do a using
Base.Markdown first). At the moment both are interchangeable but the idea
is that doc needs to treat e.g. relative image and link paths specially,
and may potentially also add things like reference syntax.

On 1 February 2015 at 11:22, Christoph Ortner christophortn...@gmail.com
wrote:


 I've tried (but failed), with Julia 0.4, to use Markdown strings doc . .
 . as messages, e.g., error messages or just to make some complex program
 output more readable.

 If, in a REPL or IJulia, I just write
docblah `code` bleh
 in a single line, then this shows ok. But of course in a function or
 script, it will just generate an object of type `Base.Markdown.MD`,
 without output.

 [1] Question: how can I actually send it to the output, correctly
 formatted?

 [2] Suggestion: could `print`, `println`, `error` etc automatically
 interpret markdown strings and output them accordingly?

 [3] Question: why doc . . . and not md . . . ? Could these at least be
 used synonymously?

 Thanks,
 Christoph





Re: [julia-users] create a multi data type structure in Julia

2015-02-01 Thread alex codoreanu
Thanks Milan, 

I defined Element as you suggested and created

*data=(Element, 2)*

but 

*julia **data[1].lambda=[1., 2.0]*

*ERROR: type DataType has no field lambda*

Now that I have the structure data how do I access the branch members?


On Sunday, February 1, 2015 at 8:07:51 PM UTC+11, Milan Bouchet-Valat wrote:

 Le dimanche 01 février 2015 à 00:35 -0800, alex codoreanu a écrit : 
  Hi all, 
  
  
  I'm a new Julia user and I'm starting to write a high-level multi data 
  type fits file reader and writer akin to IDL's mwrfits/mrdfits. I got 
  most of the dynamics figured out by I can't quite make a custom 
  structure. For example, I would like something like: 
  
  
  element = {name::string, lambda::[float_array, 5000 elements], 
  flux::[float_array, 5000 elements]} 
  
  
  structure = replicate(element, 500) 
  
  
  where structure inherits the members of element and can be called, 
  
  
  structure[1].name = the name of element 1 
  strucutre[1].lambda[1] = the first lambda element of the first 
  structure 
 Looks like a composite type would work: 

 type Element 
 name::UTF8String 
 lambda::Vector{Float64} 
 flux::Vector{Float64} 
 end 



 Regards 



Re: [julia-users] create a multi data type structure in Julia

2015-02-01 Thread Milan Bouchet-Valat
Le dimanche 01 février 2015 à 01:24 -0800, alex codoreanu a écrit :
 Thanks Milan, 
 
 
 I defined Element as you suggested and created
 
 
 data=(Element, 2)
What is this line supposed to do? All it does now is assign a tuple
containing the Element type, and the integer value 2.

If you want to create an Element instance, you should do e.g.:
element = Element(a name, [1, 2], [2, 3])

and then to create a list holding two Elements, and fill the first value
with the just created element:
data = Vector(Element, 2)
data[1] = element

Accessing members:
data[1].lambda


Regards


 but 
 
 
 julia data[1].lambda=[1., 2.0]
 
 ERROR: type DataType has no field lambda
 
 
 
 Now that I have the structure data how do I access the branch members?
 
 
 
 
 On Sunday, February 1, 2015 at 8:07:51 PM UTC+11, Milan Bouchet-Valat
 wrote:
 Le dimanche 01 février 2015 à 00:35 -0800, alex codoreanu a
 écrit : 
  Hi all, 
  
  
  I'm a new Julia user and I'm starting to write a high-level
 multi data 
  type fits file reader and writer akin to IDL's
 mwrfits/mrdfits. I got 
  most of the dynamics figured out by I can't quite make a
 custom 
  structure. For example, I would like something like: 
  
  
  element = {name::string, lambda::[float_array, 5000
 elements], 
  flux::[float_array, 5000 elements]} 
  
  
  structure = replicate(element, 500) 
  
  
  where structure inherits the members of element and can be
 called, 
  
  
  structure[1].name = the name of element 1 
  strucutre[1].lambda[1] = the first lambda element of the
 first 
  structure 
 Looks like a composite type would work: 
 
 type Element 
 name::UTF8String 
 lambda::Vector{Float64} 
 flux::Vector{Float64} 
 end 
 
 
 
 Regards 
 



[julia-users] Markdown Strings as Messages

2015-02-01 Thread Christoph Ortner

I've tried (but failed), with Julia 0.4, to use Markdown strings doc . . 
. as messages, e.g., error messages or just to make some complex program 
output more readable.

If, in a REPL or IJulia, I just write 
   docblah `code` bleh
in a single line, then this shows ok. But of course in a function or 
script, it will just generate an object of type `Base.Markdown.MD`, without 
output.

[1] Question: how can I actually send it to the output, correctly formatted?

[2] Suggestion: could `print`, `println`, `error` etc automatically 
interpret markdown strings and output them accordingly?

[3] Question: why doc . . . and not md . . . ? Could these at least be 
used synonymously?

Thanks,
Christoph




[julia-users] Robust Inner Products

2015-02-01 Thread Christoph Ortner

I was delighted to find that Julia has robust summation algorithms 
implemented in Base.

In my code I also need robust inner product. Is that implemented as well? I 
couldn't find it, but it is easy to write for myself
(trivially sum_kbn(x.*y), but this allocates extra memory)

But are there other similar summation-type algorithms that could benefit, 
and is there any interest in this?

Christoph


Re: [julia-users] create a multi data type structure in Julia

2015-02-01 Thread alex codoreanu
That worked but I had to use:

data = Array(Element, 2) 

Thanks for the info!



On Sunday, February 1, 2015 at 8:57:23 PM UTC+11, Milan Bouchet-Valat wrote:

 Le dimanche 01 février 2015 à 01:24 -0800, alex codoreanu a écrit : 
  Thanks Milan, 
  
  
  I defined Element as you suggested and created 
  
  
  data=(Element, 2) 
 What is this line supposed to do? All it does now is assign a tuple 
 containing the Element type, and the integer value 2. 

 If you want to create an Element instance, you should do e.g.: 
 element = Element(a name, [1, 2], [2, 3]) 

 and then to create a list holding two Elements, and fill the first value 
 with the just created element: 
 data = Vector(Element, 2) 
 data[1] = element 

 Accessing members: 
 data[1].lambda 


 Regards 


  but 
  
  
  julia data[1].lambda=[1., 2.0] 
  
  ERROR: type DataType has no field lambda 
  
  
  
  Now that I have the structure data how do I access the branch members? 
  
  
  
  
  On Sunday, February 1, 2015 at 8:07:51 PM UTC+11, Milan Bouchet-Valat 
  wrote: 
  Le dimanche 01 février 2015 à 00:35 -0800, alex codoreanu a 
  écrit : 
   Hi all, 
   
   
   I'm a new Julia user and I'm starting to write a high-level 
  multi data 
   type fits file reader and writer akin to IDL's 
  mwrfits/mrdfits. I got 
   most of the dynamics figured out by I can't quite make a 
  custom 
   structure. For example, I would like something like: 
   
   
   element = {name::string, lambda::[float_array, 5000 
  elements], 
   flux::[float_array, 5000 elements]} 
   
   
   structure = replicate(element, 500) 
   
   
   where structure inherits the members of element and can be 
  called, 
   
   
   structure[1].name = the name of element 1 
   strucutre[1].lambda[1] = the first lambda element of the 
  first 
   structure 
  Looks like a composite type would work: 
  
  type Element 
  name::UTF8String 
  lambda::Vector{Float64} 
  flux::Vector{Float64} 
  end 
  
  
  
  Regards 
  



Re: [julia-users] Parametric vs Abstract typed collections, design question

2015-02-01 Thread Tim Holy
James, in case you haven't seen it, this is treated _extensively_ in the FAQ. 
Make a nice cup of tea and pull up a comfy chair before you start reading :-).

--Tim

On Sunday, February 01, 2015 01:16:19 AM Jutho wrote:
 As long as not all parameters of a parametric concrete type are fully
 specified, the type is treated as abstract. So in both cases your
 collection would be of abstract elements and they would not be stored
 packed in memory. I don't think what you are requesting is possible, but I
 might be mistaken. If the elements of the collection are of a concrete
 type, then applying a function to them should always be the same function.
 If this were not the case, then type inference could not work.



Re: [julia-users] Re-size image in IJulia Markdown

2015-02-01 Thread Tim Holy
Can't help you with the markdown issue, but your Images.jl installation is 
borked. This happens all-to-often for OSX users. See 
https://github.com/timholy/Images.jl#installation for ways to fix; if none of 
the linked methods work, please do open an issue.

--Tim

On Saturday, January 31, 2015 05:40:15 PM Nathan Nesbitt wrote:
 I am using Markup in IJulia to write notes amongst my code, and want to
 control the size of an embedded image.
 
 To embed the image I'm currently using the code:
 
 ![alt text](my_pic.jpg Title)
 
 
 This works fine, but doesn't give me control over the image size. Is there
 any code to add to this line to control the image size? Supposedly for some
 implementations
 http://stackoverflow.com/questions/14675913/how-to-change-image-size-markdo
 wn of Markdown the following code works, but this failed for me
 
 ![alt text](my_pic.jpg =100x100)
 
 
 For an altenative I looked into the ImageView package
 https://github.com/timholy/ImageView.jl by Tim Holy, but got an error
 when I tried to use this in IJulia
 
 (following the suggested syntax)
 
 using ImageView
 
 
 img = imread(my_pic.jpg)
 
 libversion not defined
 
  in image2wand at /Users/Nikola/.julia/v0.3/Images/src/io.jl:304
 
  in _writemime at /Users/Nikola/.julia/v0.3/Images/src/io.jl:198
 
  in base64 at base64.jl:125
 
  in display_dict at
 /Users/Nikola/.julia/v0.3/IJulia/src/execute_request.jl:34
 
 
 view(img, pixelspacing = [1,1])
 
 view not defined
 
 while loading In[11], in expression starting on line 1
 
 
 Any suggestions?
 
 Thanks!
 
 Best,
 
 Nate



Re: [julia-users] How should nested local functions be annotated?

2015-02-01 Thread Tim Holy
It's a long-standing issue that local and anonymous functions allocate (and 
have poor performance). FastAnonymous.jl and similar packages give you one way 
around that.

--Tim

On Saturday, January 31, 2015 08:03:37 PM Kirill Ignatiev wrote:
 On Saturday, 31 January 2015 21:54:54 UTC-5, Jameson wrote:
  code_warntype is being a bit misleading there. code inference hasn't yet
  fully run on the inner function.
 
  This is a reduced test case from a larger program, which allocated a lot
 of memory, and when I got rid of the locally declared function, the memory
 usage dropped significantly. So at least empirically I think there may be a
 problem there.



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Christoph Ortner
Is this an existing issue somewhere?

On Sunday, 1 February 2015 12:18:42 UTC, Christoph Ortner wrote:


 Thanks for the clarification. It would be really great to have this 
 functionality, especially, if it can be combined with MathJax!

 Thanks,
 Christoph



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Christoph Ortner

Thanks for the clarification. It would be really great to have this 
functionality, especially, if it can be combined with MathJax!

Thanks,
Christoph


Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Mike Innes
No, feel free to open one if you want (ping me too).

On 1 February 2015 at 12:18, Christoph Ortner christophortn...@gmail.com
wrote:

 Is this an existing issue somewhere?


 On Sunday, 1 February 2015 12:18:42 UTC, Christoph Ortner wrote:


 Thanks for the clarification. It would be really great to have this
 functionality, especially, if it can be combined with MathJax!

 Thanks,
 Christoph




Re: [julia-users] create a multi data type structure in Julia

2015-02-01 Thread Milan Bouchet-Valat
Le dimanche 01 février 2015 à 00:35 -0800, alex codoreanu a écrit :
 Hi all,
 
 
 I'm a new Julia user and I'm starting to write a high-level multi data
 type fits file reader and writer akin to IDL's mwrfits/mrdfits. I got
 most of the dynamics figured out by I can't quite make a custom
 structure. For example, I would like something like:
 
 
 element = {name::string, lambda::[float_array, 5000 elements],
 flux::[float_array, 5000 elements]}
 
 
 structure = replicate(element, 500)
 
 
 where structure inherits the members of element and can be called,
 
 
 structure[1].name = the name of element 1
 strucutre[1].lambda[1] = the first lambda element of the first
 structure
Looks like a composite type would work:

type Element
name::UTF8String
lambda::Vector{Float64}
flux::Vector{Float64}
end



Regards



Re: [julia-users] Parametric vs Abstract typed collections, design question

2015-02-01 Thread Jutho
As long as not all parameters of a parametric concrete type are fully 
specified, the type is treated as abstract. So in both cases your collection 
would be of abstract elements and they would not be stored packed in memory. I 
don't think what you are requesting is possible, but I might be mistaken. If 
the elements of the collection are of a concrete type, then applying a function 
to them should always be the same function. If this were not the case, then 
type inference could not work.

[julia-users] Subtraction of the diagonal matrix, how sipler?

2015-02-01 Thread paul analyst
I have somethink like this :

rand(4,4).-eye(4,4)*5

Qestion : how to simpler subtracted from the diagonal value(5) , 
Without  multiplying matrices  I have great and I have to simplify the 
code.

Paul


Re: [julia-users] Help with the findin method

2015-02-01 Thread Diego Tapias
Thank you so much Jiahao!
I see my guess was correct.
What a nice group.

2015-01-31 16:50 GMT-06:00 Jiahao Chen jia...@mit.edu:
 Your custom particles type is not iterable, i.e. does not have start, next
 and done methods defined. See the manual for details of the iterable
 interface.

 Note that Julia numbers are iterable, so the following works:

 julia findin([1,2,3], 2)
 1-element Array{Int64,1}:
  2

 Thanks,

 Jiahao Chen
 Staff Research Scientist
 MIT Computer Science and Artificial Intelligence Laboratory


Re: [julia-users] Subtraction of the diagonal matrix, how sipler?

2015-02-01 Thread paul analyst
Thx, nice. Is the fast way in Julia ?
Paul

W dniu niedziela, 1 lutego 2015 17:18:03 UTC+1 użytkownik Andreas Noack 
napisał:

 This is cheaper: rand(4,4) - 5*I because 5I is a special type that doesn't 
 store all elements.

 2015-02-01 11:03 GMT-05:00 paul analyst paul.a...@mail.com javascript:
 :

 I have somethink like this :

 rand(4,4).-eye(4,4)*5

 Qestion : how to simpler subtracted from the diagonal value(5) , 
 Without  multiplying matrices  I have great and I have to simplify 
 the code.

 Paul




[julia-users] Re: Robust Inner Products

2015-02-01 Thread Simon Byrne
I realise I didn't actually answer your question: I can't speak as to 
whether it will be accepted in Base (you will probably have to open an 
issue or pull request to start a discussion), but at the very least it 
would be useful to at least have in a package somewhere. If you don't want 
to create your own, I'd be happy to incorporate it into DoubleDouble.jl.

Simon

On Sunday, 1 February 2015 15:20:29 UTC, Christoph Ortner wrote:

 thanks for the suggestions; indeed, Rumpf's paper is my main reference :) 
 for these things. 



[julia-users] Memory usage and speed in nested loop.

2015-02-01 Thread Kristoffer Carlsson
I have two versions of an example function that calculates a number by 
looping over all pair of points. In the first one I use a 2d-array and 
access points with [:,i] syntax to get the coordinates. In the second 
version of the function I instead creates an array of Point-types (each 
Point has a x and y coordinate). I then access the coordinate like point.x, 
point.y etc.

These two functions takes vastly different time and memory usage.

This is the first function:

function slow()
srand(1234)
points = randn(2, 5000)
n_points::Int = size(points,2)
cum = 0.0
for i in 1:n_points
for j in (i+1):n_points
point_2 = points[:, j]
cum += point_2[1]
end
  end
  return cum
end

This is the fast version with the Point types:

immutable Point
  x::Float64
  y::Float64
end


function fast()
srand(1234)
points = randn(2, 5000)
n_points = size(points, 2)
cum= 0.0


# Create array of points
points_vec = Point[]
for i in 1:n_points
push!(points_vec, Point( points [1,i], points [2,i]))
end


  for i in 1:n_points
for j in (i+1):n_points
point_2 =  points_vec[j]
cum += point_2.x
end
  end
  return cum
end


Running
@time println(slow())
@time println(fast())

now gives:

-23952.535945302105
elapsed time: 0.954317047 seconds (1055 MB allocated, 3.78% gc time in 48 
pauses with 0 full sweep)

-23952.535945302105
elapsed time: 0.025171914 seconds (1 MB allocated)

The slow version takes 50 times longer and consumes 1000x the memory. 
Running the functions with memory tacker gives:

- 
- 
- 
- function slow()
28688 srand(1234)
80048 points = randn(2, 5000)
0 n_points::Int = size(points,2)
0 cum = 0.0
0 for i in 1:n_points
0 for j in (i+1):n_points
109978 point_2 = points[:, j]
0 cum += point_2[1]
- end
-   end
0   return cum
- end
- 
- 
- 
- immutable Point
-   x::Float64
-   y::Float64
- end
- 
- function fast()
  2540964 srand(1234)
80048 points = randn(2, 5000)
0 n_points = size(points, 2)
0 cum= 0.0
- 
- # Create array of points
   48 points_vec = Point[]
0 for i in 1:n_points
   263112 push!(points_vec, Point( points [1,i], points [2,i]))
- end
- 
0   for i in 1:n_points
0 for j in (i+1):n_points
0 point_2 =  points_vec[j]
0 cum += point_2.x
- end
-   end
0   return cum
- end
- 
- 
- 
- @time println(slow())
- @time println(fast())
- 
- 



So what seems to take all the memory is
point_2 = points[:, j]

Maybe some copying is performed when slicing but I have tried replacing it 
with sub and slice etc (that shouldnt copy?) and it just get worse. Are 
there some alignment issues?

I have tried both in 0.3.5 and 0.4 with the same results.

Any help?

Best regards,
Kristoffer Carlsson





Re: [julia-users] Memory usage and speed in nested loop.

2015-02-01 Thread Tim Holy
If you're using sub or slice, your performance should be vastly better on 0.4 
than 0.3, but as you observed it will still be awful. In the long run (maybe 
even by the time 0.4 is released?), we hope that in such loops sub/slice won't 
actually create a new object and allocate memory---it will be transparently 
elided by the compiler. I think the main roadblock currently is the fact that 
immutables with tuple fields (see the definition of SubArray) currently do not 
inline the tuple, instead holding a reference to a heap-allocated object. 
That basically prevents further optimization. For improvements, the main issue 
to watch is https://github.com/JuliaLang/julia/issues/8974.

Manual devectorization or manual immutables like your Point type are currently 
your best bet.

Best,
--Tim

On Sunday, February 01, 2015 06:35:05 AM Kristoffer Carlsson wrote:
 I have two versions of an example function that calculates a number by
 looping over all pair of points. In the first one I use a 2d-array and
 access points with [:,i] syntax to get the coordinates. In the second
 version of the function I instead creates an array of Point-types (each
 Point has a x and y coordinate). I then access the coordinate like point.x,
 point.y etc.
 
 These two functions takes vastly different time and memory usage.
 
 This is the first function:
 
 function slow()
 srand(1234)
 points = randn(2, 5000)
 n_points::Int = size(points,2)
 cum = 0.0
 for i in 1:n_points
 for j in (i+1):n_points
 point_2 = points[:, j]
 cum += point_2[1]
 end
   end
   return cum
 end
 
 This is the fast version with the Point types:
 
 immutable Point
   x::Float64
   y::Float64
 end
 
 
 function fast()
 srand(1234)
 points = randn(2, 5000)
 n_points = size(points, 2)
 cum= 0.0
 
 
 # Create array of points
 points_vec = Point[]
 for i in 1:n_points
 push!(points_vec, Point( points [1,i], points [2,i]))
 end
 
 
   for i in 1:n_points
 for j in (i+1):n_points
 point_2 =  points_vec[j]
 cum += point_2.x
 end
   end
   return cum
 end
 
 
 Running
 @time println(slow())
 @time println(fast())
 
 now gives:
 
 -23952.535945302105
 elapsed time: 0.954317047 seconds (1055 MB allocated, 3.78% gc time in 48
 pauses with 0 full sweep)
 
 -23952.535945302105
 elapsed time: 0.025171914 seconds (1 MB allocated)
 
 The slow version takes 50 times longer and consumes 1000x the memory.
 Running the functions with memory tacker gives:
 
 -
 -
 -
 - function slow()
 28688 srand(1234)
 80048 points = randn(2, 5000)
 0 n_points::Int = size(points,2)
 0 cum = 0.0
 0 for i in 1:n_points
 0 for j in (i+1):n_points
 109978 point_2 = points[:, j]
 0 cum += point_2[1]
 - end
 -   end
 0   return cum
 - end
 -
 -
 -
 - immutable Point
 -   x::Float64
 -   y::Float64
 - end
 -
 - function fast()
   2540964 srand(1234)
 80048 points = randn(2, 5000)
 0 n_points = size(points, 2)
 0 cum= 0.0
 -
 - # Create array of points
48 points_vec = Point[]
 0 for i in 1:n_points
263112 push!(points_vec, Point( points [1,i], points [2,i]))
 - end
 -
 0   for i in 1:n_points
 0 for j in (i+1):n_points
 0 point_2 =  points_vec[j]
 0 cum += point_2.x
 - end
 -   end
 0   return cum
 - end
 -
 -
 -
 - @time println(slow())
 - @time println(fast())
 -
 -
 
 
 
 So what seems to take all the memory is
 point_2 = points[:, j]
 
 Maybe some copying is performed when slicing but I have tried replacing it
 with sub and slice etc (that shouldnt copy?) and it just get worse. Are
 there some alignment issues?
 
 I have tried both in 0.3.5 and 0.4 with the same results.
 
 Any help?
 
 Best regards,
 Kristoffer Carlsson



Re: [julia-users] Subtraction of the diagonal matrix, how sipler?

2015-02-01 Thread Andreas Noack
I think so, but it neither take long time

julia let

   A = rand(4,4);

   @time for i = 1:10^6;A - 5.0*eye(4);end

   end;

elapsed time: 0.177621144 seconds (534 MB allocated, 5.71% gc time in 25
pauses with 0 full sweep)


julia let

   A = rand(4,4);

   @time for i = 1:10^6;A - 5.0*I;end

   end;

elapsed time: 0.053749097 seconds (167 MB allocated, 5.04% gc time in 8
pauses with 0 full sweep)


2015-02-01 11:49 GMT-05:00 paul analyst paul.anal...@mail.com:

 Thx, nice. Is the fast way in Julia ?
 Paul

 W dniu niedziela, 1 lutego 2015 17:18:03 UTC+1 użytkownik Andreas Noack
 napisał:

 This is cheaper: rand(4,4) - 5*I because 5I is a special type that
 doesn't store all elements.

 2015-02-01 11:03 GMT-05:00 paul analyst paul.a...@mail.com:

 I have somethink like this :

 rand(4,4).-eye(4,4)*5

 Qestion : how to simpler subtracted from the diagonal value(5) ,
 Without  multiplying matrices  I have great and I have to simplify
 the code.

 Paul





Re: [julia-users] Symbol to string

2015-02-01 Thread Ivar Nesje
And more generally, for all types that makes sense to convert between, you 
can use the convert function

convert(String, :blah)

(or maybe better, to avoid type instability issues)
convert(ASCIIString, :blah)
convert(UTF8String, :blah)

In 0.4 the default constructor for all types will fallback to convert, so 
for your own types, you usually just need to add a convert definition.

søndag 1. februar 2015 16.29.48 UTC+1 skrev Milan Bouchet-Valat følgende:

 Le dimanche 01 février 2015 à 07:18 -0800, Arshak Navruzyan a écrit : 
  Hello, 
  
  A really basic question how do I turn a symbol into a string ? 
  
  :blah - blah 
 julia string(:blah) 
 blah 

 Easy, isn't it? 

 Regards 



Re: [julia-users] Subtraction of the diagonal matrix, how sipler?

2015-02-01 Thread Andreas Noack
This is cheaper: rand(4,4) - 5*I because 5I is a special type that doesn't
store all elements.

2015-02-01 11:03 GMT-05:00 paul analyst paul.anal...@mail.com:

 I have somethink like this :

 rand(4,4).-eye(4,4)*5

 Qestion : how to simpler subtracted from the diagonal value(5) ,
 Without  multiplying matrices  I have great and I have to simplify the
 code.

 Paul



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Mike Innes
Markdown.jl lives in base Julia now so it should probably go there.

On 1 February 2015 at 15:18, Christoph Ortner christophortn...@gmail.com
wrote:

 should this go in Docile, Lexicon, or Julialang?
 Christoph



[julia-users] Re: Robust Inner Products

2015-02-01 Thread Christoph Ortner
thanks for the suggestions; indeed, Rumpf's paper is my main reference :) 
for these things. 


Re: [julia-users] Symbol to string

2015-02-01 Thread Milan Bouchet-Valat
Le dimanche 01 février 2015 à 07:18 -0800, Arshak Navruzyan a écrit :
 Hello,
 
 A really basic question how do I turn a symbol into a string ? 
 
 :blah - blah
julia string(:blah)
blah

Easy, isn't it?

Regards


[julia-users] Re: select! with range creating a memory allocation

2015-02-01 Thread Ivar Nesje
This issue will fix itself when `a[r]` returns a subarray with 0.4 (this 
hasn't landed in master yet, and reminds me that the pending change will 
require a huge documentation effort that change affect APIs).

The improved garbage collection that was merged in master recently will 
likely greatly reduce the performance penalty for such short lived 
allocations.

Ivar

søndag 1. februar 2015 11.53.02 UTC+1 skrev julia.e...@gmail.com følgende:

 I realize this is a more developer issue, but I'm wondering if anyone else 
 thinks it would be a good idea for select! to be changed to not return A[r] 
 (which allocates memory).


 I think it could return (A[r.start], A[r.stop]), sub(A, r) or even 
 nothing.  As it stands it seems like a performance penalty that can be 
 avoided since the rest of the function should operate on memory in place. 


 In my own project I've made a local copy of select for this reason and 
 taken out the return value.



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Christoph Ortner
should this go in Docile, Lexicon, or Julialang?
Christoph


[julia-users] Symbol to string

2015-02-01 Thread Arshak Navruzyan
Hello,

A really basic question how do I turn a symbol into a string ?

:blah - blah

Thanks


[julia-users] Re: Data structure with variable length outputs

2015-02-01 Thread Ivar Nesje
As you probably realized vcat makes a copy of the vector, so it is 
inefficient when you use it repeatedly. Julia provides the push! and append! 
functions 
so that you can efficiently add elements to the end of a vector. There is 
also a sizehint! function if you know how many elements there will 
ultimately be, and want to save a few copies when the array grows. 

søndag 1. februar 2015 15.31.40 UTC+1 skrev Christopher Fisher følgende:

 Hi all-

 I am trying to figure out a graceful way to store output from a model 
 simulation. Each simulated trial produces a single reaction time, a single 
 choice and a vector of visual fixations generated by a Markov chain. The 
 complicating factor is that the size of the vector of visual fixations 
 varies on each simulated trial and I would like to generate multiple 
 simulated trials. Is there a way to store these outputs in a data structure 
 that will be easy to access in other functions?

 A related issue is the dynamic nature of the vector size. I am currently 
 using vcat to add new values to the vector. Is there a more efficient 
 method?

 Thank you in advance for your help.



[julia-users] The sub function behaves differently in v0.3 and master

2015-02-01 Thread Daniel Casimiro
Hi,

I noticed that the sub function behaves differently in master and version 
0.3.5, when handling data of type Array{Float64, 1}. I am not sure if the 
change is intentional.

The following works as expected on Julia master:

*julia* *sub([1.0; 2; 3; 4], 1:3, :)*

but, fails in version 0.3.5 with the following error:


 *ERROR: `sub` has no method matching sub(::Array{Float64,1}, 
 ::(UnitRange{Int64},UnitRange{Int64}))*
 * in sub at subarray.jl:80** in sub at subarray.jl:132*


Is this a bug in Julia 0.3.5 or master?

This came up in this pull request: 
https://github.com/dancasimiro/WAV.jl/pull/20. I just figured out that my 
fix is broken too. 

Thanks.


Re: [julia-users] Parametric vs Abstract typed collections, design question

2015-02-01 Thread James Crist
Thanks, that's kind of what I expected.

@Tim:

I found some stuff in there (and in the very helpful Types section), but
never saw explicitly whether a collection of unparametrized parametric type
offered any additional benefit to the compiler than one of just abstract
type. (i.e. Rational[] vs Real[]). Since the container type isn't concrete,
I figured it probably wouldn't offer any benefit, but thought I'd ask.

- Jim


On Sun, Feb 1, 2015 at 5:07 AM, Tim Holy tim.h...@gmail.com wrote:

 James, in case you haven't seen it, this is treated _extensively_ in the
 FAQ.
 Make a nice cup of tea and pull up a comfy chair before you start reading
 :-).

 --Tim

 On Sunday, February 01, 2015 01:16:19 AM Jutho wrote:
  As long as not all parameters of a parametric concrete type are fully
  specified, the type is treated as abstract. So in both cases your
  collection would be of abstract elements and they would not be stored
  packed in memory. I don't think what you are requesting is possible, but
 I
  might be mistaken. If the elements of the collection are of a concrete
  type, then applying a function to them should always be the same
 function.
  If this were not the case, then type inference could not work.




Re: [julia-users] Re: Data structure with variable length outputs

2015-02-01 Thread Tim Holy
With regards to the part about a data structure for storing your results, 
http://docs.julialang.org/en/latest/manual/types/#composite-types might be 
what you're looking for.

--Tim

On Sunday, February 01, 2015 11:34:50 AM Ivar Nesje wrote:
 As you probably realized vcat makes a copy of the vector, so it is
 inefficient when you use it repeatedly. Julia provides the push! and append!
 functions so that you can efficiently add elements to the end of a vector.
 There is also a sizehint! function if you know how many elements there will
 ultimately be, and want to save a few copies when the array grows.
 søndag 1. februar 2015 15.31.40 UTC+1 skrev Christopher Fisher følgende:
  Hi all-
  
  I am trying to figure out a graceful way to store output from a model
  simulation. Each simulated trial produces a single reaction time, a single
  choice and a vector of visual fixations generated by a Markov chain. The
  complicating factor is that the size of the vector of visual fixations
  varies on each simulated trial and I would like to generate multiple
  simulated trials. Is there a way to store these outputs in a data
  structure
  that will be easy to access in other functions?
  
  A related issue is the dynamic nature of the vector size. I am currently
  using vcat to add new values to the vector. Is there a more efficient
  method?
  
  Thank you in advance for your help.



Re: [julia-users] Markdown Strings as Messages

2015-02-01 Thread Christoph Ortner
created issue [10009](https://github.com/JuliaLang/julia/issues/10009)


[julia-users] Re: Data structure with variable length outputs

2015-02-01 Thread Christopher Fisher
Thank you Ivar and Tim. Using append gave me a 3 fold improvement in speed 
and I'll look into the composite types. 

On Sunday, February 1, 2015 at 9:31:40 AM UTC-5, Christopher Fisher wrote:

 Hi all-

 I am trying to figure out a graceful way to store output from a model 
 simulation. Each simulated trial produces a single reaction time, a single 
 choice and a vector of visual fixations generated by a Markov chain. The 
 complicating factor is that the size of the vector of visual fixations 
 varies on each simulated trial and I would like to generate multiple 
 simulated trials. Is there a way to store these outputs in a data structure 
 that will be easy to access in other functions?

 A related issue is the dynamic nature of the vector size. I am currently 
 using vcat to add new values to the vector. Is there a more efficient 
 method?

 Thank you in advance for your help.



[julia-users] Re: Robust Inner Products

2015-02-01 Thread Christoph Ortner
many thanks - well I will implement it first of all, and then see where it 
could go.
Christoph




[julia-users] Re: select! with range creating a memory allocation

2015-02-01 Thread julia . elison . 2

Thanks for the reply.  To start using 0.4 will I just have to gamble on a 
nightly windows installer?


On Sunday, February 1, 2015 at 1:11:40 PM UTC-6, Ivar Nesje wrote:

 This issue will fix itself when `a[r]` returns a subarray with 0.4 (this 
 hasn't landed in master yet, and reminds me that the pending change will 
 require a huge documentation effort that change affect APIs).

 The improved garbage collection that was merged in master recently will 
 likely greatly reduce the performance penalty for such short lived 
 allocations.

 Ivar

 søndag 1. februar 2015 11.53.02 UTC+1 skrev julia.e...@gmail.com følgende:

 I realize this is a more developer issue, but I'm wondering if anyone 
 else thinks it would be a good idea for select! to be changed to not return 
 A[r] (which allocates memory).


 I think it could return (A[r.start], A[r.stop]), sub(A, r) or even 
 nothing.  As it stands it seems like a performance penalty that can be 
 avoided since the rest of the function should operate on memory in place. 


 In my own project I've made a local copy of select for this reason and 
 taken out the return value.



[julia-users] Re: select! with range creating a memory allocation

2015-02-01 Thread Ivar Nesje
Yes, if you are on windows, the nightly installer is the easiest way. Be sure 
to keep the old downloads, and remember what versions you use so that you can 
revert if a new change causes trouble with your setup. (How careful you must 
be, depends on how acceptable it is for you to have a broken Julia install 
until you adapt your code to a breaking change)

Re: [julia-users] The sub function behaves differently in v0.3 and master

2015-02-01 Thread Tim Holy
SubArrays have been entirely rewritten in 0.4, and can do quite a few tricks 
that the old ones can't. Changing the dimensionality of the view compared to 
the parent, as in your example, is one of them. Here's another example of 
something you simply can't do on 0.3:

julia A = reshape(1:15, 3, 5)
3x5 Array{Int64,2}:
 1  4  7  10  13
 2  5  8  11  14
 3  6  9  12  15

julia b = sub(A, [2, 7, 8])
3-element SubArray{Int64,1,Array{Int64,2},(Array{Int64,1},),0}:
 2
 7
 8

julia b[2] = -1
-1

julia A
3x5 Array{Int64,2}:
 1  4  -1  10  13
 2  5   8  11  14
 3  6   9  12  15

If anything, the divergence between the two will grow further once 
https://github.com/JuliaLang/julia/issues/9874 is addressed.

Best,
--Tim

On Sunday, February 01, 2015 12:02:56 PM Daniel Casimiro wrote:
 Hi,
 
 I noticed that the sub function behaves differently in master and version
 0.3.5, when handling data of type Array{Float64, 1}. I am not sure if the
 change is intentional.
 
 The following works as expected on Julia master:
 
 *julia* *sub([1.0; 2; 3; 4], 1:3, :)*
 
 but, fails in version 0.3.5 with the following error:
  *ERROR: `sub` has no method matching sub(::Array{Float64,1},
  
  ::(UnitRange{Int64},UnitRange{Int64}))*
  
  * in sub at subarray.jl:80** in sub at subarray.jl:132*
 
 Is this a bug in Julia 0.3.5 or master?
 
 This came up in this pull request:
 https://github.com/dancasimiro/WAV.jl/pull/20. I just figured out that my
 fix is broken too.
 
 Thanks.



[julia-users] [Tips] The fastest way of reading csv files for Matlab users

2015-02-01 Thread Jung Soo Park
Hi,

This is to share my finding of using the Juila for Matlab users.

I have a few big size data in csv format to open from the Juila and it 
tooks at least 3 minutes to read. However, if I convert the file into 
Matlab format (.mat)  and load the file with Matlab, it tooks only one 
minute.

I searched the Julia and find the MAT package. 

https://github.com/simonster/MAT.jl

Instead of the csv file, I open the file with Matlab format data using the 
MAT package and it took only *a few second* !!

using MAT   

MN=@sprintf(MODEL_%02d.mat,2); # this code is to select the 2nd matlab file 
for me. 

  #check the name of the variable name you saved in Matlab first

 file = matopen(MN);  # you can type your file name directly from here like, 
MODEL_02.mat

 varnames = names(file);  #I found *MODEL* was the name of the varnames

 close(file)


# Now call the variable from Julia

 file = matopen(MN);

 MODEL=read(file, *MODEL*); # Enter the name of the varnames above at  and 
assign variable name again!! 

 close(file)


# Now whether you open the righ file

  size(MODEL)



It's fantastic and this prove Julia has something more than other languages but 
still Julia is lack of tutorials to cover. I suggest to share this kind of your 
findings for the progress of the Julia and for the convenience of the users.


Thank you Julia.


Jase