Re: [julia-users] Re: why's my julia code running slower than matlab, despite performance tips

[Tk]

Also try:
julia -O --check-bounds=no yourcode.jl

On Monday, May 9, 2016 at 2:03:58 AM UTC+9, feza wrote:
>
> Milan
>
> Script is here: 
> https://gist.github.com/musmo/27436a340b41c01d51d557a655276783
>
>
> On Sunday, May 8, 2016 at 12:40:44 PM UTC-4, feza wrote:
>>
>> Thanks for the tip (initially I just transllated the matlab verbatim)
>>
>> Now I have made all the changes. In place operations, and direct function 
>> calls.
>> Despite these changes. Matlab is 3.6 seconds, new Julia  7.6 seconds
>> TBH the results of this experiment are frustrating, I was hoping Julia 
>> was going to provide a huge speedup (on the level of c)
>>
>> Am I still missing anything in the Julia code that is crucial to speed?
>> @code_warntype looks ok sans a few red unions which i don't think are in 
>> my control
>>
>>
>> On Sunday, May 8, 2016 at 8:15:25 AM UTC-4, Tim Holy wrote:
>>>
>>> One of the really cool features of julia is that functions are allowed 
>>> to have 
>>> more than 0 arguments. It's even considered good style, and I highly 
>>> recommend 
>>> making use of this awesome feature in your code! :-) 
>>>
>>> In other words: try passing all variables as arguments to the functions. 
>>> Even 
>>> though you're wrapping everything in a function, performance-wise you're 
>>> running up against an inference problem 
>>> (https://github.com/JuliaLang/julia/issues/15276). In terms of coding 
>>> style, 
>>> you're still essentially using global variables. Honestly, these make 
>>> your 
>>> life harder in the end (
>>> http://c2.com/cgi/wiki?GlobalVariablesAreBad)---it's 
>>> not a bad thing that julia provides gentle encouragement to avoid using 
>>> them, 
>>> and you're losing out on opportunities by trying to sidestep that 
>>> encouragement. 
>>>
>>> Best, 
>>> --Tim 
>>>
>>> On Sunday, May 08, 2016 01:38:41 AM feza wrote: 
>>> > That's no surprise your CPU is better :) 
>>> > 
>>> > Regarding devectorization 
>>> > for l in 1:q 
>>> > for k in 1:nz 
>>> > for j in 1:ny 
>>> > for i in 1:nx 
>>> > u = ux[i,j,k] 
>>> > v = uy[i,j,k] 
>>> > w = uz[i,j,k] 
>>> > 
>>> > cu = c[k,1]*u  + c[k,2]*v + c[k,3]*w 
>>> > u2 = u*u + v*v + w*w 
>>> > feq[i,j,k,l] = weights[k]*ρ[i,j,k]*(1 + 3*cu + 
>>> 9/2*(cu*cu) 
>>> > - 3/2*u2) 
>>> > f[i,j,k,l] = f[i,j,k,l]*(1-ω) + ω*feq[i,j,k,l] 
>>> >   end 
>>> >   end 
>>> >   end 
>>> >  end 
>>> > 
>>> > Actually makes the code a lot slower 
>>> > 
>>> > On Sunday, May 8, 2016 at 4:37:18 AM UTC-4, Patrick Kofod Mogensen 
>>> wrote: 
>>> > > For what it's worth  it run in about 3-4 seconds on my computer on 
>>> latest 
>>> > > v0.4. 
>>> > > 
>>> > > CPU : Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz 
>>> > > 
>>> > > On Sunday, May 8, 2016 at 10:33:14 AM UTC+2, Patrick Kofod Mogensen 
>>> wrote: 
>>> > >> As for the v0.5 performance (which is horrible), I think it's the 
>>> boxing 
>>> > >> issue with closure https://github.com/JuliaLang/julia/issues/15276 
>>> . 
>>> > >> Right? 
>>> > >> 
>>> > >> On Sunday, May 8, 2016 at 10:29:59 AM UTC+2, STAR0SS wrote: 
>>> > >>> You are using a lot of vectorized operations and Julia isn't as 
>>> good as 
>>> > >>> matlab is with those. 
>>> > >>> 
>>> > >>> The usual solution is to devectorized your code and to use loops 
>>> (except 
>>> > >>> for matrix multiplication if you have large matrices). 
>>>
>>>

Re: [julia-users] Re: why's my julia code running slower than matlab, despite performance tips

[Tk]

Could you try replacing
   for i in 1:nx, j in 1:ny, k in 1:nz
to
   for k in 1:nz, j in 1:ny, i in 1:nx
because your arrays are defined like a[i,j,k]?

Another question is, how many cores is your Matlab code using?


On Monday, May 9, 2016 at 2:03:58 AM UTC+9, feza wrote:
>
> Milan
>
> Script is here: 
> https://gist.github.com/musmo/27436a340b41c01d51d557a655276783
>
>
> On Sunday, May 8, 2016 at 12:40:44 PM UTC-4, feza wrote:
>>
>> Thanks for the tip (initially I just transllated the matlab verbatim)
>>
>> Now I have made all the changes. In place operations, and direct function 
>> calls.
>> Despite these changes. Matlab is 3.6 seconds, new Julia  7.6 seconds
>> TBH the results of this experiment are frustrating, I was hoping Julia 
>> was going to provide a huge speedup (on the level of c)
>>
>> Am I still missing anything in the Julia code that is crucial to speed?
>> @code_warntype looks ok sans a few red unions which i don't think are in 
>> my control
>>
>>
>> On Sunday, May 8, 2016 at 8:15:25 AM UTC-4, Tim Holy wrote:
>>>
>>> One of the really cool features of julia is that functions are allowed 
>>> to have 
>>> more than 0 arguments. It's even considered good style, and I highly 
>>> recommend 
>>> making use of this awesome feature in your code! :-) 
>>>
>>> In other words: try passing all variables as arguments to the functions. 
>>> Even 
>>> though you're wrapping everything in a function, performance-wise you're 
>>> running up against an inference problem 
>>> (https://github.com/JuliaLang/julia/issues/15276). In terms of coding 
>>> style, 
>>> you're still essentially using global variables. Honestly, these make 
>>> your 
>>> life harder in the end (
>>> http://c2.com/cgi/wiki?GlobalVariablesAreBad)---it's 
>>> not a bad thing that julia provides gentle encouragement to avoid using 
>>> them, 
>>> and you're losing out on opportunities by trying to sidestep that 
>>> encouragement. 
>>>
>>> Best, 
>>> --Tim 
>>>
>>> On Sunday, May 08, 2016 01:38:41 AM feza wrote: 
>>> > That's no surprise your CPU is better :) 
>>> > 
>>> > Regarding devectorization 
>>> > for l in 1:q 
>>> > for k in 1:nz 
>>> > for j in 1:ny 
>>> > for i in 1:nx 
>>> > u = ux[i,j,k] 
>>> > v = uy[i,j,k] 
>>> > w = uz[i,j,k] 
>>> > 
>>> > cu = c[k,1]*u  + c[k,2]*v + c[k,3]*w 
>>> > u2 = u*u + v*v + w*w 
>>> > feq[i,j,k,l] = weights[k]*ρ[i,j,k]*(1 + 3*cu + 
>>> 9/2*(cu*cu) 
>>> > - 3/2*u2) 
>>> > f[i,j,k,l] = f[i,j,k,l]*(1-ω) + ω*feq[i,j,k,l] 
>>> >   end 
>>> >   end 
>>> >   end 
>>> >  end 
>>> > 
>>> > Actually makes the code a lot slower 
>>> > 
>>> > On Sunday, May 8, 2016 at 4:37:18 AM UTC-4, Patrick Kofod Mogensen 
>>> wrote: 
>>> > > For what it's worth  it run in about 3-4 seconds on my computer on 
>>> latest 
>>> > > v0.4. 
>>> > > 
>>> > > CPU : Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz 
>>> > > 
>>> > > On Sunday, May 8, 2016 at 10:33:14 AM UTC+2, Patrick Kofod Mogensen 
>>> wrote: 
>>> > >> As for the v0.5 performance (which is horrible), I think it's the 
>>> boxing 
>>> > >> issue with closure https://github.com/JuliaLang/julia/issues/15276 
>>> . 
>>> > >> Right? 
>>> > >> 
>>> > >> On Sunday, May 8, 2016 at 10:29:59 AM UTC+2, STAR0SS wrote: 
>>> > >>> You are using a lot of vectorized operations and Julia isn't as 
>>> good as 
>>> > >>> matlab is with those. 
>>> > >>> 
>>> > >>> The usual solution is to devectorized your code and to use loops 
>>> (except 
>>> > >>> for matrix multiplication if you have large matrices). 
>>>
>>>

[julia-users] Re: Order of multiple for-loops

[Tk]

As for "column-majorness" of matrix algebra, it seems to be natural (for 
me) because it is common to write a vector in column form. But of course, 
one could start with a row vector as the basic building block, so it looks 
like a matter of convention...

The reason for this choice might ultimately be traced back to the greater 
ratio of right-handed people (just my guess!).
As English is written from left to right so as to be convenient for 
right-handed people with pen (again a guess!!), 
old mathematician in German, France, and US (where people often write from 
left to right) might have preferred to
use right eigenvectors A x = lambda x rather than left eigenvectors x A = 
lambda x, in such a way that sentences begin with
capital letters (Hello rather than helloW). Also, people may prefer reading 
tabular data from left to right because it is
close to reading a sentence in a book. So the true question is why there 
are more right-handed people :)

And because there are countries where people write from top to bottom or 
right to left, it would have been interesting
if matrix algebra originated from such countries...

[julia-users] Re: Order of multiple for-loops

[Tk]

Thanks very much for various information. And I am sorry if my question 
might be misleading or indicate
confusion or complaints about memory order (indeed, column-major is 
perfectly fine for me).
So let me explain a bit more about my question...

Given that Julia's arrays are column-major order, the following loop is 
most efficient for m x n array A:

 for i2=1:n,
   i1=1:m
   A[ i1, i2 ] = 10 * i1 + i2
end

or equivalently,

 for i2=1:n,  i1=1:m  # (1)
   A[ i1, i2 ] = 10 * i1 + i2
end

To create the same array using comprehension, I once thought that the 
expression might be
something like this

 A = [ 10 * i1 + i2  for  i2=1:n, i1=1:m ] # (2) 

where the index most far from "for" runs faster. Then for-loops in (1) and 
(2) would take the same form.
In fact, Julia employs a different definition

A = [ 10 * i1 + i2  for i1=1:m, i2=1:n ] # (3)

where the index closer to "for" runs faster.
So part of my question was why the definition (2) was not employed (within 
the column-major order)
but (3) was employed.

But come to think of it..., the definition (2) is clearly not nice-looking 
(because the indices are swapped)
while Julia's definition in (3) is more intuitive (although the order in 
(1) and (3) is different).
So I guess this would be the reason for this choice (sorry if this is a 
silly question...).
And I now understand Stephan's comment about row-major order (thanks :)

RE how to memorize the definition (3), the following form is close to (3):

B = [ [ 10 * i1 + i2 for i1=1:m ] for i2=1:n ]

although this gives an array of arrays. So, the definition (3) may be 
viewed as a "rectangularized" or serialized
version of B. This is also easy for me to memorize it (Btw, the above 
expression is similar to implied do-loops
in Fortran, but in fact the content is very different!)

Anyway, thanks very much :)

[julia-users] Re: For loop = or in?

[Tk]

Hi,

Before closing (this thread?), I would like to write one more thing. First, 
because I am not a native English speaker, it doesn't
actually matter how 1:n etc "reads" (simply because I don't pronounce them 
in mind :) Second, when I started
learning Julia half a year ago (sorry for a slow starter...), I also felt 
awkward that I have two kinds of syntax for the same
thing. So I tried to use "in" only for the time being (to mimic Python). 
But after a while I got back to "=", because it is gives  more concise (and 
IMO more readable) expressions, particularly in array comprehension. For 
example,

a = [ f(i,j,k) for i=1:p, j=1:q, k=1:r ]

a = [ f(i,j,k) for i in 1:p, j in 1:q, k in 1:r ]

# In this sense \in (greek letter) is as concise as =, but typing in the 
greek character takes a bit more time... (though I may be
able to customize Emacs if necessary).

As for Python-like, "there should be only one syntax for one purpose 
(semantic?)" policy (sorry if my English is strange), there seem to be many 
counter examples in Julia. For example, vcat() and [ a; b ] seem 
essentially the same, while the latter is probably there for brevity. I 
think it is not bad to have a shorter way to write it unless there are 
serious problems.

Btw, apart from the = vs in stuff, I really like this language because it 
allows multiple for-loops to be written concisely as

for i = 1:p,
 j = 1:q,
 k = 1:r
 (do something)
end

Indeed, I used to use a macro like this in C++ before that allows to write

For( i, 1, p,
   j, 1, p,
   k, 1, r ) {
 (do something)
}

so Julia has become one of my most favorite languages :)
Sorry for a long post...

Re: [julia-users] Re: For loop = or in?

[Tk]

> `=` better than `in`: `a = [ f(i,j,k) for i=1:p, j=1:q, k=1:r ]`

Please note that I am not trying to claim "=" is better universally (i.e. 
for everyone) based on this example,
because some people prefer more English like syntax for clarity of meaning 
(even when a bit longer).

So anyway, it would be nice if the official document will be updated to 
emphasize that there is no distinction
between in and = (used with for-loops, as of v0.4), and that if one 
prefers, one could use "in" everywhere (this is
a great point). Then we can just cite that explanation when a question 
arises for a while :)


On Thursday, October 29, 2015 at 10:41:19 PM UTC+9, Tom Breloff wrote:
>
> Agreed with Tony.  Having lots of ways to do the same thing is fine, as 
> long as we had no other syntactical use for the `=`.  Anyone want to 
> support notation like: `for y = sin(_) in 1:10 ... end`?  Probably not, so 
> no big deal to keep both around.
>
> Also I just saw Tk's post with this great example of `=` better than `in`: 
> `a = [ f(i,j,k) for i=1:p, j=1:q, k=1:r ]`
>
> On Thu, Oct 29, 2015 at 9:10 AM, Tony Kelman  > wrote:
>
>> How is having multiple syntaxes for something automatically bad? I don't 
>> hear people complaining that for loops are redundant when you could just do 
>> while with a counter, or that enumerate is equivalent. Having style 
>> guidelines for base to say when one choice makes more sense than another is 
>> fine, but this thread's overblown.
>
>
>

[julia-users] Order of multiple for-loops

[Tk]

Hello,

In the following explicit for-loops,

for i = 1:2,
 j = 1:3,
 k = 1:4
@show i,j,k
end

the index "k" runs fastest, as if this is written as three for(...) loops 
in C etc.
On the other hand, in the following array comprehension,

a = [ (@show i,j,k; i+j+k) for i=1:2, j=1:3, k=1:4 ]

the index "i" runs fastest. To understand this behavior, is it reasonable 
to think that
"an index closer to the expression to be evaluated runs faster"? Or is 
there any different
reason that naturally determines this behavior (in the latter example)?

# If we flatten the for-loops as "for i=1:2, j=1:3, k=1:4", both cases look 
the same.
So initially I got a bit confused (though I now got used to it).

# I tried to remember this order by analogy to the implied Do-loop syntax 
in Fortran, such as

a = [((( f(i,j,k), i=1,2 ), j=1,3 ), k=1,4 )))]

shere "i" runs fastest (obviously).

[julia-users] Re: Order of multiple for-loops

[Tk]

PS. If this is concerned, I know that arrays in Julia are column-major 
order.

On Thursday, October 29, 2015 at 11:57:23 PM UTC+9, Tk wrote:
>
> Hello,
>
> In the following explicit for-loops,
>
> for i = 1:2,
>  j = 1:3,
>  k = 1:4
> @show i,j,k
> end
>
> the index "k" runs fastest, as if this is written as three for(...) loops 
> in C etc.
> On the other hand, in the following array comprehension,
>
> a = [ (@show i,j,k; i+j+k) for i=1:2, j=1:3, k=1:4 ]
>
> the index "i" runs fastest. To understand this behavior, is it reasonable 
> to think that
> "an index closer to the expression to be evaluated runs faster"? Or is 
> there any different
> reason that naturally determines this behavior (in the latter example)?
>
> # If we flatten the for-loops as "for i=1:2, j=1:3, k=1:4", both cases 
> look the same.
> So initially I got a bit confused (though I now got used to it).
>
> # I tried to remember this order by analogy to the implied Do-loop syntax 
> in Fortran, such as
>
> a = [((( f(i,j,k), i=1,2 ), j=1,3 ), k=1,4 )))]
>
> shere "i" runs fastest (obviously).
>

Re: [julia-users] Re: For loop = or in?

[Tk]

As far as I can understand, Julia also seems trying to attract people from 
Matlab,
because there are so many similarities in the syntax (.* and ./ etc) and 
the names of functions.

Also I often see questions from Matlab users posted in StackOverflow. Their 
codes are
rather Matlab-like, but it works anyway. I guess this helps people to try a 
new language
with lower barrier. (Though, personally, I don't like ".*" notation very 
much. But I can live with it.)

As for =, I also prefer using = for counters (for i = 1:n) and "in" for 
other containers.
I use Fortran and Python most often for research work, but I feel "=" is 
quite natural for indicating
that the index goes from 1 to n. On the other hand, with "in", I feel like 
the entity is just
somewhere inside the container, with no particular order enforced (even if 
it has a definite order).
My familiarity with "=" is probably coming from old languages (including 
C-like ones),
but that aside, I hope "=" remains a valid symbol for this.

So anyway, +1 for the comments from Glen O.

On Wednesday, October 28, 2015 at 3:20:19 AM UTC+9, FANG Colin wrote:
>
> Julia tries to attract people from Python & R, which use `in`. As for 
> matlab, it is not a direct competitor.
>
> Anyway, I think we only need 1 of the 2. "There should be one-- and 
> preferably only one --obvious way to do it."
>
> Maybe enhance the documentation for the time being.
>
>
> On 27 October 2015 at 16:38, Glen O  
> wrote:
>
>> "When calculating a Fibonacci number, we have to apply 
>> F_n=F_(n-1)+F_(n-2) repeatedly. So to find F_6, we apply the equation for n 
>> equals 3 through 6". Writing it as "for n in 3 through 6" or "for n in the 
>> range 3 through 6" wouldn't make nearly as much sense.
>>
>> As I said, for general iterables, like vectors, the "in" keyword makes 
>> more sense. But when you're talking about a counter variable, equals makes 
>> a much more natural expression - you're not really constructing the range 
>> object, you're just telling the program you want the counter to start at 
>> the first value, and increment until it reaches the second value.
>>
>> On Wednesday, 28 October 2015 02:23:54 UTC+10, Tom Breloff wrote:
>>>
>>> It definitely makes sense for a range.
>>>
>>>
>>> Sorry... gotta disagree... mathematical set notation is more 
>>> appropriate, especially for scientific computing.  This is coming from a 
>>> former matlab user, btw, so it's not like I was confused by the syntax.   
>>> The "for i = 1:5" syntax is actually more reminiscent of C:  "for (int i=1; 
>>> i<=5; i++)", and I'm guessing that the syntax originated more from that 
>>> rather than scientific concepts.
>>>
>>> On Tue, Oct 27, 2015 at 11:58 AM, feza  wrote:
>>>
 +1 @Tom Breloff .  
 I was confused about this when starting out. Comparing   `for i in 
 1:3` vs  `for i = 1:3`, even though I regularly use matlab if you think 
 about it for `i = 1:10` doesn't really make a lot of sense. It would be 
 nice if it was just one way as opposed to the confusion about whether = or 
 in should be used.

 On Tuesday, October 27, 2015 at 10:26:44 AM UTC-4, Tom Breloff wrote:
>
> It's harmless, sure, but I would prefer that everyone uses "in" 
> exclusively so that there's one less thing to waste brainpower on.  You 
> don't say "for each x equals the range 1 to n", you say "for each x in 
> the 
> range 1 to n".  I don't think "=" has a place here at all except to allow 
> copy/pasting of Matlab code (which creates other performance problems 
> anyways).
>
> On Tue, Oct 27, 2015 at 10:04 AM, Stefan Karpinski <
> ste...@karpinski.org> wrote:
>
>> My general approach is to only use = when the RHS is an explicit 
>> range, as in `for i = 1:n`. For everything else I use `for i in v`. I 
>> would 
>> be ok with dropping the = syntax at some point, but it seems pretty 
>> harmless to have it.
>>
>> On Tue, Oct 27, 2015 at 8:56 AM, FANG Colin  
>> wrote:
>>
>>> Thank you. In that case I will happily stick with `in`.
>>>
>>>
>>> On Monday, October 26, 2015 at 8:43:22 PM UTC, Alireza Nejati wrote:

 There is no difference, as far as I know.

 '=' seems to be used more for explicit ranges (i = 1:5) and 'in' 
 seems to be used more for variables (i in mylist). But using 'in' for 
 everything is ok too.

 The '=' is there for familiarity with matlab. Remember that julia's 
 syntax was in part designed to be familiar to matlab users.

 On Tuesday, October 27, 2015 at 8:26:07 AM UTC+13, FANG Colin wrote:
>
> Hi All
>
> I have got a stupid question:
>
> Are there any difference in "for i in 1:5" and "for i = 1:5"?
>
> Does the julia community prefer one to the other? I see use of