Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Steven G. Johnson]

On Sunday, October 18, 2015 at 10:01:20 AM UTC-4, Stefan Karpinski wrote:
>
> There are different styles and levels of writing code in every language. 
> There's highly abstract C++ and there's low-level pointer-chasing C++ 
> that's basically C. Even in C there's the void*-style of programming which 
> is effectively dynamically typed without the safety. Given this fact, I'm 
> not sure what solving the two language problem would look like from the 
> perspective this post is posing. Enforcing only one style of programming 
> sounds like a problem to me, not a solution. On the contrary, I think one 
> of Julia's greatest strengths is its ability to accommodate a very broad 
> range of programming styles and levels.
>

Even forgetting about pointers etc, the fact is that highly optimized code 
often looks very different from unoptimized code in any language.   e.g. 
the simplest way to implement a matrix multiplication in C is three loops. 
  Getting decent performance (compared to peak flops) requires 100+ lines 
of code.  Super-optimized implementations require tens of thousands of 
lines of code.  This has nothing to do with safety and pointers, and 
everything to do with locality (for caches) and unrolling (for registers).

There is no language in which optimization doesn't typically involve 
rewriting critical code.

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Tamas Papp]

On Mon, Oct 19 2015, Steven G. Johnson  wrote:

> On Sunday, October 18, 2015 at 10:01:20 AM UTC-4, Stefan Karpinski wrote:
>>
>> There are different styles and levels of writing code in every language.
>> There's highly abstract C++ and there's low-level pointer-chasing C++
>> that's basically C. Even in C there's the void*-style of programming which
>> is effectively dynamically typed without the safety. Given this fact, I'm
>> not sure what solving the two language problem would look like from the
>> perspective this post is posing. Enforcing only one style of programming
>> sounds like a problem to me, not a solution. On the contrary, I think one
>> of Julia's greatest strengths is its ability to accommodate a very broad
>> range of programming styles and levels.
>>
>
> Even forgetting about pointers etc, the fact is that highly optimized code
> often looks very different from unoptimized code in any language.   e.g.
> the simplest way to implement a matrix multiplication in C is three loops.
>   Getting decent performance (compared to peak flops) requires 100+ lines
> of code.  Super-optimized implementations require tens of thousands of
> lines of code.  This has nothing to do with safety and pointers, and
> everything to do with locality (for caches) and unrolling (for registers).
>
> There is no language in which optimization doesn't typically involve
> rewriting critical code.

Still, in a nice language the programming effort vs speed curve should
not be too steep. Eg you can get 80% of the speed with 20% of the
effort, both compared to the absolute super fastest solution.

Julia is a nice language: with a few simple rules that one pick ups up
after a bit of experience (= mistakes) [1], it does not take a whole lot
of extra effort to write reasonably fast code. Getting within a factor
of 2-5 of C with significantly less effort is enough for a lot of
applications, especially considering that in scientific computing a lot
of code is only used a few times (analyze a problem, estimate a model,
etc). For the rest, one can optimize further.

The "two language problem" is really about a discontinuity in the effort
vs speed curve. You hit the limits of Language A, you have to go to
Language B, which is significantly different. In Julia you can make a
lot of incremental optimizations.

Best,

Tamas

[1] http://docs.julialang.org/en/release-0.4/manual/performance-tips/

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[John Gibson]

Having a language where you don't worry about types and the compiler 
handles optimization is possible in Fortran because it has a dead-simple, 
nonextendable set of types. The downside is that you can't do abstraction 
in Fortran, and all your code ends up being a long sequence of low-level 
operations on the fundamental numeric types. 

>From my experience having an extensible type system and well-designed 
libraries of high-level types (strings, matrices with no structure, special 
structure) is well worth the added complexity of types, if for no other 
reason than that you can represent high-level concepts with high-level 
expressions, making code compact and readable. 

In Julia you have to think about types harder than in Python, but the 
benefit is you get extendable high-level abstraction, compact, readable 
code, polymorphic code generation, and nearly the speed of C. It's a 
winning combination!

"craziness" would apply to a language with the rich type system of Julia 
but the implicit type system of Fortran. Yikes!

John




On Monday, October 19, 2015 at 2:31:16 PM UTC-4, lawrence dworsky wrote:
>
> I certainly am not arguing against developing new languages with improved 
> programming techniques, but I think you're being too quick to consider 
> liking an older technique as heresy. I programmed in Fortran for over 4 
> decades and never had any trouble with the variable name typing system:
>
> 1.  Once you're used to it, it becomes second nature and doesn't slow down 
> program development at all. If anything, it speeds it up a bit.
>
> 2.  If you really hate it, you can turn it off - either selectively or 
> completely (implicit none).
>
> 3.  It doesn't restrict program structure or development in any way.
>
> 4.  It doesn't interact with program execution speed in any way.
>
> This last point, I think, is very important. I'm a Julia newbe so I'm just 
> reading all the stuff about the interaction between variable typing and 
> execution speed. I've gotta tell you, having a language where I don't worry 
> about this at all and then the compiler handles any optimization issues has 
> a lot to say for it. Why is this "craziness?"
>
> Larry
>
>
> On Sun, Oct 18, 2015 at 3:06 PM, Art Kuo  > wrote:
>
>>
>> Having the type of a variable be determined by the variable name is 
>>> craziness. Which is why you always run with implicit none. 
>>>
>>
>> It had its reason back in its day. For math it is/was typical to choose 
>> counter-like integers for things like a series, with variables like i, j, 
>> k, l, m, or n. It wasn't unreasonable to carry that convention into Fortran 
>> 66. Back in the days of punch cards and 1024-byte memory, variable names 
>> were often one letter and one alphanumeric, so the easier/shorter the 
>> declaration, the better. In Fortran 77, "implicit" was introduced as a way 
>> to maintain backwards compatibility and also optionally break from the i-n 
>> convention and be clear about it. Nowadays we can be thankful for upper and 
>> lower case, and the backspace key makes long variable names trivial. 
>>
>
>

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[lawrence dworsky]

I certainly am not arguing against developing new languages with improved
programming techniques, but I think you're being too quick to consider
liking an older technique as heresy. I programmed in Fortran for over 4
decades and never had any trouble with the variable name typing system:

1.  Once you're used to it, it becomes second nature and doesn't slow down
program development at all. If anything, it speeds it up a bit.

2.  If you really hate it, you can turn it off - either selectively or
completely (implicit none).

3.  It doesn't restrict program structure or development in any way.

4.  It doesn't interact with program execution speed in any way.

This last point, I think, is very important. I'm a Julia newbe so I'm just
reading all the stuff about the interaction between variable typing and
execution speed. I've gotta tell you, having a language where I don't worry
about this at all and then the compiler handles any optimization issues has
a lot to say for it. Why is this "craziness?"

Larry


On Sun, Oct 18, 2015 at 3:06 PM, Art Kuo  wrote:

>
> Having the type of a variable be determined by the variable name is
>> craziness. Which is why you always run with implicit none.
>>
>
> It had its reason back in its day. For math it is/was typical to choose
> counter-like integers for things like a series, with variables like i, j,
> k, l, m, or n. It wasn't unreasonable to carry that convention into Fortran
> 66. Back in the days of punch cards and 1024-byte memory, variable names
> were often one letter and one alphanumeric, so the easier/shorter the
> declaration, the better. In Fortran 77, "implicit" was introduced as a way
> to maintain backwards compatibility and also optionally break from the i-n
> convention and be clear about it. Nowadays we can be thankful for upper and
> lower case, and the backspace key makes long variable names trivial.
>

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Steven G. Johnson]

On Monday, October 19, 2015 at 2:31:16 PM UTC-4, lawrence dworsky wrote:
>
> This last point, I think, is very important. I'm a Julia newbe so I'm just 
> reading all the stuff about the interaction between variable typing and 
> execution speed. I've gotta tell you, having a language where I don't worry 
> about this at all and then the compiler handles any optimization issues has 
> a lot to say for it. Why is this "craziness?"
>

An untyped variable in Julia is a completely different beast from an 
untyped variable in Fortran 77 style with implicit typing.

If I write f(x) = x+1 in Julia, the code is totally *polymorphic* — the 
function f(x) works for any  x that supports addition (+) with an Int (1), 
including bigints, arrays, all the different floating-point types, complex 
numbers, external-package types like quaternions, etc.  Furthermore, f(x) 
is fast for all of those types, because the Julia compiler specializes f 
into a different compiled version for each argument type as needed.

If you use an implicitly typed variable "x" in Fortran (without "implicit 
none"), then it is *not* polymorphic.  x has one and only one type (real, 
i.e. floating-point, either single or double precision depending on a 
compiler switch), and your code only works with that type.

The advantage of the polymorphism and dynamism in Julia is that code is 
much more re-usable — we can implement a function like sum(x) *once* and it 
will work with any type.   (This is a generalization of the polymorphism 
that e.g. object-oriented programming or C++ templates give you.)The 
price you pay for this is that if you want it to be fast, you need to think 
a little bit more about types than you would with code that worked with one 
and only one type.

(If you want code that works with one and only one type in Julia, of course 
you can do it, just by explicit type declarations.  But it is more powerful 
to get used to thinking about dynamic polymorphism, and learning to follow 
the simple rules that will make such code fast.)

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Tamas Papp]

On Mon, Oct 19 2015, Sisyphuss  wrote:

> On Monday, October 19, 2015 at 5:06:45 PM UTC+2, Tamas Papp wrote:
>>
>> The "two language problem" is really about a discontinuity in the effort
>> vs speed curve. You hit the limits of Language A, you have to go to
>> Language B, which is significantly different. In Julia you can make a
>> lot of incremental optimizations.
>
> I define Language C := Union{A,B}. Then it solves the two language problem.

"I think you'll find it's a bit more complicated than that ..." :-)

Best,

Tamas

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Sisyphuss]

On Monday, October 19, 2015 at 5:06:45 PM UTC+2, Tamas Papp wrote:
>
> On Mon, Oct 19 2015, Steven G. Johnson  
> wrote: 
>
> > On Sunday, October 18, 2015 at 10:01:20 AM UTC-4, Stefan Karpinski 
> wrote: 
> >> 
> >> There are different styles and levels of writing code in every 
> language. 
> >> There's highly abstract C++ and there's low-level pointer-chasing C++ 
> >> that's basically C. Even in C there's the void*-style of programming 
> which 
> >> is effectively dynamically typed without the safety. Given this fact, 
> I'm 
> >> not sure what solving the two language problem would look like from the 
> >> perspective this post is posing. Enforcing only one style of 
> programming 
> >> sounds like a problem to me, not a solution. On the contrary, I think 
> one 
> >> of Julia's greatest strengths is its ability to accommodate a very 
> broad 
> >> range of programming styles and levels. 
> >> 
> > 
> > Even forgetting about pointers etc, the fact is that highly optimized 
> code 
> > often looks very different from unoptimized code in any language.   e.g. 
> > the simplest way to implement a matrix multiplication in C is three 
> loops. 
> >   Getting decent performance (compared to peak flops) requires 100+ 
> lines 
> > of code.  Super-optimized implementations require tens of thousands of 
> > lines of code.  This has nothing to do with safety and pointers, and 
> > everything to do with locality (for caches) and unrolling (for 
> registers). 
> > 
> > There is no language in which optimization doesn't typically involve 
> > rewriting critical code. 
>
> Still, in a nice language the programming effort vs speed curve should 
> not be too steep. Eg you can get 80% of the speed with 20% of the 
> effort, both compared to the absolute super fastest solution. 
>
> Julia is a nice language: with a few simple rules that one pick ups up 
> after a bit of experience (= mistakes) [1], it does not take a whole lot 
> of extra effort to write reasonably fast code. Getting within a factor 
> of 2-5 of C with significantly less effort is enough for a lot of 
> applications, especially considering that in scientific computing a lot 
> of code is only used a few times (analyze a problem, estimate a model, 
> etc). For the rest, one can optimize further. 
>
> The "two language problem" is really about a discontinuity in the effort 
> vs speed curve. You hit the limits of Language A, you have to go to 
> Language B, which is significantly different. In Julia you can make a 
> lot of incremental optimizations. 
>
> I define Language C := Union{A,B}. Then it solves the two language problem.
 

> Best, 
>
> Tamas 
>
> [1] http://docs.julialang.org/en/release-0.4/manual/performance-tips/ 
>

[julia-users] Re: Does Julia really solve the two-language problem?

[Páll Haraldsson]

On Sunday, October 18, 2015 at 12:51:21 PM UTC, Sisyphuss wrote:
>
> The two-language problem refers to prototyping with one slow dynamic 
> language and rewrite it with a fast static language for the final product.
>

Yes, but there is one other two (or more..) language "thing" (I wouldn't 
call it a "problem", would you?):

People using Python/Numpy, MATLAB, etc. are used to having to reimplement a 
tiny part of their code (or not so..?) in C. [Often you reuse those 
solutions (or pure ones) and to not even see a two language problem.]

In Julia (and new languages in general), theoretically, most code could be 
only in Julia, but practically you reuse slow (or fast..) libraries in 
other languages. That seems to be a great thing to me! Maybe I'm in 
minority, as I've found it difficult to "convert" people, but I can say, 
that seems to be changing..

Do you people consider it a hurdle to call, say using PyCall or whatever 
(do usres need to know (too..) much else than "Python and the relevant to 
me library" exists?)? Are wrappers (using PyCall) preferred or even 
reimplementiong stuff in pure Julia?


If you are only reusing libraries, you do not have to know (much about) the 
other languages, but still, you might have to/want to (end up) maintain 
those libraries.. I forsee that, in many cases a permanent "solution" (or 
"problem" if you will). Do you think for most things, calling other 
languages, to be a stop-gap solution?

-- 
Palli.

[julia-users] Re: Does Julia really solve the two-language problem?

[Art Kuo]

> Having the type of a variable be determined by the variable name is 
> craziness. Which is why you always run with implicit none. 
>

It had its reason back in its day. For math it is/was typical to choose 
counter-like integers for things like a series, with variables like i, j, 
k, l, m, or n. It wasn't unreasonable to carry that convention into Fortran 
66. Back in the days of punch cards and 1024-byte memory, variable names 
were often one letter and one alphanumeric, so the easier/shorter the 
declaration, the better. In Fortran 77, "implicit" was introduced as a way 
to maintain backwards compatibility and also optionally break from the i-n 
convention and be clear about it. Nowadays we can be thankful for upper and 
lower case, and the backspace key makes long variable names trivial. 

[julia-users] Re: Does Julia really solve the two-language problem?

[LarryD]

How does this compare to the "old standard?"

1.  Fortran requires every variable to have a defined type, but 
automatically defaults to floating or integer type based on the first 
letter of the variable name. Or, you can pre-define your own sets of 
first-letter defaults with the Implicit command. Once you get used to this, 
writing new code goes just as fast as if you didn't need to define types.

2.  Optimizing compilers take care of most of the clean-up phase. No work 
required.

Larry

On Sunday, October 18, 2015 at 7:51:21 AM UTC-5, Sisyphuss wrote:
>
> The two-language problem refers to prototyping with one slow dynamic 
> language and rewrite it with a fast static language for the final product.
>
> If Julia really solves the two-language problem, it should meet the 
> following criteria:
> Let A be the code written during prototyping, B be the code written for 
> the final product, with a small net increment $\Delta$, A+\Delta=B.
>
> If Julia uses one code style to do prototyping, and then uses a completely 
> different style to write final product, then it can't be called the same 
> language. At best, Julia turns the 2-language problem to a 1.5-language 
> problem.
>
>
>

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Stefan Karpinski]

There are different styles and levels of writing code in every language.
There's highly abstract C++ and there's low-level pointer-chasing C++
that's basically C. Even in C there's the void*-style of programming which
is effectively dynamically typed without the safety. Given this fact, I'm
not sure what solving the two language problem would look like from the
perspective this post is posing. Enforcing only one style of programming
sounds like a problem to me, not a solution. On the contrary, I think one
of Julia's greatest strengths is its ability to accommodate a very broad
range of programming styles and levels.

On Sun, Oct 18, 2015 at 7:14 PM, Kristoffer Carlsson 
wrote:

>
>> If Julia uses one code style to do prototyping, and then uses a
>> completely different style to write final product, then it can't be called
>> the same language. At best, Julia turns the 2-language problem to a
>> 1.5-language problem.
>>
>>
>  What is it with Julia that requires you to write in two completely
> different styles? If what you are saying is that you can't just copy paste
> code from Matlab and get 10x speed up then that is not really fair.
>
> For all the codes I am writing, sticking to good idiomatic julia code, it
> is very rarely needed to rewrite large portions of the good for
> performance. Tweaking some stuff in hot loops maybe but not starting from
> scratch.
>

[julia-users] Re: Does Julia really solve the two-language problem?

[Christoph Ortner]

Just speaking from my own experience in scientific computing (PDEs and 
molecular simulation): A+\Delta=B. with \Delta small is *not* necessary for 
Julia to "solve the two-language problem". In fact, I tend to write three 
iterations: (i) a crappy piece of code that solves the core problem but is 
neither fast nor elegant; (ii) a complete rewrite, using Julia's 
type-system to make sure that the code is readable and easily generalisable 
in the future; (iii) find the bottlenecks and optimise the performance. All 
three steps work brilliantly for me; there are usually very small parts 
that I need to optimise for performance, but even if they were bigger, it 
is extremely useful for productivity to have a working code in the same 
language to incrementally optimise it. The only "problem" I have, is that I 
have to be very disciplined in step (i) to not try to incorporate aspects 
of (ii) or (iii) too early. 

Christoph

[julia-users] Re: Does Julia really solve the two-language problem?

[Kristoffer Carlsson]

>
>
> If Julia uses one code style to do prototyping, and then uses a completely 
> different style to write final product, then it can't be called the same 
> language. At best, Julia turns the 2-language problem to a 1.5-language 
> problem.
>
>
 What is it with Julia that requires you to write in two completely 
different styles? If what you are saying is that you can't just copy paste 
code from Matlab and get 10x speed up then that is not really fair. 

For all the codes I am writing, sticking to good idiomatic julia code, it 
is very rarely needed to rewrite large portions of the good for 
performance. Tweaking some stuff in hot loops maybe but not starting from 
scratch.

[julia-users] Re: Does Julia really solve the two-language problem?

[Kristoffer Carlsson]

Having the type of a variable be determined by the variable name is 
craziness. Which is why you always run with implicit none.

On Sunday, October 18, 2015 at 4:02:02 PM UTC+2, LarryD wrote:
>
> How does this compare to the "old standard?"
>
> 1.  Fortran requires every variable to have a defined type, but 
> automatically defaults to floating or integer type based on the first 
> letter of the variable name. Or, you can pre-define your own sets of 
> first-letter defaults with the Implicit command. Once you get used to this, 
> writing new code goes just as fast as if you didn't need to define types.
>
> 2.  Optimizing compilers take care of most of the clean-up phase. No work 
> required.
>
> Larry
>
> On Sunday, October 18, 2015 at 7:51:21 AM UTC-5, Sisyphuss wrote:
>>
>> The two-language problem refers to prototyping with one slow dynamic 
>> language and rewrite it with a fast static language for the final product.
>>
>> If Julia really solves the two-language problem, it should meet the 
>> following criteria:
>> Let A be the code written during prototyping, B be the code written for 
>> the final product, with a small net increment $\Delta$, A+\Delta=B.
>>
>> If Julia uses one code style to do prototyping, and then uses a 
>> completely different style to write final product, then it can't be called 
>> the same language. At best, Julia turns the 2-language problem to a 
>> 1.5-language problem.
>>
>>
>>

Re: [julia-users] Re: Does Julia really solve the two-language problem?

[Stefan Karpinski]

Fortran is an awesome language, but clearly there's demand for something
easier. Keep in mind that every single popular technical computing
environment is dynamically typed.

On Sun, Oct 18, 2015 at 7:32 PM, LarryD  wrote:

> How does this compare to the "old standard?"
>
> 1.  Fortran requires every variable to have a defined type, but
> automatically defaults to floating or integer type based on the first
> letter of the variable name. Or, you can pre-define your own sets of
> first-letter defaults with the Implicit command. Once you get used to this,
> writing new code goes just as fast as if you didn't need to define types.
>
> 2.  Optimizing compilers take care of most of the clean-up phase. No work
> required.
>
> Larry
>
>
> On Sunday, October 18, 2015 at 7:51:21 AM UTC-5, Sisyphuss wrote:
>>
>> The two-language problem refers to prototyping with one slow dynamic
>> language and rewrite it with a fast static language for the final product.
>>
>> If Julia really solves the two-language problem, it should meet the
>> following criteria:
>> Let A be the code written during prototyping, B be the code written for
>> the final product, with a small net increment $\Delta$, A+\Delta=B.
>>
>> If Julia uses one code style to do prototyping, and then uses a
>> completely different style to write final product, then it can't be called
>> the same language. At best, Julia turns the 2-language problem to a
>> 1.5-language problem.
>>
>>
>>