Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

[Michael Jones]

I'm sorry Isaac, I meant multi-language benchmarking generally, nothing
about the specific case you mention so i was slightly tangential to your
original post.

On Thu, Mar 7, 2019 at 9:41 AM 'Isaac Gouy' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> On Wednesday, March 6, 2019 at 7:22:41 PM UTC-8, Michael Jones wrote:
>>
>> There is another problem about these microbenchmarks as well--they often
>> are ports of an originating C-version.
>>
>
> Which microbenchmarks?
>
> You quoted a reply to a question about "Performance comparison of Go, C++,
> and Java for biological sequencing tool".
>
> For those who haven't looked, that is about an evaluation (done by the
> authors of the elPrep tool) to select a new implementation language for the
> particular case of elPrep.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

['Isaac Gouy' via golang-nuts]

On Wednesday, March 6, 2019 at 7:22:41 PM UTC-8, Michael Jones wrote:
>
> There is another problem about these microbenchmarks as well--they often 
> are ports of an originating C-version. 
>

Which microbenchmarks? 

You quoted a reply to a question about "Performance comparison of Go, C++, 
and Java for biological sequencing tool".

For those who haven't looked, that is about an evaluation (done by the 
authors of the elPrep tool) to select a new implementation language for the 
particular case of elPrep.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

[Robert Engels]

I wholeheartedly agree and would add an important point, that ease of 
development/understanding leads to easier refactoring allowing for improvements 
in the algorithm- which are usually far more important to performance - which 
is exactly what you’ve demonstrated. 

> On Mar 6, 2019, at 9:22 PM, Michael Jones  wrote:
> 
> There is another problem about these microbenchmarks as well--they often are 
> ports of an originating C-version. 
> 
> I just implemented the Rabbit stream cipher and after reading the papers I 
> reviewed several versions online in Java and two in Go, as well as the now 
> open-source C version. It seems that they all started with the C version and 
> did a "c-to-Java" or "c-to-go-ization" of it. The results are not strong for 
> the language. My code took me a day and runs at 830 MB/s vs 620 MB/s peak for 
> any of these. Now this is not a microbenchmark exactly and I may have more 
> experience than some, but it still shows that language-X would seem to do a 
> job at 620 that it could do at 830 with cleaner code. That kind of difference 
> is probably more than the difference between the upper half of the 
> performance reports. 
> 
> The base code has the C-ism "... + (a false/true. That's a nice thing and I understood it in 1975 at BTL in 
> Guilford Center NC. But it is not in Go or whatever else. So these port 
> versions all have a function that takes a boolean, has an if statement, and 
> returns 1 or 0 as appropriate. That works. But not anywhere fast enough. I 
> reorganized the code to get rid of that. That made it much faster than any 
> compiler mode or great language design special benefit. A core computational 
> element of Rabbit is its custom highly non-linear G-operator. Every version I 
> can find online does this with four 32-bit multiplies, 2 shifts, 2 adds, and 
> an XOR.I did it with one 64-bit squaring, a shift, and an XOR. It is 20% 
> faster on laptop/desktop/server CPUs. It is no big deal but the reason for 
> the universal implementation was lost to all who ported without understanding 
> and revisiting the needs of the algorithm. This is a common outcome. It makes 
> me look at all such benchmarks and imagine such huge error bars of 
> performance/memory/size/... that direct comparisons of those is not too 
> useful.
> 
> On the other hand, I absolutely love such suites because they are probably 
> the best teacher of how programming language concepts work, help/hurt, and 
> are expressed across languages. I've a long history in programming (Fortran 
> II on PDP-8i, BTL CARDIAC, DG Nova, CDC 3300, ...) but learn new and helpful 
> things whenever I see how people answer Project Euler questions or port 
> benchmarks in languages I don't really know well.
> 
> My advice is to imagine there is always somebody who could use any 
> computer/language and double whatever you can do. Figure that in when you see 
> that some code is 4% faster in some 20-option GCC compile mode (no frame 
> pointer, no ...) OTOH, if Go is 20x slower than language X that is like 
> waving a red flag at a bull; I must figure out why. So far, there has never 
> been a meaningful reason. It is most often problem redefinition, softness in 
> specification, and sometimes, a very clever data structure or coding 
> technique. Rarely is it that X is just-unfixably-slow and Y is inherently 
> fast. (There are some of those, but few, and even if, it is usually slightly 
> unhelpful like places where Python is really fast and it's because the whole 
> benchmark is a giant matrix SVD done by one Python call to 
> BLAS/LINPACK/LAPACK under the hood. That is a valid result, but does not say 
> much about Python vs anything else for actual code-in-Python.)
> 
> A hidden surprise of these kinds of suites (and my coding of Rabbit) is that 
> I'm looking at the generated code and asking, "how can this be really 
> faster?" That made me realize a way to make Go code really faster for 32-bit 
> integer intensive code like this, which is to make the compiler's rewrite 
> rules and register assignments be able to express that two 32-bit variables 
> are "roommates" in a 64-bit register. That's what I'll need to do by hand to 
> get this over 1 GB/s and it would not be difficult. But if the compiler could 
> do it that would make everything in this realm faster. Now I don't understand 
> how to do that, I may not myself be able to do it, but a day of porting and 
> benchmarking taught me what the next barrier to generated code performance 
> may be. That was a surprise and generally a valuable one. If 100 people here 
> have varied realizations like this, we'd really be able to make things fly.
> 
> So nothing against small benchmarks and puzzle suites...just don't take the 
> numbers to literally.
> 
> Michael
> 
>> On Wed, Mar 6, 2019 at 4:32 PM 'Isaac Gouy' via golang-nuts 
>>  wrote:
>>> On Wednesday, March 6, 2019 at 4:03:52 PM UTC-8, Bakul Shah wrote:
>>> Thanks for an interesting read!
>>> 
>>> 

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

[Michael Jones]

There is another problem about these microbenchmarks as well--they often
are ports of an originating C-version.

I just implemented the Rabbit stream cipher and after reading the papers I
reviewed several versions online in Java and two in Go, as well as the now
open-source C version. It seems that they all started with the C version
and did a "c-to-Java" or "c-to-go-ization" of it. The results are not
strong for the language. My code took me a day and runs at 830 MB/s vs 620
MB/s peak for any of these. Now this is not a microbenchmark exactly and I
may have more experience than some, but it still shows that language-X
would seem to do a job at 620 that it could do at 830 with cleaner code.
That kind of difference is probably more than the difference between the
upper half of the performance reports.

The base code has the C-ism "... + (a wrote:

> On Wednesday, March 6, 2019 at 4:03:52 PM UTC-8, Bakul Shah wrote:
>>
>> Thanks for an interesting read!
>>
>> Curious to know if you guys have any estimates on the number of lines,
>> development time and number of bugs for each language implementation? I
>> realize this is subjective but this comparison may be quite meaningful
>> given that the authors had an existing reference implementation of a sort
>> done in CL. It is not often one sees real world examples of multiple
>> implementations done by a small team with the same goals.
>>
>
>
> One of the authors Pascal Costanza showed-up on programming reddit to
> remedy some misconceptions.
>
> You could ask him in-that-discussion
> 
> or email him directly.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

['Isaac Gouy' via golang-nuts]

On Wednesday, March 6, 2019 at 4:03:52 PM UTC-8, Bakul Shah wrote:
>
> Thanks for an interesting read!
>
> Curious to know if you guys have any estimates on the number of lines, 
> development time and number of bugs for each language implementation? I 
> realize this is subjective but this comparison may be quite meaningful 
> given that the authors had an existing reference implementation of a sort 
> done in CL. It is not often one sees real world examples of multiple 
> implementations done by a small team with the same goals.
>


One of the authors Pascal Costanza showed-up on programming reddit to 
remedy some misconceptions.

You could ask him in-that-discussion 

 
or email him directly.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

[Bakul Shah]

Thanks for an interesting read!

Curious to know if you guys have any estimates on the number of lines, 
development time and number of bugs for each language implementation? I realize 
this is subjective but this comparison may be quite meaningful given that the 
authors had an existing reference implementation of a sort done in CL. It is 
not often one sees real world examples of multiple implementations done by a 
small team with the same goals.

Thanks!

> On Feb 28, 2019, at 9:05 AM, 'Isaac Gouy' via golang-nuts 
>  wrote:
> 
> "We reimplemented elPrep in all three languages and benchmarked their runtime 
> performance and memory use. Results: The Go implementation performs best, 
> yielding the best balance between runtime performance and memory use. While 
> the Java benchmarks report a somewhat faster runtime than the Go benchmarks, 
> the memory use of the Java runs is significantly higher."
> 
> proggit discussion
> 
> article
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Performance comparison of Go, C++, and Java for biological sequencing tool

[Dan Kortschak]

It should be pointed out that these three implementations have close to
zero testing. In the absence of that, there is little that should be
drawn from the integration benchmarks that this suggests.

If we relax correct correctness requirements we can get answers in O(1)
with small constants.

On Thu, 2019-02-28 at 09:05 -0800, 'Isaac Gouy' via golang-nuts wrote:
> "We reimplemented elPrep in all three languages and benchmarked
> their 
> runtime performance and memory use. Results: *The Go implementation 
> performs best*, yielding the best balance between runtime performance
> and 
> memory use. While the Java benchmarks report a somewhat faster
> runtime than 
> the Go benchmarks, the memory use of the Java runs is significantly
> higher."
> 
> proggit discussion 
>  parison_of_go_c_and_java_for/>
> 
> article 
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.