Indeed, I am not a fan of overengineering either. But the rules of data structure largely expounded by Knuth and others a long time ago were in the context of an entirely different hardware architecture to what we have today. CPUs didn't even have caches until late in the 16 bit era. The caches in my i5 are bigger than the memory was in computers main memory only 15 years ago. They have a specific geometry and block size and they are the fastest type of memory that exists currently.
Exploiting their geometries to gain at least 3-4x speedups in algorithms is definitely a thing as Varnish proves. A much more simple example and that led to a change in the way that disk partitioners work has to do with pages in flash memory storage devices. If you create a partition not on those boundaries you literally halve the performance of the drive. I did a quick and dirty set of tests to compare how fast a read/write cycle works with repeatedly writing to one variable, an operation that would entirely take place in cache, and necessarily also meant a read cycle, I got 14Gb/s write speed. The same thing done by writing (linear) to a very large preallocated cache that would live in main memory only got 4Gb/s. Algorithms that make better use of cache locality will cause a fairly big jump in the performance of software, on that rough benchmark probably 3.5x speedup. Almost everything depends on sorting algorithms, from DOM trees to databases to GUIs in general, and possibly even low level in kernels on systems with very large memory maps and cache management would be a boon to the industry. The current generation of file systems only minimally has been engineered to work better on SSD at this point, likely there is room for improvement there as well by alignment of access and sequential access queuing. On Saturday, 5 May 2018 17:12:34 UTC+3, matthe...@gmail.com wrote: > > I sorta only vaguely understand what you mean about being 'too big'. > > > I’m being vague and I haven’t read this thread in detail besides seeing > type B in the first email. I plan to read it in detail and send another > response. I’m speaking from the idea behind a version of this story: > http://cs.txstate.edu/~br02/cs1428/ShortStoryForEngineers.htm > > Matt > > On Friday, May 4, 2018 at 5:28:30 PM UTC-5, Louki Sumirniy wrote: >> >> I sorta only vaguely understand what you mean about being 'too big'. I >> only have 4 comparators, 5 walk operators and a small collection maybe >> around 8 helper functions. But now that I'm working with inter-convertible >> slice types, I don't have to do any interfaces in fact. I only have 5 types >> I actually need to use now, byte, uint8, 16, 32 and 64. I can use switch >> blocks to implement each one depending on the data being worked on. There >> won't be any package scoping issues either nor any closure scoping issues >> as I won't need to do this, the configurations in the main struct will be >> used as conditions in the switches. >> >> Scoping and side effects from it were causing me huge headaches up to >> now. Data was not being read or written to the right locations and I >> couldn't figure out why, something to do with interfaces and package >> scoping I think. None of this will be a problem as I kinda don't even need >> to use type interfaces at all, I will try to avoid this. It's a very low >> level code I will be writing, something that will evoke halcyon days of >> assembly programming, but with GC, type safety and neat infix operators. >> >> On Saturday, 5 May 2018 00:26:11 UTC+3, matthe...@gmail.com wrote: >>> >>> With byte slices you’ll need to pay attention to reallocation of the >>> backing array as a possible source of performance problems ( >>> https://blog.golang.org/go-slices-usage-and-internals). >>> >>> I can see clearly how freakin complex code gets when you try to do >>>> generics and polymorphism. >>> >>> >>> I think it’s only complex because you’re not using the best possible >>> idioms. For example I mentioned in the other thread that I think your >>> interface is too big. Have you thought of other ways to do the typing? >>> >>> Perhaps try another language like Rust? >>> >>> >>> I think Go could work. >>> >>> Matt >>> >>> On Friday, May 4, 2018 at 1:45:52 PM UTC-5, atomly wrote: >>>> >>>> Perhaps try another language like Rust? >>>> >>>> atomly >>>> >>>> On Fri, May 4, 2018 at 10:44 AM, Louki Sumirniy < >>>> louki.sumir...@gmail.com> wrote: >>>> >>>>> Well after bashing my head against a brick wall I finally understand >>>>> what a mess it is for my specific purpose to work with Go. Nobody really >>>>> specifically said it in any discussions up to now but what my design >>>>> needs >>>>> is generics. If code generation could be done more transparently I could >>>>> use that but it really kicks interactivity and flow in the nuts. I'm >>>>> going >>>>> to probably abandon Golang on this basis. I am inclined to go with C but >>>>> I >>>>> am nervous about issues relating to pointers and runtime breaks but oh >>>>> well. </discussion for me>. >>>>> >>>>> If there was a way to transparently add code generation that would be >>>>> sufficient and remain type safe. *sigh* >>>>> >>>>> The biggest headache I am running up against is that between packages. >>>>> I think that before I throw in the towel I am going to study exactly how >>>>> the sort package is implemented, since in essence my library does the >>>>> same >>>>> thing, so maybe I just don't understand well enough how to use >>>>> interfaces, >>>>> embedding and composition. >>>>> >>>>> >>>>> On Friday, 4 May 2018 13:13:56 UTC+3, Louki Sumirniy wrote: >>>>>> >>>>>> Yep, you are correct, more or less. My code is using interfaces and >>>>>> panicking when it's fed the wrong thing. >>>>>> >>>>>> For example, in my test I had a literal number going into a write >>>>>> function to put a value into a node in the tree. The insert >>>>>> implementation >>>>>> panicked saying it couldn't put an int into a uint32 slot. Eventually I >>>>>> worked out it was the assumed int typing of the literal causing the >>>>>> problem >>>>>> (happens all the time, I should be used to it by now). >>>>>> >>>>>> The thing is, it is intended that the user of the type specific >>>>>> library that imports the tree library will not make a type assumption >>>>>> like >>>>>> the compiler did about a literal. The type specific part has to be >>>>>> written >>>>>> for any given type. There is a little verbosity in the part that wraps >>>>>> the >>>>>> injected functions with syntactic sugar so the base library has the >>>>>> functions to work on the concrete data agnostic of its content and only >>>>>> the >>>>>> return values provided by the functions it gets to do various >>>>>> comparisons >>>>>> and whatnot. It's not excessively wordy for my tastes and I know it's >>>>>> doing >>>>>> it the best way possible. >>>>>> >>>>>> It's just learning how to play with interfaces that I was struggling >>>>>> with and I seem to have pretty much got it all figured out now. I'm >>>>>> happy >>>>>> with how the code is developing and pretty much joyous that I can do >>>>>> this >>>>>> inside a type safe language. Type safety is a big part of how Go is such >>>>>> a >>>>>> fast compiling language. Implicit casts and inheritance have a massive >>>>>> cost >>>>>> in complexity not just a naive translation to binary code, but also in >>>>>> optimising it. When can it cut out the dereferencing to methods and when >>>>>> not? Since one could be using a method to do a relatively simple thing, >>>>>> in >>>>>> very long, tight loops, if the OOP compiler guesses wrong and puts >>>>>> dereferences in where they are not needed, it's a big cost over 1 >>>>>> million >>>>>> iterations compared to being aware of it, like in my code, and avoiding >>>>>> the >>>>>> need for dereferencing. >>>>>> >>>>>> I firmly believe that Go could replace C as a language to write >>>>>> systems kernels in. I saw an amusing post recently relating to Dan >>>>>> Larimer's work on EOS mentioning an automated code correctness checking >>>>>> application for C++... Having intimately dealt with Dan's C++ source >>>>>> code, >>>>>> I can see why he might be waking up to this, especially with the ether >>>>>> leaks going on, that EOS has inherited (literally, I presume). >>>>>> >>>>>> This would not happen if it were written in Go. Note also that Geth >>>>>> lacks this problem and there was almost going to be a split over code >>>>>> changes from the Go side. >>>>>> >>>>>> Linus Torvalds famously said he wrote the Git CVS in C specifically >>>>>> to exclude C++ programmers. But as any avid gopher would know, C has all >>>>>> kinds of gotchas to do with busting out of stacks, heaps, allocated >>>>>> memory, >>>>>> and forgetting to free memory. This does not happen in Go. >>>>>> >>>>>> So to cut a long story short, this is why I choose to program in Go. >>>>>> I have a background in low level programming, I used to do a bit of M68k >>>>>> assembler on my amiga when I was a teenager. I don't mind verbosity that >>>>>> is >>>>>> necessary due to hardware architectural constraints. And as to the >>>>>> potential users of my code, I am not responsible for them having a lack >>>>>> of >>>>>> awareness of the exposed inner workings and them scratching their head >>>>>> at >>>>>> panics when they try to push the wrong type of value into one of the >>>>>> functions I wrote. They probably need to do some more study. >>>>>> >>>>>> I think of programming a bit like carpentry. Sure, you can just whack >>>>>> together an Ikea table in half an hour, but if you want to make a nice >>>>>> one >>>>>> that will last hundreds of years, you probably will need to at least >>>>>> spend >>>>>> a week or two practicing before you even have something simply crude and >>>>>> functional. C++ and other OOP and interpreted language users are like >>>>>> Ikea >>>>>> customers. Gophers are like people who go to the sawmill and the >>>>>> specialist >>>>>> tool store. Disregarding the zero cost of reproduction of course (that's >>>>>> what Ikea does). >>>>>> >>>>>> On Friday, 4 May 2018 11:36:33 UTC+3, M P r a d e s wrote: >>>>>>> >>>>>>> > Eliminating code duplication shrinks the base of code that must be >>>>>>> changed >>>>>>> >>>>>>> Except go was designed a certain way that will make whatever you are >>>>>>> attempting to do here burdensome. We've all been there at first trying >>>>>>> to >>>>>>> fit a square into a round hole, it doesn't work. Unless you start >>>>>>> ignoring >>>>>>> Go type system with interface {} everywhere, you're not going to go far >>>>>>> trying to emulate method overloading in Go. The price of using Go is >>>>>>> absolutely code duplication and a certain amount of verbosity in order >>>>>>> to >>>>>>> remain type safe. The only real way to achieve method polymorphism in >>>>>>> go is >>>>>>> to use interfaces, since Go supports neither inheritance nor generic >>>>>>> programming. Go does not allow developers to write "clever" code >>>>>>> without >>>>>>> sacrificing readability or compile time type safety. >>>>>>> >>>>>>> Le vendredi 4 mai 2018 06:26:59 UTC+2, Louki Sumirniy a écrit : >>>>>>>> >>>>>>>> The specifics of how it is done is not as important as why to do it: >>>>>>>> >>>>>>>> I am writing this for a later project. It will need at least 5 >>>>>>>> different, sometimes nearly not very different, short data types. They >>>>>>>> all >>>>>>>> have the property of being comparable. The code that keeps them stored >>>>>>>> and >>>>>>>> sorted does not need to know anything other than this comparability, >>>>>>>> and to >>>>>>>> be able to pass the data back to the caller. >>>>>>>> >>>>>>>> I am not making claims about something, trying to sell some snake >>>>>>>> oil, I am attempting to solve a problem and make use of low level >>>>>>>> architecture of the hardware to improve performance. It worked for >>>>>>>> B-heaps >>>>>>>> so maybe MAYBE it will also benefit binary trees. The way some people >>>>>>>> respond you'd think I was running an ICO! >>>>>>>> >>>>>>>> Programming is usually 5% writing and 95% debugging. Eliminating >>>>>>>> code duplication shrinks the base of code that must be changed, >>>>>>>> otherwise I >>>>>>>> have to delve into code generators or macros. If I have an intended >>>>>>>> use for >>>>>>>> a library I am writing, that would require 5 different source files >>>>>>>> for >>>>>>>> each data type, yet 90% of the code was the same, what happens when I >>>>>>>> realise I made a mistake in this common code? >>>>>>>> >>>>>>>> I pretty much concluded that it is better to just pass an 8 element >>>>>>>> function pointer struct as value anyway. Whether the compiler does it >>>>>>>> differently doesn't really matter since I can't control that. What >>>>>>>> matters >>>>>>>> to me is that it's readable so I don't have to spend more time reading >>>>>>>> than >>>>>>>> understanding. >>>>>>>> >>>>>>>> On Thursday, 3 May 2018 23:40:07 UTC+3, Jan Mercl wrote: >>>>>>>>> >>>>>>>>> On Thu, May 3, 2018 at 10:14 PM Louki Sumirniy < >>>>>>>>> louki.sumir...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> > As some here may know, I am implementing a novel binary tree >>>>>>>>> based on complete trees and using ideas from B-heaps in order to >>>>>>>>> achieve >>>>>>>>> maximum data cache locality in a search and sort algorithm. >>>>>>>>> >>>>>>>>> ... >>>>>>>>> >>>>>>>>> > Note that this is not really at all an OOP paradigm that I am >>>>>>>>> trying to ape with this. >>>>>>>>> >>>>>>>>> ... >>>>>>>>> >>>>>>>>> > I have the base type that concerns itself with the walk, search, >>>>>>>>> insert and delete functions in an interface. >>>>>>>>> >>>>>>>>> .... >>>>>>>>> >>>>>>>>> > The b.Overloads - I am not sure, whether it should be pass by >>>>>>>>> value or pass by reference. I think pass by reference adds an extra >>>>>>>>> resolution step, but it is probably more expensive as every call of >>>>>>>>> the >>>>>>>>> functions will have to perform this dereference. >>>>>>>>> >>>>>>>>> ... >>>>>>>>> >>>>>>>>> > Thanks in advance for any feedback on this. >>>>>>>>> >>>>>>>>> - Novelty is only valid when established by peer review. >>>>>>>>> Self-claimed novelty often means just lack of research. Even if >>>>>>>>> eventually >>>>>>>>> correct, it's a warning sign for me meanwhile. >>>>>>>>> >>>>>>>>> - Attempts to make non-interface types in Go behave in the >>>>>>>>> OOP-style inheritance is usually a telling sign of using the wrong >>>>>>>>> language >>>>>>>>> - or abusing one - to solve the problem. >>>>>>>>> >>>>>>>>> - Pass by reference has not even a well defined meaning in Go. >>>>>>>>> >>>>>>>>> No insult intended, you've explicitly asked for _any_ feedback. My >>>>>>>>> apologies if you feel offended anyway, >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> -j >>>>>>>>> >>>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
