Doh. Thanks. I did the setup but didnt click "execute". Revisiting this because its now a bottleneck since it directly impact user experience (how long a request will take to process) and scalability (requests per second a single instance can handle). It wasn't pre-mature optimization, rather proper architecture planning :)
In C, the request would come into a ring buffer of Struct of Arrays (read SoAs vs AoS on Intel x86) -> a pointer to the post data is kept. This is used to check the mime type as well as compute the md5. Then it passed to be written to disk before it is released. No copies are needed. How can I accomplish this in idiomatically Go? When I say idiomatic, I mean efficient in space, time and verbosity depending on the requirements and most importantly, not fighting the language. I'm having difficulty grokking whether a command copies data or uses a reference to the underlying buffer (pointer). Or does everything copy because data needs to be in each stack for each go routine? I've read the source code of io.copy, if there is a reader.ReadFrom or writer.WriteTo, the copy uses the existing buffer, avoiding allocation and a copy. However crypto/md5 does not have either of these, so its not possible to compute the md5 without copying data. Is this because the md5 library is written for streaming data vs static data? Is there a way to accomplish this? i.e. here's a buffer of data, compute the md5 on it. Re: the mimetype, I should be able to create a 1024 byte slice of the file and pass it to mimemagic. This should avoid the copy. On Saturday, July 2, 2016 at 9:27:15 PM UTC-4, Dave Cheney wrote: > > The hash is always the same because you ask for the hash value before > writing any data through it with io.Copy. On Saturday, July 2, 2016 at 9:27:15 PM UTC-4, Dave Cheney wrote: > > The hash is always the same because you ask for the hash value before > writing any data through it with io.Copy. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
