[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-08-04 Thread Tamás Gulácsi
If md5 is enough at the end, use an io.T eeReader. If not, you need to buffer it, with bytes.Buffer. That can be reused with sync.Pool (don't forget the Reset). For mime, the first 1024 bytes is enough. Read that into a [1024]byte and create a Reader with io.MultiReader. -- You received this

[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-08-03 Thread Sri G
Doh. Thanks. I did the setup but didnt click "execute". Revisiting this because its now a bottleneck since it directly impact user experience (how long a request will take to process) and scalability (requests per second a single instance can handle). It wasn't pre-mature optimization, rather

[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-07-02 Thread Dave Cheney
The hash is always the same because you ask for the hash value before writing any data through it with io.Copy. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to

[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-07-02 Thread Sri G
Update: Adding file.Seek(0,0) does fix the issue in Version 2. The uploaded file is the correct size on disk with the correct md5. Without it, the uploaded file which is saved is missing the first 1024 bytes. This makes sense. There is something wrong with the way the md5 is calculated, it

[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-07-02 Thread Sri G
Thanks for the pointer. I also found this helpful Asynchronously Split an io.Reader in Go (golang) « Rodaine but I'm still missing something. Version 1: the uploaded file is 1024 bytes extra at the end (too big): mimebuf :=

[go-nuts] Re: files, readers, byte arrays (slices?), byte buffers and http.requests

2016-07-02 Thread Tamás Gulácsi
2016. július 2., szombat 8:15:19 UTC+2 időpontban Sri G a következőt írta: > > I'm working on receiving uploads through a form. > > The tricky part is validation. > > I attempt to read the first 1024 bytes to check the mime of the file and > then if valid read the rest and hash it and also save