Thanks Rob, Yes you are right these images are small. <100kb each with an example below. And as I mentioned I have millions of these.
I am able to max the CPUs out with a worker pool but that is when I noticed a lot of the work was actually decoding the PNG images which made we wonder if if was able to reuse decoders at all and even if that would help! On Sat, 27 Jun 2020 at 01:30, Rob Pike <r...@golang.org> wrote: > Without information about the particular images - the data set you're > working with - it's hard to provide any accurate advice. If the images are > small, maybe it's all overhead. If the images are of a particular form of > PNG, maybe a fast path is missing. And so on. > > To get meaningful help for problems like this, the more information you > can provide, the better. > > -rob > > > On Fri, Jun 26, 2020 at 10:59 PM Lee Armstrong <l...@pinkfroot.com> wrote: > >> Thanks, I am already maxing out some servers but wondered if it could be >> sped up. >> >> I will give the bimg package a go! >> >> On Friday, June 26, 2020 at 1:55:40 PM UTC+1 ren...@ix.netcom.com wrote: >> >>> Just parallelize in the cloud. At a minimum parallelize locally with >>> multiple Go routines. >>> >>> On Jun 26, 2020, at 7:51 AM, howar...@gmail.com wrote: >>> >>> >>> >>> I don't know if the CGo transitions for 16 million images will >>> completely swamp the speed gains, or how much post-processing you need to >>> do to these images, but you might try https://github.com/h2non/bimg and >>> see if it gives you any wins. It claims 'typically 4x faster' then the Go >>> Image package. It is a CGO interface to libvips, which is an experimental >>> branch of libspng, which is a performance focused alternative to libpng. >>> >>> Howard >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com >>> <https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. > > >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com >> <https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAC5LoL_yWxgfDUCs5UwStep-dOMoWovpF_RK0K_MuPNzsG6Unw%40mail.gmail.com.
