Re: [go-nuts] Re: try to avoid additional memory allocations
You could avoid allocations altogether by reusing the same byte array and
passing it to the function.
To retain readability, use a buffer and fmt.Fprintf
On Saturday, 11 November 2017 04:00:48 UTC+13, Vasiliy Tolstov wrote:
>
> Thanks, i know that optimization can lead obscurity, but for critical
> paths i think that it useful =)
>
> 2017-11-10 8:41 GMT+03:00 peterGo >:
> > Vasiliy,
> >
> > For portability, when you initialize cfg.WorkDir, clean it: cfg.WorkDir
> =
> > filepath.Clean(cfg.WorkDir)
> >
> > You can reduce the allocations from 5 to 1. However, optimization can
> lead
> > to obscurity. For example,
> >
> > $ go test oid_test.go -bench=.
> > BenchmarkVasiliy-4 200700 ns/op88 B/op5 allocs/op
> > BenchmarkKrolaw-4 300550 ns/op80 B/op4 allocs/op
> > BenchmarkPeter-4 1000201 ns/op48 B/op1 allocs/op
> >
> > Playground: https://play.golang.org/p/DnGoM8PYd6
> >
> > Peter
> >
> >
> > On Thursday, November 9, 2017 at 3:58:03 PM UTC-5, Vasiliy Tolstov
> wrote:
> >>
> >> Hi. I have server that read/write data to many files (each is about
> >> 4-32Mb). To determine on which file i need to read/write i'm use this
> >> function to os.OpenFile
> >>
> >> func oid2filepath(cfg *Config, oID uint64) string {
> >> file_path := filepath.Join(cfg.WorkDir, fmt.Sprintf("%x",
> >> oid2vid(oID)), fmt.Sprintf("%016x", oID))
> >> return file_path
> >> }
> >>
> >> oid2vid function get uint32 and return uint32 with shift some bits
> >> path looks like (for one of 4Mb chunk)
> store/obj/7c2b25/007c2b250180
> >>
> >> When i'm write 4Gb data i need to call this function 1024 times for
> >> single big chunk of data.
> >> Does it possible to write function that does not have additional
> >> allocations?
> >> And does it possible to write it in portable way (i mean path
> separator).
> >>
> >>
> >> --
> >> Vasiliy Tolstov,
> >> e-mail: v.to...@selfip.ru
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to golang-nuts...@googlegroups.com .
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Vasiliy Tolstov,
> e-mail: v.to...@selfip.ru
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: try to avoid additional memory allocations
Thanks, i know that optimization can lead obscurity, but for critical
paths i think that it useful =)
2017-11-10 8:41 GMT+03:00 peterGo :
> Vasiliy,
>
> For portability, when you initialize cfg.WorkDir, clean it: cfg.WorkDir =
> filepath.Clean(cfg.WorkDir)
>
> You can reduce the allocations from 5 to 1. However, optimization can lead
> to obscurity. For example,
>
> $ go test oid_test.go -bench=.
> BenchmarkVasiliy-4 200700 ns/op88 B/op5 allocs/op
> BenchmarkKrolaw-4 300550 ns/op80 B/op4 allocs/op
> BenchmarkPeter-4 1000201 ns/op48 B/op1 allocs/op
>
> Playground: https://play.golang.org/p/DnGoM8PYd6
>
> Peter
>
>
> On Thursday, November 9, 2017 at 3:58:03 PM UTC-5, Vasiliy Tolstov wrote:
>>
>> Hi. I have server that read/write data to many files (each is about
>> 4-32Mb). To determine on which file i need to read/write i'm use this
>> function to os.OpenFile
>>
>> func oid2filepath(cfg *Config, oID uint64) string {
>> file_path := filepath.Join(cfg.WorkDir, fmt.Sprintf("%x",
>> oid2vid(oID)), fmt.Sprintf("%016x", oID))
>> return file_path
>> }
>>
>> oid2vid function get uint32 and return uint32 with shift some bits
>> path looks like (for one of 4Mb chunk) store/obj/7c2b25/007c2b250180
>>
>> When i'm write 4Gb data i need to call this function 1024 times for
>> single big chunk of data.
>> Does it possible to write function that does not have additional
>> allocations?
>> And does it possible to write it in portable way (i mean path separator).
>>
>>
>> --
>> Vasiliy Tolstov,
>> e-mail: v.to...@selfip.ru
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
Vasiliy Tolstov,
e-mail: v.tols...@selfip.ru
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: try to avoid additional memory allocations
Thanks!
2017-11-10 3:52 GMT+03:00 krolaw :
> func oid2filepath(cfg *Config, oID uint64) string {
>return fmt.Sprintf("%s%c%x%c%016x", cfg.WorkDir, filepath.Seperator,
> oid2vid(oID), filepath.Seperator, oID)
> }
>
> On Friday, 10 November 2017 09:58:03 UTC+13, Vasiliy Tolstov wrote:
>>
>> Hi. I have server that read/write data to many files (each is about
>> 4-32Mb). To determine on which file i need to read/write i'm use this
>> function to os.OpenFile
>>
>> func oid2filepath(cfg *Config, oID uint64) string {
>> file_path := filepath.Join(cfg.WorkDir, fmt.Sprintf("%x",
>> oid2vid(oID)), fmt.Sprintf("%016x", oID))
>> return file_path
>> }
>>
>> oid2vid function get uint32 and return uint32 with shift some bits
>> path looks like (for one of 4Mb chunk) store/obj/7c2b25/007c2b250180
>>
>> When i'm write 4Gb data i need to call this function 1024 times for
>> single big chunk of data.
>> Does it possible to write function that does not have additional
>> allocations?
>> And does it possible to write it in portable way (i mean path separator).
>>
>>
>> --
>> Vasiliy Tolstov,
>> e-mail: v.to...@selfip.ru
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
Vasiliy Tolstov,
e-mail: v.tols...@selfip.ru
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
