Re: [go-nuts] Re: Efficient to copy Hash?

[roger peppe]

On 22 November 2017 at 18:57, 'simon place' via golang-nuts
 wrote:
> thanks,
>
> this cloning is an optimisation, slowing it down a lot makes it pointless to
> begin with, but luckily really, the program produced has a built-in list of
> selectable, std.lib., hash routines to select from.
>
> i will add a warning for someone forking it about this.
>
>
> BTW your improved cloning reflection code proved much faster;

That's good to hear :)

>
> func cloneHash(h hash.Hash) hash.Hash{
> return clone(h).(hash.Hash)
> }
>
> func clone(i interface{}) interface{} {
> hv := reflect.ValueOf(i)
> hv1 := reflect.New(hv.Type().Elem())
> hv1.Elem().Set(hv.Elem())
> return hv1.Interface()
> }
>
>
>
> On Wednesday, 22 November 2017 13:16:01 UTC, rog wrote:
>>
>> On 21 November 2017 at 21:56, 'simon place' via golang-nuts
>>  wrote:
>> > i found this code, and use it on hash.Hash
>> >
>> > // copy an interface value using reflect (here for pointers to
>> > interfaces)
>> > func clone(i interface{}) interface{} {
>> > indirect := reflect.Indirect(reflect.ValueOf(i))
>> > newIndirect := reflect.New(indirect.Type())
>> > newIndirect.Elem().Set(reflect.ValueOf(indirect.Interface()))
>> > return newIndirect.Interface()
>> > }
>> >
>> > like this;
>> >
>> > branchHasher = clone(hasher).(hash.Hash)
>>
>> Although this works on most hash implementations in the standard library,
>> I'm not sure I'd recommend doing this, as it depends on the fact that
>> all the state in the hash implementations will copied with a shallow copy.
>>
>> Better (albeit less efficient) would be something like this, I think:
>>
>> func CloneHash(h hash.Hash) hash.Hash {
>> type codec interface {
>> hash.Hash
>> encoding.BinaryMarshaler
>> encoding.BinaryUnmarshaler
>> }
>> mh, ok := h.(codec)
>> if !ok {
>> panic(fmt.Errorf("hash %T cannot be cloned", h))
>> }
>> data, err := mh.MarshalBinary()
>> if err != nil {
>> panic(fmt.Errorf("hash %T marshal failed: %v", h, err))
>> }
>> t := reflect.TypeOf(h)
>> if t.Kind() != reflect.Ptr {
>> panic(fmt.Errorf("hash %T is not of pointer type", h))
>> }
>>
>> mh1 := reflect.New(t.Elem()).Interface().(codec)
>> if err := mh1.UnmarshalBinary(data); err != nil {
>> panic(fmt.Errorf("hash %T unmarshal failed: %v", mh1, err))
>> }
>> return mh1
>> }
>>
>> I could understand why you might use your original version for performance
>> reasons, though. It could be a little simpler, I think:
>>
>> // CloneHash clones the current state of the given
>> // hash. It works with most implementations in the standard
>> // library but is not guaranteed to work with all Hash
>> // implementations.
>> func CloneHash(h hash.Hash) hash.Hash {
>> hv := reflect.ValueOf(h)
>> hv1 := reflect.New(hv.Type().Elem())
>> hv1.Elem().Set(hv.Elem())
>> return hv1.Interface().(hash.Hash)
>> }
>>
>> If I was using it in production code, I'd probably be defensive and do
>> something
>> like this:
>>
>> https://play.golang.org/p/QDUlwuFAuv
>>
>> I think that even now that hashes that implement encoding.BinaryMarshaler
>> and
>> encoding.BinaryUnmarshaler, there's probably still room for a Clone method
>> (or perhaps a top level hash.Clone function) in the standard library to
>> avoid
>> the necessity for this kind of thing.
>>
>> >
>> >
>> > On Wednesday, 8 November 2017 12:54:18 UTC, Christian LeMoussel wrote:
>> >>
>> >> Hi,
>> >>
>> >> I want to calculate hash on 3 strings. First string is always the same,
>> >> the other may vary.
>> >> The first approach is to calculate the hash each time for the 3 strings
>> >> (BenchmarkHash)
>> >> Another approach would be to calculate once for the first string, and
>> >> then
>> >> reuse this hash to calculate the hash with the other 2
>> >> strings(BenchmarkCopyHash)
>> >> The difficulty is that sha256.New() returns a pointer, we have to copy
>> >> the
>> >> first hash. To do this, I created the function copyHash()
>> >> But the performances are not exceptional.
>> >>
>> >> Do you have another idea to do this in efficient way?
>> >>
>> >>
>> >> BenchmarkHash-8  100  1761 ns/op
>> >> 176 B/op  4 allocs/op
>> >> BenchmarkCopyHash-8  100  1519 ns/op
>> >> 240 B/op  4 allocs/op
>> >>
>> >>
>> >> var m1 = strings.Repeat("a", 64)
>> >> var m2 = strings.Repeat("b", 48)
>> >> var m3 = strings.Repeat("c", 32)
>> >>
>> >> func BenchmarkHash(b *testing.B) {
>> >> var (
>> >> d hash.Hash
>> >> )
>> >>
>> >> d = sha256.New()
>> >> for n := 0; n < b.N; n++ {
>> >> d.Reset()
>> >> d.Write([]byte(m1))
>> >> d.Write([]byte(m2))
>> >> 

Re: [go-nuts] Re: Efficient to copy Hash?

['simon place' via golang-nuts]

thanks, 

this cloning is an optimisation, slowing it down a lot makes it pointless 
to begin with, but luckily really, the program produced has a built-in list 
of selectable, std.lib., hash routines to select from.

i will add a warning for someone forking it about this. 


BTW your improved cloning reflection code proved much faster;

func cloneHash(h hash.Hash) hash.Hash{
return clone(h).(hash.Hash)
}

func clone(i interface{}) interface{} {
hv := reflect.ValueOf(i)
hv1 := reflect.New(hv.Type().Elem())
hv1.Elem().Set(hv.Elem())
return hv1.Interface()
}   



On Wednesday, 22 November 2017 13:16:01 UTC, rog wrote:
>
> On 21 November 2017 at 21:56, 'simon place' via golang-nuts 
>  wrote: 
> > i found this code, and use it on hash.Hash 
> > 
> > // copy an interface value using reflect (here for pointers to 
> interfaces) 
> > func clone(i interface{}) interface{} { 
> > indirect := reflect.Indirect(reflect.ValueOf(i)) 
> > newIndirect := reflect.New(indirect.Type()) 
> > newIndirect.Elem().Set(reflect.ValueOf(indirect.Interface())) 
> > return newIndirect.Interface() 
> > } 
> > 
> > like this; 
> > 
> > branchHasher = clone(hasher).(hash.Hash) 
>
> Although this works on most hash implementations in the standard library, 
> I'm not sure I'd recommend doing this, as it depends on the fact that 
> all the state in the hash implementations will copied with a shallow copy. 
>
> Better (albeit less efficient) would be something like this, I think: 
>
> func CloneHash(h hash.Hash) hash.Hash { 
> type codec interface { 
> hash.Hash 
> encoding.BinaryMarshaler 
> encoding.BinaryUnmarshaler 
> } 
> mh, ok := h.(codec) 
> if !ok { 
> panic(fmt.Errorf("hash %T cannot be cloned", h)) 
> } 
> data, err := mh.MarshalBinary() 
> if err != nil { 
> panic(fmt.Errorf("hash %T marshal failed: %v", h, err)) 
> } 
> t := reflect.TypeOf(h) 
> if t.Kind() != reflect.Ptr { 
> panic(fmt.Errorf("hash %T is not of pointer type", h)) 
> } 
>
> mh1 := reflect.New(t.Elem()).Interface().(codec) 
> if err := mh1.UnmarshalBinary(data); err != nil { 
> panic(fmt.Errorf("hash %T unmarshal failed: %v", mh1, err)) 
> } 
> return mh1 
> } 
>
> I could understand why you might use your original version for performance 
> reasons, though. It could be a little simpler, I think: 
>
> // CloneHash clones the current state of the given 
> // hash. It works with most implementations in the standard 
> // library but is not guaranteed to work with all Hash 
> // implementations. 
> func CloneHash(h hash.Hash) hash.Hash { 
> hv := reflect.ValueOf(h) 
> hv1 := reflect.New(hv.Type().Elem()) 
> hv1.Elem().Set(hv.Elem()) 
> return hv1.Interface().(hash.Hash) 
> } 
>
> If I was using it in production code, I'd probably be defensive and do 
> something 
> like this: 
>
> https://play.golang.org/p/QDUlwuFAuv 
>
> I think that even now that hashes that implement encoding.BinaryMarshaler 
> and 
> encoding.BinaryUnmarshaler, there's probably still room for a Clone method 
> (or perhaps a top level hash.Clone function) in the standard library to 
> avoid 
> the necessity for this kind of thing. 
>
> > 
> > 
> > On Wednesday, 8 November 2017 12:54:18 UTC, Christian LeMoussel wrote: 
> >> 
> >> Hi, 
> >> 
> >> I want to calculate hash on 3 strings. First string is always the same, 
> >> the other may vary. 
> >> The first approach is to calculate the hash each time for the 3 strings 
> >> (BenchmarkHash) 
> >> Another approach would be to calculate once for the first string, and 
> then 
> >> reuse this hash to calculate the hash with the other 2 
> >> strings(BenchmarkCopyHash) 
> >> The difficulty is that sha256.New() returns a pointer, we have to copy 
> the 
> >> first hash. To do this, I created the function copyHash() 
> >> But the performances are not exceptional. 
> >> 
> >> Do you have another idea to do this in efficient way? 
> >> 
> >> 
> >> BenchmarkHash-8  100  1761 ns/op 
> >> 176 B/op  4 allocs/op 
> >> BenchmarkCopyHash-8  100  1519 ns/op 
> >> 240 B/op  4 allocs/op 
> >> 
> >> 
> >> var m1 = strings.Repeat("a", 64) 
> >> var m2 = strings.Repeat("b", 48) 
> >> var m3 = strings.Repeat("c", 32) 
> >> 
> >> func BenchmarkHash(b *testing.B) { 
> >> var ( 
> >> d hash.Hash 
> >> ) 
> >> 
> >> d = sha256.New() 
> >> for n := 0; n < b.N; n++ { 
> >> d.Reset() 
> >> d.Write([]byte(m1)) 
> >> d.Write([]byte(m2)) 
> >> d.Write([]byte(m3)) 
> >> d.Sum(nil) 
> >> } 
> >> } 
> >> func BenchmarkCopyHash(b *testing.B) { 
> >> var ( 
> >> d1 hash.Hash 
> >> d2 

Re: [go-nuts] Re: Efficient to copy Hash?

[roger peppe]

On 21 November 2017 at 21:56, 'simon place' via golang-nuts
 wrote:
> i found this code, and use it on hash.Hash
>
> // copy an interface value using reflect (here for pointers to interfaces)
> func clone(i interface{}) interface{} {
> indirect := reflect.Indirect(reflect.ValueOf(i))
> newIndirect := reflect.New(indirect.Type())
> newIndirect.Elem().Set(reflect.ValueOf(indirect.Interface()))
> return newIndirect.Interface()
> }
>
> like this;
>
> branchHasher = clone(hasher).(hash.Hash)

Although this works on most hash implementations in the standard library,
I'm not sure I'd recommend doing this, as it depends on the fact that
all the state in the hash implementations will copied with a shallow copy.

Better (albeit less efficient) would be something like this, I think:

func CloneHash(h hash.Hash) hash.Hash {
type codec interface {
hash.Hash
encoding.BinaryMarshaler
encoding.BinaryUnmarshaler
}
mh, ok := h.(codec)
if !ok {
panic(fmt.Errorf("hash %T cannot be cloned", h))
}
data, err := mh.MarshalBinary()
if err != nil {
panic(fmt.Errorf("hash %T marshal failed: %v", h, err))
}
t := reflect.TypeOf(h)
if t.Kind() != reflect.Ptr {
panic(fmt.Errorf("hash %T is not of pointer type", h))
}

mh1 := reflect.New(t.Elem()).Interface().(codec)
if err := mh1.UnmarshalBinary(data); err != nil {
panic(fmt.Errorf("hash %T unmarshal failed: %v", mh1, err))
}
return mh1
}

I could understand why you might use your original version for performance
reasons, though. It could be a little simpler, I think:

// CloneHash clones the current state of the given
// hash. It works with most implementations in the standard
// library but is not guaranteed to work with all Hash
// implementations.
func CloneHash(h hash.Hash) hash.Hash {
hv := reflect.ValueOf(h)
hv1 := reflect.New(hv.Type().Elem())
hv1.Elem().Set(hv.Elem())
return hv1.Interface().(hash.Hash)
}

If I was using it in production code, I'd probably be defensive and do something
like this:

https://play.golang.org/p/QDUlwuFAuv

I think that even now that hashes that implement encoding.BinaryMarshaler and
encoding.BinaryUnmarshaler, there's probably still room for a Clone method
(or perhaps a top level hash.Clone function) in the standard library to avoid
the necessity for this kind of thing.

>
>
> On Wednesday, 8 November 2017 12:54:18 UTC, Christian LeMoussel wrote:
>>
>> Hi,
>>
>> I want to calculate hash on 3 strings. First string is always the same,
>> the other may vary.
>> The first approach is to calculate the hash each time for the 3 strings
>> (BenchmarkHash)
>> Another approach would be to calculate once for the first string, and then
>> reuse this hash to calculate the hash with the other 2
>> strings(BenchmarkCopyHash)
>> The difficulty is that sha256.New() returns a pointer, we have to copy the
>> first hash. To do this, I created the function copyHash()
>> But the performances are not exceptional.
>>
>> Do you have another idea to do this in efficient way?
>>
>>
>> BenchmarkHash-8  100  1761 ns/op
>> 176 B/op  4 allocs/op
>> BenchmarkCopyHash-8  100  1519 ns/op
>> 240 B/op  4 allocs/op
>>
>>
>> var m1 = strings.Repeat("a", 64)
>> var m2 = strings.Repeat("b", 48)
>> var m3 = strings.Repeat("c", 32)
>>
>> func BenchmarkHash(b *testing.B) {
>> var (
>> d hash.Hash
>> )
>>
>> d = sha256.New()
>> for n := 0; n < b.N; n++ {
>> d.Reset()
>> d.Write([]byte(m1))
>> d.Write([]byte(m2))
>> d.Write([]byte(m3))
>> d.Sum(nil)
>> }
>> }
>> func BenchmarkCopyHash(b *testing.B) {
>> var (
>> d1 hash.Hash
>> d2 hash.Hash
>> )
>>
>> d1 = sha256.New()
>> d1.Write([]byte(m1))
>>
>> for n := 0; n < b.N; n++ {
>> d2 = copyHash(d1)
>> d2.Write([]byte(m2))
>> d2.Write([]byte(m3))
>> d2.Sum(nil)
>> }
>> }
>>
>> func copyHash(src hash.Hash) hash.Hash {
>> typ := reflect.TypeOf(src).Elem()
>> val := reflect.ValueOf(src).Elem()
>> elem := reflect.New(typ).Elem()
>> elem.Set(val)
>> return elem.Addr().Interface().(hash.Hash)
>> }
>>
>>
>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

[go-nuts] Re: Efficient to copy Hash?

['simon place' via golang-nuts]

i found this code, and use it on hash.Hash

// copy an interface value using reflect (here for pointers to interfaces)
func clone(i interface{}) interface{} {
indirect := reflect.Indirect(reflect.ValueOf(i))
newIndirect := reflect.New(indirect.Type())
newIndirect.Elem().Set(reflect.ValueOf(indirect.Interface()))
return newIndirect.Interface()
}

like this;

branchHasher = clone(hasher).(hash.Hash)


On Wednesday, 8 November 2017 12:54:18 UTC, Christian LeMoussel wrote:
>
> Hi,
>
> I want to calculate hash on 3 strings. First string is always the same, 
> the other may vary. 
> The first approach is to calculate the hash each time for the 3 strings (
> BenchmarkHash)
> Another approach would be to calculate once for the first string, and then 
> reuse this hash to calculate the hash with the other 2 strings
> (BenchmarkCopyHash)
> The difficulty is that sha256.New() returns a pointer, we have to copy 
> the first hash. To do this, I created the function copyHash()
> But the performances are not exceptional.
>
> Do you have another idea to do this in efficient way?
>
>
> BenchmarkHash-8  100  1761 
> ns/op 176 B/op  4 allocs/op
> BenchmarkCopyHash-8  100  1519 
> ns/op 240 B/op  4 allocs/op
>
>
> var m1 = strings.Repeat("a", 64)
> var m2 = strings.Repeat("b", 48)
> var m3 = strings.Repeat("c", 32)
>
> func BenchmarkHash(b *testing.B) {
> var (
> d hash.Hash
> )
>
> d = sha256.New()
> for n := 0; n < b.N; n++ {
> d.Reset()
> d.Write([]byte(m1))
> d.Write([]byte(m2))
> d.Write([]byte(m3))
> d.Sum(nil)
> }
> }
> func BenchmarkCopyHash(b *testing.B) {
> var (
> d1 hash.Hash
> d2 hash.Hash
> )
>
> d1 = sha256.New()
> d1.Write([]byte(m1))
>
> for n := 0; n < b.N; n++ {
> d2 = copyHash(d1)
> d2.Write([]byte(m2))
> d2.Write([]byte(m3))
> d2.Sum(nil)
> }
> }
>
> func copyHash(src hash.Hash) hash.Hash {
> typ := reflect.TypeOf(src).Elem()
> val := reflect.ValueOf(src).Elem()
> elem := reflect.New(typ).Elem()
> elem.Set(val)
> return elem.Addr().Interface().(hash.Hash)
> }
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.