[go-nuts] runtime.readmemstats -> buckhash_sys increase

['Kevin Malachowski' via golang-nuts]

Is your application leaking memory allocated in C? (Are you using cgo-related 
packages at all?)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] scanner.Scan is holding my goroutine ;(

['Kevin Malachowski' via golang-nuts]

Can you post a short, self-contained (compile-able and runnable) example which 
shows the problem? Preferably hosted on play.golang.org for easy sharing and 
editing.

If the scanner is reading from a reader which has been closed (returns io.EOF), 
the sc.Scan should return false (and the sc.Err would return nil). Depending on 
the surrounding code, that should mean that the for loop you presented should 
terminate. It's possible the code which contains your bug has not been shared, 
so posting a self-contained reproduction program would help move the 
conversation forward.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Best way to handle database transactions? (commit, or rollback on error)

['Kevin Malachowski' via golang-nuts]

The best way to access the return value of a function is to name the return 
variable:

func DoTwoThings(db *Database) (err error) {
tx, err := db.Begin()
if err != nil {
return err
}
defer tx.Close()

...
}

Then any accidental shadowing doesn't matter because the "defer" is 
referring to the actual return variable; it also means that you can support 
returning a new error directly (i.e. `return nil, fmt.Errorf(...)`) rather 
than needing to set the outer 'err' variable and then return it.

On Monday, December 3, 2018 at 3:55:03 PM UTC-8, Ben Hoyt wrote:
>
> Robert, here is code (actual working code this time) similar to what we 
> have: https://play.golang.org/p/jUPqgnk6Ttk
>
> Leonel, yep, I understand that, which is why we have this problem. The 
> thing is, this pattern makes it very easy to forget to always re-use the 
> same error variable, especially in code that's a bit more complex with a 
> bunch of nested ifs, etc. So we're trying to come up with a pattern that's 
> hard or impossible to misuse, instead of easy to misuse.
>
> -Ben
>
> On Mon, Dec 3, 2018 at 6:46 PM Leonel Quinteros  > wrote:
>
>> Hi Ben, 
>>
>> I'm pretty sure that the *err* variable is getting shadowed on your 
>> Update construct. 
>> Instead of doing: 
>>
>> if _, err := UpdateBar(tx); err != nil {
>> return err
>> }
>>
>> You should do something like: 
>>
>> _, err = UpdateBar(tx)
>> if err != nil {
>> return err
>> }
>>
>> Just like you do with the insert and avoiding the *:=* operand
>>
>> Let me know if that works. 
>>
>>
>> Best! 
>> Leonel
>>
>>
>> El lunes, 3 de diciembre de 2018, 18:53:30 (UTC-3), Ben Hoyt escribió:
>>>
>>> Hi folks,
>>>
>>> We found some subtle bugs in our db transaction code for handling 
>>> commits/rollbacks. Here's the pattern we were using (not real, but shows 
>>> the issue):
>>>
>>> func DoTwoThings() error {
>>> tx, err := db.Begin()
>>> if err != nil {
>>> return err
>>> }
>>> // commit or rollback the transaction before we return
>>> defer tx.Close()
>>>
>>> err := InsertFoo(tx)
>>> if err != nil {
>>> return err
>>> }
>>> if _, err := UpdateBar(tx); err != nil {
>>> return err
>>> }
>>> return nil
>>> }
>>>
>>> The problem is there's a subtle but potentially quite bad bug with this 
>>> usage pattern -- if the InsertFoo succeeds but UpdateBar fails, the first 
>>> "err" variable will be nil, so the deferred tx.Close() will COMMIT the 
>>> transaction rather than ROLLBACK, and the database will be in an 
>>> inconsistent state.
>>>
>>> The code above is a bit contrived, and you can easily fix it by moving 
>>> the "_, err := UpdateBar()" outside of the if so it's referring to the same 
>>> "err" variable, but it's very easy to miss and get it wrong. So we decided 
>>> it was a bad pattern and started thinking about the best way to fix.
>>>
>>> One idea is a RollbackUnlessCommitted() function which you can defer, 
>>> and then you call Commit() once manually (stolen from gocraft/dbr):
>>>
>>> func DoTwoThings() error {
>>> tx, err := db.Begin()
>>> if err != nil {
>>> return err
>>> }
>>> defer tx.RollbackUnlessCommitted()
>>>
>>> err := InsertFoo(tx)
>>> if err != nil {
>>> return err
>>> }
>>> if _, err := UpdateBar(tx); err != nil {
>>> return err
>>> }
>>> tx.Commit()
>>> return nil
>>> }
>>>
>>> Another idea is to create a "Transact" function which takes an anonymous 
>>> function and does all the transaction handling:
>>>
>>> func (db *DatabaseImpl) Transact(txFunc func() error) (err error) {
>>> tx, err := db.Begin()
>>> if err != nil {
>>> return
>>> }
>>> defer func() {
>>> if p := recover(); p != nil {
>>> tx.Rollback()
>>> panic(p) // re-throw panic after Rollback
>>> } else if err != nil {
>>> tx.Rollback() // err is non-nil; don't change it
>>> } else {
>>> err = tx.Commit() // err is nil; if Commit returns error 
>>> update err
>>> }
>>> }()
>>> err = txFunc(tx)
>>> return err
>>> }
>>>
>>> And then the DoTwoThings function becomes:
>>>
>>> func DoTwoThings() error {
>>> return db.Transact(func() error) {
>>> err := InsertFoo(tx)
>>> if err != nil {
>>> return err
>>> }
>>> if _, err := UpdateBar(tx); err != nil {
>>> return err
>>> }
>>> })
>>> }
>>>
>>> I think the second is probably safer and nicer, but it's slightly 
>>> awkward in that it requires an extra level of indentation. Still, awkward 
>>> is better than buggy.
>>>
>>> Does anyone else have a better pattern for this kind of thing, or 
>>> feedback on the above?
>>>
>>> -Ben
>>>
>>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> 

[go-nuts] Re: [RFC] Dynamically instrumentation Go binaries

['Kevin Malachowski' via golang-nuts]

If you can't recompile the binary, an option is to use a generic dynamic 
instrumentation platform like DynamoRIO 
 (which I think works for Go 
programs after a recent bugfix). It works on windows, mac, and linux, and 
has been used in the past for various security-based testing and 
verification. Plenty of papers available on the 
website: http://dynamorio.org/pubs.html



On Thursday, November 15, 2018 at 3:06:11 PM UTC-8, Damian Gryski wrote:
>
> One approach would be to augment the compiler to instrument the binary 
> with the techniques outlined in:
>
> XRay: A Function Call Tracing System
> https://ai.google/research/pubs/pub45287
>
> Damian
>
> On Thursday, November 15, 2018 at 9:09:19 AM UTC-8, ju...@sqreen.io wrote:
>>
>> Hi,
>>
>> I am working on dynamic instrumentation of Go programs at run time, 
>> possibly without static source-code instrumentation. As I would like a 
>> solution as close to Go and standard as possible, I was first thinking of 
>> using `go generate` to generate a file adding things `reflect` doesn't 
>> provide such as the list of packages, functions, global variables... That 
>> way, I should be able to use `reflect` to modify any dynamic calls by 
>> modifying the method tables of their underlying type representations.
>>
>> But regarding statically linked calls, the less intrusive technique I 
>> found are uprobes, which is linux-specific. And at the opposite, there are 
>> user-space binary code instrumentation libraries such as dyninst that 
>> modify the code at run time...
>>
>> So I am wondering if anyone here has any thoughts on this subject, that 
>> doesn't seem to be solved for Go programs.
>>
>> Thanks!
>>
>> Julio
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Garbage collector and Slices

['Kevin Malachowski' via golang-nuts]

That information is old now - Go has a precise GC now. There is some sort 
of metadata stored for each variable related to whether the type contains 
pointers (as Jan alluded to earlier in the thread).

On Monday, August 20, 2018 at 7:28:51 AM UTC-7, Florian Uekermann wrote:
>
> From the GC POV, the only interesting thing about an unsafe.Pointer is if 
>> its value falls anywhere into a GC managed memory block for liveness 
>> analysis, AFAIK.
>>
> And then it will check that memory block for stuff that looks like a 
> pointer (not caring what the actual type being pointed to is) afaik. That 
> is what I meant with conservative GC.
> I found a thread from  2013, where Ian says the same 
> https://groups.google.com/forum/#!topic/golang-nuts/yNis7bQG_rY.
> Maybe my information is outdated, I haven't followed the GC evolution in 
> recent years that closely.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] New identifiers and selectors in short variable declarations

['Kevin Malachowski' via golang-nuts]

It is not a new declaration, it is definitely an assignment. This can be 
determined because (in Go) a new declaration has effects when closing over 
variables: https://play.golang.org/p/a_IZdOWeqYf

(ignore the obvious race condition; it works the same but looks uglier with 
the requisite locks: https://play.golang.org/p/OXBK5XEg9Yf)

On Tuesday, January 9, 2018 at 2:20:06 PM UTC-8, Ayan George wrote:
>
>
>
> On 01/09/2018 04:45 PM, Ian Lance Taylor wrote: 
> > On Tue, Jan 9, 2018 at 1:02 PM, Jim Bishopp 
> >  wrote: 
> >> 
> >> Has there ever been a discussion about allowing new identifiers and 
> >> selectors in short variable declarations? 
> >> 
> >> var a struct { x int } b, a.x := 1, 2 
> >> 
> >> ERROR: expected identifier on left side of := 
> > 
> > That idea appears in https://golang.org/issue/377, along with many 
> > others. 
> > 
>
> I had the same question myself. I settled on the idea that that is a 
> short declaration -- not an assignment -- though it does re-declare 
> existing variables (of the same type!). That behaves like assignment 
> but I think it is still declaration. 
>
> But its behavior makes sense if you consider that you can't declare or 
> re-declare struct member. 
>
> -ayan 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] use of builtin print for debug

['Kevin Malachowski' via golang-nuts]

You can declare a func "print" in each package which just calls the real print 
func. When you want to turn off printing, you just have one line to comment out 
in that func to affect all callers.

I'm curious why fmt.Printf doesn't work for you, though. In that case you'd 
still probably want a way to turn off that specific output, but I'm still 
curious.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why were tabs chosen for indentation?

['Kevin Malachowski' via golang-nuts]

I love that Go uses tabs because I use 3 spaces for my tabstop, and very few 
people share that preference.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a way to check if all pending finalizers have been executed?

['Kevin Malachowski' via golang-nuts]

In that case it's generally "better" to have an explicit Close on your Go type 
which causes the explicit freeing of C memory. This can be tedious depending on 
your specific code, though.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Duplicate File Checker Performance

['Kevin Malachowski' via golang-nuts]

Sorry, I meant that calling Write on the hash type might be slower if it's 
called more often.

(I'm on mobile right now. When I get back to a keyboard I'll try to come up 
with an example)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Duplicate File Checker Performance

['Kevin Malachowski' via golang-nuts]

It also might be inefficient to call Sum multiple times with smaller slices 
compared to calling it once with a larger slice. Try using io.CopyBuffer and 
passing in larger buffer sizes to see if the CPU and memory usage start to 
converge.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are receivers from a channel fifo?

['Kevin Malachowski' via golang-nuts]

Try it out :) seems like a simple enough test program to write.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] defer at struct level

['Kevin Malachowski' via golang-nuts]

Do you have a code sample? I don't quite understand what you mean by something 
being "wrapped" in a struct. Will 'a' be a field in a struct? If so, you can 
just access it:

defer myStructValue.a.Close()

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.