Re: [go-nuts] Temporary files in go

[Greg Saylor]

If I remember correctly, it would be something like this:
fd = open("/tmp", O_TMPFILE | O_RDWR, 0600);
linkat(fd, "", AT_FDCWD, "/tmp/test", AT_EMPTY_PATH);
This is pretty specific to OS/kernel version and quite possibly the filesystem 
too.  This could be entirely too much of an edge case to be reasonably done.

- Greg


> On Oct 11, 2018, at 9:02 PM, Justin Israel  wrote:
> 
> 
> 
>> On Fri, Oct 12, 2018, 2:31 PM Ian Lance Taylor  wrote:
>> On Thu, Oct 11, 2018 at 4:48 PM, Greg Saylor  
>> wrote:
>> >
>> > In other programming languages (this is specific to Linux/Unix systems), in
>> > the past to ensure security in the even of a program crash, we would do
>> > something like:
>> >
>> > 1. Create a temporary file and squirrel away the file handle
>> > 2. Unlink the temporary file by name
>> > 3. Various functions would write stuff to the file
>> > 4. If the programs completes to some successful state, create a hardlink to
>> > the file handle with the final filename
>> >
>> > I'm finding this very difficult to do in Go, as there does not seem to be a
>> > way to do #4.  And this is a very important consideration for this piece of
>> > the system.
>> >
>> > For example, os.Rename takes filenames as the old/new filename.
>> >
>> > I figured looking in that code might reveal something lower level that 
>> > could
>> > be used, which lead me to syscal_linuxl.Rename()
>> >
>> > That lead me to syscall_linux.RenameAt()
>> >
>> > Which led me to zsyscall_linux_amd64.go.
>> >
>> > .. at this point I got pretty lost on how to do any of this.  _AT_FDCWD and
>> > fishing around in what appears to be some pretty low-level internals of
>> > Go...
>> >
>> > Is there some way to achieve this or a way that can ensure these files are
>> > always removed if the program is kill -9'd, terminates from a panic, etc.
>> 
>> Can you show us how you do this in C?
> 
> 
> Probably linkat(2) ? 
> 
> http://man7.org/linux/man-pages/man2/linkat.2.html
> 
>> 
>> I expect that will point toward how to do it in Go.
>> 
>> Ian
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Temporary files in go

[robert engels]

I think a more “standard” and certainly portable way to do this is with a tmp 
file, and a rename/move at the end - along with a clean-up of tmp at 
application start (if worried about crashes).

Using the proper file and process permissions on the tmp file - since you need 
these to protect the final file anyway.

> On Oct 12, 2018, at 12:21 AM, Sam Whited  wrote:
> 
> On Thu, Oct 11, 2018, at 18:48, Greg Saylor wrote:
>> 1. Create a temporary file and squirrel away the file handle
>> 2. Unlink the temporary file by name
>> 3. Various functions would write stuff to the file
>> 4. If the programs completes to some successful state, create a hardlink to 
>> the file handle with the final filename
> 
> The golang.org/x/sys/unix [1] package gives you a lower level interface that 
> might be familiar to you if you're used to using C. Something like this is 
> probably similar to what you were doing (though you'll need /proc mounted, 
> I'm not sure if there's a better way to do that):
> 
>   oldfd, err := unix.Open(".", unix.O_TMPFILE|unix.O_RDWR, 
> unix.S_IRUSR|unix.S_IWUSR)
>   if err != nil {
>   panic(err)
>   }
> 
>   f := os.NewFile(uintptr(oldfd), "myoldfile")
>   _, err = f.Write([]byte("Hello, World"))
>   if err != nil {
>   panic(err)
>   }
> 
>   dir, err := os.Open(".")
>   if err != nil {
>   panic(err)
>   }
>   procdir, err := os.Open("/proc/self/fd/")
>   if err != nil {
>   panic(err)
>   }
> 
>   err = unix.Linkat(int(procdir.Fd()), strconv.Itoa(oldfd), 
> int(dir.Fd()), "mynewfile", unix.AT_SYMLINK_FOLLOW)
>   if err != nil {
>   panic(err)
>   }
> 
> —Sam
> 
> [1]: https://godoc.org/golang.org/x/sys/unix
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Temporary files in go

[Sam Whited]

On Thu, Oct 11, 2018, at 18:48, Greg Saylor wrote:
> 1. Create a temporary file and squirrel away the file handle
> 2. Unlink the temporary file by name
> 3. Various functions would write stuff to the file
> 4. If the programs completes to some successful state, create a hardlink to 
> the file handle with the final filename

The golang.org/x/sys/unix [1] package gives you a lower level interface that 
might be familiar to you if you're used to using C. Something like this is 
probably similar to what you were doing (though you'll need /proc mounted, I'm 
not sure if there's a better way to do that):

oldfd, err := unix.Open(".", unix.O_TMPFILE|unix.O_RDWR, 
unix.S_IRUSR|unix.S_IWUSR)
if err != nil {
panic(err)
}

f := os.NewFile(uintptr(oldfd), "myoldfile")
_, err = f.Write([]byte("Hello, World"))
if err != nil {
panic(err)
}

dir, err := os.Open(".")
if err != nil {
panic(err)
}
procdir, err := os.Open("/proc/self/fd/")
if err != nil {
panic(err)
}

err = unix.Linkat(int(procdir.Fd()), strconv.Itoa(oldfd), 
int(dir.Fd()), "mynewfile", unix.AT_SYMLINK_FOLLOW)
if err != nil {
panic(err)
}

—Sam

[1]: https://godoc.org/golang.org/x/sys/unix

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Golang short variable declaration needs at least one must not be declared?

[qaq1362211689]

Thank you ! I will record what you said to my bookmark, also I will update 
my question in Stackoverflow to let more people see your answer.

On Friday, October 12, 2018 at 10:02:55 AM UTC+8, Burak Serdar wrote:
>
>
>
> On Thu, Oct 11, 2018 at 7:51 PM iamwwc > 
> wrote:
>
>> Did you mean -> 
>> --
>>
>> For all typed object such as type *** struct, type *** func(), type  
>> int, their field cannot be redeclare? and cannot be used at left side of 
>> :=?
>>
> That is correct. A member field cannot be on the left side of :=
>
>
> This is the from the spec:
>
> It is shorthand for a regular variable declaration with initializer 
> expressions but no types:
>
> "var" IdentifierList = ExpressionList .
>
> So, 
>
>x, y:=values
>
> is identical to
>
>var x, y = values
>
> Applying this rule to your example, it becomes:
>
>var c.file, err=os.OpenFile...
>
> which is not valid. 
>
>
>  
>
>>
>> But what puzzles me is Golang docs said "at least one of the non-blank 
>> variables is new.
>>
>>
>> *Is it because the documentation is not rigorous?*
>>
>>
>> Sorry for my bad English
>>
>> On Friday, October 12, 2018 at 9:32:28 AM UTC+8, Burak Serdar wrote:
>>>
>>> The IDE is probably buggy, but you are wrong as well. The problem is 
>>> c.file, not err. 
>>>
>>> c.file cannot be on the left side of :=, it is not a name you can 
>>> redeclare. You have to write:
>>>
>>> var err error
>>> c.file, err= os.OpenFile...
>>>
>>> or:
>>>
>>> f, err:=os.OpenFile
>>> c.file=f
>>>
>>> On Thu, Oct 11, 2018 at 7:21 PM  wrote:
>>>
 Hi, everyone, I have in trouble about `short variable declaration`

 Here is my codes snippet

 package utils
 import "os"

 type FileController struct{
 file *os.File}

 func (c *FileController)OpenFile(path string)error{
 c.file, err := os.OpenFile(path,os.O_CREATE | os.O_RDWR,0755)

 //return some value these}


 c.file have been declared, but err not.


 From golang spec docs



> Unlike regular variable declarations, a short variable declaration may 
> redeclare variables provided they were originally declared earlier in the 
> same block (or the parameter lists if the block is the function body) 
> with 
> the same type, and at least one of the non-blank variables is new. 



 My codes snippet should works, but `GolandIDE` tell me "unresolved 
 reference 'err' "


 [image: image_3.png]


 It's a bug about Goland IDE or just I'm wrong?

 -- 
 You received this message because you are subscribed to the Google 
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to golang-nuts...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Temporary files in go

[Justin Israel]

On Fri, Oct 12, 2018, 2:31 PM Ian Lance Taylor  wrote:

> On Thu, Oct 11, 2018 at 4:48 PM, Greg Saylor 
> wrote:
> >
> > In other programming languages (this is specific to Linux/Unix systems),
> in
> > the past to ensure security in the even of a program crash, we would do
> > something like:
> >
> > 1. Create a temporary file and squirrel away the file handle
> > 2. Unlink the temporary file by name
> > 3. Various functions would write stuff to the file
> > 4. If the programs completes to some successful state, create a hardlink
> to
> > the file handle with the final filename
> >
> > I'm finding this very difficult to do in Go, as there does not seem to
> be a
> > way to do #4.  And this is a very important consideration for this piece
> of
> > the system.
> >
> > For example, os.Rename takes filenames as the old/new filename.
> >
> > I figured looking in that code might reveal something lower level that
> could
> > be used, which lead me to syscal_linuxl.Rename()
> >
> > That lead me to syscall_linux.RenameAt()
> >
> > Which led me to zsyscall_linux_amd64.go.
> >
> > .. at this point I got pretty lost on how to do any of this.  _AT_FDCWD
> and
> > fishing around in what appears to be some pretty low-level internals of
> > Go...
> >
> > Is there some way to achieve this or a way that can ensure these files
> are
> > always removed if the program is kill -9'd, terminates from a panic, etc.
>
> Can you show us how you do this in C?
>

Probably linkat(2) ?

http://man7.org/linux/man-pages/man2/linkat.2.html


> I expect that will point toward how to do it in Go.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go language should become an ANSI and ISO standard.

[Beoran]

So no matter if I say yes or no, both ways are bad? I think is not a very fair 
way to argue.

Anyway, with the Ruby standard you can do either. The Ruby standard defines 
that there are strictly conforming Ruby processors, which implement the 
standard and conforming Ruby processors which may have any number of additional 
implementation defined extensions, alternate syntax and language features. 

After the standard was written, mruby was implemented to be a strictly 
conforming Ruby processor, which doesn't influence or hold back the development 
of the other Ruby implementations at all.

And all other Ruby implementations can be considered confirming, which is worth 
millions of $$$ to Ruby developers. The organizations and governments I 
mentioned   tend to have deep pockets, and the Ruby standard enables us to gain 
approval from said bureaucrats. So, we can now use Ruby for these well funded 
projects, since now it is an international standard.

So actually, because the Ruby standard was carefully written to enable this, it 
has been win/win for Ruby developers. You can use a strictly conforming mruby 
if you like or need to, or use any other Ruby implementations as conforming 
ones and please the bureaucrats.

I consider that we should do the same for Go. When done carefully it will also 
be a win/win.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Golang short variable declaration needs at least one must not be declared?

[Burak Serdar]

On Thu, Oct 11, 2018 at 7:51 PM iamwwc  wrote:

> Did you mean ->
> --
>
> For all typed object such as type *** struct, type *** func(), type 
> int, their field cannot be redeclare? and cannot be used at left side of
> :=?
>
That is correct. A member field cannot be on the left side of :=


This is the from the spec:

It is shorthand for a regular variable declaration with initializer
expressions but no types:

"var" IdentifierList = ExpressionList .

So,

   x, y:=values

is identical to

   var x, y = values

Applying this rule to your example, it becomes:

   var c.file, err=os.OpenFile...

which is not valid.




>
> But what puzzles me is Golang docs said "at least one of the non-blank
> variables is new.
>
>
> *Is it because the documentation is not rigorous?*
>
>
> Sorry for my bad English
>
> On Friday, October 12, 2018 at 9:32:28 AM UTC+8, Burak Serdar wrote:
>>
>> The IDE is probably buggy, but you are wrong as well. The problem is
>> c.file, not err.
>>
>> c.file cannot be on the left side of :=, it is not a name you can
>> redeclare. You have to write:
>>
>> var err error
>> c.file, err= os.OpenFile...
>>
>> or:
>>
>> f, err:=os.OpenFile
>> c.file=f
>>
>> On Thu, Oct 11, 2018 at 7:21 PM  wrote:
>>
>>> Hi, everyone, I have in trouble about `short variable declaration`
>>>
>>> Here is my codes snippet
>>>
>>> package utils
>>> import "os"
>>>
>>> type FileController struct{
>>> file *os.File}
>>>
>>> func (c *FileController)OpenFile(path string)error{
>>> c.file, err := os.OpenFile(path,os.O_CREATE | os.O_RDWR,0755)
>>>
>>> //return some value these}
>>>
>>>
>>> c.file have been declared, but err not.
>>>
>>>
>>> From golang spec docs
>>>
>>>
>>>
 Unlike regular variable declarations, a short variable declaration may
 redeclare variables provided they were originally declared earlier in the
 same block (or the parameter lists if the block is the function body) with
 the same type, and at least one of the non-blank variables is new.
>>>
>>>
>>>
>>> My codes snippet should works, but `GolandIDE` tell me "unresolved
>>> reference 'err' "
>>>
>>>
>>> [image: image_3.png]
>>>
>>>
>>> It's a bug about Goland IDE or just I'm wrong?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Golang short variable declaration needs at least one must not be declared?

[iamwwc]

Did you mean -> 
--

For all typed object such as type *** struct, type *** func(), type  int, 
their field cannot be redeclare? and cannot be used at left side of :=?


But what puzzles me is Golang docs said "at least one of the non-blank 
variables is new.


*Is it because the documentation is not rigorous?*


Sorry for my bad English

On Friday, October 12, 2018 at 9:32:28 AM UTC+8, Burak Serdar wrote:
>
> The IDE is probably buggy, but you are wrong as well. The problem is 
> c.file, not err. 
>
> c.file cannot be on the left side of :=, it is not a name you can 
> redeclare. You have to write:
>
> var err error
> c.file, err= os.OpenFile...
>
> or:
>
> f, err:=os.OpenFile
> c.file=f
>
> On Thu, Oct 11, 2018 at 7:21 PM > wrote:
>
>> Hi, everyone, I have in trouble about `short variable declaration`
>>
>> Here is my codes snippet
>>
>> package utils
>> import "os"
>>
>> type FileController struct{
>> file *os.File}
>>
>> func (c *FileController)OpenFile(path string)error{
>> c.file, err := os.OpenFile(path,os.O_CREATE | os.O_RDWR,0755)
>>
>> //return some value these}
>>
>>
>> c.file have been declared, but err not.
>>
>>
>> From golang spec docs
>>
>>
>>
>>> Unlike regular variable declarations, a short variable declaration may 
>>> redeclare variables provided they were originally declared earlier in the 
>>> same block (or the parameter lists if the block is the function body) with 
>>> the same type, and at least one of the non-blank variables is new. 
>>
>>
>>
>> My codes snippet should works, but `GolandIDE` tell me "unresolved 
>> reference 'err' "
>>
>>
>> [image: image_3.png]
>>
>>
>> It's a bug about Goland IDE or just I'm wrong?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Golang short variable declaration needs at least one must not be declared?

[Burak Serdar]

The IDE is probably buggy, but you are wrong as well. The problem is
c.file, not err.

c.file cannot be on the left side of :=, it is not a name you can
redeclare. You have to write:

var err error
c.file, err= os.OpenFile...

or:

f, err:=os.OpenFile
c.file=f

On Thu, Oct 11, 2018 at 7:21 PM  wrote:

> Hi, everyone, I have in trouble about `short variable declaration`
>
> Here is my codes snippet
>
> package utils
> import "os"
>
> type FileController struct{
> file *os.File}
>
> func (c *FileController)OpenFile(path string)error{
> c.file, err := os.OpenFile(path,os.O_CREATE | os.O_RDWR,0755)
>
> //return some value these}
>
>
> c.file have been declared, but err not.
>
>
> From golang spec docs
>
>
>
>> Unlike regular variable declarations, a short variable declaration may
>> redeclare variables provided they were originally declared earlier in the
>> same block (or the parameter lists if the block is the function body) with
>> the same type, and at least one of the non-blank variables is new.
>
>
>
> My codes snippet should works, but `GolandIDE` tell me "unresolved
> reference 'err' "
>
>
> [image: image_3.png]
>
>
> It's a bug about Goland IDE or just I'm wrong?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Temporary files in go

[Ian Lance Taylor]

On Thu, Oct 11, 2018 at 4:48 PM, Greg Saylor  wrote:
>
> In other programming languages (this is specific to Linux/Unix systems), in
> the past to ensure security in the even of a program crash, we would do
> something like:
>
> 1. Create a temporary file and squirrel away the file handle
> 2. Unlink the temporary file by name
> 3. Various functions would write stuff to the file
> 4. If the programs completes to some successful state, create a hardlink to
> the file handle with the final filename
>
> I'm finding this very difficult to do in Go, as there does not seem to be a
> way to do #4.  And this is a very important consideration for this piece of
> the system.
>
> For example, os.Rename takes filenames as the old/new filename.
>
> I figured looking in that code might reveal something lower level that could
> be used, which lead me to syscal_linuxl.Rename()
>
> That lead me to syscall_linux.RenameAt()
>
> Which led me to zsyscall_linux_amd64.go.
>
> .. at this point I got pretty lost on how to do any of this.  _AT_FDCWD and
> fishing around in what appears to be some pretty low-level internals of
> Go...
>
> Is there some way to achieve this or a way that can ensure these files are
> always removed if the program is kill -9'd, terminates from a panic, etc.

Can you show us how you do this in C?

I expect that will point toward how to do it in Go.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Golang short variable declaration needs at least one must not be declared?

[qaq1362211689]

Hi, everyone, I have in trouble about `short variable declaration`

Here is my codes snippet

package utils
import "os"

type FileController struct{
file *os.File}

func (c *FileController)OpenFile(path string)error{
c.file, err := os.OpenFile(path,os.O_CREATE | os.O_RDWR,0755)

//return some value these}


c.file have been declared, but err not.


>From golang spec docs



> Unlike regular variable declarations, a short variable declaration may 
> redeclare variables provided they were originally declared earlier in the 
> same block (or the parameter lists if the block is the function body) with 
> the same type, and at least one of the non-blank variables is new. 



My codes snippet should works, but `GolandIDE` tell me "unresolved 
reference 'err' "


[image: image_3.png]


It's a bug about Goland IDE or just I'm wrong?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Why can runtime.procyield declare its argument stack size to be zero?

[Jeffrey Baker]

I'm trying to learn how to safely use this asm, and I don't understand some 
of the existing asm in the runtime package.  From amd64.s:

TEXT runtime·procyield(SB),NOSPLIT,$0-0
MOVL cycles+0(FP), AX

My understanding of $0-0 is it's supposed to indicate the sizes of the 
arguments, in this case would seem to be $0-4 since the argument is uint32. 
Indeed, specializations of this function on other platforms use $0-4, such 
as ppc64:

TEXT runtime·procyield(SB),NOSPLIT|NOFRAME,$0-4
MOVW cycles+0(FP), R7

Why does the amd64 version work? Why doesn't `go vet` complain about it?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Announcing a Fyne GUI toolkit

[R Srinivasan]

Thanks a bunch. I will give a serious "go".

Background - a tool that is command line now. Developed on Linux but 
typical user in Windows. Just cross built for windows. Want to add a 
minimal frontend - simple dialog. Being like a calculator - don't want to 
have elaborate "install" procedures. In the case of the cli - one .exe is 
all it takes. 

will experiment and report back.

srini

On Thursday, October 11, 2018 at 3:54:29 PM UTC-4, Andrew Williams wrote:
>
> Hi,
>
> Great questions, thanks!
>
> 1. We can bundle in the libraries for Windows and for macOS - but on linux 
> probably not (unless your app is LGPL/GPL due to a licensing issue that I 
> have not found a better solution to). But distribution on Linux it is 
> normal to depend on external libraries :).
> There will be a tool included to package final binaries, but I have not 
> got it to a satisfactory level to share yet.
>
> 2. Yes, we should be able to do cross-compilation using the standard Go 
> tools. There is the usual challenge of enabling CGO for a GOOS build, but 
> beyond that it should be OK. It's easier than, for example, andlabs UI as 
> we are not linking to OS specific functionality, just the libefl 
> abstraction.
>
> Point 2 may change over time - as we may wish to add certain OS 
> abstraction directly which may make cross compilaton harder.
>
> What I was thinking about, however, was creating some app metadata format 
> and a centralised build server to work around both build and package 
> distribution issues...
>
> I hope that helps,
> Andrew
>
> On Sat, 6 Oct 2018 at 11:41 R Srinivasan > 
> wrote:
>
>> 1. What are the "distribution" considerations?
>>
>> Considering Windows targets - can we have all the required libraries 
>> bundled in the final executable? 
>>
>> 2. Can the apps be cross built? i.e. Built on Linux for macOS and 
>> windows`targets?
>>
>> thanks for pointers, srini
>>
>>
>> On Friday, September 14, 2018 at 1:17:00 PM UTC-4, Andrew Williams wrote:
>>>
>>> Hi,
>>>
>>> Some time ago I realised that Go was a great language for building new 
>>> applications, as I wanted to use something powerful but fast to learn. I 
>>> also wanted a really simple to use GUI toolkit that worked cross platform - 
>>> this was a little more difficult to satisfy!
>>>
>>> The aim was to create an API that was:
>>>
>>>- Simple to learn
>>>- Great looking with theme options
>>>- Truly cross platform with identical look across platforms
>>>- Solved all of the complicated GUI challenges (threading, scaling 
>>>etc)
>>>
>>> And so the Fyne project was created https://github.com/fyne-io/fyne !
>>> The design language is basically material design and the rendering is 
>>> currently EFL with support for Windows, macOS and Linux.
>>> It's entirely vector based (though you can draw a Raster space if you 
>>> need) and scales beautifully.
>>>
>>> For a taste of what that looks like here you go :)
>>>
>>> [image: widgets-dark.png]
>>>
>>> It's now well into development and ready for people to get involved. 
>>> There is a long way to go but it feels like a solid base.
>>> Instructions for getting started, if you need them, are at 
>>> https://github.com/fyne-io/bootstrap/blob/master/README.md .
>>> If you want to know more we're also in the #fyne channel on the gopher 
>>> Slack server.
>>>
>>> Thanks in advance for your thoughts :)
>>> Andrew
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> http://andywilliams.me
> http://ajwillia.ms
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Temporary files in go

[Greg Saylor]

Hello,

In other programming languages (this is specific to Linux/Unix systems), in 
the past to ensure security in the even of a program crash, we would do 
something like:

1. Create a temporary file and squirrel away the file handle
2. Unlink the temporary file by name
3. Various functions would write stuff to the file
4. If the programs completes to some successful state, create a hardlink to 
the file handle with the final filename

I'm finding this very difficult to do in Go, as there does not seem to be a 
way to do #4.  And this is a very important consideration for this piece of 
the system.

For example, os.Rename takes filenames as the old/new filename.

I figured looking in that code might reveal something lower level that 
could be used, which lead me to syscal_linuxl.Rename()

That lead me to syscall_linux.RenameAt()

Which led me to zsyscall_linux_amd64.go.

.. at this point I got pretty lost on how to do any of this.  _AT_FDCWD and 
fishing around in what appears to be some pretty low-level internals of 
Go...

Is there some way to achieve this or a way that can ensure these files are 
always removed if the program is kill -9'd, terminates from a panic, etc.

Thanks!

Greg


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go language should become an ANSI and ISO standard.

[Gerald Henriksen]

On Thu, 11 Oct 2018 06:15:16 +0200, you wrote:

>But that is not what I am getting at. Five years ago I was somewhat
>involved in the effort to make Ruby an ISO standard language. From the get
>go it was a largely bureaucratic effort. We specified a common sub set of a
>then already old version of Ruby and made enough loopholes to make sure
>that any Ruby out there could be considered confirming to the standard.
>
>Why? You can perhaps not appreciate how powerful the bureaucrats can be in
>some countries and corporations. Since Ruby became an ISO standard, I have
>been able to convince those bureaucrats to let me use Ruby where I was
>barred from doing so before.

So, the question is are you able to use any version of Ruby, or are
you restricted to the ISO version of Ruby?

If you are restricted to the ISO version that it is a very bad thing
because you are inherently then holding back the forward progress of
the language by forcing people to use an old version.

If you can use any version of Ruby then you have demonstrated that the
ISO certification isn't worth the paper its printed on.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Announcing a Fyne GUI toolkit

[Andrew Williams]

Hi,

Many thanks for that feedback! We have a long way to go for a full toolkit
but it's close to supporting basic applications.
Being designed for the Go language makes it so much quicker and easier to
develop with than other possible toolkit choices - and I'm not just saying
that from a biased point of view :)

Thanks,
Andrew

On Tue, 9 Oct 2018 at 02:43 ct via golang-nuts 
wrote:

> Just wanted to drop in and say that I'm very impressed with the design and
> quality of the examples I've seen.
>
> Great work, and thanks for sharing!
>
> On Friday, September 14, 2018 at 5:17:00 PM UTC, Andrew Williams wrote:
>>
>> Hi,
>>
>> Some time ago I realised that Go was a great language for building new
>> applications, as I wanted to use something powerful but fast to learn. I
>> also wanted a really simple to use GUI toolkit that worked cross platform -
>> this was a little more difficult to satisfy!
>>
>> The aim was to create an API that was:
>>
>>- Simple to learn
>>- Great looking with theme options
>>- Truly cross platform with identical look across platforms
>>- Solved all of the complicated GUI challenges (threading, scaling
>>etc)
>>
>> And so the Fyne project was created https://github.com/fyne-io/fyne !
>> The design language is basically material design and the rendering is
>> currently EFL with support for Windows, macOS and Linux.
>> It's entirely vector based (though you can draw a Raster space if you
>> need) and scales beautifully.
>>
>> For a taste of what that looks like here you go :)
>>
>> [image: widgets-dark.png]
>>
>> It's now well into development and ready for people to get involved.
>> There is a long way to go but it feels like a solid base.
>> Instructions for getting started, if you need them, are at
>> https://github.com/fyne-io/bootstrap/blob/master/README.md .
>> If you want to know more we're also in the #fyne channel on the gopher
>> Slack server.
>>
>> Thanks in advance for your thoughts :)
>> Andrew
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 
http://andywilliams.me
http://ajwillia.ms

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Announcing a Fyne GUI toolkit

[Andrew Williams]

Hi,

As the Fyne toolkit depends on EFL you need to either install that yourself
or use our bootstrap tool[1]. If you have done one or other of those then
the error you are seeing is strange, I can try to help you off-list (if so
please provide info about OS etc).

I am trying to find ways to provide more helpful errors when the native
deps cannot be found...!

Andrew

On Mon, 8 Oct 2018 at 16:21  wrote:

> Hi There,
>
> I tried it to test it.
>
> But i am getting the below error :
>
> # pkg-config --cflags  -- eina evas ecore-evas ecore-evas ecore-input
> ecore ecore-evas eina evas ecore-evas ecore-input ecore evas ecore
> ecore-evas ecore-input evas
> Package eina was not found in the pkg-config search path.
> Perhaps you should add the directory containing `eina.pc'
> to the PKG_CONFIG_PATH environment variable
> No package 'eina' found
>
>
> what path should be exported to PKG_CONFIG_PATH?
>
> Thank u
>
>
>
> On Friday, September 14, 2018 at 10:47:00 PM UTC+5:30, Andrew Williams
> wrote:
>>
>> Hi,
>>
>> Some time ago I realised that Go was a great language for building new
>> applications, as I wanted to use something powerful but fast to learn. I
>> also wanted a really simple to use GUI toolkit that worked cross platform -
>> this was a little more difficult to satisfy!
>>
>> The aim was to create an API that was:
>>
>>- Simple to learn
>>- Great looking with theme options
>>- Truly cross platform with identical look across platforms
>>- Solved all of the complicated GUI challenges (threading, scaling
>>etc)
>>
>> And so the Fyne project was created https://github.com/fyne-io/fyne !
>> The design language is basically material design and the rendering is
>> currently EFL with support for Windows, macOS and Linux.
>> It's entirely vector based (though you can draw a Raster space if you
>> need) and scales beautifully.
>>
>> For a taste of what that looks like here you go :)
>>
>> [image: widgets-dark.png]
>>
>> It's now well into development and ready for people to get involved.
>> There is a long way to go but it feels like a solid base.
>> Instructions for getting started, if you need them, are at
>> https://github.com/fyne-io/bootstrap/blob/master/README.md .
>> If you want to know more we're also in the #fyne channel on the gopher
>> Slack server.
>>
>> Thanks in advance for your thoughts :)
>> Andrew
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 
http://andywilliams.me
http://ajwillia.ms

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Announcing a Fyne GUI toolkit

[Andrew Williams]

Hi,

Great questions, thanks!

1. We can bundle in the libraries for Windows and for macOS - but on linux
probably not (unless your app is LGPL/GPL due to a licensing issue that I
have not found a better solution to). But distribution on Linux it is
normal to depend on external libraries :).
There will be a tool included to package final binaries, but I have not got
it to a satisfactory level to share yet.

2. Yes, we should be able to do cross-compilation using the standard Go
tools. There is the usual challenge of enabling CGO for a GOOS build, but
beyond that it should be OK. It's easier than, for example, andlabs UI as
we are not linking to OS specific functionality, just the libefl
abstraction.

Point 2 may change over time - as we may wish to add certain OS abstraction
directly which may make cross compilaton harder.

What I was thinking about, however, was creating some app metadata format
and a centralised build server to work around both build and package
distribution issues...

I hope that helps,
Andrew

On Sat, 6 Oct 2018 at 11:41 R Srinivasan  wrote:

> 1. What are the "distribution" considerations?
>
> Considering Windows targets - can we have all the required libraries
> bundled in the final executable?
>
> 2. Can the apps be cross built? i.e. Built on Linux for macOS and
> windows`targets?
>
> thanks for pointers, srini
>
>
> On Friday, September 14, 2018 at 1:17:00 PM UTC-4, Andrew Williams wrote:
>>
>> Hi,
>>
>> Some time ago I realised that Go was a great language for building new
>> applications, as I wanted to use something powerful but fast to learn. I
>> also wanted a really simple to use GUI toolkit that worked cross platform -
>> this was a little more difficult to satisfy!
>>
>> The aim was to create an API that was:
>>
>>- Simple to learn
>>- Great looking with theme options
>>- Truly cross platform with identical look across platforms
>>- Solved all of the complicated GUI challenges (threading, scaling
>>etc)
>>
>> And so the Fyne project was created https://github.com/fyne-io/fyne !
>> The design language is basically material design and the rendering is
>> currently EFL with support for Windows, macOS and Linux.
>> It's entirely vector based (though you can draw a Raster space if you
>> need) and scales beautifully.
>>
>> For a taste of what that looks like here you go :)
>>
>> [image: widgets-dark.png]
>>
>> It's now well into development and ready for people to get involved.
>> There is a long way to go but it feels like a solid base.
>> Instructions for getting started, if you need them, are at
>> https://github.com/fyne-io/bootstrap/blob/master/README.md .
>> If you want to know more we're also in the #fyne channel on the gopher
>> Slack server.
>>
>> Thanks in advance for your thoughts :)
>> Andrew
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 
http://andywilliams.me
http://ajwillia.ms

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[Ian Denhardt]

Quoting gary.willoug...@victoriaplumb.com (2018-10-11 10:38:38)
>Add it to this: https://github.com/golang/go/wiki/Go2GenericsFeedback

Already did (before posting to the list); it's at the top.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[Ian Denhardt]

Quoting Dean B (2018-10-11 14:58:01)

> Yeah that's why I was about how operators were the primary reason
> interfaces couldn't work out. Your proposal doesn't include operators,
> which was the reason interfaces aren't used. Syntax in Go is pretty
> important, which I imagine is the reason that operators didn't work
> out.

As it stands, I don't really feel like the draft design's solution re:
operators is better than just punting on them. Much of this discussion
is already in my proposal, but:

You still have to either write different versions of your code for types
that support `<` and types that support `.Less()`, or only support one
of these.

Supporting the former means your code can only ever work with basic
types, and I don't feel like the meager extra expressiveness you get
is worth the complexity.

The extensible option is to only support the latter, and if you have
a function that expects something with a `.Less()` method, and you want
to pass it an int64, you define a wrapper type around the int64.

I would imagine in this scenario before long someone would write a
package with a standard set of wrappers around the basic types, covering
the boilerplate once and for all:

```
package ops

type Int64 int64

func (lhs int64) Less(rhs int64) bool {
return lhs < rhs
}

..
```

It probably makes sense in this case to just have part of the generics
design be to add methods to the basic  types that are aliases for the
operators, so we can skip the above.

> I'm working on another proposal with a `self` type introduced which
> should be simple. The issue with a `self` type is that interfaces are
> comparable with every other comparable value, so it can't be used in
> that case... I'll figure something out haha.

I considered adding something like self, but my proposal is actually
able to cover the cases where you'd want self as is. Again, this is in
the proposal:

```
// The ordered interface is parametrized over another type. You could
// define a type A that implements Ordered(B), but see below.
type Ordered(type T) interface {
Less(other T) bool
}

// Sort is still able to constrain its parameter to only types which
// implemented Ordered *for themselves*, so it would not accept an A
// which implements Ordered(B). I believe this solves the problems
// that you would want 'self' for, without adding an extra construct.
func Sort(type T Ordered(T))(slice []T) {
// ...
}
```

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

Yes, that's acceptable.

Le jeu. 11 oct. 2018 à 21:28, Sam Whited  a écrit :

> On Thu, Oct 11, 2018, at 14:14, Thomas Bruyelle wrote:
> > That say, the downside is I have to keep both hash system, until all
> users
> > connect, which can take a long time, or can never happen!
>
> Or at least until enough users have updated that you don't mind forcing
> the rest to update their password if they ever log in again.
>
> —Sam
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/Lx672zPwqSQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Sam Whited]

On Thu, Oct 11, 2018, at 14:14, Thomas Bruyelle wrote:
> That say, the downside is I have to keep both hash system, until all users
> connect, which can take a long time, or can never happen!

Or at least until enough users have updated that you don't mind forcing the 
rest to update their password if they ever log in again.

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

That say, the downside is I have to keep both hash system, until all users
connect, which can take a long time, or can never happen!

Le jeu. 11 oct. 2018 à 21:08, Thomas Bruyelle  a
écrit :

> This is brillant, thanks again Sam.
> I think I'll go for something like that. The argon2 hash can hold the
> version, just behind the mode, I could use that to distinguish old and new
> hash.
>
> $argon2i$*v=13*
> $m=65536,t=3,p=4$SZ30vQfC522jpGssj92FkQ$xO4vPBrnd+DW/CbhiGjWW7u0s/nf7PcGUjS5bWQElYo
>
>
>
> Le jeu. 11 oct. 2018 à 21:01, Sam Whited  a écrit :
>
>> On Thu, Oct 11, 2018, at 13:56, Thomas Bruyelle wrote:
>> > Unfortunately, because of that version mismatch, all my users' hashes
>> were
>> > created with a version not supported by golang.org/x/crypto/argon2, so
>> I
>> > can't migrate :/
>>
>> I hope no problems are ever discovered in Argon2 then, it's generally a
>> good idea to have some sort of system for migrating hashes :)
>>
>> For example, when the user next logs in you could verify that he hash is
>> correct, but also calculate the new hash and update it and set a prefix or
>> a bit in the database somewhere saying that they're on "hash mechanism v2".
>> There's no need to force reset every password all at once since this isn't
>> a security issue.
>>
>> —Sam
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/golang-nuts/Lx672zPwqSQ/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

This is brillant, thanks again Sam.
I think I'll go for something like that. The argon2 hash can hold the
version, just behind the mode, I could use that to distinguish old and new
hash.

$argon2i$*v=13*
$m=65536,t=3,p=4$SZ30vQfC522jpGssj92FkQ$xO4vPBrnd+DW/CbhiGjWW7u0s/nf7PcGUjS5bWQElYo



Le jeu. 11 oct. 2018 à 21:01, Sam Whited  a écrit :

> On Thu, Oct 11, 2018, at 13:56, Thomas Bruyelle wrote:
> > Unfortunately, because of that version mismatch, all my users' hashes
> were
> > created with a version not supported by golang.org/x/crypto/argon2, so
> I
> > can't migrate :/
>
> I hope no problems are ever discovered in Argon2 then, it's generally a
> good idea to have some sort of system for migrating hashes :)
>
> For example, when the user next logs in you could verify that he hash is
> correct, but also calculate the new hash and update it and set a prefix or
> a bit in the database somewhere saying that they're on "hash mechanism v2".
> There's no need to force reset every password all at once since this isn't
> a security issue.
>
> —Sam
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/Lx672zPwqSQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Sam Whited]

On Thu, Oct 11, 2018, at 13:56, Thomas Bruyelle wrote:
> Unfortunately, because of that version mismatch, all my users' hashes were 
> created with a version not supported by golang.org/x/crypto/argon2, so I 
> can't migrate :/

I hope no problems are ever discovered in Argon2 then, it's generally a good 
idea to have some sort of system for migrating hashes :)

For example, when the user next logs in you could verify that he hash is 
correct, but also calculate the new hash and update it and set a prefix or a 
bit in the database somewhere saying that they're on "hash mechanism v2". 
There's no need to force reset every password all at once since this isn't a 
security issue.

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

Wow indeed Sam you're completely right, with Version13 the test passes. 
Thank you a lot for help.

Unfortunately, because of that version mismatch, all my users' hashes were 
created with a version not supported by golang.org/x/crypto/argon2, so I 
can't migrate :/

Le jeudi 11 octobre 2018 19:10:45 UTC+2, Sam Whited a écrit :
>
> On Thu, Oct 11, 2018, at 11:37, Thomas Bruyelle wrote: 
> > I want to update my authentication code by using 
> > golang.org/x/crypto/argon2, as a replacement of 
> > github.com/tvdburgt/go-argon2 which uses the legacy C lib under the 
> hood 
> > through CGO. 
> > The main benefit is of course to drop the usage of CGO. 
> > 
> > But I encounter a serious issue, with the same inputs (password, salt, 
> > times, threads, memory), the 2 libraries give me 2 different hashes. 
> > That means it's impossible for me to migrate, or else I have to ask all 
> my 
> > users to regenerate their passwords, which is not acceptable for me. 
>
> The x/crypto/argon2 package implements version 13, if you add "Version: 
>  Version13," to your context they should be the same. The docs claim this 
> is the default for the legacy version, but they appear to be wrong. 
>
> —Sam 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[Dean B]

Yeah that's why I was about how operators were the primary reason 
interfaces couldn't work out.

Your proposal doesn't include operators, which was the reason interfaces 
aren't used. Syntax in Go is pretty important, which I imagine is the 
reason that operators didn't work out. I'm working on another proposal with 
a `self` type introduced which should be simple.

The issue with a `self` type is that interfaces are comparable with every 
other comparable value, so it can't be used in that case... I'll figure 
something out haha.

On Thursday, October 11, 2018 at 1:42:25 PM UTC-4, Ian Denhardt wrote:
>
> Hmm, I suppose I imagined having read a deeper reason than syntax :/. 
> That does seem like an incredibly shallow reason to introduce a whole 
> new non-orthogonal concept (I see others have attempted to tackle the 
> syntax issue, including yourself). 
>
> There are actually deeper difficulties than syntax with using interfaces 
> to capture the built-in operators, in that some of them have semantics 
> that can't be captured as methods. One approach is to only use 
> interfaces to describe "well-behaved" operators, leaving the rest as 
> special constructs that can't be abstracted over, like today. But `==` 
> falls into this category, since it works on both pointers and structs, 
> and does different things -- whereas it is not generally possible for a 
> method M to do different things for type T vs type *T. 
>
> The design I sketched actually just punts on operators entirely, which I 
> don't like but I think is better than where contracts put us (esp. since 
> you still can't actually define alternative notions of `<` with 
> contracts, just abstract over existing ones, so it doesn't even give you 
> much extra expressive power)`.  I'd like to find a good solution to the 
> operator problem though. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

Wow indeed Sam you're completely right, with Version13 the test passes. 
Thank you a lot for help.

Unfortunately, because of that version mismatch, all my users' hashes were 
created with a version not supported by golang.org/x/crypto/argon2, so I 
can't migrate :/

Le jeudi 11 octobre 2018 18:37:07 UTC+2, Thomas Bruyelle a écrit :
>
> Hi all,
>
> I want to update my authentication code by using 
> golang.org/x/crypto/argon2, as a replacement of 
> github.com/tvdburgt/go-argon2 which uses the legacy C lib under the hood 
> through CGO.
> The main benefit is of course to drop the usage of CGO.
>
> But I encounter a serious issue, with the same inputs (password, salt, 
> times, threads, memory), the 2 libraries give me 2 different hashes. 
> That means it's impossible for me to migrate, or else I have to ask all my 
> users to regenerate their passwords, which is not acceptable for me.
>
> I wonder if I did something wrong, to illustrate that I wrote a small 
> test, runnable inside  github.com/tvdburgt/go-argon2 :
>
> package argon2
> import (
>   "bytes"
>   "encoding/base64"
>   "testing"
>
>   xargon2 "golang.org/x/crypto/argon2"
> )
> func TestCompat(t *testing.T) {
>   salt := []byte("0123456789abcdef")
>   pwd := []byte("some password")
>
>   ctx := &Context{
>   Iterations:  3,
>   Memory:  1 << 16,
>   Parallelism: 4,
>   HashLen: 32,
>   Mode:ModeArgon2i,
>   }
>   hash, err := Hash(ctx, pwd, salt)
>   if err != nil {
>   t.Fatal(err)
>   }
>
>   xhash := xargon2.Key(pwd, salt, 3, 1<<16, 4, 32)
>
>   if !bytes.Equal(hash, xhash) {
>   t.Errorf("Compat failed:\n%s\n%s\n",
>   base64.StdEncoding.EncodeToString(hash),
>   base64.StdEncoding.EncodeToString(xhash),
>   )
>   }
> }
>
> And the result is :
>
> $ go test .
> --- FAIL: TestCompat (0.08s)
> compat_test.go:30: Compat failed:
> UgjfozCx6kU6vTNKoN8Ic2UTh7Ckphy0Dc79+1xlT/0=
> mf9LVGs+VH62cWpoLZVfCvtBWye6uMD7sWJfhYQk3fo=
> FAIL
> FAIL  github.com/tvdburgt/go-argon2   0.362s
>
>
>
>
> Thanks in advance
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[Ian Denhardt]

Hmm, I suppose I imagined having read a deeper reason than syntax :/.
That does seem like an incredibly shallow reason to introduce a whole
new non-orthogonal concept (I see others have attempted to tackle the
syntax issue, including yourself).

There are actually deeper difficulties than syntax with using interfaces
to capture the built-in operators, in that some of them have semantics
that can't be captured as methods. One approach is to only use
interfaces to describe "well-behaved" operators, leaving the rest as
special constructs that can't be abstracted over, like today. But `==`
falls into this category, since it works on both pointers and structs,
and does different things -- whereas it is not generally possible for a
method M to do different things for type T vs type *T.

The design I sketched actually just punts on operators entirely, which I
don't like but I think is better than where contracts put us (esp. since
you still can't actually define alternative notions of `<` with
contracts, just abstract over existing ones, so it doesn't even give you
much extra expressive power)`.  I'd like to find a good solution to the
operator problem though.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Sam Whited]

On Thu, Oct 11, 2018, at 11:37, Thomas Bruyelle wrote: 
> I want to update my authentication code by using 
> golang.org/x/crypto/argon2, as a replacement of 
> github.com/tvdburgt/go-argon2 which uses the legacy C lib under the hood 
> through CGO.
> The main benefit is of course to drop the usage of CGO.
> 
> But I encounter a serious issue, with the same inputs (password, salt, 
> times, threads, memory), the 2 libraries give me 2 different hashes. 
> That means it's impossible for me to migrate, or else I have to ask all my 
> users to regenerate their passwords, which is not acceptable for me.

The x/crypto/argon2 package implements version 13, if you add "Version:  
Version13," to your context they should be the same. The docs claim this is the 
default for the legacy version, but they appear to be wrong.

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Compatibility between golang.org/x/crypto/argon2 and the legacy one

[Thomas Bruyelle]

Hi all,

I want to update my authentication code by using 
golang.org/x/crypto/argon2, as a replacement of 
github.com/tvdburgt/go-argon2 which uses the legacy C lib under the hood 
through CGO.
The main benefit is of course to drop the usage of CGO.

But I encounter a serious issue, with the same inputs (password, salt, 
times, threads, memory), the 2 libraries give me 2 different hashes. 
That means it's impossible for me to migrate, or else I have to ask all my 
users to regenerate their passwords, which is not acceptable for me.

I wonder if I did something wrong, to illustrate that I wrote a small test, 
runnable inside  github.com/tvdburgt/go-argon2 :

package argon2
import (
"bytes"
"encoding/base64"
"testing"

xargon2 "golang.org/x/crypto/argon2"
)
func TestCompat(t *testing.T) {
salt := []byte("0123456789abcdef")
pwd := []byte("some password")

ctx := &Context{
Iterations:  3,
Memory:  1 << 16,
Parallelism: 4,
HashLen: 32,
Mode:ModeArgon2i,
}
hash, err := Hash(ctx, pwd, salt)
if err != nil {
t.Fatal(err)
}

xhash := xargon2.Key(pwd, salt, 3, 1<<16, 4, 32)

if !bytes.Equal(hash, xhash) {
t.Errorf("Compat failed:\n%s\n%s\n",
base64.StdEncoding.EncodeToString(hash),
base64.StdEncoding.EncodeToString(xhash),
)
}
}

And the result is :

$ go test .
--- FAIL: TestCompat (0.08s)
compat_test.go:30: Compat failed:
UgjfozCx6kU6vTNKoN8Ic2UTh7Ckphy0Dc79+1xlT/0=
mf9LVGs+VH62cWpoLZVfCvtBWye6uMD7sWJfhYQk3fo=
FAIL
FAILgithub.com/tvdburgt/go-argon2   0.362s




Thanks in advance

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go list returning directory starting with underscore on Ubuntu

[Daniel Martí]

I've encountered similar behavior with testdata directories within GOPATH,
when not in module-aware mode.

I found this thread while searching for answers, so I'm linking the issue I 
just
created in case anyone else runs into the same bug:
https://github.com/golang/go/issues/28155

On Wednesday, April 25, 2018 at 4:21:43 PM UTC+1, run...@uber.com wrote:
>
> Not accurate. 
>
> $ cat ~/code/scratch/repro-underscore-issue.sh
> echo "GOPATH: $GOPATH"
> mkdir -p $GOPATH/src/github.com/testcase/testdata
> echo "package testdata" > $GOPATH/src/
> github.com/testcase/testdata/testdata.go
> echo "" >> $GOPATH/src/github.com/testcase/testdata/testdata.go
> cd $GOPATH/src/github.com/testcase/testdata
> echo "Running go list ./..."
> go list ./...
>
> $ ~/code/scratch/repro-underscore-issue.sh
> GOPATH: /Users/prungta/code/gocode
> Running go list ./...
> _/Users/prungta/code/gocode/src/github.com/testcase/testdata
>
>
> Should this be a bug report? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[Dean B]

Looking at the proposal, operators are the reason that contracts are used 
rather than interfaces.

Why not use interfaces instead of contracts?
> *The interface method syntax is familiar.* *Writing contract bodies 
> with x + x is ordinary Go syntax, but it* 
> *is stylized, repetitive, and looks weird.*It is unclear how to represent 
> operators using interface methods. We considered syntaxes like +(T, T) T, 
> but that is confusing and repetitive. Also, a minor point, but ==(T, T) 
> bool does not correspond to the == operator, which returns an untyped 
> boolean value, not bool. We also considered writing simply + or ==. That 
> seems to work but unfortunately the semicolon insertion rules require 
> writing a semicolon after each operator at the end of a line. Using 
> contracts that look like functions gives us a familiar syntax at the cost 
> of some repetition. These are not fatal problems, but they are difficulties.

*https://go.googlesource.com/proposal/+/master/design/go2draft-contracts.md#why-not-use-interfaces-instead-of-contracts*
 

On Wednesday, October 10, 2018 at 5:14:35 PM UTC-4, Ian Denhardt wrote:
>
> I've seen a lot of folks expressing the sentiment "we should just use 
> interfaces instead of this new contract thing." Most of the discussion 
> I've seen around this jumps right to operator overloading without being 
> terribly explicit about the general case; I've written a proposal 
> focusing on the latter: 
>
> https://gist.github.com/zenhack/ad508d08c72fce6df945a49945ad826d 
>
> (It is also linked from the feedback wiki page). 
>
> It punts on operator overloading, though discusses some related 
> subtleties. 
>
> Thoughts? 
>
> -Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Golang Modules/vgo - local changes push, merge and tags

[Guy Brandwine]

We are trying to use Go-Modules



We have reusable repo : "github/mycompany/funcs" many repo's use this, 
example : "github/mycompany/eCom/basket.go"



A new ticket says : "basket should support export to xls" .

In order to do that, we want to : 

1. Add funcs.go a function "StructToXls(interface{})" 

2. basket.go should use funcs.StructToXls(myBasket)



in the current modules format we need to, push and TAG of funcs and then 
update github/mycompany/eCom dependencies.



If we have a small bug we would need to repeatedly push and tag funcs, 
update eCom etc.



Is there a better way (other than go.mod -> replace which is dangerous and 
"developer must remember to use it everytime he changes a basic repo") to 
work locally and when all done push and TAG the 2 repos ?

Moreover, in our current process (git flow style with some modifications) 
the developer only pushes the code to a branch, whereas the TL, approves 
and merges the code.

If the developer tags the library (funcs) locally and does not push that 
(theoretically its not his duty neither he is permitted to do that), that 
may create a false dependency, moreover, two different developer may do 
diff changes in funcs both changing ver from 1.2.3 => 1.2.4 since the 
increment is not centralized (which BTW needs a better tooling anyhow than 
manually inc.).



A better suggested flow would be greatly appreciated.



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: A concrete proposal re: using interfaces instead of contracts in Go 2 generics.

[gary . willoughby]

Add it to this: https://github.com/golang/go/wiki/Go2GenericsFeedback

On Wednesday, 10 October 2018 22:14:35 UTC+1, Ian Denhardt wrote:
>
> I've seen a lot of folks expressing the sentiment "we should just use 
> interfaces instead of this new contract thing." Most of the discussion 
> I've seen around this jumps right to operator overloading without being 
> terribly explicit about the general case; I've written a proposal 
> focusing on the latter: 
>
> https://gist.github.com/zenhack/ad508d08c72fce6df945a49945ad826d 
>
> (It is also linked from the feedback wiki page). 
>
> It punts on operator overloading, though discusses some related 
> subtleties. 
>
> Thoughts? 
>
> -Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [Proposal] Goroutine Scoped Context

[alex . besogonov]

Historically Go developers resist any attempts to add goroutine-local 
storage. Unless they change their opinion, this proposal is DOA.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[Nathan Davies]

Ok, that's an interesting line of investigation - I'll take a look into
that and let you know if I find anything.

Thanks
Nathan

On Thu, 11 Oct 2018 at 13:04, Robert Engels  wrote:

> It is not necessarily the open doc that is freeing the memory, could be
> the close doc as well - meaning it needs the file and opt reference while
> it is open. Probably not file, but possible opt. The other thing are you
> sure that the opt doesn’t contain a reference that is freed to early...
>
> On Oct 11, 2018, at 6:57 AM, ndav...@turnitin.com wrote:
>
> Hi,
>
> That was at least a working theory there for a while; that perhaps the
> open document was doing some freeing of the memory before it returned. The
> reason we discounted that was some docs the defer func() free works fine.
> It could be a weird bug in that library, and I'm still considering that
> option :) What moved us away from that is that when we had the defer func()
> approach a certain set of docs failed every time with this issue. Move to
> the code we have now and all those docs work as expected.
>
> It's a really odd one. I'm ok with the code as it is now. I just would
> like to understand some more. Thanks for taking a look and for asking
> questions. Please keep doing so, you never know we may get to the bottom of
> this.
>
> Thanks
> Nathan
>
>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
Nathan Davies
Senior Software Engineer

Turnitin
36 Gallowgate, Wellbar Central, Newcastle upon Tyne, NE1 4TD, UK
Direct: +44 (0) 191 681-0334 <0191%20681%200235> (internally Ext: 2344)
Office: +44 (0) 191 681-0200
Email: ndav...@turnitin.com
Web: www.turnitin.com

Turnitin is a Registered product of Turnitin UK, Ltd which is a wholly
owned subsidiary of Turnitin, LLC. Turnitin UK, Ltd is a company registered
in England and Wales with company number   07321841.



DISCLAIMER: This e-mail and any files transmitted with it are confidential
and intended solely for use by the recipient to which it is addressed. If
this email has been misdirected in error please contact us on+44 (0)
191 681 0200. This e-mail and any attachments have been scanned for viruses
prior to sending. No liability will be accepted for any loss incurred as a
result of any viruses being passed on. Any form of unauthorised publication
or distribution is strictly prohibited.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[Robert Engels]

It is not necessarily the open doc that is freeing the memory, could be the 
close doc as well - meaning it needs the file and opt reference while it is 
open. Probably not file, but possible opt. The other thing are you sure that 
the opt doesn’t contain a reference that is freed to early...

> On Oct 11, 2018, at 6:57 AM, ndav...@turnitin.com wrote:
> 
> Hi, 
> 
> That was at least a working theory there for a while; that perhaps the open 
> document was doing some freeing of the memory before it returned. The reason 
> we discounted that was some docs the defer func() free works fine. It could 
> be a weird bug in that library, and I'm still considering that option :) What 
> moved us away from that is that when we had the defer func() approach a 
> certain set of docs failed every time with this issue. Move to the code we 
> have now and all those docs work as expected.
> 
> It's a really odd one. I'm ok with the code as it is now. I just would like 
> to understand some more. Thanks for taking a look and for asking questions. 
> Please keep doing so, you never know we may get to the bottom of this.
> 
> Thanks
> Nathan
> 
>>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[ndavies]

Hi, 

That was at least a working theory there for a while; that perhaps the open 
document was doing some freeing of the memory before it returned. The 
reason we discounted that was some docs the defer func() free works fine. 
It could be a weird bug in that library, and I'm still considering that 
option :) What moved us away from that is that when we had the defer func() 
approach a certain set of docs failed every time with this issue. Move to 
the code we have now and all those docs work as expected.

It's a really odd one. I'm ok with the code as it is now. I just would like 
to understand some more. Thanks for taking a look and for asking questions. 
Please keep doing so, you never know we may get to the bottom of this.

Thanks
Nathan


>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[Robert Engels]

Are you sure the document is not doing the free of the file and the opt when it 
is destroyed? It might need those references beyond the function scope. 

> On Oct 11, 2018, at 3:52 AM, ndav...@turnitin.com wrote:
> 
> 
> package main
> 
> /*
> #cgo CFLAGS: -g -Wall -I/usr/local/include
> #cgo LDFLAGS: -L/usr/local/lib -lm -lstdc++
> #include 
> #include 
> #include 
> #include 
> 
> // tet_pcos_get_string wraps the TET_pcos_get_string so we can use it in Go, 
> cgo does not support
> // variadic args, this is the solution
> char* str_add(const char *s1, const char *s2) {
> char *result = malloc(strlen(s1) + strlen(s2) + 1); // +1 for the 
> null-terminator
> // in real code you would check for errors in malloc here
> strcpy(result, s1);
> strcat(result, s2);
> return result;
> }
> */
> import "C"
> import (
> "fmt"
> "unsafe"
> )
> 
> func  main() {
> str := doTheCStuff()
> fmt.Println(str)
> }
> 
> func doTheCStuff() string{
> s1 := C.CString("string one")
> //defer C.free(unsafe.Pointer(file))
> s2 := C.CString("string 2")
> //defer C.free(unsafe.Pointer(opt))
> 
> defer func(){
> C.free(unsafe.Pointer(s1))
> C.free(unsafe.Pointer(s2))
> }()
> 
> s3 := C.GoString(C.str_add(s1, s2))
> return s3
> }
> 
> This is the sort of thing, this will run just fine and no re-produce the 
> error. The exact use case in production where I see this involves using a 3rd 
> party C library that opens a document. This then, sometimes, results in the 
> error. The trouble is, using the 3rd party lib requires a license key so I'm 
> not sure how much I can post on here.
> 
> The actual snippet from prod that occasionally causes this issue is:
> // openDoc returns the document handle for the metadata extraction
> func (e *Extractor) openDoc(fp string) (*doc, error) {
> file := C.CString(fp)
> defer C.free(unsafe.Pointer(file))
> opt := C.CString("shrug checkglyphlists=true decompose={none} 
> tetml={elements={docinfo=true}} fold={{[U+0640] preserve} {[:Private_Use:] 
> unknownchar} {[:space:] remove} {[U+0009] remove} {[U+000D] remove} 
> {[:Control:] remove} {[:Unassigned:] preserve} {[U+] remove}} 
> unknownchar=U+003F allowjpeg2000=true")
> defer C.free(unsafe.Pointer(opt))
> 
> // For documentation see page 175
> docNum := C.TET_open_document(e.tet, file, 0, opt)
> if docNum < 0 {
> return nil, fmt.Errorf("failed to open document")
> }
> doc := &doc{
> num: docNum,
> tet: e.tet, // added for convenience
> }
> return doc, nil
> }
> 
> 
> For this to work you need to have an instance ofTET and a license key. I 
> can't share that, for obvious reason. But What you can see in this code is 
> that I now have the more common pattern for create and free. Yet when I use 
> the defer closure pattern here, after creating both file and opt, I sometimes 
> get the double free. We have have noticed that the issue was reproducible 
> with a specific set of files. Using this pattern fixes that issue, I'm just 
> curious as to why the defer would cause trouble.
> 
>> On Thursday, October 11, 2018 at 9:23:23 AM UTC+1, Jan Mercl wrote:
>> 
>> On Thu, Oct 11, 2018 at 10:15 AM  wrote:
>> 
>> > What I am wondering is why the closure approach does not work?
>> 
>> ISTM it should work. Please post a full, standalone reproduction code here 
>> or at the issue tracker, thanks.
>> 
>> 
>> -- 
>> -j
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Is it safe to run go build in parallel?

[Yulrizka]

Ok, to give some update apparently our Makefile uses `build -i` where it 
tries to install package dependency. 
This explains the conflicted `.a` files.

It was there before GOCACHE to speed up `go build`

Thanks for the help!


On Wednesday, October 10, 2018 at 9:40:28 PM UTC+2, Yulrizka wrote:
>
> It seems that after searching in google, I found couple of similar issue:
>
> most of them are related to parallel build with "GNU parallel" which is in 
> our case as well.
>
> https://github.com/gravitational/teleport/issues/1708
> https://github.com/grpc/grpc-go/issues/368
> https://github.com/vmware/vic/issues/5230
>
> This is even before GO 1.11 or go mod.
>
> And one thing, I noticed that this is not about GOCACHE, looks like 
> package installed binaries example:
>
> /var/lib/jenkins/workspace/project/src/go/pkg/linux_
> amd64_netgo/project/thrift/gen/denormalized.a
>
> while in our jenkins, `go env` says: 
> ...
> GOCACHE="/var/lib/jenkins/.cache/go-build"
> ...
>
>
>
> On Monday, October 1, 2018 at 7:56:46 PM UTC+2, Mark Rushakoff wrote:
>>
>> I saw that you wrote
>>
>> > The reason that I'm using shared GOPATH for this Jenkins step/job is 
>> that to now download the packages in `pkg/mod`.
>>
>> And I thought that meant you were using modules.
>>
>> Discussion in https://github.com/golang/go/issues/26677 says that the 
>> build cache is explicitly safe, but if you have a reproducer that shows 
>> otherwise, that's probably worthy of its own issue.
>>
>> On Mon, Oct 1, 2018 at 10:50 AM Yulrizka  wrote:
>>
>>> Hi Mark, 
>>>
>>> what I find interesting is that I haven't enable go module yet. 
>>> As you can see in the errors that I'm still using a vendor directory
>>>
>>> On Monday, October 1, 2018 at 6:27:57 PM UTC+2, Mark Rushakoff wrote:

 https://github.com/golang/go/issues/26794 is "can't run go builds 
 concurrently if they download modules".

 In that issue, Russ says:
 > There is a plan to make downloading of modules by parallel go 
 commands safe but we haven't done that yet.

 On Monday, October 1, 2018 at 8:43:00 AM UTC-7, Yulrizka wrote:
>
> Hi,
>
> Recently, I played around with our Jenkins pipeline. In the new 
> set-up, I build multiple packages in parallel. This step executes "go 
> build" for each of the services that we have at the same time.
> One thing I also change is that for each service, it uses the same 
> GOPATH.
>
> Sometimes, I see failures like this
>
> src/go/pkg/linux_amd64_netgo/project/vendor/
> github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg.a" 
> already exists and is not an object file
>
> go build project/vendor/github.com/prometheus/common/model: build 
> output 
> "/var/lib/jenkins/workspace/project/src/go/pkg/linux_amd64_netgo/project/vendor/
> github.com/prometheus/common/model.a" already exists and is not an 
> object file
>
> go build project/vendor/github.com/struCoder/pidusage: build output 
> "/var/lib/jenkins/workspace/project/src/go/pkg/linux_amd64_netgo/project/vendor/
> github.com/struCoder/pidusage.a" already exists and is not an object 
> file
>
> go build project/thrift/gen/denormalized: build output 
> "/var/lib/jenkins/workspace/project/src/go/pkg/linux_amd64_netgo/project/thrift/gen/denormalized.a"
>  
> already exists and is not an object file
>
>
> My current guess is that this is because running multiple go build at 
> the same time, but I'm not sure if this is actually the case.
>
> If it is the case, what is the recommended way to build multiple 
> packages when you have multiple Jenkins step/job that build the binary at 
> the same time?
>
> The reason that I'm using shared GOPATH for this Jenkins step/job is 
> that to now download the packages in `pkg/mod`.
>
>
> Cheers!
>
 -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "golang-nuts" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/golang-nuts/Pj7BVrrBSuw/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to 
>>> golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[ndavies]

package main

/*
#cgo CFLAGS: -g -Wall -I/usr/local/include
#cgo LDFLAGS: -L/usr/local/lib -lm -lstdc++
#include 
#include 
#include 
#include 

// tet_pcos_get_string wraps the TET_pcos_get_string so we can use it in 
Go, cgo does not support
// variadic args, this is the solution
char* str_add(const char *s1, const char *s2) {
   char *result = malloc(strlen(s1) + strlen(s2) + 1); // +1 for the 
null-terminator
   // in real code you would check for errors in malloc here
   strcpy(result, s1);
   strcat(result, s2);
   return result;
}
*/
import "C"
import (
   "fmt"
   "unsafe"
)

func  main() {
   str := doTheCStuff()
   fmt.Println(str)
}

func doTheCStuff() string{
   s1 := C.CString("string one")
   //defer C.free(unsafe.Pointer(file))
   s2 := C.CString("string 2")
   //defer C.free(unsafe.Pointer(opt))

defer func(){
   C.free(unsafe.Pointer(s1))
   C.free(unsafe.Pointer(s2))
   }()
   
   s3 := C.GoString(C.str_add(s1, s2))
   return s3
}

This is the sort of thing, this will run just fine and no re-produce the 
error. The exact use case in production where I see this involves using a 
3rd party C library that opens a document. This then, sometimes, results in 
the error. The trouble is, using the 3rd party lib requires a license key 
so I'm not sure how much I can post on here.

The actual snippet from prod that occasionally causes this issue is:
// openDoc returns the document handle for the metadata extraction
func (e *Extractor) openDoc(fp string) (*doc, error) {
file := C.CString(fp)
defer C.free(unsafe.Pointer(file))
opt := C.CString("shrug checkglyphlists=true decompose={none} 
tetml={elements={docinfo=true}} fold={{[U+0640] preserve} {[:Private_Use:] 
unknownchar} {[:space:] remove} {[U+0009] remove} {[U+000D] remove} 
{[:Control:] remove} {[:Unassigned:] preserve} {[U+] remove}} 
unknownchar=U+003F allowjpeg2000=true")
defer C.free(unsafe.Pointer(opt))

// For documentation see page 175
docNum := C.TET_open_document(e.tet, file, 0, opt)
if docNum < 0 {
return nil, fmt.Errorf("failed to open document")
}
doc := &doc{
num: docNum,
tet: e.tet, // added for convenience
}
return doc, nil
}


For this to work you need to have an instance ofTET and a license key. I 
can't share that, for obvious reason. But What you can see in this code is 
that I now have the more common pattern for create and free. Yet when I use 
the defer closure pattern here, after creating both file and opt, I 
sometimes get the double free. We have have noticed that the issue was 
reproducible with a specific set of files. Using this pattern fixes that 
issue, I'm just curious as to why the defer would cause trouble.

On Thursday, October 11, 2018 at 9:23:23 AM UTC+1, Jan Mercl wrote:
>
>
> On Thu, Oct 11, 2018 at 10:15 AM > 
> wrote:
>
> > What I am wondering is why the closure approach does not work?
>
> ISTM it should work. Please post a full, standalone reproduction code here 
> or at the issue tracker, thanks.
>
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Using a defer func() closure to call c.free

[Jan Mercl]

On Thu, Oct 11, 2018 at 10:15 AM  wrote:

> What I am wondering is why the closure approach does not work?

ISTM it should work. Please post a full, standalone reproduction code here
or at the issue tracker, thanks.


-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Using a defer func() closure to call c.free

[ndavies]

Hi all, 

Apologies if this has already been asked and answered elsewhere, I took a 
look and couldn't find anything.

I have a simple cgo program that looks a little something like:

func doTheCGo(){
f := C.CString("/path/to/file.ext")
o := C.CString("useful values for c call")
defer func(){
C.free(unsafe.Pointer(f))
C.free(unsafe.Pointer(o))
}()
// call a C function with f and o; this function does NOT mutate this 
objects
}

This results in a double free and then a stack underflow exception.

I have tried the code such that I do defer C.free(unsafe.Pointer(f) 
immediately after f is created, and the same for o. That works. I have 
tried freeing right at the end of my function, with no defer, and that 
works too. 

I understand that the convention is:
f := C.CString("/path/to/file.ext")
defer C.free(unsafe.Pointer(f))

I have now moved the code to that pattern. 

I have tried printing the addresses in a closure and they are the same as 
they are on creation, and the same as they are immediately prior to return.

What I am wondering is why the closure approach does not work?

Thanks
Nathan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.