[go-nuts] Encrypting credentials config file in production and pseudo key rotation

[Ivan Buljan]

Hello World

This relates to that never ending question of securing the credentials in 
production/staging envs, that is, avoiding storing them as plain text

I am wondering if anyone could share their thoughts about the following 
approach we are thinking of taking.

Here we go:

During build phase, an encryption key is generated and credentials are 
encrypted with it.

Once deployed, the instance decrypts credentials with the provided key and 
does what it needs with them. Just before destroying the original files 
(creds & key), the instance then generates a new encryption key and 
re-encrypts a copy of credentials, which it keeps in memory. Newly 
encrypted credentials along with the key are only dumped onto a filesystem 
if the application panics and requires to be restarted, at which point the 
same cycle key rotation decryption/encryption happens again.

Is any security benefit with such approach?


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1340d748-5284-493d-9c94-7b5844ba1a62n%40googlegroups.com.

Re: [go-nuts] gofmt 1.19 reformats comments in a way which breaks go-swagger

[Ivan Porto Carrero]

I think that indenting everything should work, we are detecting the yaml 
marker, collect all the lines in the comment until we see a thing that 
starts with swagger: or the comment block ends and then proceed to 
uncomment and remove the indentation based on the first line after the --- 
marker.

The branch I'm working in was reformatted by go 1.19 fmt and still passes 
the tests but I don't think that our test cases capture all the real-world 
use cases.

On Wednesday, August 17, 2022 at 8:33:08 PM UTC-7 crod...@gmail.com wrote:

> I think the parsing rules for go-swagger are described here:
>
> https://goswagger.io/use/spec.html#parsing-rules
>
> It looks like this issue has affected other codebases such as Cockroachdb:
>
> https://github.com/cockroachdb/cockroach/pull/86049#issuecomment-1213573808
>
> —
> Craig
>
>
> On Wednesday, August 17, 2022 at 7:20:11 PM UTC-7 Ian Lance Taylor wrote:
>
>> What are the constraints on go-swagger? Can one simply indent 
>> everything to get a code block? 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/04c316e1-7b37-4732-8e34-67cb8a673400n%40googlegroups.com.

[go-nuts] gollvm: meaningless "unsupported llvm::CallingConv::ID"

[Ivan Serdyuk]

Hi.

I caught the following error:
https://go.googlesource.com/gollvm/+/refs/heads/master/bridge/go-llvm-cabi-oracle.cpp#162
.

Among other things: people has nothing to do with an ID. There is a 
specific list of options:
https://llvm.org/doxygen/namespacellvm_1_1CallingConv.html
.
But providing a human friendly name *with* an ID could have some sense.

Another problem is that this is an intersection with the class, dedicated 
to a specific struct alignment. I have doubts that all (at least - user 
mode, not kernel mode ) calling conventions would assume all possible 
options, for aligning structs/unions. Also there are OS specific issues - 
so it more about OS-convention-alignment triple.

I would propose some re-design, to offer a purpose specific output, 
depending on developer's goals. So the output would help to navigate 
against the next step.

Ivan

Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2ddf5cde-c66d-4c9c-b317-fc113335a1d8n%40googlegroups.com.

Re: [go-nuts] gollvm: Issues on Alpine linux. Tentative definition has type 'struct sigstack' that is never completed, in sysinfo.c

[Ivan Serdyuk]

A patch could be submitted

On Wednesday, November 18, 2020 at 4:27:20 AM UTC+2, Ian Lance Taylor wrote:
>
> On Tue, Nov 17, 2020 at 6:11 AM Than McIntosh  > wrote: 
> > 
> > > 
> /home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/sysinfo.c:462:1:
>  
> error: tentative definition has type 'struct sigstack' that is never 
> completed 
> > > SREF(sigstack); 
> > > ^ 
> > 
> > I don't know why sysinfo.c has the line "SREF(sigstack)".  What 
> > happens if you just remove it? 
> > 
> > I agree, I would take it out and see what happens. 
> > 
> > The SREF(...) constructs in sysinfo.c are there to generate explicit 
> references to types mentioned in the C code from system header files, so as 
> to insure that DWARF gets generated for them (in case the C++ compiler 
> being used is clang, which does not support 
> -fno-eliminate-unused-debug-types).  Looks like on Alpine (which is 
> musl-based) there is no such type.   Not clear whether it is actually 
> needed at this point, so it is worth trying without it. 
>
> As far as I can tell we never use the sigstack type in libgo, so my 
> guess is that we don't need it.  The sigstack type (really struct 
> sigstack) was used for the old sigstack function, but that function 
> has been replaced by sigaltstack. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/93011e85-8777-48b7-86c7-10b953bdc3d5o%40googlegroups.com.

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

CC'ing to Jayesh, Sreenivas, Adarsh from Intel.
I am BCC'ing to Sameeran, Tejas from AMD.


On Thu, Jan 7, 2021 at 12:21 AM Ivan Serdyuk 
wrote:

> I assume that this should be enforced by CMake - so some build flag would
> be provided. So the "default" schema would be to cover all per-project
> architectures (not the llc derived ones).
>
> So here is what was generated:
>
> $ cat ../driver/ArchCpusAttrs.h
>> // DO NOT EDIT: this file auto-generated by the following command:
>> //
>> //./capture-fcn-attributes -o ../driver/ArchCpusAttrs.h -triples
>> i686-pc-linux-gnu
>> //
>> // in combination with clang:
>> //
>> //  clang version 12.0.0
>> //
>>
>> typedef struct {
>>   const char *cpu;
>>   const char *attrs;
>> } CpuAttrs;
>>
>> typedef struct {
>>   const char *triple;
>>   const CpuAttrs *cpuattrs;
>> } TripleCpus;
>>
>> // triple: i686-pc-linux-gnu
>> static const CpuAttrs attrs0[] = {
>>   // first entry is default cpu
>>   *{ "pentium4", "+cx8,+fxsr,+mmx,+sse,+sse2,+x87"}*,
>>   { "alderlake",
>> "+adx,+aes,+avx,+avx2,+avxvnni,+bmi,+bmi2,+cldemote,+clflushopt,+cx16,+cx8,+f16c,+fma,+fsgsbase,+fxsr,+hreset,+invpcid,+lzcnt,+mmx,+movbe,+pclmul,+popcnt,+prfchw,+ptwrite,+rdrnd,+rdseed,+sahf,+serialize,+sgx,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+waitpkg,+x87,+xsave,+xsavec,+xsaveopt,+xsaves"},
>>   { "amdfam10",
>> "+3dnow,+3dnowa,+cx16,+cx8,+fxsr,+lzcnt,+mmx,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4a,+x87"},
>>   { "athlon", "+3dnow,+3dnowa,+cx8,+mmx,+x87"},
>>   { "athlon-4", "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+x87"},
>>   { "athlon-fx", "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+sse2,+x87"},
>>   { "athlon-mp", "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+x87"},
>>   { "athlon-tbird", "+3dnow,+3dnowa,+cx8,+mmx,+x87"},
>>   { "athlon-xp", "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+x87"},
>>   { "athlon64", "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+sse2,+x87"},
>>   { "athlon64-sse3",
>> "+3dnow,+3dnowa,+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
>>   { "atom",
>> "+cx16,+cx8,+fxsr,+mmx,+movbe,+sahf,+sse,+sse2,+sse3,+ssse3,+x87"},
>>   { "barcelona",
>> "+3dnow,+3dnowa,+cx16,+cx8,+fxsr,+lzcnt,+mmx,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4a,+x87"},
>>   { "bdver1",
>> "+aes,+avx,+cx16,+cx8,+fma4,+fxsr,+lwp,+lzcnt,+mmx,+pclmul,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+x87,+xop,+xsave"},
>>   { "bdver2",
>> "+aes,+avx,+bmi,+cx16,+cx8,+f16c,+fma,+fma4,+fxsr,+lwp,+lzcnt,+mmx,+pclmul,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+tbm,+x87,+xop,+xsave"},
>>   { "bdver3",
>> "+aes,+avx,+bmi,+cx16,+cx8,+f16c,+fma,+fma4,+fsgsbase,+fxsr,+lwp,+lzcnt,+mmx,+pclmul,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+tbm,+x87,+xop,+xsave,+xsaveopt"},
>>   { "bdver4",
>> "+aes,+avx,+avx2,+bmi,+bmi2,+cx16,+cx8,+f16c,+fma,+fma4,+fsgsbase,+fxsr,+lwp,+lzcnt,+mmx,+movbe,+mwaitx,+pclmul,+popcnt,+prfchw,+rdrnd,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+tbm,+x87,+xop,+xsave,+xsaveopt"},
>>   { "bonnell",
>> "+cx16,+cx8,+fxsr,+mmx,+movbe,+sahf,+sse,+sse2,+sse3,+ssse3,+x87"},
>>   { "broadwell",
>> "+adx,+avx,+avx2,+bmi,+bmi2,+cx16,+cx8,+f16c,+fma,+fsgsbase,+fxsr,+invpcid,+lzcnt,+mmx,+movbe,+pclmul,+popcnt,+prfchw,+rdrnd,+rdseed,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsaveopt"},
>>   { "btver1",
>> "+cx16,+cx8,+fxsr,+lzcnt,+mmx,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4a,+ssse3,+x87"},
>>   { "btver2",
>> "+aes,+avx,+bmi,+cx16,+cx8,+f16c,+fxsr,+lzcnt,+mmx,+movbe,+pclmul,+popcnt,+prfchw,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+x87,+xsave,+xsaveopt"},
>>   { "c3", "+3dnow,+cx8,+mmx,+x87"},
>>   { "c3-2", "+cx8,+fxsr,+mmx,+sse,+x87"},
>>   { "cannonlake",
>> "+adx,+aes,+avx,+avx2,+avx512bw,+avx512cd,+avx512dq,+avx512f,+avx512ifma,+avx512vbmi,+avx512vl,+bmi,+bmi2,+clflushopt,+cx16,+cx8,+f16c,+fma,+fsgsbase,+fxsr,+invpcid,+lzcnt,+mmx,+movbe,+pclmul,+pku,+popcnt,+prfchw,+rdrnd,+rdseed,+sahf,+sgx,+sha,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsavec,+xsaveopt,+xsaves"},
>>   { "cascadelake",
>> "+adx,+aes,+avx,+avx2,+avx512bw,+avx512

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

mwaitx,+pclmul,+pku,+popcnt,+prfchw,+rdpid,+rdrnd,+rdseed,+sahf,+sha,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+vaes,+vpclmulqdq,+wbnoinvd,+x87,+xsave,+xsavec,+xsaveopt,+xsaves"},
>   { "", "" } // sentinel
> };
>
> const TripleCpus triples[] = {
>   { "i686-pc-linux-gnu", [0] },
>   { "", nullptr } // sentinel
> };
>
.
So the value was set to "pentium4", while

> $ llc --version | grep CPU
>   Host CPU: yonah
>
. It is even more confusing.

Now, I made further steps:

> [3/3] Linking CXX executable bin/llvm-goc
> $ ./bin/llvm-goc --version
> gollvm version 1 (experimental) [LLVM version 12]
>

and

$ ninja libgo_golang.org_x_crypto_chacha20
> [1/120] Building Go package 'unicode' (non-PIC)
> FAILED: tools/gollvm/libgo/unicode.o
> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo &&
> /usr/local/bin/cmake -E make_directory ./. &&
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/./unicode.o
> -fgo-pkgpath=unicode -I .
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/casetables.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/digit.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/graphic.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/letter.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/tables.go
>
.
I do not see any proof that


> currently Gollvm is not supported on architecture i686
> unsupported llvm::CallingConv::ID 1023
> unsupported llvm::CallingConv::ID 1023
> unsupported llvm::CallingConv::ID 1023
>

is beyond the ranks of the current issue.

#0 0x087a548a llvm::sys::PrintStackTrace(llvm::raw_ostream&, int)
> (/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc+0x87a548a)
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the
> crash backtrace.
> Stack dump:
> 0. Program arguments:
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/./unicode.o
> -fgo-pkgpath=unicode -I .
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/casetables.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/digit.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/graphic.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/letter.go
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/tables.go
> Segmentation fault (core dumped)
> [2/120] Building Go package 'internal/unsafeheader' (PIC)
> FAILED: tools/gollvm/libgo/internal/.pic/unsafeheader.o
> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo &&
> /usr/local/bin/cmake -E make_directory ./internal/.pic &&
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/internal/.pic/unsafeheader.o
> -fPIC -fgo-pkgpath=internal/unsafeheader -I .
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/internal/unsafeheader/unsafeheader.go
> currently Gollvm is not supported on architecture i686
> unsupported llvm::CallingConv::ID 1023
> unsupported llvm::CallingConv::ID 1023
> unsupported llvm::CallingConv::ID 1023
> #0 0x087a548a llvm::sys::PrintStackTrace(llvm::raw_ostream&, int)
> (/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc+0x87a548a)
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the
> crash backtrace.
> Stack dump:
> 0. Program arguments:
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/internal/.pic/unsafeheader.o
> -fPIC -fgo-pkgpath=internal/unsafeheader -I .
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/internal/unsafeheader/unsafeheader.go
> Segmentation fault (core dumped)
>
.

So I don't see what is the next step (and what to do with "pentium4" versus
"yonah").
I am super sure that some patch is required, to provide consistent
automation of build.


On Wed, Jan 6, 2021 at 11:40 PM Than McIntosh  wrote:

>
> You can of course run capture-fcn-attributes with an explicit set of
> triples, e.g.
>
>  $ ./capture-fcn-attributes -o ../driver/ArchCpusAttrs.h -triples
> i686-pc-linux-gnu
>
> whic

Re: [go-nuts] Generics and formatting

[Ivan Majeru]

If I understand right the panic is because the `m := NewMap[string,
int]` is a function not a function call and after that you try to
assign to this function `m[int]`. This
works https://go2goplay.golang.org/p/NOaPH-dJyWd .
On Tue, 2021-01-05 at 08:10 -0800, Brian Candler wrote:
> I don't know if this is expected, but this (bad) program causes the
> go2go playground to give "panic: assertion failed"
> 
> https://go2goplay.golang.org/p/VH5SLJNxw3X
> 
> - I think it's the compiler itself that's panicing, not the runtime
> (since it doesn't get as far as the next line).
> 
> 
> 
> Aside: my general feeling so far is that the error messages given
> from go programs which misapply generic syntax can be very
> unhelpful.  Here's one example:
> https://go2goplay.golang.org/p/m9YUDBTWwpA
> 
> Error given: "missing ',' in type argument list"
> Actual problem: a constraint was used in a place where it's not
> allowed
> -- 
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/aa0acea2-7e52-457b-b22f-92f0e19bf922n%40googlegroups.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3c1ea2f3aa5d9483a876fcc01b131ae3c438fff5.camel%40gmail.com.

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

My current scenario assumes that I have compiled for x86 targets, to run
Clang on an x86 machine. What should be done if it is not assumed to
involve any other back-ends, during the build?


On Wed, Jan 6, 2021 at 10:27 PM Than McIntosh  wrote:

> Right, that would be expected.   The easiest thing to do is make sure that
> you're running cmake in your build area with
>
>-DLLVM_TARGETS_TO_BUILD="all"
>
> then you can build "llc" and "clang" , then update your path so that the
> freshly built llc/clang are picked up first. Once you have that set up, it
> should take care of the problem running "llc" from capture-fcn-attributes.
>
> Than
>
>
> On Wed, Jan 6, 2021 at 3:14 PM Ivan Serdyuk 
> wrote:
>
>> I think a caught an error:
>>
>> $ ./capture-fcn-attributes
>>> llc run failed: llc: error: : error: unable to get target for
>>> 'aarch64-unknown-linux-gnu', see --version and --triple.
>>>
>>> capture-fcn-attributes: err = exit status 1
>>>
>>
>> . Hence that I didn't compile Aarch64 back-end, for LLVM:
>>
>> $ llc --version
>>> LLVM (http://llvm.org/):
>>>   LLVM version 12.0.0git
>>>   Optimized build.
>>>   Default target: i686-pc-linux-gnu
>>>   Host CPU: yonah
>>>
>>>   Registered Targets:
>>> x86- 32-bit X86: Pentium-Pro and above
>>> x86-64 - 64-bit X86: EM64T and AMD64
>>>
>> . So this would not run out-of-the-box.
>>
>>
>>
>> On Wed, Jan 6, 2021 at 8:21 PM Than McIntosh  wrote:
>>
>>> Thanks. The error you're seeing
>>>
>>>   "unable to determine target CPU features for target i686-pc-linux-gnu"
>>>
>>> looks as though it's because i686-pc-linux-gnu isn't listed as a target
>>> in
>>> driver/ArchCpusAttrs.h.
>>>
>>> From the root of your repo, try
>>>
>>>   $ cd tools
>>>   $ go build capture-fcn-attributes.go
>>>   $ ./capture-fcn-attributes -o ../driver/ArchCpusAttrs.h
>>>
>>> and then give the gollvm build another try.
>>>
>>> Thanks, Than
>>>
>>> On Wed, Jan 6, 2021 at 12:53 PM Ivan Serdyuk <
>>> local.tourist.k...@gmail.com> wrote:
>>>
>>>> Here you go:
>>>>
>>>> ~/Desktop/workarea/llvm-project/llvm/tools/gollvm$ git diff
>>>> diff --git a/cmake/modules/AutoGenGo.cmake
>>>> b/cmake/modules/AutoGenGo.cmake
>>>> index 3e3ab83..097a2ad 100644
>>>> --- a/cmake/modules/AutoGenGo.cmake
>>>> +++ b/cmake/modules/AutoGenGo.cmake
>>>> @@ -51,6 +51,17 @@ function(mkversion goos goarch outfile bindir
>>>> srcroot scriptroot)
>>>>  set(pcquantum "1")
>>>>  set(int64align "8")
>>>>  set(minframesize 0)
>>>> +  elseif( ${goarch} STREQUAL "386")
>>>> +# Simply picking up one typical setting
>>>> +# Align with current sets in gofrontend/libgo/goarch.sh
>>>> +set(archfamily "I386")
>>>> +set(bigendian "false")
>>>> +set(cachelinesize "64")
>>>> +set(physpagesize "4096")
>>>> +set(pcquantum "1")
>>>> +set(int64align "4")
>>>> +set(minframesize 4)
>>>> +set(ptrsize 4)
>>>>elseif( ${goarch} STREQUAL "arm64")
>>>>  # Simply picking up one typical setting
>>>>  # Align with current sets in gofrontend/libgo/goarch.sh
>>>> diff --git a/cmake/modules/GoVars.cmake b/cmake/modules/GoVars.cmake
>>>> index ec6f6b2..0cea38a 100644
>>>> --- a/cmake/modules/GoVars.cmake
>>>> +++ b/cmake/modules/GoVars.cmake
>>>> @@ -8,6 +8,9 @@ list(GET lht_components 2 goos)
>>>>  # LLVM's "x86_64" is the same as Go's "amd64".
>>>>  if( ${llarch} STREQUAL "x86_64" )
>>>>set(goarch "amd64")
>>>> +# For i386/i686
>>>> +elseif( ${llarch} STREQUAL "i686" )
>>>> +  set(goarch "386")
>>>>  # LLVM's "aarch64" is the same as Go's "arm64".
>>>>  elseif( ${llarch} STREQUAL "aarch64" )
>>>>set(goarch "arm64")
>>>> @@ -23,6 +26,10 @@ set(allgoos "aix" "android" "darwin" "dragonfly"
>>>> "freebsd" "irix" &q

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

I think a caught an error:

$ ./capture-fcn-attributes
> llc run failed: llc: error: : error: unable to get target for
> 'aarch64-unknown-linux-gnu', see --version and --triple.
>
> capture-fcn-attributes: err = exit status 1
>

. Hence that I didn't compile Aarch64 back-end, for LLVM:

$ llc --version
> LLVM (http://llvm.org/):
>   LLVM version 12.0.0git
>   Optimized build.
>   Default target: i686-pc-linux-gnu
>   Host CPU: yonah
>
>   Registered Targets:
> x86- 32-bit X86: Pentium-Pro and above
> x86-64 - 64-bit X86: EM64T and AMD64
>
. So this would not run out-of-the-box.



On Wed, Jan 6, 2021 at 8:21 PM Than McIntosh  wrote:

> Thanks. The error you're seeing
>
>   "unable to determine target CPU features for target i686-pc-linux-gnu"
>
> looks as though it's because i686-pc-linux-gnu isn't listed as a target in
> driver/ArchCpusAttrs.h.
>
> From the root of your repo, try
>
>   $ cd tools
>   $ go build capture-fcn-attributes.go
>   $ ./capture-fcn-attributes -o ../driver/ArchCpusAttrs.h
>
> and then give the gollvm build another try.
>
> Thanks, Than
>
> On Wed, Jan 6, 2021 at 12:53 PM Ivan Serdyuk 
> wrote:
>
>> Here you go:
>>
>> ~/Desktop/workarea/llvm-project/llvm/tools/gollvm$ git diff
>> diff --git a/cmake/modules/AutoGenGo.cmake b/cmake/modules/AutoGenGo.cmake
>> index 3e3ab83..097a2ad 100644
>> --- a/cmake/modules/AutoGenGo.cmake
>> +++ b/cmake/modules/AutoGenGo.cmake
>> @@ -51,6 +51,17 @@ function(mkversion goos goarch outfile bindir srcroot
>> scriptroot)
>>  set(pcquantum "1")
>>  set(int64align "8")
>>  set(minframesize 0)
>> +  elseif( ${goarch} STREQUAL "386")
>> +# Simply picking up one typical setting
>> +# Align with current sets in gofrontend/libgo/goarch.sh
>> +set(archfamily "I386")
>> +set(bigendian "false")
>> +set(cachelinesize "64")
>> +set(physpagesize "4096")
>> +set(pcquantum "1")
>> +set(int64align "4")
>> +set(minframesize 4)
>> +set(ptrsize 4)
>>elseif( ${goarch} STREQUAL "arm64")
>>  # Simply picking up one typical setting
>>  # Align with current sets in gofrontend/libgo/goarch.sh
>> diff --git a/cmake/modules/GoVars.cmake b/cmake/modules/GoVars.cmake
>> index ec6f6b2..0cea38a 100644
>> --- a/cmake/modules/GoVars.cmake
>> +++ b/cmake/modules/GoVars.cmake
>> @@ -8,6 +8,9 @@ list(GET lht_components 2 goos)
>>  # LLVM's "x86_64" is the same as Go's "amd64".
>>  if( ${llarch} STREQUAL "x86_64" )
>>set(goarch "amd64")
>> +# For i386/i686
>> +elseif( ${llarch} STREQUAL "i686" )
>> +  set(goarch "386")
>>  # LLVM's "aarch64" is the same as Go's "arm64".
>>  elseif( ${llarch} STREQUAL "aarch64" )
>>set(goarch "arm64")
>> @@ -23,6 +26,10 @@ set(allgoos "aix" "android" "darwin" "dragonfly"
>> "freebsd" "irix" "linux" "netbs
>>  # Set library suffix based on target triple
>>  if( ${llarch} STREQUAL "x86_64" )
>>set(library_suffix "64")
>> +elseif( ${llarch} STREQUAL "i686" )
>> +# Driver::installedLibDir honors ./lib only, on Ubuntu 16
>> +# But we can stick for ./lib32 (along with those in AddGollvm.cmake)
>> +  set(library_suffix "32")
>>  elseif( ${llarch} STREQUAL "aarch64" )
>>  # Driver::installedLibDir honors ./lib64 only
>>  # Future change needed (along with those in AddGollvm.cmake)
>> diff --git a/cmake/modules/LibbacktraceUtils.cmake
>> b/cmake/modules/LibbacktraceUtils.cmake
>> index dc54f18..42287a9 100644
>> --- a/cmake/modules/LibbacktraceUtils.cmake
>> +++ b/cmake/modules/LibbacktraceUtils.cmake
>> @@ -8,6 +8,9 @@ function(setup_libbacktrace)
>>if( ${goarch} STREQUAL "amd64")
>>  set(BACKTRACE_ELF_SIZE 64)
>>  set(HAVE_GETIPINFO 1)
>> +  elseif( ${goarch} STREQUAL "386")
>> +set(BACKTRACE_ELF_SIZE 32)
>> +set(HAVE_GETIPINFO 1)
>>elseif( ${goarch} STREQUAL "arm64")
>>  set(BACKTRACE_ELF_SIZE 64)
>>  set(HAVE_GETIPINFO 1)
>> diff --git a/cmake/modules/LibffiUtils.cmake
>> b/cmake/modules/LibffiUtils.cmake
>> index b3fa697..fb34c36 100644
>> --- a/cmake/modules/LibffiUtils.cmake
>> --- a/cmake/modules/LibffiUtils.cmake
>> +++ b/cmake/modules/LibffiU

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

Here you go:

~/Desktop/workarea/llvm-project/llvm/tools/gollvm$ git diff
diff --git a/cmake/modules/AutoGenGo.cmake b/cmake/modules/AutoGenGo.cmake
index 3e3ab83..097a2ad 100644
--- a/cmake/modules/AutoGenGo.cmake
+++ b/cmake/modules/AutoGenGo.cmake
@@ -51,6 +51,17 @@ function(mkversion goos goarch outfile bindir srcroot 
scriptroot)
 set(pcquantum "1")
 set(int64align "8")
 set(minframesize 0)
+  elseif( ${goarch} STREQUAL "386")
+# Simply picking up one typical setting
+# Align with current sets in gofrontend/libgo/goarch.sh
+set(archfamily "I386")
+set(bigendian "false")
+set(cachelinesize "64")
+set(physpagesize "4096")
+set(pcquantum "1")
+set(int64align "4")
+set(minframesize 4)
+set(ptrsize 4)
   elseif( ${goarch} STREQUAL "arm64")
 # Simply picking up one typical setting
 # Align with current sets in gofrontend/libgo/goarch.sh
diff --git a/cmake/modules/GoVars.cmake b/cmake/modules/GoVars.cmake
index ec6f6b2..0cea38a 100644
--- a/cmake/modules/GoVars.cmake
+++ b/cmake/modules/GoVars.cmake
@@ -8,6 +8,9 @@ list(GET lht_components 2 goos)
 # LLVM's "x86_64" is the same as Go's "amd64".
 if( ${llarch} STREQUAL "x86_64" )
   set(goarch "amd64")
+# For i386/i686
+elseif( ${llarch} STREQUAL "i686" )
+  set(goarch "386")
 # LLVM's "aarch64" is the same as Go's "arm64".
 elseif( ${llarch} STREQUAL "aarch64" )
   set(goarch "arm64")
@@ -23,6 +26,10 @@ set(allgoos "aix" "android" "darwin" "dragonfly" 
"freebsd" "irix" "linux" "netbs
 # Set library suffix based on target triple
 if( ${llarch} STREQUAL "x86_64" )
   set(library_suffix "64")
+elseif( ${llarch} STREQUAL "i686" )
+# Driver::installedLibDir honors ./lib only, on Ubuntu 16
+# But we can stick for ./lib32 (along with those in AddGollvm.cmake)
+  set(library_suffix "32")
 elseif( ${llarch} STREQUAL "aarch64" )
 # Driver::installedLibDir honors ./lib64 only
 # Future change needed (along with those in AddGollvm.cmake)
diff --git a/cmake/modules/LibbacktraceUtils.cmake 
b/cmake/modules/LibbacktraceUtils.cmake
index dc54f18..42287a9 100644
--- a/cmake/modules/LibbacktraceUtils.cmake
+++ b/cmake/modules/LibbacktraceUtils.cmake
@@ -8,6 +8,9 @@ function(setup_libbacktrace)
   if( ${goarch} STREQUAL "amd64")
 set(BACKTRACE_ELF_SIZE 64)
 set(HAVE_GETIPINFO 1)
+  elseif( ${goarch} STREQUAL "386")
+set(BACKTRACE_ELF_SIZE 32)
+set(HAVE_GETIPINFO 1)
   elseif( ${goarch} STREQUAL "arm64")
 set(BACKTRACE_ELF_SIZE 64)
 set(HAVE_GETIPINFO 1)
diff --git a/cmake/modules/LibffiUtils.cmake 
b/cmake/modules/LibffiUtils.cmake
index b3fa697..fb34c36 100644
--- a/cmake/modules/LibffiUtils.cmake
--- a/cmake/modules/LibffiUtils.cmake
+++ b/cmake/modules/LibffiUtils.cmake
@@ -9,6 +9,8 @@ function(setup_libffi libffi_srcroot)
 set(arch_dir "aarch64")
   elseif(${llarch} STREQUAL "x86_64")
 set(arch_dir "x86")
+  elseif(${llarch} STREQUAL "i686")
+set(arch_dir "x86")
   else()
 message(SEND_ERROR "Arch ${llarch} not yet supported")
   endif()
diff --git a/driver/Driver.cpp b/driver/Driver.cpp
index 8debbab..aa83fb1 100644
--- a/driver/Driver.cpp
+++ b/driver/Driver.cpp
@@ -76,7 +76,7 @@ std::string Driver::installedLibDir()
   llvm::sys::path::append(ldir, "../lib64");
   break;
 default:
-  llvm::sys::path::append(ldir, "../lib64");
+  llvm::sys::path::append(ldir, "../lib");
   break;
   }
:
--- a/cmake/modules/LibffiUtils.cmake
+++ b/cmake/modules/LibffiUtils.cmake
@@ -9,6 +9,8 @@ function(setup_libffi libffi_srcroot)
 set(arch_dir "aarch64")
   elseif(${llarch} STREQUAL "x86_64")
 set(arch_dir "x86")
+  elseif(${llarch} STREQUAL "i686")
+set(arch_dir "x86")
   else()
 message(SEND_ERROR "Arch ${llarch} not yet supported")
   endif()
diff --git a/driver/Driver.cpp b/driver/Driver.cpp
index 8debbab..aa83fb1 100644
--- a/driver/Driver.cpp
+++ b/driver/Driver.cpp
@@ -76,7 +76,7 @@ std::string Driver::installedLibDir()
   llvm::sys::path::append(ldir, "../lib64");
   break;
 default:
-  llvm::sys::path::append(ldir, "../lib64");
+  llvm::sys::path::append(ldir, "../lib");
   break;
   }
   return std::string(ldir.str());

and

$ git log -1
commit 850255c5cd7f9df79a148d537fd395089b7caf29 (HEAD -> master, 
origin/master, origin/HEAD)
Author: Ivan Serdyuk 
Date:   Fri Dec 4 17:14:00 2020 +0200

gollvm: Updating README.md, to elevate the min. required version of 
Clang compiler

T

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

Than,
I could perform "git diff" and provide the generated file.
I could also compress & share the build folder.
What else would be required?

Ivan

On Wednesday, January 6, 2021 at 5:07:06 PM UTC+2 th...@google.com wrote:

>
> It would be helpful if you could put up a CL that includes all of the 
> changes you're testing -- at this point do you have any modifications to 
> driver/ArchCpusAttrs.h ?
>
> Thanks
>
>
>
> On Wed, Jan 6, 2021 at 10:00 AM Ivan Serdyuk  
> wrote:
>
>> Ping
>>
>> On Friday, January 1, 2021 at 5:45:02 AM UTC+2 Ivan Serdyuk wrote:
>>
>>> Happy New Year, Than.
>>>
>>> So I have rebuilt llvm-goc, after applying 
>>> https://go-review.googlesource.com/c/gollvm/+/270219.
>>> Here 
>>> <https://drive.google.com/file/d/1qDSKwKORZjH824gVeSXUknVm0rpKZ9j-/view?usp=sharing>
>>>  
>>> is my compressed build folder.
>>>
>>> I am using 
>>> $ clang --version
>>> clang version 12.0.0
>>> Target: i686-pc-linux-gnu
>>> Thread model: posix
>>> on
>>> $ uname -a
>>> Linux oceanfish81-A8He 4.15.0-129-generic #132-Ubuntu SMP Thu Dec 10 
>>> 14:07:35 UTC 2020 i686 i686 i686 GNU/Linux
>>>
>>> You can obtain a pre-compiler "MinSizeRel" build of LLVM (including 
>>> Clang) here 
>>> <https://drive.google.com/file/d/1c7wredQbaX4p20Ee15WnY3SblbjiQHMD/view?usp=sharing>
>>> .
>>>
>>> I tried to build an avarage libgo package - and here is what I got:
>>>
>>> $ ninja libgo_golang.org_x_crypto_chacha20
>>> [1/120] Building Go package 'unicode' (non-PIC)
>>> FAILED: tools/gollvm/libgo/unicode.o 
>>> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
>>> /usr/local/bin/cmake -E make_directory ./. && 
>>> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
>>> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/./unicode.o 
>>> -fgo-pkgpath=unicode -I . 
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/casetables.go
>>>  
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/digit.go
>>>  
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/graphic.go
>>>  
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/letter.go
>>>  
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/tables.go
>>> currently Gollvm is not supported on architecture i686
>>> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
>>> determine target CPU features for target i686-pc-linux-gnu*
>>> [2/120] Building Go package 'internal/unsafeheader' (PIC)
>>> FAILED: tools/gollvm/libgo/internal/.pic/unsafeheader.o 
>>> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
>>> /usr/local/bin/cmake -E make_directory ./internal/.pic && 
>>> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
>>> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/internal/.pic/unsafeheader.o
>>>  
>>> -fPIC -fgo-pkgpath=internal/unsafeheader -I . 
>>> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/internal/unsafeheader/unsafeheader.go
>>> currently Gollvm is not supported on architecture i686
>>> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
>>> determine target CPU features for target i686-pc-linux-gnu*
>>> ninja: build stopped: subcommand failed.
>>>
>>> Ivan
>>> On Tuesday, December 8, 2020 at 3:30:05 PM UTC+2 th...@google.com wrote:
>>>
>>>> Hello,
>>>>
>>>> Thanks for the note.
>>>>
>>>> I am still not completely sure what the problem is.
>>>>
>>>>
>>>> You wrote:
>>>>
>>>>  | I found
>>>>  | 
>>>>  | // triple: i686-pc-linux-gnu
>>>>  | static const CpuAttrs attrs1[] = {
>>>>  | // first entry is default cpu
>>>>  | { "i686", "+cx8,+x87"},
>>>>  | 
>>>>  | and (inside the hashmap)
>>>>  | 
>>>>  | { "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
>>>>  | , which is not what I have supported (for Intel Celeron M440).

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

Ping

On Friday, January 1, 2021 at 5:45:02 AM UTC+2 Ivan Serdyuk wrote:

> Happy New Year, Than.
>
> So I have rebuilt llvm-goc, after applying 
> https://go-review.googlesource.com/c/gollvm/+/270219.
> Here 
> <https://drive.google.com/file/d/1qDSKwKORZjH824gVeSXUknVm0rpKZ9j-/view?usp=sharing>
>  
> is my compressed build folder.
>
> I am using 
> $ clang --version
> clang version 12.0.0
> Target: i686-pc-linux-gnu
> Thread model: posix
> on
> $ uname -a
> Linux oceanfish81-A8He 4.15.0-129-generic #132-Ubuntu SMP Thu Dec 10 
> 14:07:35 UTC 2020 i686 i686 i686 GNU/Linux
>
> You can obtain a pre-compiler "MinSizeRel" build of LLVM (including Clang) 
> here 
> <https://drive.google.com/file/d/1c7wredQbaX4p20Ee15WnY3SblbjiQHMD/view?usp=sharing>
> .
>
> I tried to build an avarage libgo package - and here is what I got:
>
> $ ninja libgo_golang.org_x_crypto_chacha20
> [1/120] Building Go package 'unicode' (non-PIC)
> FAILED: tools/gollvm/libgo/unicode.o 
> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
> /usr/local/bin/cmake -E make_directory ./. && 
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/./unicode.o 
> -fgo-pkgpath=unicode -I . 
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/casetables.go
>  
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/digit.go
>  
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/graphic.go
>  
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/letter.go
>  
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/tables.go
> currently Gollvm is not supported on architecture i686
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
> determine target CPU features for target i686-pc-linux-gnu*
> [2/120] Building Go package 'internal/unsafeheader' (PIC)
> FAILED: tools/gollvm/libgo/internal/.pic/unsafeheader.o 
> cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
> /usr/local/bin/cmake -E make_directory ./internal/.pic && 
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
> /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/internal/.pic/unsafeheader.o
>  
> -fPIC -fgo-pkgpath=internal/unsafeheader -I . 
> /home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/internal/unsafeheader/unsafeheader.go
> currently Gollvm is not supported on architecture i686
> /home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
> determine target CPU features for target i686-pc-linux-gnu*
> ninja: build stopped: subcommand failed.
>
> Ivan
> On Tuesday, December 8, 2020 at 3:30:05 PM UTC+2 th...@google.com wrote:
>
>> Hello,
>>
>> Thanks for the note.
>>
>> I am still not completely sure what the problem is.
>>
>>
>> You wrote:
>>
>>  | I found
>>  | 
>>  | // triple: i686-pc-linux-gnu
>>  | static const CpuAttrs attrs1[] = {
>>  | // first entry is default cpu
>>  | { "i686", "+cx8,+x87"},
>>  | 
>>  | and (inside the hashmap)
>>  | 
>>  | { "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
>>  | , which is not what I have supported (for Intel Celeron M440).
>>
>> What makes you say that this is not what you have supported? Are you
>> saying that the cpu clang calls "yonah" doesn't actually have one of these
>> features (ex: sse3)?
>>
>>
>>  | Clang reports "unsupported CPU features" on any non-provided one.
>>  | So that is one big problem.
>>
>> Not sure what you mean here. Can you please post the complete clang 
>> invocation and error message you are getting?
>>
>>
>>
>>  | const TripleCpus triples[] = {
>>  | { "x86_64-unknown-linux-gnu", [0] },
>>  | { "i686-pc-linux-gnu", [0] },
>>  | { "aarch64-unknown-linux-gnu", [0] },
>>  | { "", nullptr } // sentinel
>>  | };
>>  | is not targeting to yonah, while llc is targeting it.
>>  | It is always some "default" CPU model and, in fact, your code never 
>> provided extraction of the CPU model (from llc).
>>  | 
>>
>> When llc emits the line
>>
>>   Host CPU: yonah
>>
>> this does not mean that clang/llc will automatically target 'yonah' when 
>> compilin

Re: [go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

Happy New Year, Than.

So I have rebuilt llvm-goc, after applying 
https://go-review.googlesource.com/c/gollvm/+/270219 <http://this patch>.
Here 
<https://drive.google.com/file/d/1qDSKwKORZjH824gVeSXUknVm0rpKZ9j-/view?usp=sharing>
 
is my compressed build folder.

I am using 
$ clang --version
clang version 12.0.0
Target: i686-pc-linux-gnu
Thread model: posix
on
$ uname -a
Linux oceanfish81-A8He 4.15.0-129-generic #132-Ubuntu SMP Thu Dec 10 
14:07:35 UTC 2020 i686 i686 i686 GNU/Linux

You can obtain a pre-compiler "MinSizeRel" build of LLVM (including Clang) 
here 
<https://drive.google.com/file/d/1c7wredQbaX4p20Ee15WnY3SblbjiQHMD/view?usp=sharing>
.

I tried to build an avarage libgo package - and here is what I got:

$ ninja libgo_golang.org_x_crypto_chacha20
[1/120] Building Go package 'unicode' (non-PIC)
FAILED: tools/gollvm/libgo/unicode.o 
cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
/usr/local/bin/cmake -E make_directory ./. && 
/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
/home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/./unicode.o 
-fgo-pkgpath=unicode -I . 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/casetables.go
 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/digit.go
 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/graphic.go
 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/letter.go
 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/unicode/tables.go
currently Gollvm is not supported on architecture i686
/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
determine target CPU features for target i686-pc-linux-gnu*
[2/120] Building Go package 'internal/unsafeheader' (PIC)
FAILED: tools/gollvm/libgo/internal/.pic/unsafeheader.o 
cd /home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo && 
/usr/local/bin/cmake -E make_directory ./internal/.pic && 
/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc -c -o 
/home/oceanfish81/Desktop/workarea/release/tools/gollvm/libgo/internal/.pic/unsafeheader.o
 
-fPIC -fgo-pkgpath=internal/unsafeheader -I . 
/home/oceanfish81/Desktop/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/internal/unsafeheader/unsafeheader.go
currently Gollvm is not supported on architecture i686
/home/oceanfish81/Desktop/workarea/release/./bin/llvm-goc: *unable to 
determine target CPU features for target i686-pc-linux-gnu*
ninja: build stopped: subcommand failed.

Ivan
On Tuesday, December 8, 2020 at 3:30:05 PM UTC+2 th...@google.com wrote:

> Hello,
>
> Thanks for the note.
>
> I am still not completely sure what the problem is.
>
>
> You wrote:
>
>  | I found
>  | 
>  | // triple: i686-pc-linux-gnu
>  | static const CpuAttrs attrs1[] = {
>  | // first entry is default cpu
>  | { "i686", "+cx8,+x87"},
>  | 
>  | and (inside the hashmap)
>  | 
>  | { "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
>  | , which is not what I have supported (for Intel Celeron M440).
>
> What makes you say that this is not what you have supported? Are you
> saying that the cpu clang calls "yonah" doesn't actually have one of these
> features (ex: sse3)?
>
>
>  | Clang reports "unsupported CPU features" on any non-provided one.
>  | So that is one big problem.
>
> Not sure what you mean here. Can you please post the complete clang 
> invocation and error message you are getting?
>
>
>
>  | const TripleCpus triples[] = {
>  | { "x86_64-unknown-linux-gnu", [0] },
>  | { "i686-pc-linux-gnu", [0] },
>  | { "aarch64-unknown-linux-gnu", [0] },
>  | { "", nullptr } // sentinel
>  | };
>  | is not targeting to yonah, while llc is targeting it.
>  | It is always some "default" CPU model and, in fact, your code never 
> provided extraction of the CPU model (from llc).
>  | 
>
> When llc emits the line
>
>   Host CPU: yonah
>
> this does not mean that clang/llc will automatically target 'yonah' when 
> compiling, it just means that the program has detected the host CPU.
>
> Generally speaking if you want clang to produce code targeted specifically 
> to the host CPU, you have to use -march=native or -mtune-native.
>
> Thanks, Than
>
> On Fri, Dec 4, 2020 at 8:08 PM Ivan Serdyuk  
> wrote:
>
>> Hello.
>>
>> This issue is related to 
>> https://go-review.googlesource.com/c/gollvm/+/274574
>> .
>>
>> I think I have some misunderstanding on how you used to deal with CPU 
>> models, for LLVM.
&g

[go-nuts] gollvm: capture-fcn-attributes.go generates the header file with improperly chosen CPU model (i686; yonah)

[Ivan Serdyuk]

Hello.

This issue is related to 
https://go-review.googlesource.com/c/gollvm/+/274574
.

I think I have some misunderstanding on how you used to deal with CPU 
models, for LLVM.

First things first - I had success with using

#include "llvm/ADT/StringRef.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/MC/SubtargetFeature.h"
#include "llvm/Support/Host.h"

using namespace llvm;
SubtargetFeatures Features1;

int main (int argc, char **argv)
{
sys::getHostCPUName();
StringMap HostFeatures;
if (sys::getHostCPUFeatures(HostFeatures))
for (auto  : HostFeatures)
Features1.AddFeature(F.first(), F.second);

printf("test %s", Features1.getString().c_str());
printf("\nsomething else\n");
return 0;
}
. It gives me such a set of CPU features:

+sse2,-tsxldtrk,-cx16,-sahf,-tbm,-avx512ifma,-sha,-gfni,-fma4,-vpclmulqdq,-prfchw,-bmi2,-cldemote,-fsgsbase,-ptwrite,-amx-tile,-avx512bf16,-popcnt,-aes,-avx512bitalg,-movdiri,-xsaves,-avx512er,-xsavec,-avx512vnni,-amx-bf16,-avx512vpopcntdq,-pconfig,-clwb,-avx512f,-clzero,-pku,+mmx,-lwp,-rdpid,-xop,-rdseed,-waitpkg,-movdir64b,-sse4a,-avx512bw,-clflushopt,-xsave,-avx512vbmi2,-64bit,-avx512vl,-serialize,-invpcid,-avx512cd,-avx,-vaes,+cx8,-fma,-rtm,-bmi,-enqcmd,-rdrnd,-mwaitx,-sse4.1,-sse4.2,-avx2,+fxsr,-wbnoinvd,+sse,-lzcnt,-pclmul,-prefetchwt1,-f16c,-ssse3,-sgx,-shstk,+cmov,-avx512vbmi,-amx-int8,-movbe,-avx512vp2intersect,-xsaveopt,-avx512dq,-adx,-avx512pf,+sse3

$ llc --version
provides
Default target: i686-pc-linux-gnu
Host CPU: yonah
.

I tried to update the capture-fcn-attributes.go file, like this:

var supportedTriples []string = []string{
"x86_64-unknown-linux-gnu",
"i686-pc-linux-gnu",
"aarch64-unknown-linux-gnu",
}
.

When I tried the generator

capture-fcn-attributes -o /tmp/cpu_feature_list
it generated me a broad list.
The header contained

Ubuntu clang version 11.0.0-++20200721055954+cebd637c886-1exp1
20200721161335.13
.
I found

// triple: i686-pc-linux-gnu
static const CpuAttrs attrs1[] = {
// first entry is default cpu
{ "i686", "+cx8,+x87"},

and (inside the hashmap)

{ "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
, which is not what I have supported (for Intel Celeron M440).
Clang reports "unsupported CPU features" on any non-provided one.
So that is one big problem.
Next problem is that

const TripleCpus triples[] = {
{ "x86_64-unknown-linux-gnu", [0] },
{ "i686-pc-linux-gnu", [0] },
{ "aarch64-unknown-linux-gnu", [0] },
{ "", nullptr } // sentinel
};
is not targeting to yonah, while llc is targeting it.
It is always some "default" CPU model and, in fact, your code never 
provided extraction of the CPU model (from llc).

To make my observation complete - I am providing what is generated via

capture-fcn-attributes -cpu yonah
:

// triple: x86_64-unknown-linux-gnu
static const CpuAttrs attrs0[] = {
// first entry is default cpu
{ "x86-64", "+cx8,+fxsr,+mmx,+sse,+sse2,+x87"},
{ "", "" } // sentinel
};

// triple: i686-pc-linux-gnu
static const CpuAttrs attrs1[] = {
// first entry is default cpu
{ "i686", "+cx8,+x87"},
{ "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
{ "", "" } // sentinel
};

// triple: aarch64-unknown-linux-gnu
static const CpuAttrs attrs2[] = {
// first entry is default cpu
{ "generic", "+neon"},
{ "", "" } // sentinel
};

const TripleCpus triples[] = {
{ "x86_64-unknown-linux-gnu", [0] },
{ "i686-pc-linux-gnu", [0] },
{ "aarch64-unknown-linux-gnu", [0] },
{ "", nullptr } // sentinel
};
.

I tried

capture-fcn-attributes -cpu yonah -triples i686-pc-linux-gnu
and got

// triple: i686-pc-linux-gnu
static const CpuAttrs attrs0[] = {
// first entry is default cpu
{ "i686", "+cx8,+x87"},
{ "yonah", "+cx8,+fxsr,+mmx,+sse,+sse2,+sse3,+x87"},
{ "", "" } // sentinel
};

const TripleCpus triples[] = {
{ "i686-pc-linux-gnu", [0] },
{ "", nullptr } // sentinel
};
.

I understand that your strategy worked find on Intel based system-on-board 
machines - but didn't try something for AMD (yet).
Nevertheless I have these issues on i686 - so I am proposing to perform a 
review.


Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/aac8c576-9763-4bba-96a1-51f545084630n%40googlegroups.com.

Re: [go-nuts] Estimating blockers for supporting gollvm on RISC-V (qemu). Fedora 33

[Ivan Serdyuk]

Jim,
adding you to the discussion.

I was not able to build gold, by configuring binutils, using

> ./configure --enable-gold
>
did not allow GNU make tool to build gold: many sub-directories were
processed, to compile the sources/generate files - but not the sub-dir. for
gold.
When I entered the "gold" dir. and ran "configure" script - it failed with
an error, which reported an unsupported arch. , under qemu.

Is there any way to compile gold?
Perhaps I have improper identification of the arch.?
Perhaps there is a working Makefile, for this (until "configure" would be
patched properly)?

What about cross-compiling (Linux x86_64 -> Linux 64bit RISC-V ) ?


 Ivan

On Tue, Nov 24, 2020 at 4:20 PM Ian Lance Taylor  wrote:

> On Tue, Nov 24, 2020 at 6:14 AM Richard W.M. Jones 
> wrote:
> >
> > One thing I'm missing: Why does golang need gold?
>
> The gccgo and GoLLVM compilers (which are not the most widely used Go
> compilers) require the gold linker for full support of stack
> splitting.  Stack splitting is used to ensure that goroutines do not
> run out of stack space.  Stack splitting is supported by gold and lld,
> but, as far as I know, is not supported by the GNU linker.
>
> However, the bulk of stack splitting support is in the compiler,   At
> least for GCC, stack splitting is only supported on x86, PPC, and
> S/390.  So at least until stack splitting support is added for RISC/V,
> there is no particular reason to require the gold linker for RISC/V.
>
> (It does seem a shame that people are thinking of deprecating gold,
> but it's certainly true that I have not had time to maintain it.)
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANFCsz9EdJWL87oX%2BeLrzBbK5m%3DFzwVs6Z8RQ55feJ34v%2BH42Q%40mail.gmail.com.

Re: [go-nuts] Estimating blockers for supporting gollvm on RISC-V (qemu). Fedora 33

[Ivan Serdyuk]

Richard,
could you tell why gold linker is not available on RISC-V port of Fedora 33?


On Sat, Nov 21, 2020 at 12:10 PM Ivan Serdyuk 
wrote:

> Wei, Carlos: adding you to this discussion.
>
> Carlos, please ask your questions. What was not understood, regarding my
> attempt?
>
> Hence that QEMU's host OS could be, say, Fedora 32+ x86_64.
>
> Ivan
>
> On Sat, Nov 21, 2020 at 5:36 AM Ivan Serdyuk 
> wrote:
>
>> Hello.
>>
>> It is just a brief survey - but I am currently dealing with a messy
>> setup, for qemu.
>>
>> I have upgraded, after bootstrapping( the manual
>> <https://fedoraproject.org/wiki/Architectures/RISC-V/Installing> ) and I
>> am able to boot
>>
>> > Fedora (5.8.0-1.0.riscv64.fc33.riscv64) 33 (Rawhide)
>>
>> kernel, under qemu.
>>
>> I see a very abstract
>>
>> > $ cat /proc/cpuinfo
>> > processor : 0
>> > hart : 0
>> > isa : rv64imafdcsu
>> > mmu : sv48
>> .
>>
>> Doesn't really help.
>> I will suggest trying other emulators - but would have to compile those
>> myself.
>>
>> I have some concerns about gmp, mpfr and mpc (since RISC-V assumes some
>> extensions, to cover such demands) - I feel that it might have more sense
>> to get rid of overlaying, for the end-user use cases (which gollvm project
>> itself achieves).
>>
>>
>> Ivan
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/2c9788e3-5349-4f5e-bbfc-53f300cf439bn%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/2c9788e3-5349-4f5e-bbfc-53f300cf439bn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANFCsz9FzZzET7rWqQyH%2B%2Bt3kDhzQ6H_gZU6S14Ha1Y13451Yg%40mail.gmail.com.

Re: [go-nuts] Estimating blockers for supporting gollvm on RISC-V (qemu). Fedora 33

[Ivan Serdyuk]

Wei, Carlos: adding you to this discussion.

Carlos, please ask your questions. What was not understood, regarding my
attempt?

Hence that QEMU's host OS could be, say, Fedora 32+ x86_64.

Ivan

On Sat, Nov 21, 2020 at 5:36 AM Ivan Serdyuk 
wrote:

> Hello.
>
> It is just a brief survey - but I am currently dealing with a messy setup,
> for qemu.
>
> I have upgraded, after bootstrapping( the manual
> <https://fedoraproject.org/wiki/Architectures/RISC-V/Installing> ) and I
> am able to boot
>
> > Fedora (5.8.0-1.0.riscv64.fc33.riscv64) 33 (Rawhide)
>
> kernel, under qemu.
>
> I see a very abstract
>
> > $ cat /proc/cpuinfo
> > processor : 0
> > hart : 0
> > isa : rv64imafdcsu
> > mmu : sv48
> .
>
> Doesn't really help.
> I will suggest trying other emulators - but would have to compile those
> myself.
>
> I have some concerns about gmp, mpfr and mpc (since RISC-V assumes some
> extensions, to cover such demands) - I feel that it might have more sense
> to get rid of overlaying, for the end-user use cases (which gollvm project
> itself achieves).
>
>
> Ivan
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2c9788e3-5349-4f5e-bbfc-53f300cf439bn%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/2c9788e3-5349-4f5e-bbfc-53f300cf439bn%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANFCsz_PxE7Oy%3Dzb%3DrF-2ZsyVspggA%2B928NiE5QkVfLb5t97fg%40mail.gmail.com.

[go-nuts] Estimating blockers for supporting gollvm on RISC-V (qemu). Fedora 33

[Ivan Serdyuk]

Hello.

It is just a brief survey - but I am currently dealing with a messy setup, 
for qemu.

I have upgraded, after bootstrapping( the manual 
<https://fedoraproject.org/wiki/Architectures/RISC-V/Installing> ) and I am 
able to boot

> Fedora (5.8.0-1.0.riscv64.fc33.riscv64) 33 (Rawhide)

kernel, under qemu.

I see a very abstract

> $ cat /proc/cpuinfo
> processor : 0
> hart : 0
> isa : rv64imafdcsu
> mmu : sv48
. 

Doesn't really help.
I will suggest trying other emulators - but would have to compile those 
myself.

I have some concerns about gmp, mpfr and mpc (since RISC-V assumes some 
extensions, to cover such demands) - I feel that it might have more sense 
to get rid of overlaying, for the end-user use cases (which gollvm project 
itself achieves).


Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2c9788e3-5349-4f5e-bbfc-53f300cf439bn%40googlegroups.com.

[go-nuts] gollvm: Issues on Alpine linux. Tentative definition has type 'struct sigstack' that is never completed, in sysinfo.c

[Ivan Serdyuk]

Hello.

I was trying to check what is required to build gollvm on Alpine linux (it 
has musl instead of glibc).

Look like recently revealed  
set of minimal 
requirements allowed to build the configuration file (for Ninja).

heading to check whether Ninja would compile.
I wasn't able to find a pre-compiled Clang 11, for Alpine - but Clang 10 
was available, which was pre-compiled against musl (C++ standard library 
has to rely on musl, instead of glibc - as I suppose).

Maybe I would recompile Clang 11, for Alpine distro - but not today.

Must admit that CMake worked much faster, compared to Ubuntu.

Comments would be much appreciated.

So, I am stuck:

[1/1084] Building sysinfo.o
FAILED: tools/gollvm/libgo/sysinfo.o
cd /home/oceanfish81/workarea/build_release/tools/gollvm/libgo && 
/usr/bin/clang -g3 -c -fno-eliminate-unused-debug-types 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/sysinfo.c
 
-o /home/oceanfish81/workarea/build_release/tools/gollvm/libgo/sysinfo.o 
-D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 
-I/home/oceanfish81/workarea/build_release/tools/gollvm/libgo 
-I/home/oceanfish81/workarea/build_release/tools/gollvm/libgo/runtime
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/sysinfo.c:462:1:
 
error: tentative definition has type 'struct sigstack' that is never 
completed
SREF(sigstack);
^
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/sysinfo.c:374:32:
 
note: expanded from macro 'SREF'
#define SREF(stag) struct stag stag ## ref
^
:52:1: note: expanded from here
sigstackref
^
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/sysinfo.c:462:6:
 
note: forward declaration of 'struct sigstack'
SREF(sigstack);
^
1 error generated.
ninja: build stopped: subcommand failed.  

You can find my compressed build folder out here 

  

Hence using sigaltstack 

 ( description  ) 
versus sigstack 

 ( description 
 ).  

I have replaced the line - so now I am getting the following:

[1/1075] Building Go package 'runtime' (non-PIC)
FAILED: tools/gollvm/libgo/runtime.o
cd /home/oceanfish81/workarea/build_release/tools/gollvm/libgo && 
/usr/bin/cmake -E make_directory ./. && 
/home/oceanfish81/workarea/build_release/./bin/llvm-goc -c -o 
/home/oceanfish81/workarea/build_release/tools/gollvm/libgo/./runtime.o 
-fgo-pkgpath=runtime -fgo-c-header=runtime.inc.raw -fgo-compiling-runtime 
-I . 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/alg.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/atomic_pointer.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/cgo_gccgo.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/cgocall.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/cgocheck.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/chan.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/compiler.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/cpuprof.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/cputicks.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/debug.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/debuglog.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/debuglog_off.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/env_posix.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/eqtype.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/error.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/extern.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/fastlog2.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/fastlog2table.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/ffi.go
 
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/float.go
 

Re: [go-nuts] Build kubernetes with gollvm

[Ivan Serdyuk]

Brendan,
could you review this and express your vision against the usage of

https://godoc.org/k8s.io/kubernetes/third_party/forked/golang/reflect
<https://godoc.org/k8s.io/kubernetes/third_party/forked/golang/reflect>
https://golang.org/pkg/reflect/
https://github.com/modern-go/reflect2
packages?

What is required to patch, within the reflect2 package, so Kubernetes could
be compiled via gollvm?

Ivan

On Sat, Jul 20, 2019 at 5:32 AM Yuan Ting  wrote:

> OK, I will check out the reflect2 package and take a look. Thanks a lot
> for your direction!
>
> Best,
> Ting
>
> On Friday, July 19, 2019 at 10:30:32 PM UTC+8, Ian Lance Taylor wrote:
>>
>> On Fri, Jul 19, 2019 at 1:11 AM Yuan Ting  wrote:
>> >
>> >  Sorry to have troubled you again, I found that many famous Go projects
>> also import github.com/modern-go/reflect2 , such as kubernetes and moby.
>> According to the second paragraph in your explanation, these popular
>> projects will also failed in compilation. I think this may be a big
>> challenge for gccgo/gollvm's promotion.
>> >
>> > To solve this compiler-sensitive problem, the way I can come up with is
>> replacing all unsafe mechanisms as mentioned above in reflect2 to safe or
>> standard mechanisms (and may go against the purpose of reflect2). I'm not
>> familiar with runtime, but is it possible to support the same symbol in
>> gccgo/gollvm style?
>>
>> Simply supporting runtime.typelinks is not enough.  The reflect
>> package is closely tied to the compiler implementation.  The reflect
>> package is not identical in the gc library and the GoLLVM library.  If
>> people need reflect2 to work with GoLLVM, then the only workable
>> option is for the reflect2 authors to write a version of their package
>> that uses the gccgo build tag to build a version that works with
>> GoLLVM.  Perhaps you could help them with that.  Sorry this isn't very
>> helpful.  I don't really understand why people are using the package.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1fd95c73-6e30-4089-bad4-b21c261d788a%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/1fd95c73-6e30-4089-bad4-b21c261d788a%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANFCsz_Ropfv93%2B_g%3DsPCa_yzrQuQJSQcWTjvLbTB-GcMhBkyw%40mail.gmail.com.

Re: [go-nuts] [proposal] Make go compiler work with different syntax versions in same project to support adaptivity

[Ivan Ivanyuk]

Ok, thanks for the clarification.

On Mon, Aug 10, 2020 at 9:38 PM Ian Lance Taylor  wrote:

> On Mon, Aug 10, 2020 at 1:30 AM Ivan Ivanyuk 
> wrote:
> >
> > Thank you for answering
> >
> > Why does it take so long to implement generics then? There is already an
> instrument in playground that works fine. Why not just roll it out and
> improve design, if needed, in next version?
> >
> > Having generics in 2021 means many projects will choose other languages
> in 2020, which will effectively mean 1 year of work in other language
>
> Besides what other people have said, we believe, rightly or wrongly,
> that language stability and backward compatibility are important
> features for Go programmers.  We don't want to roll out a version of
> generics that we then have to change in some incompatible way.  That
> would force people to rewrite programs over time, which is painful,
> and will decrease people's perception of Go as a stable and reliable
> language.
>
> Ian
>
>
>
> > On Mon, Aug 10, 2020, 04:13 Ian Lance Taylor  wrote:
> >>
> >> On Sat, Aug 8, 2020 at 7:51 PM  wrote:
> >> >
> >> > Add support of optional syntax_version to file beginning, allow
> compiler to tokenize/compile file by the compiler of syntax_version, using
> any number of syntax_version-s in same project
> >> >
> >> > Why? Because go takes ridiculously long to implement generics, and
> implementation is very likely to not be perfect or to become not perfect
> when world changes. Having support of several syntaxes/compilers in same
> project will allow to adapt syntax faster
> >>
> >> We already support something like this on a per-module basis via the
> >> "go" line in a go.mod file
> >> (https://golang.org/cmd/go/#hdr-The_go_mod_file).  See also
> >>
> https://go.googlesource.com/proposal/+/refs/heads/master/design/28221-go2-transitions.md
> >> .
> >>
> >> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BXqAJxX_Xq6vjmmYZrwtEDmsSQLjg3sqB0kQ2VPbEdb_M0TfQ%40mail.gmail.com.

Re: [go-nuts] [proposal] Make go compiler work with different syntax versions in same project to support adaptivity

[Ivan Ivanyuk]

Thank you for answering

Why does it take so long to implement generics then? There is already an
instrument in playground that works fine. Why not just roll it out and
improve design, if needed, in next version?

Having generics in 2021 means many projects will choose other languages in
2020, which will effectively mean 1 year of work in other language

On Mon, Aug 10, 2020, 04:13 Ian Lance Taylor  wrote:

> On Sat, Aug 8, 2020 at 7:51 PM  wrote:
> >
> > Add support of optional syntax_version to file beginning, allow compiler
> to tokenize/compile file by the compiler of syntax_version, using any
> number of syntax_version-s in same project
> >
> > Why? Because go takes ridiculously long to implement generics, and
> implementation is very likely to not be perfect or to become not perfect
> when world changes. Having support of several syntaxes/compilers in same
> project will allow to adapt syntax faster
>
> We already support something like this on a per-module basis via the
> "go" line in a go.mod file
> (https://golang.org/cmd/go/#hdr-The_go_mod_file).  See also
>
> https://go.googlesource.com/proposal/+/refs/heads/master/design/28221-go2-transitions.md
> .
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BXqAJwADTgR31tg422dtn5vEYP%3DQUyzOzBtYs44mHCHZF0wSA%40mail.gmail.com.

Re: [go-nuts] Failed to build gollvm in a docker container

[Ivan Serdyuk]

Yuan,
do you have any progress here?

Ivan

On Thursday, May 28, 2020 at 6:47:41 AM UTC+3 Yuan Ting wrote:

> Thanks for your correction, overriding CMAKE_C_FLAGS and CMAKE_ASM_FLAGS 
> with "-fcf-protection=none" works for me.
> I also try to build gollvm from a clean slate (by recreating a build area 
> and rerun cmake without overriding CMAKE_C_FLAGS and CMAKE_ASM_FLAGS). 
> However, I still failed to build gollvm. I can even see the notice during 
> the configuration:
>
> -- trying -fcf-protection=none workaround
>
> -- Performing Test SPLIT_STACK_WORKAROUND
>
> -- Performing Test SPLIT_STACK_WORKAROUND - Success
>
> -- applying -fcf-protection=none workaround
>
> -- starting libgo configuration.
>
>
> but during building, the same error happened:
>
>
> [2427/2600] Linking C shared library tools/gollvm/libgo/libgo.so.11git
>
> FAILED: tools/gollvm/libgo/libgo.so.11git
>
> : && /usr/bin/cc -fPIC -fPIC -Werror=date-time -Wall -Wextra 
> -Wno-unused-parameter -Wwrite-strings -Wno-missing-field-initializers 
> -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-comment 
> -fdiagnostics-color -ffunction-sections -fdata-sections -O3  
> -Wl,-z,nodelete -fuse-ld=gold   -Wl,-O3 -Wl,--gc-sections -shared 
> -Wl,-soname,libgo.so.11git -o tools/gollvm/libgo/libgo.so.11git ...*(omits 
> all object files)...* -Wl,-rpath,"\$ORIGIN/../lib"  -lpthread  -lm  
> -fsplit-stack && :
>
> ... ...
>
> /usr/bin/ld.gold: error: 
> tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: 
> failed to match split-stack sequence at section 22 offset 0
>
> /usr/bin/ld.gold: error: 
> tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: 
> failed to match split-stack sequence at section 26 offset 0
>
> /usr/bin/ld.gold: error: 
> tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffiw64.c.o: 
> failed to match split-stack sequence at section 16 offset 0
>
> collect2: error: ld returned 1 exit status
>
> [2429/2600] Linking C static library tools/gollvm/libgo/libgo.a
>
> ninja: build stopped: subcommand failed.
>
> On Wednesday, May 27, 2020 at 10:36:23 PM UTC+8, Than McIntosh wrote:
>
>> >> I can successfully compile and link gollvm by setting 
>> CMAKE_C_FLAGS="-fcf-protection=none".
>>
>> The manual override of  CMAKE_C_FLAGS shouldn't be needed, if you rerun 
>> cmake from a clean slate (removing the CMakeCache.txt, etc) it should 
>> detect that "-fcf-protection=branch" is hard-wired on, and will add the 
>> right flags under the hood to work around it.
>>
>> >>root@a78010e8da5d:~# go version
>> >>go: symbol lookup error: 
>> /root/llvm-project/install-release/lib64/libgo.so.11git: undefined symbol: 
>> __go_getcontext
>>
>> This symbol is defined in an assembly source file in libgo. What's 
>> happening is that by setting CMAKE_C_FLAGS="-fcf-protection=none" but not 
>> also setting CMAKE_ASM_FLAGS="-fcf-protection=none" you've introduced a 
>> discrepancy between the C and Assembly compiles in libgo. If you 
>> set CMAKE_C_FLAGS, you should also set CMAKE_ASM_FLAGS.
>>
>> Thanks, Than
>>
>>
>> On Wed, May 27, 2020 at 10:02 AM Yuan Ting  wrote:
>>
> Thank you for your guidance, now I can successfully compile and link 
>>> gollvm by setting CMAKE_C_FLAGS="-fcf-protection=none". But after 
>>> installing gollvm, there is still a weird problem:
>>>
>>> root@a78010e8da5d:~# go version
>>> go: symbol lookup error: 
>>> /root/llvm-project/install-release/lib64/libgo.so.11git: undefined symbol: 
>>> __go_getcontext
>>>
>>> On Wednesday, May 27, 2020 at 12:01:06 AM UTC+8, Than McIntosh wrote:
>>>>
>>>>
>>>> This looks exactly like the bug in 
>>>> https://github.com/golang/go/issues/38728. The prolog of the function 
>>>> contains
>>>>
>>>>0: f3 0f 1e fa   endbr64
>>>>
>>>> which is confusing the gold linker. This instruction doesn't get added 
>>>> unless the host compiler has "-fcf-protection=branch" turned on.  I would 
>>>> suggest deleting and recreating your build area or removing your 
>>>> CMakeCache.txt file to insure that you have a rebuild from a clean start.
>>>>
>>>> Thanks, Than
>>>>
>>>>
>>>> On Tue, May 26, 2020 at 11:00 AM Yuan Ting  wrote:
>>>>
>>>>> OK, as follows
>>>>>
>>>>> ./tools/gollvm/libg

Re: [go-nuts] Build kubernetes with gollvm

[Ivan Serdyuk]

Yuan,
are you still working on gollvm based port of k8s?

Ivan

On Saturday, July 20, 2019 at 5:32:03 AM UTC+3 Yuan Ting wrote:

> OK, I will check out the reflect2 package and take a look. Thanks a lot 
> for your direction!
>
> Best,
> Ting
>
> On Friday, July 19, 2019 at 10:30:32 PM UTC+8, Ian Lance Taylor wrote:
>>
>> On Fri, Jul 19, 2019 at 1:11 AM Yuan Ting  wrote: 
>> > 
>> >  Sorry to have troubled you again, I found that many famous Go projects 
>> also import github.com/modern-go/reflect2 , such as kubernetes and moby. 
>> According to the second paragraph in your explanation, these popular 
>> projects will also failed in compilation. I think this may be a big 
>> challenge for gccgo/gollvm's promotion. 
>> > 
>> > To solve this compiler-sensitive problem, the way I can come up with is 
>> replacing all unsafe mechanisms as mentioned above in reflect2 to safe or 
>> standard mechanisms (and may go against the purpose of reflect2). I'm not 
>> familiar with runtime, but is it possible to support the same symbol in 
>> gccgo/gollvm style? 
>>
>> Simply supporting runtime.typelinks is not enough.  The reflect 
>> package is closely tied to the compiler implementation.  The reflect 
>> package is not identical in the gc library and the GoLLVM library.  If 
>> people need reflect2 to work with GoLLVM, then the only workable 
>> option is for the reflect2 authors to write a version of their package 
>> that uses the gccgo build tag to build a version that works with 
>> GoLLVM.  Perhaps you could help them with that.  Sorry this isn't very 
>> helpful.  I don't really understand why people are using the package. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/58a75dcf-661e-4dd1-af80-fbbfadfb4906n%40googlegroups.com.

[go-nuts] Re: [generics] Zero value

[Ivan Ivanyuk]

What about something like int.CreateZeroValue(), or T.CreateZeroValue() and 
CreateZeroValueInterface?

On Thursday, June 18, 2020 at 7:34:55 PM UTC+3, bruno...@gmail.com wrote:
>
> First, congratulations on the working compiler! Very cool to try out 
> generics in Go.
>
> I was experimenting  with 
> some simple generic functions and felt the need for the zero value of a 
> generic type. I've read the discussion anticipated in your proposal 
> ,
>  
> so I'd like to use this topic just to provide my sentiment, and perhaps 
> gather it from others.
>
> In my opinion, the most Go-like proposal 
>  is using '_' to signify the 
> zero value in RHS. We are already used to having '_' in LHS to accept 
> whatever type, and although the meaning is slightly changed (from 
> 'whatever' to 'zero'), it's conceptually close.
>
> I'm opposed to a change that uses '{}' or 'T{}' to mean zero type, as in 
> this proposal , because in my 
> mind curly braces are used for composite types. Imagining that 'T' is a 
> placeholder, It's weird replacing 'T{}' for 'int{}', for example. I'm super 
> in favor of having a leaner syntax for composite types, though .
>
> Whatever is decided, please don't punt on this issue and recommend 
> '*new(T)'. That's really ugly and feels wrong to allocate and immediately 
> dereference the value just to circumvent a syntax change. Even if the 
> compiler is smart enough not to allocate anything, that's what's written in 
> the source code.
>
> Thanks.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/df88fcc6-3422-4260-bcc7-b499f6c912ddo%40googlegroups.com.

[go-nuts] Re: Dlopen without Cgo on Darwin (#18296)

['Ivan Trubach' via golang-nuts]

I plan to finish FFI (the Call/Invoke method) and add dladdr(3) for the sake of 
completeness, and then the usual README/LICENSE stuff.

Still not sure what to do about the C to Go callbacks. Though it seems doable 
now (runtime.cgocallback).

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F6647D0D-5F07-465A-828C-2CDBD507B84C%40icloud.com.

Re: [go-nuts] Dlopen without Cgo on Darwin (#18296)

['Ivan Trubach' via golang-nuts]

Sure, I will. The dyld package should work for dlsym C calls, however I don’t 
think it’s possible to do callbacks with the functionality runtime exposes 
(i.e. I am talking about API that sys/windows package provides).

So, yeah, while there is no need for Darwin dlopen in x/sys/unix, we still need 
callbacks for non-trivial code. And at this point adding Dlopen to sys may come 
in handy.

Either that, or there is a way to make a callback without Cgo that I don’t know 
about. I would be grateful for any hints.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F6F40B11-B16E-4AB8-889C-2554918E2766%40icloud.com.

[go-nuts] Dlopen without Cgo on Darwin (#18296)

['Ivan Trubach' via golang-nuts]

Hi, I’ve stubbled upon issue 18296 (dlopen/dlsym without CGo). The discussion 
focuses on Linux, but since Go on Darwin already co-exists with libSystem, 
wouldn’t it be trivial to add Darwin dlopen support to e.g. x/sys/unix?

I’ve also set up a small standalone dlopen demo: https://github.com/tie/dyld

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/517AC180-A835-4802-8766-BB2F23A39D34%40icloud.com.

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Ivan Bertona]

You are totally right on that, sorry. It's just > 4k.

On Thu, Nov 21, 2019 at 7:13 PM burak serdar  wrote:

> On Thu, Nov 21, 2019 at 4:59 PM Ivan Bertona  wrote:
> >
> > 1) Yes if you set NumberOfWorkers high enough (> 4k / num CPUs), and
> your machine is actually capable of handling this workload. Based on
> experience I'd say you shouldn't expect significant overhead for job
> scheduling.
>
> Not divided by nCPUs though, right?
>
> If there are w workers, and with w workers each goroutine takes 2secs,
> and if you're getting work at a rate of 2k/sec, you need at least 4k
> goroutines to keep up, regardless of the cpu count. After the 2nd
> second, you'll have all 4k goroutines busy. Am I missing something?
>
> The important thing is: does it take 2 secs for each goroutine to
> complete when w is > 4k
>
>
> > 2) Not sure this is a question
> > 3) No
> > 4) What you are doing is totally fine at 2k/s
> >
> > I'll add that you shouldn't trust me, you can easily measure the
> overhead yourself by making the consumer work be a 2s sleep, setting
> NumberOfWorkers to 4k / num CPUs, pushing 2k/s jobs, and looking at how
> your system load looks like when it's running. As for whether this would
> work with your actual workload, again the only way is to try and measure.
> >
> > Best,
> > Ivan
> >
> > On Thursday, November 21, 2019 at 3:24:26 PM UTC-5, Michael Jones wrote:
> >>
> >> Agree. Essentially I'm saying the "channel aspect" is not an issue.
> >>
> >> On Thu, Nov 21, 2019 at 12:12 PM Robert Engels 
> wrote:
> >>>
> >>> He stated "each request takes 2 secs to process" - what's involved in
> that is the important aspect imo.
> >>>
> >>> -Original Message-
> >>> From: Michael Jones
> >>> Sent: Nov 21, 2019 2:06 PM
> >>> To: Robert Engels
> >>> Cc: Sankar , golang-nuts
> >>> Subject: Re: [go-nuts] golang multiple go routines reading from a
> channel and performance implications
> >>>
> >>> In my (past) benchmarking, I got ~3M channel send/receive operations
> per second on my MacBook Pro. It is faster on faster computers. 2k
> requests/src is much less than 3M, clearly, and the 1/1000 duty cycle
> suggests that you'll have 99.9% non-overhead to do your processing. This is
> back of the envelope thinking, but what I go through for every path
> explored. What should it be? How is in in fact? What explains the
> difference? ... that kind of thing.
> >>>
> >>> On Thu, Nov 21, 2019 at 11:25 AM Robert Engels 
> wrote:
> >>>>
> >>>> You need to determine how well they parallelize and what the resource
> consumption of a request is. For example, if every request can run
> concurrently at 100% (not possible btw because of switching overhead), and
> each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall
> time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs.
> IO can run concurrently on most modern setups, so you can essentially
> factor this out, less so if most of the operations are writes.
> >>>>
> >>>> Your local CPU requirements may be less if the request is then
> handled by a cluster (over the network, database etc), but you will still
> need 1000 cpus in the cluster (probably a lot more due to the network
> overhead).
> >>>>
> >>>> You can look at github.com/robaho/go-trader for an example of very
> high CPU based processing using Go and channels (and other concurrency
> structs).
> >>>>
> >>>>
> >>>>
> >>>> -Original Message-
> >>>> From: Sankar
> >>>> Sent: Nov 21, 2019 12:30 PM
> >>>> To: golang-nuts
> >>>> Subject: [go-nuts] golang multiple go routines reading from a channel
> and performance implications
> >>>>
> >>>> We have a setup where we have a producer goroutine pumping in a few
> thousand objects into a channel (Approximately 2k requests per second).
> There are a configurable number of goroutines that work as consumers,
> consuming from this single channel. If none of the consumer threads could
> receive the message, the message gets discarded. Each consumer go routine
> takes about 2 seconds for each work to be completed, by which they will
> come back to read the next item in the channel. The channel is sized such
> that it can hold up to 10,000 messages.
> >>>>
> >>>> The code is roughly something like:
> >>>>
> >>&g

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Ivan Bertona]

1) Yes if you set NumberOfWorkers high enough (> 4k / num CPUs), and your 
machine is actually capable of handling this workload. Based on experience 
I'd say you shouldn't expect significant overhead for job scheduling.
2) Not sure this is a question
3) No
4) What you are doing is totally fine at 2k/s

I'll add that you shouldn't trust me, you can easily measure the overhead 
yourself by making the consumer work be a 2s sleep, setting NumberOfWorkers 
to 4k / num CPUs, pushing 2k/s jobs, and looking at how your system load 
looks like when it's running. As for whether this would work with your 
actual workload, again the only way is to try and measure.

Best,
Ivan

On Thursday, November 21, 2019 at 3:24:26 PM UTC-5, Michael Jones wrote:
>
> Agree. Essentially I'm saying the "channel aspect" is not an issue.
>
> On Thu, Nov 21, 2019 at 12:12 PM Robert Engels  > wrote:
>
>> He stated "each request takes 2 secs to process" - what's involved in 
>> that is the important aspect imo.
>>
>> -Original Message- 
>> From: Michael Jones 
>> Sent: Nov 21, 2019 2:06 PM 
>> To: Robert Engels 
>> Cc: Sankar , golang-nuts 
>> Subject: Re: [go-nuts] golang multiple go routines reading from a channel 
>> and performance implications 
>>
>> In my (past) benchmarking, I got ~3M channel send/receive operations per 
>> second on my MacBook Pro. It is faster on faster computers. 2k requests/src 
>> is much less than 3M, clearly, and the 1/1000 duty cycle suggests that 
>> you'll have 99.9% non-overhead to do your processing. This is back of the 
>> envelope thinking, but what I go through for every path explored. What 
>> should it be? How is in in fact? What explains the difference? ... that 
>> kind of thing.
>>
>> On Thu, Nov 21, 2019 at 11:25 AM Robert Engels > > wrote:
>>
>>> You need to determine how well they parallelize and what the resource 
>>> consumption of a request is. For example, if every request can run 
>>> concurrently at 100% (not possible btw because of switching overhead), and 
>>> each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall 
>>> time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs. 
>>> IO can run concurrently on most modern setups, so you can essentially 
>>> factor this out, less so if most of the operations are writes.
>>>
>>> Your local CPU requirements may be less if the request is then handled 
>>> by a cluster (over the network, database etc), but you will still need 1000 
>>> cpus in the cluster (probably a lot more due to the network overhead).
>>>
>>> You can look at github.com/robaho/go-trader for an example of very high 
>>> CPU based processing using Go and channels (and other concurrency structs).
>>>
>>>
>>>
>>> -Original Message- 
>>> From: Sankar 
>>> Sent: Nov 21, 2019 12:30 PM 
>>> To: golang-nuts 
>>> Subject: [go-nuts] golang multiple go routines reading from a channel 
>>> and performance implications 
>>>
>>> We have a setup where we have a producer goroutine pumping in a few 
>>> thousand objects into a channel (Approximately 2k requests per second). 
>>> There are a configurable number of goroutines that work as consumers, 
>>> consuming from this single channel. If none of the consumer threads could 
>>> receive the message, the message gets discarded. Each consumer go routine 
>>> takes about 2 seconds for each work to be completed, by which they will 
>>> come back to read the next item in the channel. The channel is sized such 
>>> that it can hold up to 10,000 messages.
>>>
>>> The code is roughly something like:
>>>
>>> producer.go:
>>> func produce() {
>>>  ch <- item
>>> }
>>>
>>> func consumer() {
>>>  for i:=0; i < NumberOfWorkers; i ++ {
>>>go func() {
>>>   for _, item := range ch {
>>>  // process item
>>>   }
>>>} ()
>>>  }
>>> }
>>>
>>> With this above setup, we are seeing about 40% of our messages getting 
>>> dropped.
>>>
>>> So my questions are:
>>>
>>> 1) In such a high velocity incoming data, will this above design work ? 
>>> (Producer, Consumer Worker Threads)
>>> 2) We did not go for an external middleware for saving the message and 
>>> processing data later, as we are concerned about latency for now.
>>> 3) Are channels bad for such an approach ? Are 

Re: [go-nuts] Automated Gerrit reviews

[Ivan Borshukov]

Thanks for sharing, it looks quite interesting.

Do you think it would be possible (and easy) to extend the tool to support
posting comments in GitLab or GitHub?

There are times that some static check failures are ok, but that’s now
always the case. So instead of failing the build or tests it would be great
to have a bot commenting on the change itself, leaving an option for the
submitter to resolve the discussion (possibly with explanation why).

Thanks,
Ivan

On Fri, 12 Jul 2019 at 15:55,  wrote:

> https://github.com/heppu/go-review
>
> I wrote small tool which takes various linter reports as input and posts
> those as comments to gerrit reviews. I use it at work with golangci-lint in
> our Jenkins CI pipe.
> Any feedback is much appreciated. =)
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2a66f4e0-284a-499e-8388-0e7ced6fe772%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/2a66f4e0-284a-499e-8388-0e7ced6fe772%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Ivan Borshukov, bo0mer...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFph64dxWUrDXw2fpX7kh4NeEddr%3D5a1W51KihCh8etBbyqJ7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go: how to input two-dimensional string array form keyboard??

[Ivan Fraixedes]

Could you post your non-working peice of code? it should help to understand 
what's the problem.

On Monday, 10 June 2019 07:19:02 UTC+2, shine sun wrote:
>
> Firstly,  I try to input N and M from keyboard, creating a two-dimensional 
> string array (var nums [N][M]string). Then, for i := 0; i < N; i++ , how to 
> use fmt.Scanf to input M strings for each nums[i]? Finally, it can 
> successfully  output the two-dimensional array nums. Thank you for helping!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b68188a4-3be1-48a3-b8d3-45678dc55c43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Go will shine on huge web projects, but how about simple ones?

[Ivan Bertona]

Looking a the two code samples I wouldn't say this is an apples to apples 
comparison... The Perl script seems to be a simple single-threaded loop 
that understands a tiny subset of HTTP vs. a fully-fledged (and secure) web 
server from the Go standard library. I would definitely not run that Perl 
script in production, even if it was for a simple project. My bet is that 
if you actually port the Perl script to a Go program that does more or less 
the same thing you'll see more or less the same performance (because the 
example is fundamentally I/O-bound).

Best,
Ivan

On Friday, June 7, 2019 at 9:36:49 AM UTC-4, Tong Sun wrote:
>
> I had always believed that the web projects build with Go should be much 
> faster than Perl, since Go is a compiled language. 
>
> However that belief was crushed brutally last night, when I did a 
> comparison -- the Go implementation is *8 times worse 
> <https://dev.to/suntong/simple-web-server-implemented-in-perl-and-go-b43>* 
> than the *Perl*! -- the *mean *response time jumped from 6ms to 48ms. 
>
> I know this is the simplest possible web server, but still, when it comes 
> to simple web servers like this, I have to say that Perl performs much 
> better than Go.
>
> I don't think there is much I can twist on the Go side, since it can't be 
> more simpler than that. However, I also believe it won't hurt to ask and 
> confirm. So,
>
> Have I missed anything? Is it possible for me to make my Go implementation 
> anywhere near the Perl's performance?
>
> Thanks
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/12e7666a-385f-47ef-b061-9738802e7a88%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [48]byte should be make to empty

[ivan . fraixedes]

Or even just declare the var without any value assignation because it's 
zero value is an array of zero byte value, which is 0.

var array [48]byte


On Thursday, 6 June 2019 17:38:14 UTC+2, Michel Levieux wrote:
>
> Hi,
>
> It is not quite clear what you are trying to do, but if I understand it 
> well:
>
> var array = [48]byte{}
>
> should initialize your array with zero values.
>
> Le jeu. 6 juin 2019 à 15:48, sasikala tholisam  > a écrit :
>
>> test [48]byte make to empty..
>> How to do it?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/f5a26feb-8946-415c-9eb3-f80e8da73d3c%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/946cca47-a106-473d-bc63-03e7b7abfbba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[Ivan Medoedov]

Thanks very much. This is all very interesting.

On Sat, May 4, 2019 at 2:20 AM  wrote:

> I don't have hard numbers on slowdowns. It will depend a lot on the binary
> in question.
> At the time I checked the compiler and linker and it did not slow those
> down (to the accuracy of our benchmarks). The linker particularly is fairly
> read syscall intensive.
>
> We've seen benchmarks that have significant slowdowns though. There was a
> factor of 4 slowdown on 1.12 for a (IMO a not very realistic) http
> benchmark. This wasn't conclusively pinned on the libSystem changes, but it
> seems likely.
>
> On Friday, May 3, 2019 at 4:52:37 PM UTC-7, Ian Lance Taylor wrote:
>>
>> On Fri, May 3, 2019 at 3:36 PM  wrote:
>> >
>> > I know that calling C functions via cgo is relatively expensive because
>> of the goroutines, so there must be some optimizations/tricks to make all
>> these libc calls fast.
>>
>> Yes, this only uses part of the cgo call path, not all of it.  In
>> particular there's no need to notify the scheduler about these calls
>> (beyond what was already done before this change), just as we don't
>> need to notify the scheduler about all system calls.
>>
>> Ian
>>
>>
>> > On Saturday, May 4, 2019 at 12:21:27 AM UTC+2, ivan.m...@gmail.com
>> wrote:
>> >>
>> >> Indeed, it works with Go 1.12. I have 3 different Go versions, and my
>> aliasing was messed up. Sorry.
>> >>
>> >> > No.  The Linux kernel, fortunately, supports static linking and
>> provides a stable syscall API.  There is no reason to make this change on
>> GNU/Linux.
>> >>
>> >> Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms
>> using raw syscalls.
>> >>
>> >> Last question: did performance drop after the switch? There must be
>> benchmarks.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to golan...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/uX8eUeyuuAY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Is it possible to use the Go linker to link C object files?

[ivan . medoedov]

Go linker is amazing. It's the only truly cross platform linker.

Is it possible to use it to create binaries from object files compiled with 
C?

For example

main.c:
int main() { puts("Hello"); return 0; }

gcc -c -o main.o main.c

go tool link main.o
main.o: not package main


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[ivan . medoedov]

I know that calling C functions via cgo is relatively expensive because of 
the goroutines, so there must be some optimizations/tricks to make all 
these libc calls fast.

On Saturday, May 4, 2019 at 12:21:27 AM UTC+2, ivan.m...@gmail.com wrote:
>
> Indeed, it works with Go 1.12. I have 3 different Go versions, and my 
> aliasing was messed up. Sorry.
>
> > No.  The Linux kernel, fortunately, supports static linking and provides 
> a stable syscall API.  There is no reason to make this change on GNU/Linux. 
>
> Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms 
> using raw syscalls. 
>
> Last question: did performance drop after the switch? There must be 
> benchmarks.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[ivan . medoedov]

Indeed, it works with Go 1.12. I have 3 different Go versions, and my 
aliasing was messed up. Sorry.

> No.  The Linux kernel, fortunately, supports static linking and provides 
a stable syscall API.  There is no reason to make this change on GNU/Linux. 

Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms using 
raw syscalls. 

Last question: did performance drop after the switch? There must be 
benchmarks.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[ivan . medoedov]

Hello,

I read the following from one of the contributors:

> On Solaris (and Windows), and more recently in macOS as well we link with 
libc (or equivalent).
> Go used to do raw system calls on macOS, and binaries were occasionally 
broken by kernel updates. Now Go uses libc on macOS.

I just checked syscall/zsyscall_darwin_amd64.go, and indeed in Go 1.10 we 
had

Syscall(SYS_WRITE, uintptr(fd), uintptr(_p0), uintptr(len(p)))

now we have

syscall(funcPC(libc_write_trampoline), uintptr(fd), uintptr(_p0), 
uintptr(len(p)))

But when I run otool -L on the binary to see if /usr/lib/libSystem.B.dylib 
is linked, I get nothing.

Is it because Go uses its own linker that's not supported by macOS tools?

P.S. Are there plans to do the same transition for Linux for consistency 
and stability?

Thanks.

-Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is Go a single pass compiler?

[ivan . medoedov]

Thanks, Ian.

I remember reading in some compiler book that languages should be designed 
for a single pass to reduce compilation speed.

Go proves that wrong :) It's amazingly fast, looks like computers are 
pretty good at traversing AST trees.

On Wednesday, February 27, 2019 at 11:50:05 PM UTC+1, Ian Lance Taylor 
wrote:
>
> On Wed, Feb 27, 2019 at 2:42 PM > 
> wrote: 
> > 
> > In Go functions can be used before they are defined, but as I 
> understand, it's still possible to have a single pass compiler. 
>
> I don't think it's possible to compile Go in a single pass compiler, 
> unless you consider a separate parsing and code generation step to be 
> a single pass compiler.  In the general case, you can't generate code 
> for any Go function until you've seen all the functions in the 
> package. 
>
> There are multiple Go compilers.  All the ones I am aware of are have 
> many passes. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Is Go a single pass compiler?

[ivan . medoedov]

Hello,

In Go functions can be used before they are defined, but as I understand, 
it's still possible to have a single pass compiler. 

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Alternative of defer?

[Ivan Bertona]

Recovering from panics is a common and IMO perfectly acceptable practice...
For example in a web server where you do not want a panic that occurs while
handling a request to affect other request, especially ones that are in
flight.

On Fri, Feb 8, 2019 at 10:47 PM Robert Engels  wrote:

> The way Go is designed a panic must terminate the application. Anything
> else is so indeterminate to be unusable.
>
> On Feb 8, 2019, at 8:08 PM, Michael Jones  wrote:
>
> Agree to that.
>
> From the original blog post:
>
> The convention in the Go libraries is that even when a package uses panic
> internally, its external API still presents explicit error return values.
>
> But yes, agree with the problem in the situation that you describe
>
> On Fri, Feb 8, 2019 at 4:24 PM Ivan Bertona  wrote:
>
>> What's suboptimal with the first one (or the second one) is that if
>> performOperation1() panics the lock will not be released. It may or may not
>> be a problem depending on the situation. Your assessment of defer used with
>> locks is correct - it works well only if the lock doesn't need to be
>> released before the end of the function - but in my experience you can
>> always extract a function to achieve that. If you can't, it's usually a
>> sign that the code needs some reworking.
>>
>> On Friday, February 8, 2019 at 7:03:28 PM UTC-5, Michael Jones wrote:
>>>
>>> I don’t see anything suboptimal about the first one.
>>>
>>> Defer is a function-scope magic thing.
>>>
>>> Go designers chose not to have lexical scope magic, so you would either
>>> force function scope (prior answer) or be happy with the normal code.
>>>
>>> On Fri, Feb 8, 2019 at 10:31 AM Burak Serdar  wrote:
>>>
>>>> On Fri, Feb 8, 2019 at 11:28 AM vincent163  wrote:
>>>> >
>>>> > I am thinking about how to write programs like this:
>>>> > lock1.Lock()
>>>> > err = performOperation1()
>>>> > if err != nil {
>>>> >   lock1.Unlock()
>>>> >   return err
>>>> > }
>>>> > lock1.Unlock()
>>>> > performExpensiveOperation2()
>>>>
>>>> How about this:
>>>>
>>>> lock1.Lock()
>>>> err = performOperation1()
>>>> lock1.Unlock()
>>>> if err != nil {
>>>>   return err
>>>> }
>>>> performExpensiveOperation2()
>>>>
>>>>
>>>> >
>>>> >
>>>> > The lock1 must be locked while performing operation1, and I need to
>>>> use its result to perform operation2. Since operation2 is expensive, I
>>>> don't want to hold the lock while performing it, and lock1.Unlock() needs
>>>> to be called before calling operation2.
>>>> > Go's defer mechanism doesn't seem to handle this case well since the
>>>> resource is used only within a block and not throughout the function. Is
>>>> there a recommended way to write programs in this case?
>>>> > I know I could wrap the lock block in a closure, but that creates a
>>>> completely new scope, so I can't return directly or break out of a loop
>>>> within the closure, etc.
>>>> >
>>>> > --
>>>> > You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to golang-nuts...@googlegroups.com.
>>>> > For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golang-nuts...@googlegroups.com.
>>>
>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>>>
>>> *Michael T. jonesmichae...@gmail.com*
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
>
> *Michael T. jonesmichael.jo...@gmail.com *
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Alternative of defer?

[Ivan Bertona]

What's suboptimal with the first one (or the second one) is that if 
performOperation1() panics the lock will not be released. It may or may not 
be a problem depending on the situation. Your assessment of defer used with 
locks is correct - it works well only if the lock doesn't need to be 
released before the end of the function - but in my experience you can 
always extract a function to achieve that. If you can't, it's usually a 
sign that the code needs some reworking.

On Friday, February 8, 2019 at 7:03:28 PM UTC-5, Michael Jones wrote:
>
> I don’t see anything suboptimal about the first one. 
>
> Defer is a function-scope magic thing. 
>
> Go designers chose not to have lexical scope magic, so you would either 
> force function scope (prior answer) or be happy with the normal code. 
>
> On Fri, Feb 8, 2019 at 10:31 AM Burak Serdar  > wrote:
>
>> On Fri, Feb 8, 2019 at 11:28 AM vincent163 > > wrote:
>> >
>> > I am thinking about how to write programs like this:
>> > lock1.Lock()
>> > err = performOperation1()
>> > if err != nil {
>> >   lock1.Unlock()
>> >   return err
>> > }
>> > lock1.Unlock()
>> > performExpensiveOperation2()
>>
>> How about this:
>>
>> lock1.Lock()
>> err = performOperation1()
>> lock1.Unlock()
>> if err != nil {
>>   return err
>> }
>> performExpensiveOperation2()
>>
>>
>> >
>> >
>> > The lock1 must be locked while performing operation1, and I need to use 
>> its result to perform operation2. Since operation2 is expensive, I don't 
>> want to hold the lock while performing it, and lock1.Unlock() needs to be 
>> called before calling operation2.
>> > Go's defer mechanism doesn't seem to handle this case well since the 
>> resource is used only within a block and not throughout the function. Is 
>> there a recommended way to write programs in this case?
>> > I know I could wrap the lock block in a closure, but that creates a 
>> completely new scope, so I can't return directly or break out of a loop 
>> within the closure, etc.
>> >
>> > --
>> > You received this message because you are subscribed to the Google 
>> Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an email to golang-nuts...@googlegroups.com .
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
>
> *Michael T. jonesmichae...@gmail.com *
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Request Content Length zero when copy from another Request

[ivan]

What you are seeing here is an edge case caused by the fact that the first 
NewRequest wraps your strings.Reader into a ioutil.NopCloser. When 
NewRequest is called the second time, the type switch no longer matches a 
type for which the length is known, so ContentLength is left to 0, which 
according to the docs means unknown. NewRequest implements this "best 
effort" method of determining the content length to avoid using the less 
efficient chunked encoding transfer method unless necessary, so you don't 
need to explicitly set the Content-Length header every time you create a 
new request.

That said, this is an artificial example which doesn't really reflect the 
way this API is used. If you were reverse proxying an incoming HTTP 
request, you would know the length of the body before reading it only if 
the client set a Content-Length header, which is optional. To properly 
proxy the request you would pass through the body reader, and copy the 
Content-Length header to the upstream request. You can check out the source 
code httputil.ReverseProxy 
 for a working 
implementation. An unset Request.ContentLength does not affect the ability 
to actually perform the request, although it might cause it to be sent with 
the somewhat less efficient chunked encoding method.

Hope this helps


On Tuesday, February 5, 2019 at 11:12:36 AM UTC-5, Matteo Biagetti wrote:
>
> I've the following situation: 
> I proxy a request to another server and when I made a POST and create a 
> new request, the contentLength is zero:
>
> req2, _ := http.NewRequest(req.Method, newApiUrl , req.Body)
> fmt.Println("New request from body:", req2.ContentLength) // print 
> 0
>
> Checking in the source code of the NewRequest func Body don't respect some 
> interface and populate the ContentLength field.
>
> Could be a bug? Which could be a valid approach in order to create a new 
> request from an existing one and correct set the Body length?
>
> A working example here:
>
> https://play.golang.org/p/SvCDLj0NrXb
>
> Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why not a mixture of value and pointer receiver?

[Ivan Fraixedes]

Perhaps 

There is a handy exception, though. When the value is addressable, the 
> language takes care of the common case of invoking a pointer method on a 
> value by inserting the address operator automatically. In our example, the 
> variable b is addressable, so we can call its Write method with just 
> b.Write. The compiler will rewrite that to ().Write for us.


clarifies your confusion.

You can find such paragraph 
in https://golang.org/doc/effective_go.html#pointers_vs_values 

On Wednesday, 23 January 2019 09:52:02 UTC+1, Xinhu Liu wrote:
>
> Why is that a problem? I can indeed call it on value (at line 20). I got 
> result:
>>
>> hello
>> hello
>> Program exited.
>>
>>
> I am now more confusing when I come to 
> https://golang.org/doc/faq#different_method_sets. 
>
> Am Mi., 23. Jan. 2019 um 06:47 Uhr schrieb Andrei Avram <
> andrei.a...@gmail.com >:
>
>> If it's about a method you've defined on a pointer and you want to call 
>> it on the value: https://play.golang.org/p/zMVivcaXrf3
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/golang-nuts/xOsuXPe1IUo/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go test linking time issue

[Ivan Tolstosheyev]

It's go version go1.11.2 linux/amd64 now.
We are flexible with go versions.

On Fri, Jan 11, 2019 at 5:52 PM Ian Lance Taylor  wrote:

> On Fri, Jan 11, 2019 at 5:49 AM  wrote:
> >
> > I'm working on pretty big golang project. This project consists of
> several hundreds of packages, most of them are with tests. When we run
> tests, go test tool does the following:
> >
> > * for each package with test files it extends this package with *_test
> files of current package, create two packages (pxtest and pmain), details
> in code.
> > * compiles those packages and dependencies
> > * links them to test binary
> >
> > The latest (link) stage is very CPU and memory consuming for our
> project, because all tests have a lot of dependencies. We've got pipeline
> that build tests in parallel, used all known speed optimizations (like -w
> -s flags, etc).
> > We even (as an experiment) forked go test tool to add ability to build
> tests from different packages into one binary (yep, there are some caveats,
> like: circular dependencies between xtest-modules, clashes with global data
> structures like flags, etc, but this is manageable). So now we can build
> tests into one binary (with limitations), and instead of N linking
> operations (where N is number of packages with tests) we got only one.
> >
> > But this approach is a little hacky, so the question is, is there any
> other ways to optimize linking time of tests? Maybe it is possible to build
> whole project to shared library, and link test dynamically? Maybe some
> other ways?
>
> Which version of Go are you using?
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: scanner.Scan is holding my goroutine ;(

[Ivan Korjavin]

Guys, you are right. It should work.
I made a mistake, the problem was in the surrounding code.

I posted it here https://github.com/korjavin/goscan just for educational 
purpose.

The actual problem wasn't scanner.Scan
it was `errs <- err`
but it took me hours to understand.

Sorry, and thank you!

On Friday, December 14, 2018 at 3:28:20 AM UTC+5, Ivan Korjavin wrote:
>
> I have a goroutenie with  scanner.Scan
> It looks like:
>
> ```
> cnn.SetReadDeadline(time.Now().Add(c.rTimeout)) // 10s
> scanner := bufio.NewScanner(cnn)
> for scanner.Scan() {
> ...
> }
> ```
> cnn is a net.Conn there
>
> My problem is when I call cnn.Close() (and I tried 
> cnn.SetReadDeadline(time.Now()) in addition as well)
> this scanner.Scan is still running and running and running.
>
> I would like to close it immediately, but can't find a way how.
>
>
> Please, help me!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] scanner.Scan is holding my goroutine ;(

[Ivan Korjavin]

I have a goroutenie with  scanner.Scan
It looks like:

```
cnn.SetReadDeadline(time.Now().Add(c.rTimeout)) // 10s
scanner := bufio.NewScanner(cnn)
for scanner.Scan() {
...
}
```
cnn is a net.Conn there

My problem is when I call cnn.Close() (and I tried 
cnn.SetReadDeadline(time.Now()) in addition as well)
this scanner.Scan is still running and running and running.

I would like to close it immediately, but can't find a way how.


Please, help me!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] Koazee a library inspired by functional programming and lazy evaluation that takes the hassle out of working with arrays

[ivan . corrales]

Last weeks I've been working on improving performance in Koazee. I put in 
practice all your advices and even though I don't have a second release 
ready, I wrote an article to share my experience trying to improve 
performance

https://www.linkedin.com/pulse/go-performance-tips-iv%C3%A1n-corrales-solera/

I hope this can be useful for someone!

Cheers


El domingo, 11 de noviembre de 2018, 20:27:16 (UTC+1), Iván Corrales Solera 
escribió:
>
> Hey guys, last weeks I've been working on Koazee and I just released a 
> very first version Titi, v0.0.1 . Koazee is a golang library inspired in 
> Lazy evaluation and functional programming that provides us a rich set of 
> operations that can be done over arrays. If you like the clean code and the 
> functional programming I am sure you enjoy it!
>
> Documentation is hosted http://wesovilabs.com/koazee/
>
> And the full code can be found on Github, 
> https://github.com/wesovilabs/koazee
> Any feedback or recommendation will be appreciated! Cheers
>

-- 


"Este mensaje está
dirigido de manera exclusiva a su destinatario y puede 
contener información
privada y confidencial. No lo reenvíe, copie o 
distribuya a terceros que no
deban conocer su contenido. En caso de haberlo 
recibido por error,  rogamos
lo notifique al remitente y proceda a su 
borrado, así como al de cualquier
documento que pudiera adjuntarse.



 Por 
favor tenga en cuenta que
los correos enviados vía Internet no permiten 
garantizar la confidencialidad de
los mensajes ni su transmisión de forma 
íntegra.



 Las opiniones expresadas en el
presente correo pertenecen 
únicamente al remitente y no representan
necesariamente la opinión del 
Grupo BBVA."



 "This
message is intended exclusively for the adressee and 
may contain privileged and
confidential information. Please, do not 
disseminate, copy or distribute it to
third parties who should not receive 
it. In case you have received it by
mistake, please inform the sender and 
delete the message and attachments from
your system.



 Please
keep in 
mind that e-mails sent by Internet do not allow to guarantee neither
the 
confidentiality or the integrity of the messages sent."

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] Koazee a library inspired by functional programming and lazy evaluation that takes the hassle out of working with arrays

[ivan . corrales]

Thanks a million rog!

I completely understand what you say. Regarding to the performance, I 
didn't expect as bad performance as I found yesterday when running my 
benchmark test and I will be focused on being able to improve it as much as 
I can. 

I found a way to improve performance in primitive arrays but keeping the 
generic interface. I just started with the reduce operation

The below test, yesterday took  7.08 ns/op vs 4000ns/op aprox. Today I am 
in 400ns/op vs 7

var numbers10 = utils.ArrayOfInt(0, 1, 10)


func BenchmarkSum10(b *testing.B) {
   for i := 0; i < b.N; i++ {
  result := 0
  for _, v := range numbers10 {
 result += v
  }
  output = result
   }

}
func BenchmarkSum10Koazee(b *testing.B) {
   for i := 0; i < b.N; i++ {
  output2 := koazee.
 StreamOf(numbers10).
 Reduce(func(acc, val int) int { return acc + val }).
 Val()
  
   }
}


To improve the performance I am  focused on making the operations smarter. 
As soon as I got it working I will share with you,

Thank you again for all the suggestion and recommendation guys! Very 
appreciated.


El martes, 13 de noviembre de 2018, 21:15:17 (UTC+1), rog escribió:
>
> It's obvious that you've put a lot of effort into this! Here's some 
> feedback.
>
> I have to say this first: this *is* a very non-Go-idiomatic API. It's 
> fundamentally based around reflection, and reflection in Go is generally 
> considered to be best avoided apart from some core libraries. Calling a 
> function with reflect is orders of magnitude slower than calling it 
> directly, and the lack of compiler type checking can make for code that's 
> hard to understand when it goes wrong. So I wouldn't use this in production 
> code.
>
> That said, it's a fun API and a great way to experiment with some of the 
> things that can be done in Go if we sacrifice compile-type type checking.
>
> Taking it at face value, here are a few random thoughts:
>
> - improve the doc comments. Go automatically generates documentation for 
> you (for example https://godoc.org/github.com/wesovilabs/koazee/stream) 
> and that's the usual place for people to read package documentation. 
> Everything that's exported should have a relevant doc comment. There 
> shouldn't be any need to generate custom API documentation.
>
> - avoid making public methods return private types. When a public method 
> returns a private type, the documentation fails because it does not display 
> methods on private types.
>
> - it's almost always better to use the builtin sort package (
> https://golang.org/pkg/sort) rather than rolling your own.
>
> - it's probably best not to use an arbitrary definition of equality 
> (stream.equalsValues), but to either use plain "==" or accept an equality 
> function.
>
> - it would be nice for it to work on arbitrary streams. This actually 
> isn't too hard. Here's a way of doing it that works on an arbitrary 
> Iterator interface, so it can iterate over channels, bufio.Scanner, etc. 
> https://play.golang.org/p/PXHuD1pR5uC
>
> Finally, if/when generics eventually make it into Go, this kind of API 
> will no longer have a huge overhead. I think I'd probably wait until 
> then... :)
>
>   cheers,
> rog.
>
> On Sun, 11 Nov 2018 at 19:27, Iván Corrales Solera <
> ivan.corra...@gmail.com > wrote:
>
>> Hey guys, last weeks I've been working on Koazee and I just released a 
>> very first version Titi, v0.0.1 . Koazee is a golang library inspired in 
>> Lazy evaluation and functional programming that provides us a rich set of 
>> operations that can be done over arrays. If you like the clean code and the 
>> functional programming I am sure you enjoy it!
>>
>> Documentation is hosted http://wesovilabs.com/koazee/
>>
>> And the full code can be found on Github, 
>> https://github.com/wesovilabs/koazee
>> Any feedback or recommendation will be appreciated! Cheers
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
-- 


"Este mensaje está
dirigido de manera exclusiva a su destinatario y puede 
contener información
privada y confidencial. No lo reenvíe, copie o 
distribuya a terceros que no
deban conocer su contenido. En caso de haberlo 
recibido por error,  rogamos
lo notifique al remitente y proceda a su 
borrado, así como al de cualquier
documento que pudiera adjuntarse.



 Por 
favor tenga en cuenta que
los correos enviados vía Internet no permiten 
garantizar la confidencialidad de
los mensajes ni su transmisión de forma 
íntegra.



 Las opiniones expresadas en el
presente correo pertenecen 
únicamente al remitente y no representan
necesariamente la opinión del 
Grupo BBVA."



 "This
message is intended exclusively for the adressee and 
may contain privileged and

Re: [go-nuts] [ANN] Koazee a library inspired by functional programming and lazy evaluation that takes the hassle out of working with arrays

[ivan . corrales]

Hey Bakul again,

I've been having a look at kona, and the project looks so good, I will look 
in deep trying to get some good ideas as you recommend me,

In regards to the code generation, I've never done to be honest, but 
obviously the performance should be better.  By the way, @Marko_Ristin  
created an issue asking for some performance testing. I think I will face 
this issue before considering code generation. But in case of the 
performance wasn't good at all, for sure that I consider your suggestion 
about generate the code

And, what you mean when say "arbitrary streams (e.g. something built on top 
of channels)?" Sorry for my bad understanding, Thanks in advance

El lunes, 12 de noviembre de 2018, 12:17:12 (UTC+1), Bakul Shah escribió:
>
> On Nov 11, 2018, at 11:27 AM, Iván Corrales Solera <
> ivan.corra...@gmail.com > wrote: 
> > 
> > Hey guys, last weeks I've been working on Koazee and I just released a 
> very first version Titi, v0.0.1 . 
> > 
> > Koazee is a golang library inspired in Lazy evaluation and functional 
> programming that provides us a rich set of operations that can be done over 
> arrays.  If you like the clean code and the functional programming I am 
> sure you enjoy it! 
> > 
> > 
> > Documentation is hosted http://wesovilabs.com/koazee/ 
> > 
> > And the full code can be found on Github, 
> https://github.com/wesovilabs/koazee 
> > 
> > Any feedback or recommendation will be appreciated! 
>
> Very nice! I like the clean interface. 
>
> Some comments: 
>
> - Can this handle arbitrary streams (e.g. something built on top of 
> channels)? 
>   That would make a nice extension! 
> - You may wish to look at k3 (open source version @ 
> https://github.com/kevinlawler/kona) 
>   In particular https://github.com/kevinlawler/kona/wiki#verbs 
>   There are some additional functions (verbs) and higher order functions 
> (adverbs) 
>   that may be of interest and will fit in nicely within your framework. 
>   PDFs of K 2.0 User and Reference manuals (listed under links) have much 
> more detail. 
> - May be you can provide another interface that *generates* Go code 
> instead of 
>   executing it! If it can be made to generate more efficient code, devoid 
> of 
>   reflect. 
>
>
-- 


"Este mensaje está
dirigido de manera exclusiva a su destinatario y puede 
contener información
privada y confidencial. No lo reenvíe, copie o 
distribuya a terceros que no
deban conocer su contenido. En caso de haberlo 
recibido por error,  rogamos
lo notifique al remitente y proceda a su 
borrado, así como al de cualquier
documento que pudiera adjuntarse.



 Por 
favor tenga en cuenta que
los correos enviados vía Internet no permiten 
garantizar la confidencialidad de
los mensajes ni su transmisión de forma 
íntegra.



 Las opiniones expresadas en el
presente correo pertenecen 
únicamente al remitente y no representan
necesariamente la opinión del 
Grupo BBVA."



 "This
message is intended exclusively for the adressee and 
may contain privileged and
confidential information. Please, do not 
disseminate, copy or distribute it to
third parties who should not receive 
it. In case you have received it by
mistake, please inform the sender and 
delete the message and attachments from
your system.



 Please
keep in 
mind that e-mails sent by Internet do not allow to guarantee neither
the 
confidentiality or the integrity of the messages sent."

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] Koazee a library inspired by functional programming and lazy evaluation that takes the hassle out of working with arrays

[ivan . corrales]

Hey Bakul thank you so much for your feedback,

It's so great receive these kind of suggestions, I will create the required 
issue for implementing it.

Cheers!





El lunes, 12 de noviembre de 2018, 12:17:12 (UTC+1), Bakul Shah escribió:
>
> On Nov 11, 2018, at 11:27 AM, Iván Corrales Solera <
> ivan.corra...@gmail.com > wrote: 
> > 
> > Hey guys, last weeks I've been working on Koazee and I just released a 
> very first version Titi, v0.0.1 . 
> > 
> > Koazee is a golang library inspired in Lazy evaluation and functional 
> programming that provides us a rich set of operations that can be done over 
> arrays.  If you like the clean code and the functional programming I am 
> sure you enjoy it! 
> > 
> > 
> > Documentation is hosted http://wesovilabs.com/koazee/ 
> > 
> > And the full code can be found on Github, 
> https://github.com/wesovilabs/koazee 
> > 
> > Any feedback or recommendation will be appreciated! 
>
> Very nice! I like the clean interface. 
>
> Some comments: 
>
> - Can this handle arbitrary streams (e.g. something built on top of 
> channels)? 
>   That would make a nice extension! 
> - You may wish to look at k3 (open source version @ 
> https://github.com/kevinlawler/kona) 
>   In particular https://github.com/kevinlawler/kona/wiki#verbs 
>   There are some additional functions (verbs) and higher order functions 
> (adverbs) 
>   that may be of interest and will fit in nicely within your framework. 
>   PDFs of K 2.0 User and Reference manuals (listed under links) have much 
> more detail. 
> - May be you can provide another interface that *generates* Go code 
> instead of 
>   executing it! If it can be made to generate more efficient code, devoid 
> of 
>   reflect. 
>
>
-- 


"Este mensaje está
dirigido de manera exclusiva a su destinatario y puede 
contener información
privada y confidencial. No lo reenvíe, copie o 
distribuya a terceros que no
deban conocer su contenido. En caso de haberlo 
recibido por error,  rogamos
lo notifique al remitente y proceda a su 
borrado, así como al de cualquier
documento que pudiera adjuntarse.



 Por 
favor tenga en cuenta que
los correos enviados vía Internet no permiten 
garantizar la confidencialidad de
los mensajes ni su transmisión de forma 
íntegra.



 Las opiniones expresadas en el
presente correo pertenecen 
únicamente al remitente y no representan
necesariamente la opinión del 
Grupo BBVA."



 "This
message is intended exclusively for the adressee and 
may contain privileged and
confidential information. Please, do not 
disseminate, copy or distribute it to
third parties who should not receive 
it. In case you have received it by
mistake, please inform the sender and 
delete the message and attachments from
your system.



 Please
keep in 
mind that e-mails sent by Internet do not allow to guarantee neither
the 
confidentiality or the integrity of the messages sent."

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Reproducible builds and rewriting buildid in binary

[Ivan Daniluk]

Wrong link to the related Go's issue. 
It's https://github.com/golang/go/issues/16860.

On Sunday, September 16, 2018 at 2:49:19 PM UTC+3, Ivan Daniluk wrote:
>
> I needed to find a way to create reproducible builds, regardless of the 
> dev environment user uses. Luckily, Go gives almost everything needed for 
> that out-of-the-box, and there is a great blog post by Filippo on the 
> topic: https://blog.filippo.io/reproducing-go-binaries-byte-by-byte. If 
> we have the same Go version and the same set of dependencies (which is easy 
> when using vendor/ approach), the only problem is the difference in the 
> absolute path of the working directory. In other words, the same code, 
> built on the same dev environment in `GOPATH/src/project1` and 
> `GOPATH/src/project2` will yield different binaries. There is an open issue 
> for that in Go, and it will be hopefully addressed in Go 1.12 (
> https://github.com/status-im/status-react/issues/5587).
>
> For now, the easy approach, of course, is to use docker for the build, but 
> that feels too heavy just for ensuring the same dir. Spoofing directory 
> with LD_PRELOAD hacks or using `chroot` approach also have obvious 
> drawbacks – the need of C toolchain and root access, respectively.
>
> After analyzing the binaries, I realized that they differ only in buildid 
> stamp, the rest is the same. BuildiD is very well explained here: 
> https://github.com/golang/go/blob/master/src/cmd/go/internal/work/buildid.go#L24
>
> For a quick recap, every Go package or binary is stamped with buildid 
> value, which is essentially a 4 hash value:
>
>actionID(binary)/actionID(main.a)/contentID(main.a)/contentID(binary)
>
> where:
>  - actionID means a unique identifier of the inputs (sources, file names, 
> go version, etc)
>  - contentID means a unique identifier of the outputs (actual content 
> output by compiler/linker)
>
> So my thought went in the following direction – *I don't care if the 
> actionID (inputs) is different, but do care if contentID (outputs) are 
> different.*
>
> If contentID is equal, I can just rewrite actionID with "expected" one and 
> get the same binary byte-by-byte. This can be fully automated in Makefile 
> or script. So the steps for the reproducible build are the following:
>
>  - build binary - `go build -ldflags "-s -w" -asmflags=-trimpath="$(pwd)" 
> -gcflags=-trimpath="$(pwd)"`
>  - extract buildid - `go tool buildid myapp`
>  - compare buildid's contentID values to known ones - `diff <(go tool 
> buildid ./myapp  | cut -d'/' -f3) <(cat release.buildid.txt  | cut -d'/' 
> -f3)`
>  - if they're equal, assume that build is the same, and just rewrite the 
> buildid value inside the binary - `objcopy --update-section 
> .note.go.buildid=release.buildid.bin ./myapp` for ELF
>
> In my tests that result in byte-by-byte equal binaries.
>
> I have two concerns with this approach:
>  1) I might be missing some corner cases, especially with hacking binaries 
> of different formats. What perils of patching binary can be here?
>  2) buildID hash is actually a truncated version of real hash (259 to 67 
> bytes), which increases the collision probability and is totally fine for 
> the task "determine if binary should be rebuilt", but might be a concern 
> for the task "guarantee that the build is the same". More explanation here: 
> https://github.com/golang/go/blob/master/src/cmd/go/internal/work/buildid.go#L113
>
> Any thoughts on that? What else am I missing? Would this be a viable 
> workaround for having reproducible build until #5587 is solved?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Reproducible builds and rewriting buildid in binary

[Ivan Daniluk]

I needed to find a way to create reproducible builds, regardless of the dev 
environment user uses. Luckily, Go gives almost everything needed for that 
out-of-the-box, and there is a great blog post by Filippo on the 
topic: https://blog.filippo.io/reproducing-go-binaries-byte-by-byte. If we 
have the same Go version and the same set of dependencies (which is easy 
when using vendor/ approach), the only problem is the difference in the 
absolute path of the working directory. In other words, the same code, 
built on the same dev environment in `GOPATH/src/project1` and 
`GOPATH/src/project2` will yield different binaries. There is an open issue 
for that in Go, and it will be hopefully addressed in Go 1.12 
(https://github.com/status-im/status-react/issues/5587).

For now, the easy approach, of course, is to use docker for the build, but 
that feels too heavy just for ensuring the same dir. Spoofing directory 
with LD_PRELOAD hacks or using `chroot` approach also have obvious 
drawbacks – the need of C toolchain and root access, respectively.

After analyzing the binaries, I realized that they differ only in buildid 
stamp, the rest is the same. BuildiD is very well explained here: 
https://github.com/golang/go/blob/master/src/cmd/go/internal/work/buildid.go#L24

For a quick recap, every Go package or binary is stamped with buildid 
value, which is essentially a 4 hash value:

   actionID(binary)/actionID(main.a)/contentID(main.a)/contentID(binary)

where:
 - actionID means a unique identifier of the inputs (sources, file names, 
go version, etc)
 - contentID means a unique identifier of the outputs (actual content 
output by compiler/linker)

So my thought went in the following direction – *I don't care if the 
actionID (inputs) is different, but do care if contentID (outputs) are 
different.*

If contentID is equal, I can just rewrite actionID with "expected" one and 
get the same binary byte-by-byte. This can be fully automated in Makefile 
or script. So the steps for the reproducible build are the following:

 - build binary - `go build -ldflags "-s -w" -asmflags=-trimpath="$(pwd)" 
-gcflags=-trimpath="$(pwd)"`
 - extract buildid - `go tool buildid myapp`
 - compare buildid's contentID values to known ones - `diff <(go tool 
buildid ./myapp  | cut -d'/' -f3) <(cat release.buildid.txt  | cut -d'/' 
-f3)`
 - if they're equal, assume that build is the same, and just rewrite the 
buildid value inside the binary - `objcopy --update-section 
.note.go.buildid=release.buildid.bin ./myapp` for ELF

In my tests that result in byte-by-byte equal binaries.

I have two concerns with this approach:
 1) I might be missing some corner cases, especially with hacking binaries 
of different formats. What perils of patching binary can be here?
 2) buildID hash is actually a truncated version of real hash (259 to 67 
bytes), which increases the collision probability and is totally fine for 
the task "determine if binary should be rebuilt", but might be a concern 
for the task "guarantee that the build is the same". More explanation 
here: 
https://github.com/golang/go/blob/master/src/cmd/go/internal/work/buildid.go#L113

Any thoughts on that? What else am I missing? Would this be a viable 
workaround for having reproducible build until #5587 is solved?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [ANN] configuration file database mollydb

[ivan . corrales . solera]

Hey guys,

 A few weeks ago I was working on implementing a database that  managed 
configuration files (in the very first version would only deal with yml) . 
This database provide a GraphQL Api  which could be used to access to 
information but also to register hooks. 

I have some ideas to make this product more robust and *user-friendly *but  
not sure if this could be useful for other peopleI. I would appreciate some 
feedback from you

And in case of you like the idea and are interested in collaborating on ii 
please let me know!

*Links*

   - Code https://github.com/wesovilabs/mollydb 
   - How to use hooks on MollyDB? 
   
https://medium.com/wesovilabs/mollydb-in-a-microservice-architecture-ac19b79c6b4
   
Thanks in advanced

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Avoiding html/template's escaping of the + in

[ivan]

Hi

I have an HTML template where I'd like to provide  tags externally. 
I'd like to
leverage html/template's escaping for href. Therefore, it seemed easiest to 
just
create a struct with fields for rel, href, title and type.

Trying to just use string types means I get everything escaped. 
Surprisingly, this
means that + gets escaped into .

Since I've never before seen "application/atomxml", I'm not convinced 
that
everything I'd like interop with will correctly turn the HTML entity back 
into a +.

As the types don't actually vary that much currently, I am fine documenting 
in code that
either they should be constants, or that they should be separately 
sanitized. So I tried
using template.HTMLAttr and, when that failed, template.HTML to prevent 
escaping of the
plus.

This still gets escaped:

https://play.golang.org/p/eZxQrHy1vCE

Is this a bug? How do I avoid html/template escaping this, while still 
escaping
href?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [mysql] 2017/12/23 11:09:37 packets.go:141: write tcp 127.0.0.1:20630->127.0.0.1:3306: write: broken pipe

[Ivan Mladenovic]

Write:broken pipe is usually an indication that an idle connection has been 
closed by the server. 

There is an  API in database/sql package for this reason.
See https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime


Assuming you are using MySQL, MariaDB, you can read current timeout 
settings using command 


show variables like '%timeout%';

And the result will look lise this:


+-+--+
| Variable_name   | Value|
+-+--+
| connect_timeout | 5|
| deadlock_timeout_long   | 5000 |
| deadlock_timeout_short  | 1|
| delayed_insert_timeout  | 300  |
| innodb_flush_log_at_timeout | 1|
| innodb_lock_wait_timeout| 50   |
| innodb_rollback_on_timeout  | OFF  |
| interactive_timeout | 28800|
| lock_wait_timeout   | 31536000 |
| net_read_timeout| 30   |
| net_write_timeout   | 60   |
| slave_net_timeout   | 3600 |
| thread_pool_idle_timeout| 60   |
| wait_timeout| 28800|
+-+--+


In this case, the interactive_timeout  value is basically idle connection 
timeout. 

After 28800 seconds of inactivity, the server will automatically close the 
connection.


To prevent further errors set connection lifetime to value less than 
configured.


eg. db.SetConnMaxLifetime(time.Second * 14400)


On Saturday, December 23, 2017 at 2:23:21 PM UTC+1, jobs jobs wrote:
>
> how to fix it?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Github wiki pages publicly editable

[Ivan Borshukov]

Forwarding to the mailing list this time.

Cheers.

On Dec 14, 2017 18:14, "Ivan Borshukov" <bo0mer...@gmail.com> wrote:

Ok, thanks.

If so, I want to announce the change that I've maid. It is regarding the
rate limiting section [0].

The wiki page states that when the created ticker is Stop-ed, the created
goroutine will exit,
which I think is not true, since the Ticker.Stop's documentation[1] states
that Stop does not
result in Ticker.C being closed. A minimal example confirms my
understanding and results
in a panic due to deadlock.

```go
package main

import (
"sync"
"time"
)

func main() {
tick := time.NewTicker(time.Second)
throttle := make(chan time.Time, 1)

wg := sync.WaitGroup{}
wg.Add(1)
go func() {
defer wg.Done()
for t := range tick.C {
select {
case throttle <- t:
default:
}
} // Wiki page says: exits after tick.Stop()
}()

tick.Stop()
wg.Wait()
}
```

The above code in the playground: https://play.golang.org/p/oWrcCa25ki.

The diff can be viewed at [2].


[0] https://github.com/golang/go/wiki/RateLimiting
[1] https://godoc.org/time#Ticker.Stop
[2] https://github.com/golang/go/wiki/RateLimiting/_compare/
1a0ca7c57e7845c9ff05d296b721b46afa7c4f9e...67bbbe92527c42753
b45775ac981e14d89b2b95e

Cheers,
Ivan

2017-12-14 18:06 GMT+02:00 Jan Mercl <0xj...@gmail.com>:

> On Thu, Dec 14, 2017 at 5:05 PM Ivan Borshukov <bo0mer...@gmail.com>
> wrote:
>
> > I've noticed that I'm able to edit the wiki pages on
> https://github.com/golang/go/wiki and I wonder whether this is
> intentional.
>
> IINM, it's like that from the beginning of the Github repository, so
> probably intentional.
>
> --
>
> -j
>



-- 
Ivan Borshukov, bo0mer...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Variadic assignable values and covariance

[Ivan Kurnosov]

Jesse,

what you quoted is an optimisation for the "Otherwise, the value passed is
a new slice of type []T with a new underlying array whose successive
elements ".

Your quote says: if the final argument is assignable - it may be passed as
is, without any copy involving.


On 15 December 2017 at 12:59, Jesse McNelis <jes...@jessta.id.au> wrote:

> On Fri, Dec 15, 2017 at 8:18 AM, Ivan Kurnosov <zerk...@gmail.com> wrote:
> > Why does this code not compile: https://play.golang.org/p/f5fMvO8Ns7
> > func f(items ...interface{}) {
> >  fmt.Println(items)
> > }
>
> The spec also says:
>
> "If the final argument is assignable to a slice type []T, it may be
> passed unchanged as the value for a ...T parameter if the argument is
> followed by  In this case no new slice is created."
>
> https://golang.org/ref/spec#Passing_arguments_to_..._parameters
>
> That is, you can pass in multiple arguments which will be combined in
> to the items parameters as a slice or tab... can be passed as the
> items parameter. You can't do both.
>



-- 
With best regards, Ivan Kurnosov

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Variadic assignable values and covariance

[Ivan Kurnosov]

Why does this code not compile: https://play.golang.org/p/f5fMvO8Ns7

package main


import (
 "fmt"
)


func f(items ...interface{}) {
 fmt.Println(items)
}


func main() {
 f("a", "b", "c")
 tab := []interface{}{"d"}
 f("a", "b", "c", tab...)
}


The spec says:
> Otherwise, the value passed is a new slice of type []T with a new 
underlying array whose successive elements are the actual arguments, which 
all must be assignable  to T.

The elements are assignable. So what spec clause exactly prevents this from 
compile?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Github wiki pages publicly editable

[Ivan Borshukov]

Hello,

I've noticed that I'm able to edit the wiki pages 
on https://github.com/golang/go/wiki and I wonder whether this is 
intentional.

Cheers,
Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Golang on Android

[ivan . matmati]

Hi there,


I'm sure you already tried to find an apk to run your go code on your 
Android tablet.
Certainly, like me you hit a wall. But I've juste discovered something 
unnoticed by most of us.
You can install golang on Android !
How ? Look at my post : https://www.linkedin.com/groups/3712244
Bye.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] roger, an app configuration library experiment

[Ivan Fraixedes]

Thanks for this library, I also found some of those missing features that 
you listed in the README.

On Monday, 9 October 2017 00:56:32 UTC+2, Alex Buchanan wrote:
>
> My weekend project for that past couple weeks has been roger: 
> https://github.com/buchanae/roger
>
> This is another take on application configuration management: flags, env 
> vars, YAML, etc.
>
> The core ideas are:
> - define config as structs
> - defaults are defined by functions which return instances of structs
> - static code analysis and generation provides some extra metadata, 
> pulling docs from code comments on struct fields
> - able to dump YAML with docs and exclude default/empty values
>
> Other than that, it's probably just a more buggy version of existing 
> libraries (such as viper) with less features. I'll probably keep chipping 
> away at it though.
>
> It also:
> - generates flags
> - loads values from environment variables
> - loads values from a YAML file
> - coerces time.Duration and github.com/alecthomas/units and others via 
> github.com/spf13/cast
>
> More in the README
>
> Thanks!
> Alex
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] github.com/golang/glog: Using replacement flag packages

[Ivan Vučica]

Hello,

Since glog's readme recommends firing off an email to golang-nuts to
report possible issues, here I go.

I really like the idea of specifying flags in flagfiles and
environment variables. This Go package seems to let me do that, while
staying pretty much API-compatible with the standard flag package:
  github.com/namsral/flag

However, if I start using it as a replacement for "flag" package, the
program (naturally) only recognizes flags specified using namsral's
flag package. This means that the flags defined in the glog package
are not visible to namsral's package, and thus are not visible to the
user.

Since variables backing the glog flags are not part of the public
interface, I can't, really, write a nice wrapper. Instead I could
write a wrapper that reads the values from namsral/flags-created
variables, and uses flag.Set() to set them into Go's flag package, and
then calls Go's flag.Parse().

That feels bad. Does anyone have any suggestions on alternative
approaches? Could the variables backing the flags be exposed, so I can
do my own calls to flag.BoolVar()/flag.Var()?

Would it make sense to extend Go's flag package to use the environment
and flagfiles in addition to just command line arguments?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Implicit cast of string vs. []byte

[Ivan Bertona]

I see, thank you!

On Saturday, June 24, 2017 at 5:46:55 PM UTC-7, Ayan George wrote:
>
>
> On 06/24/2017 08:16 PM, Ivan Bertona wrote: 
> > Hello, 
> > 
> > When I try to compile this piece of code: 
> > 
>
> I think it violates the second assignability rule since both 'string' 
> and MyString are named types.  string happens to be a a pre-declared 
> named type: 
>
>   https://golang.org/ref/spec#Assignability 
>
> Whereas []byte is unnamed therefore at least one of the types in the 
> expression is unnamed. 
>
> Here are a couple of relevant blog posts about it: 
>
>   https://medium.com/learning-the-go-programming-language 
>   http://www.laktek.com/2012/01/27/learning-go-types/ 
>
> Someone please correct me if I'm wrong. 
>
> -ayan 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Implicit cast of string vs. []byte

[Ivan Bertona]

Hello,

When I try to compile this piece of code:

package main

type MyString string
type MyBytes []byte

func GetString() (string, error) {
return "", nil
}

func GetBytes() ([]byte, error) {
return nil, nil
}

func main() {
var s MyString
var b MyBytes
var err error

s, err = GetString() 
b, err = GetBytes()
}

I get this error: tmp/sandbox008752331/main.go:19: cannot assign string to 
s (type MyString) in multiple assignment 

Does anyone know why implicit casting from []byte to MyBytes works, but it 
doesn't for string to MyString?

Best,
Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to find all func X calls using AST parser/types?

[Ivan Daniluk]

Hi,

let's say you want to find all calls to the specific function 
(strconv.Atoi, for example) using AST. It's kinda easy to do using 
Info.Uses, but how to go from Object in Uses to the whole expression with 
call?

Let's say, we have a code:
package main


import "strconv"


var fn = strconv.Atoi


func foo() func(string) (int, error) {
 return fn
}


var fn2 = foo()


func main() {
 strconv.Atoi("42")
 fn("42")
 fn2("42")
}

And I want to get this output:
 Found Atoi call at: line:10:2: strconv.Atoi("42")
 Found Atoi call at: line:11:2: fn("42")
 Found Atoi call at: line:12:2: fn2("42")

Using 'Info.Uses' only gives two entries - one is where Atoi is explicitly 
called, second, when it's assigned to fn, which is far from what I need.

Here is an example to start: https://play.golang.org/p/fIs3xWYfxk

Any advice?


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] encoding/asn1: Accept the non-PrintableString character underscore

[Ivan Borshukov]

Hi Gohpers,

I need to deal with certificates which contain '_'. Unfortunately, the '_' 
is not treated like a printable string character from Go.
I read through the discussions in [1], [2], but unfortunately none of them 
proposes a work around.

Can you suggest a solution other than forking the x509 and asn1 packages or 
building with custom standard library.

The exact piece of functionality i need is x509.ParseCertificate, which 
internally relies on asn1.Marshal.


Thanks in advance!
Regards,
Ivan

[1] https://github.com/golang/go/issues/14017
[2] https://groups.google.com/forum/#!msg/golang-nuts/_5LLpAZpwck/AiClr5ZvCwAJ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Turning unicode code string to rune

[Ivan Kurnosov]

1. Convert it to a number
2. Use `rune()`

On Sunday, April 23, 2017 at 6:51:09 AM UTC+12, Tong Sun wrote:
>
> Hi, 
>
> Given a unicode code string, be it "4e16", or "0x4e16", or "u4e16", how to 
> turn it into a single char rune?
>
> You can finish the code at https://play.golang.org/p/AFIEz3eJVz
>
> Thanks. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why sort.IsSorted implemented with decrement?

[Ivan Kurnosov]

Speaking low level - how about memory prefetch algorithms (os, hardware)? 
Do they work equally good when one iterates backward?

On Friday, April 21, 2017 at 3:50:39 PM UTC+12, andrey mirtchovski wrote:
>
> >  297 for i := n - 1; i > 0; i-- { 
>
> "i > 0" is cheaper than "i < n" on some processors :) 
>
> On Thu, Apr 20, 2017 at 3:14 AM,   wrote: 
> > Hi 
> > 
> > At the moment it is implemented as 
> > 
> >295func IsSorted(data Interface) bool { 
> >296n := data.Len() 
> >297for i := n - 1; i > 0; i-- { 
> >298if data.Less(i, i-1) { 
> >299return false 
> >300} 
> >301} 
> >302return true 
> >303} 
> > 
> > 
> > Is there any practical (technical) reason for this loop to be 
> > end-to-beginning, or is it just a preference of someone who implemented 
> it? 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why sort.IsSorted implemented with decrement?

[Ivan Kurnosov]

@Rob,

honestly to me they look the same:

func IsSorted(data Interface) bool {
n := data.Len()
for i := n - 1; i > 0; i-- {
if data.Less(i, i-1) {
return false
}
}
return true
}


func IsSortedForward(data sort.Interface) bool {
n := data.Len()
for i := 1; i < n; i++ {
if data.Less(i, i-1) {
return false
}
}
return true
}

https://play.golang.org/p/KSwwMk67ew

On Friday, April 21, 2017 at 2:19:26 PM UTC+12, Rob 'Commander' Pike wrote:
>
> Try it the other way. You'll see it's not so clean.
>
> -rob
>
>
> On Thu, Apr 20, 2017 at 7:47 PM, Ian Lance Taylor  > wrote:
>
>> On Thu, Apr 20, 2017 at 2:14 AM,   wrote:
>> >
>> > At the moment it is implemented as
>> >
>> >295 func IsSorted(data Interface) bool {
>> >296 n := data.Len()
>> >297 for i := n - 1; i > 0; i-- {
>> >298 if data.Less(i, i-1) {
>> >299 return false
>> >300 }
>> >301 }
>> >302 return true
>> >303 }
>> >
>> >
>> > Is there any practical (technical) reason for this loop to be
>> > end-to-beginning, or is it just a preference of someone who implemented 
>> it?
>>
>> Robert wrote it in
>>
>> https://github.com/golang/go/commit/18852cf6d3f23a4fbcf2756836eb929283126827
>> .  Let's ask him.  It was less than ten years ago.
>>
>> Ian
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Reflection: Constructing a struct type that satisfies an interface

[Ivan Vučica]

Cool, that's exactly the stuff I was interested in.

(I was actually trying to figure out why implementing "grpc-web" is not a
trivial problem. :))

On Thu, Feb 23, 2017 at 12:33 PM, Axel Wagner <axel.wagner...@googlemail.com
> wrote:

> From what I know, a) with the normal protobuf package, you can not en- or
> decode and then reflect messages by the proto descriptor, you need to
> compile in the generated code - adding methods to a reflect-created struct
> alone doesn't help, as the package also has some assumptions about the
> representation of the type, from what I can tell. And b) you can not invoke
> reflected RPC calls with the go grpc implementation. Related issues are
> https://github.com/golang/protobuf/issues/199
> https://github.com/grpc/grpc-go/issues/866
>
> On Thu, Feb 23, 2017 at 1:16 PM, Ivan Vučica <ivuc...@gmail.com> wrote:
>
>> What I want, in the end, is to send a gRPC request based on an ASCII or
>> JSON representation of a protobuf, receive a response and display it.
>>
>> Essentially, if you take grpc_cli tool present in the repo of C
>> implementation of gRPC, I would like to implement its "call" functionality.
>>
>> Even though trying to construct a binary protobuf /might/ be a dead end
>> depending on how I would have to invoke the RPC itself, building a
>> proto.Message out of a remote protobuf descriptor happens to be an
>> interesting problem in itself.
>>
>> It'd be less than fun if the solution is to write yet another proto
>> serializer/deserializer. And I didn't even check if, when using grpc-go, I
>> can invoke a remote gRPC method using just its name...
>>
>> Of course, if you have another shortcut that I missed in grpc-go's APIs
>> or in Go protobuf's APIs, that'd be great, too ☺️
>>
>> On Thu, Feb 23, 2017, 02:57 Matt Harden <matt.har...@gmail.com> wrote:
>>
>>> Is the intermediate Go struct necessary, or do you just want to convert
>>> a text proto to a binary representation?
>>>
>>> On Wed, Feb 22, 2017 at 6:10 PM <ivuc...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I'm fiddling with gRPC and its service reflection. I discovered a neat
>>> package (github.com/jhump/protoreflect) that let me quickly enumerate
>>> the services that are exposed by the gRPC server, the RPCs that are in
>>> those gRPC services, and finally the proto messages that are used as inputs
>>> and outputs.
>>>
>>> Later, I'll worry about how to actually send out the RPC.
>>>
>>> For now, I'm trying to unserialize a text proto message into a
>>> dynamically constructed Go struct, which I'd then serialize back into a
>>> binary proto.
>>>
>>> I'd like to make the generated struct satisfy the proto.Message
>>> interface. There are three methods required by that interface.
>>>
>>> How would one go about attaching the required methods onto the newly
>>> constructed type?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Reflection: Constructing a struct type that satisfies an interface

[Ivan Vučica]

What I want, in the end, is to send a gRPC request based on an ASCII or
JSON representation of a protobuf, receive a response and display it.

Essentially, if you take grpc_cli tool present in the repo of C
implementation of gRPC, I would like to implement its "call" functionality.

Even though trying to construct a binary protobuf /might/ be a dead end
depending on how I would have to invoke the RPC itself, building a
proto.Message out of a remote protobuf descriptor happens to be an
interesting problem in itself.

It'd be less than fun if the solution is to write yet another proto
serializer/deserializer. And I didn't even check if, when using grpc-go, I
can invoke a remote gRPC method using just its name...

Of course, if you have another shortcut that I missed in grpc-go's APIs or
in Go protobuf's APIs, that'd be great, too ☺️

On Thu, Feb 23, 2017, 02:57 Matt Harden  wrote:

> Is the intermediate Go struct necessary, or do you just want to convert a
> text proto to a binary representation?
>
> On Wed, Feb 22, 2017 at 6:10 PM  wrote:
>
> Hi,
>
> I'm fiddling with gRPC and its service reflection. I discovered a neat
> package (github.com/jhump/protoreflect) that let me quickly enumerate the
> services that are exposed by the gRPC server, the RPCs that are in those
> gRPC services, and finally the proto messages that are used as inputs and
> outputs.
>
> Later, I'll worry about how to actually send out the RPC.
>
> For now, I'm trying to unserialize a text proto message into a dynamically
> constructed Go struct, which I'd then serialize back into a binary proto.
>
> I'd like to make the generated struct satisfy the proto.Message interface.
> There are three methods required by that interface.
>
> How would one go about attaching the required methods onto the newly
> constructed type?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Using React with Golang? and does it make sense?

[ivan]

You don't have to use server-side rendering. You can have the React client 
make ajax requests to your Go backend.

On Monday, February 6, 2017 at 12:09:10 PM UTC-5, so.q...@gmail.com wrote:
>
> Correct me if I'm wrong, but in serving web apps and sites.
>
> Golang simply fills in the blanks for predefined templates.
>
> React appears to be a bit more complicated requiring the creation of 
> JavaScript/JSX classes representing the view. Having that rendered on the 
> server-side and the difference applied on the client.
>
> To combine the two, do I need to run Node.js on the server to process the 
> React/JSX classes before piping the results to Golang for responding to the 
> client? So Golang would be the conduit/middle-man filling in for Express 
> here.
>
> Does this even make sense? If I'm already running Node.js, does the use of 
> Golang feel superfluous?
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Golang and WebAssembly

[Ivan Perevezentsev]

Yes, WebAssembly would be really nice replacement for gopherjs, especially 
for game development.

Go already reached realtime garbage collection using native runtime, so 
theoretically we can make smooth games with 60 FPS. But gopherjs transpiles 
Go into JavaScript, which adds additional overhead. WebAssembly, on the 
other hand, is basically a sandbox for low-level bytecode which allows 
manual memory management.

I am personally interested in such feature, because I would like to try Go 
for creating mobile/web graphics app. Go is perfect candidate for gamedev 
and other realtime graphics apps, but it's potential isn't leveraged yet.

On Wednesday, July 8, 2015 at 4:08:22 AM UTC+3, Ilya Kowalewski wrote:
>
> Salute nuts,
>
> I read about WebAssembly (WA) recently and this topic haven't left my head 
> ever since. Am I right that if we had a Go-to-WA translator when actually 
> becomes standard, we'd been able to run Go applications on the front-end? 
> That makes hell lot of sense to me. We could kill JavaScript, once and for 
> all. Among all the modern languages, Go seems to be a pretty great 
> candidate for such a JS-killer.
>
> Go team, are you generally going to look into this topic in the near 
> future? IIRC, Google loved Go, e.g. for networking stuff, who knows, maybe 
> you guys would love it for front-end?
>
> Best regards,
> Ilya Kowalewski
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: playground URL to fetch code only

[Ivan Anfilatov]

https://play.golang.org/p/Vg6f0gSs3l.go

compile - post request to https://play.golang.org/compile in body request - 
source 
response - json


вторник, 20 сентября 2016 г., 16:09:47 UTC+3 пользователь Marvin Renich 
написал:
>
> Is there a way to take a playground URL, such as 
> https://play.golang.org/p/Vg6f0gSs3l, and modify it (e.g. with a query 
> string) so that you can retrieve just the code as text without the 
> surrounding html?  Something like 
>
> curl 'https://play.golang.org/p/Vg6f0gSs3l?text' 
>
> If not, would such a feature request be likely to get implemented? 
>
> Thanks...Marvin 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.