Re: [go-nuts] Go 1.16 and modules

2021-03-28 Thread Amit Saha
On Mon, Mar 29, 2021 at 12:15 PM Reto  wrote:
>
> On Mon, Mar 29, 2021 at 11:18:28AM +1100, Amit Saha wrote:
> > "Module-aware mode is enabled by default, regardless of whether a
> > go.mod file is present in the current working directory or a parent
> > directory. More precisely, the GO111MODULE environment variable now
> > defaults to on. To switch to the previous behavior, set GO111MODULE to
> > auto. "
> > Is that implying the above behavior?
>
> Yes: https://golang.org/ref/mod#mod-commands
>
> Note that you can still run the hello world programs via `go run` without
> having a module.

Thanks.

>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/20210329011453.3dddpz3syc6wv636%40feather.localdomain.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANODV3kx-%2BeP%3DjGYDMv0g%2BrNZ5vcrCpOrmc9mit0FJj-x2iFFw%40mail.gmail.com.


Re: [go-nuts] Go 1.16 and modules

2021-03-28 Thread Reto
On Mon, Mar 29, 2021 at 11:18:28AM +1100, Amit Saha wrote:
> "Module-aware mode is enabled by default, regardless of whether a
> go.mod file is present in the current working directory or a parent
> directory. More precisely, the GO111MODULE environment variable now
> defaults to on. To switch to the previous behavior, set GO111MODULE to
> auto. "
> Is that implying the above behavior?

Yes: https://golang.org/ref/mod#mod-commands

Note that you can still run the hello world programs via `go run` without
having a module.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20210329011453.3dddpz3syc6wv636%40feather.localdomain.


[go-nuts] Go 1.16 and modules

2021-03-28 Thread Amit Saha
Hi Folks,

I just realized that with Go 1.16, you must create a module via go mod
init, even if you are writing a simple hello world program.

The most surprising aspect to me was that I can only know that's the
case if I don't have a go.mod file anywhere in the parent directory
chain. It becomes quite a bit of a surprise element for folks writing
materials for others to follow.

Was this change documented in the release notes? We do  have this:

"Module-aware mode is enabled by default, regardless of whether a
go.mod file is present in the current working directory or a parent
directory. More precisely, the GO111MODULE environment variable now
defaults to on. To switch to the previous behavior, set GO111MODULE to
auto. "

Is that implying the above behavior?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANODV3m%2BLCDQem%2Bewny29SHXxvH-TtzW2zHuJ9-08TFX%3DSmYKw%40mail.gmail.com.


[go-nuts] Re: How to stream across json-seq RFC-7464

2021-03-28 Thread Greg Saylor

I've tried this suggestion and although its certainly a bit more 
refactoring then I expected - the outcome looks to be exactly as you 
described here.

Thank you so much for the suggestion, take a bow!

- Greg


On Sunday, March 28, 2021 at 12:15:34 PM UTC-7 Brian Candler wrote:

> No, it's even simpler than that:
>
> * The first call to decoder.Decode() will return the first object in the 
> stream.
> * The second call to decoder.Decode() will return the second object in the 
> stream.
> * And so on...
>
> By "object" I mean top-level object: everything between the opening "{" 
> and its matching closing "}", including all its nested values.  (Define a 
> struct which contains all the nested attributes, for it to be deserialized 
> into).
>
> If an io.Reader stream consists of a series of separate JSON objects - as 
> yours does - then you get one object at a time.  They don't have to be 
> separated by whitespace or newlines, but they can be.
>
> Don't think about seeking.  I don't know the internals of 
> decoder.Decode(), but I would expect that it reads in chunks from the 
> io.Reader.  This means it will likely overshoot the object boundaries, but 
> will buffer the excess and process it on the next call to Decode.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d3d1a980-deeb-4d03-afca-65e32a242bb9n%40googlegroups.com.


[go-nuts] Re: Difference between cmd/compile/internal/syntax/parser.go and pkg/go/parser/parser.go

2021-03-28 Thread philne...@gmail.com
The interesting thing furthermore is that it seems like the 
pkg/go/parser/parser.go has been around since Go 1.0 whereas the 
cmd/compile/internal was introduced in 1.6 when they translated the yacc 
parser to handwritten Go.

https://go-review.googlesource.com/c/go/+/16665/

On Saturday, March 27, 2021 at 11:47:05 PM UTC-4 aind...@gmail.com wrote:

> I won't speak for the maintainers of cmd/compile/internal/syntax, but 
> packages in the standard library have to be backwards compatible with 
> previous releases. This makes it difficult to make changes to its interface 
> (possibly for performance reasons) and major re-architectures of its 
> implementation (like coupling it with other parts of cmd/compile).
>
> On Saturday, March 27, 2021 at 8:07:38 PM UTC-4 philne...@gmail.com wrote:
>
>> Hey folks,
>>
>> Why does Go reimplement the parser in pkg/go/parser on top of the one in 
>> cmd/compile/internal? Why have two packages with somewhat duplicate code? 
>> My guess is that it's easier to control what is public-public (available to 
>> authors of Go programs) vs public within the compiler by having the partial 
>> duplicate two packages?
>>
>> Happy for links if this question has been asked before.
>>
>> Thank you!
>> Phil
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3c15088e-d701-4cca-be0e-204cd30caa33n%40googlegroups.com.


[go-nuts] Re: How to stream across json-seq RFC-7464

2021-03-28 Thread Brian Candler
No, it's even simpler than that:

* The first call to decoder.Decode() will return the first object in the 
stream.
* The second call to decoder.Decode() will return the second object in the 
stream.
* And so on...

By "object" I mean top-level object: everything between the opening "{" and 
its matching closing "}", including all its nested values.  (Define a 
struct which contains all the nested attributes, for it to be deserialized 
into).

If an io.Reader stream consists of a series of separate JSON objects - as 
yours does - then you get one object at a time.  They don't have to be 
separated by whitespace or newlines, but they can be.

Don't think about seeking.  I don't know the internals of decoder.Decode(), 
but I would expect that it reads in chunks from the io.Reader.  This means 
it will likely overshoot the object boundaries, but will buffer the excess 
and process it on the next call to Decode.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4faddcbc-1eb4-48f1-8320-fe9661811eeen%40googlegroups.com.


[go-nuts] Re: How to stream across json-seq RFC-7464

2021-03-28 Thread Greg Saylor
The inner blob is expecting an io.Reader.  But, perhaps I can change that 
to pass a Decoder based on what you are saying.   For some reason I hadn't 
grokked that is how Decoder was working.  Just to re-iterate what I think 
you are saying (and in case anyone stumbles across this thread later), 
assuming a file that has this type of structure (call each of the outer 
blobs A, B, C for reference):

{
  [
   {...},
   {...}
  ]
}
{
  [
   {...},
   {...}
  ]
}
[
  {...},
  {...}
]


The first call to Decoder() will move the pointer to the first `{` in A.
   Something like exponen-io.jsonpath Seek() could be used to advance to 
A's `[`
   The second call to Decoder(), with the embedded reader,  will set the 
position at A's first inner {...}
   Each subsequent call to Decode() will process each inner {...} of A one 
at a time until More() is false,  at which point the position is at A's `]`

The third call to Decoder() will move the pointer to the first `{` in B.  
*Question: 
Is this in fact correct?  If not how to I get reader to this point of the 
stream?*
   The fourth call to Decoder() will allow me to stream read to B's `[` (in 
this case using exponent-io.jsonpath SeekTo() or some other mechanism)
   Each subsequent call to Decode() will process each inner {...} of B one 
at a time  until More() is false, at which point the position is at B's `]`

The fifth call to Decoder() will move the pointer to the first `[` in C.
   Each subsequent call to Decode() will process each inner {...} of C one 
at a time until More() is false


I realize this may not what actually is going internally inside these 
packages, but at a high level is that conceptually something approaching 
what is going on?

If this is true, I gotta say this is one of the things I *LOVE* about Go. 
 I cannot count the number of times I had some complicated problem which, 
which Go makes a  whole lot easier.  Or put another way: I was 
over-complicating the problem and not recognizing the underlying code 
defect which should change.   In fact, even refactoring this code even 
though its used in about 100 places would be trivial.  I could probably 
just use perl -pie to fix the code.  And also, if I may be a bit indulgent 
here, the quality of the answers that come out of the Golang community are 
just amazing.  I love reading this mailing list even though I've only 
posted to it a few times.

- Greg


On Sunday, March 28, 2021 at 1:26:17 AM UTC-7 Brian Candler wrote:

> > This works, but the downside is that each {...} of bytes has to be 
> pulled into memory.  And the functions that is called is already designed 
> to receive an io.Reader and parse the VERY large inner blob in an efficient 
> manner.
>
> Is the inner blob decoder actually using a json.Decoder, as shown in your 
> example func secondDecoder()?  In that case, the simplest and most 
> efficient answer is to create a persistent json.Decoder which wraps the 
> underlying io.Reader directly, and just keep calling w2.Decode() on each 
> call.  It will happily consume the stream, one object at a time.
>
> If that's not possible for some reason, then it sounds like you want to 
> break the outer stream at outer object boundaries, i.e. { ... }, without 
> fully parsing it.  You can do that with json.RawMessage:
> https://play.golang.org/p/BitE6l27160
>
> However, you've still read each object as a stream of bytes into memory, 
> and you've still done some of the work of parsing the JSON to find the 
> start and end of each object.  You can turn it back into an io.Reader by 
> creating a bytes.NewBuffer around it, if that's what the inner parser 
> requires.   However if each object is large, and you really need to avoid 
> reading it into memory at all, then you'd need some sort of rewindable 
> stream.
>
> Another approach is to stop the source generating pretty-printed JSON, and 
> make it generate in JSON-Lines  format instead.  
> It sounds like you're unable to change the source, but you might be able to 
> un-prettyprint the JSON by using an external tool (perhaps jq can do 
> this).  Then I am thinking you could make a custom io.Reader which returns 
> data up to a newline, then sends EOF and sends you a fresh io.Reader for 
> the next line.
>
> But this is all very complicated, when keeping the inner Decoder around 
> from object to object is a simple solution to the problem that you 
> described.  Is there some other constraint which prevents you from doing 
> this?
>
> On Saturday, 27 March 2021 at 19:42:40 UTC greg.sa...@gmail.com wrote:
>
>> Good afternoon,
>>
>> For a case where there's a file containing a sequence of hashes (it could 
>> be arrays too, as the underlying object type seems irrelevant) as per 
>> RFC-7464.  I cannot figure out how to handle this in a memory efficient way 
>> that doesn't involve pulling each blob 
>>
>> I've tried to express this on Go playground here: 
>> https://play.golang.org/p/Aqx0gnc39rn
>> Note that I'm 

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

2021-03-28 Thread Ian Lance Taylor
On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote:
>
> For s2-geojson, I meet:
> ```
> # github.com/pantrif/s2-geojson/cmd/s2-geojson
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
> reference to 'reflect.unsafe_New'
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
> undefined reference to 'reflect.unsafe_New'
> decode.go:509: error: undefined reference to 'reflect.unsafe_New'
> decode.go:14935: error: undefined reference to 'reflect.unsafe_New'

This is a problem with github.com/ugorji/go.  The file
code/helper_unsafe.go uses go:linkname to refer directly to functions
that are defined but not exported by the standard library.  This is
not supported and is likely to break with any new release.  It
evidently breaks with GoLLVM.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU6VDXs02NQwooJY63GLC3794%2BBdX__UOaTnBj5KbhX2Q%40mail.gmail.com.


[go-nuts] gollvm hasn't a good support to reflect and pthread?

2021-03-28 Thread 张嘉熙
I made a issue at github, and the details are as followed:

### What version of Go are you using (`go version`)?


$ go version
go version go1.16 gollvm LLVM 13.0.0git linux/amd64


### Does this issue reproduce with the latest release?
yes


### What operating system and processor architecture are you using (`go 
env`)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/home/jx/workspace/gowork/bin"
GOCACHE="/home/jx/.cache/go-build"
GOENV="/home/jx/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jx/workspace/gowork/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jx/workspace/gowork"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct;
GOROOT="/home/jx/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jx/.local/gollvm/tools"
GOVCS=""
GOVERSION="go1.16 gollvm LLVM 13.0.0git"
GCCGO="/home/jx/.local/gollvm/bin/llvm-goc"
AR="ar"
CC="/home/jx/workspace/llvm_area/llvm-project/build/bin/clang"
CXX="/home/jx/workspace/llvm_area/llvm-project/build/bin/clang++"
CGO_ENABLED="1"
GOMOD="/home/jx/workspace/batch_compile/awesome-go/picfit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments 
-fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4288114194=/tmp/go-build 
-gno-record-gcc-switches -funwind-tables"



### What did you do?


I attempted to compile mounts of go projects with gollvm, but some of them 
meet reference error to `reflect` or `pthread`
Some cases are followed:
1. https://github.com/pantrif/s2-geojson
This case meet `error: undefined reference to 'reflect.unsafe_New` while 
compiling
```
git clone https://github.com/pantrif/s2-geojson.git
cd s2-geojson/cmd/s2-geojson
go build .
```
`Go` mentioned above is just gollvm

2. https://github.com/thoas/picfit
This case meet 'undefined reference to 'pthread_once' and many other 
functions start with `pthread_`
```
git clone https://github.com/thoas/picfit.git
cd picfit/cmd/picfit
go build .
```

### What did you expect to see?
Projects are compiled successfully as the original go compiler


### What did you see instead?
For s2-geojson, I meet:
```
# github.com/pantrif/s2-geojson/cmd/s2-geojson
/home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
 
github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
reference to 'reflect.unsafe_New'
/home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
 
github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
undefined reference to 'reflect.unsafe_New'
decode.go:509: error: undefined reference to 'reflect.unsafe_New'
decode.go:14935: error: undefined reference to 'reflect.unsafe_New'
```

For picfit, I meet:
build details Output
# github.com/thoas/picfit/cmd/picfit
src/libavcodec/aacdec_template.c:1143: error: undefined reference to 
'pthread_once'
src/libavcodec/h264dec.c:400: error: undefined reference to 'pthread_once'
src/libavcodec/pthread_frame.c:675: error: undefined reference to 
'pthread_join'
src/libavcodec/pthread_slice.c:116: error: undefined reference to 
'pthread_join'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 'pthread_mutexattr_init'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 
'pthread_mutexattr_settype'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 
'pthread_mutexattr_destroy'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::trylock(): error: undefined reference to 'pthread_mutex_trylock'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::TlsAbstraction(): error: undefined reference to 
'pthread_key_create'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::~TlsAbstraction(): error: undefined reference to 
'pthread_key_delete'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::GetData() const: error: undefined reference to 

[go-nuts] Re: io/fs adapters?

2021-03-28 Thread aind...@gmail.com
Thank you! I completely missed the memo on fstest :)

On Sunday, March 28, 2021 at 6:31:24 AM UTC-4 manlio@gmail.com wrote:

> For a project I wrote a simplified MapFS, where only the file data needs 
> to be specified, as a string:
> https://gist.github.com/perillo/45dfe7eb1c87e2d436574cab0d11221c
>
>
> Manlio
>
> Il giorno domenica 28 marzo 2021 alle 10:06:21 UTC+2 seank...@gmail.com 
> ha scritto:
>
>> https://pkg.go.dev/testing/fstest ?
>>
>> On Sunday, March 28, 2021 at 5:09:12 AM UTC+2 aind...@gmail.com wrote:
>>
>>> For testing library functions that accept fs.FS, I've been creating mock 
>>> FS implementations. However, I'm surprised by the amount of code I've had 
>>> to write for something like an FS with a single file in it. See below:
>>>
>>> *type singleFileFS struct {*
>>> *f stringFile*
>>> *}*
>>>
>>> *func (r singleFileFS) Open(name string) (fs.File, error) { return 
>>> nil, nil }*
>>> *func (r singleFileFS) Stat(name string) (fs.FileInfo, error) { return 
>>> r, nil }*
>>> *func (r singleFileFS) Name() string  { return 
>>> "." }*
>>> *func (r singleFileFS) Size() int64   { return 
>>> -1 }*
>>> *func (r singleFileFS) Mode() (m fs.FileMode) { return }*
>>> *func (r singleFileFS) ModTime() (t time.Time){ return }*
>>> *func (r singleFileFS) IsDir() bool   { return 
>>> true }*
>>> *func (r singleFileFS) Sys() interface{}  { return 
>>> nil }*
>>> *func (r singleFileFS) ReadDir(string) ([]fs.DirEntry, error) { return 
>>> []fs.DirEntry{r.f}, nil }*
>>> *func (r singleFileFS) ReadFile(name string) ([]byte, error)  { return 
>>> r.f.body, nil }*
>>>
>>> *type stringFile struct {*
>>> *name string*
>>> *body []byte*
>>> *}*
>>>
>>> *func (s stringFile) Stat() (fs.FileInfo, error)   { return s, nil }*
>>> *func (s stringFile) Name() string { return s.name 
>>>  }*
>>> *func (s stringFile) Size() int64  { return 
>>> int64(len(s.body)) }*
>>> *func (s stringFile) Mode() (m fs.FileMode){ return }*
>>> *func (s stringFile) Type() (m fs.FileMode){ return }*
>>> *func (s stringFile) ModTime() (t time.Time)   { return }*
>>> *func (s stringFile) IsDir() (false bool)  { return }*
>>> *func (s stringFile) Sys() (i interface{}) { return }*
>>> *func (s stringFile) Read([]byte) (i int, e error) { return }*
>>> *func (s stringFile) Close() (err error)   { return }*
>>> *func (s stringFile) Info() (fs.FileInfo, error)   { return s, nil }*
>>>
>>> Does anyone have any tips for creating a smaller version of this kind of 
>>> adapter? Or are there any FS adapter libraries that can help with this sort 
>>> of thing?
>>>
>>> Thanks,
>>> Akhil
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fcdf37ed-b43d-4632-a8e8-e3641c166c73n%40googlegroups.com.


Re: [go-nuts] I'm missing godoc for local packages after converting to modules.

2021-03-28 Thread Sean Liao
Assuming a recent enough version of `godoc`, running it from within a 
module should include the documentation for the standard library (standard 
library section), that module, and all its dependencies (third party 
section)
Expected output should be similar to:

$ godoc
using module mode; GOMOD=/home/arccy/testrepo-337/go.mod

It doesn't handle documentation across multiple, unrelated modules, but you 
could work around it by defining a dummy package & module with imports to 
the modules you wish to include (use replace to point to the local version) 
and running `godoc` from the dummy module

On Sunday, March 28, 2021 at 4:01:37 PM UTC+2 Hotei wrote:

> Clarification.  As mentioned earlier, $GOPATH is supposed to go away in 
> the (possibly near) future so one of my main goals is to get it working 
> outside the $GOPATH tree.  At present my solution has been to copy my 
> source tree (about 4 GB) to a "non-module-aware" go ecosystem on a 
> different machine and browse over my local network to the old godoc version 
> there.While it works (for the moment) it's not a very efficient or 
> maintainable solution.  The possibility of easy to maintain documentation 
> has always been one of go's strengths thanks to gofmt and godoc.  Hate to 
> lose half that advantage if there's a way around it.
> David Rook
>
> On Sunday, March 28, 2021 at 7:05:01 AM UTC-4 m8il...@gmail.com wrote:
>
>> I found you had to cd to each directory with a .mod file and run it 
>> there. A global option would be nice to know about.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/79476439-6459-42e9-bf6c-27f4ad962744n%40googlegroups.com.


[go-nuts] Re: How to wait for HTTP server to start?

2021-03-28 Thread Hotei
jerome - not sure that the code you provided fully answers the OP's 
problem.  I think you'd need to craft an http request and get a response to 
really KNOW that the server is indeed responding.  If you do that as a 
replacement for where you have the "select {}" then I think you've got it.

Note, the http request need not be complicated.  A simple "echo" style 
handler would do the trick.  As soon as you get the echo back then you know 
your server is ready to start handling other types of potentially more 
complex requests.  If the server has a TON of stuff (seconds to minutes of 
initialization)  you can make the echo server wait till all the prelim's 
are done first before sending the echo reply.  After startup you can also 
use the echo as a simple test of whether the server is at least running - 
even if more complicated requests are failing.
 
On Saturday, March 27, 2021 at 1:53:47 PM UTC-4 jerome@gmail.com wrote:

> When I want to ensure that the HTTP server is started (or if I want 
> additional stuff), I do this :
>
> https://play.golang.org/p/mX0OsNWFf-f
>
> Le samedi 27 mars 2021 à 15:13:40 UTC+1, cpu...@gmail.com a écrit :
>
>> The typical Go tutorials pattern for starting a server is something like 
>>
>> log.Fatal(http.ListenAndServe(":8080"))
>>
>> But what if the application needs to do other things after the server is 
>> started? It seems there is virtually no method to wait for the server 
>> actually start listening to requests?
>>
>> I'm probably missing something and would appreciate a hint.
>>
>> Thanks,
>> Andi
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ffeaa43b-bde3-4524-ae88-b1e7292108f1n%40googlegroups.com.


Re: [go-nuts] I'm missing godoc for local packages after converting to modules.

2021-03-28 Thread Hotei
Clarification.  As mentioned earlier, $GOPATH is supposed to go away in the 
(possibly near) future so one of my main goals is to get it working outside 
the $GOPATH tree.  At present my solution has been to copy my source tree 
(about 4 GB) to a "non-module-aware" go ecosystem on a different machine 
and browse over my local network to the old godoc version there.While 
it works (for the moment) it's not a very efficient or maintainable 
solution.  The possibility of easy to maintain documentation has always 
been one of go's strengths thanks to gofmt and godoc.  Hate to lose half 
that advantage if there's a way around it.
David Rook

On Sunday, March 28, 2021 at 7:05:01 AM UTC-4 m8il...@gmail.com wrote:

> I found you had to cd to each directory with a .mod file and run it there. 
> A global option would be nice to know about.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4015dad3-f54a-44e1-bb26-6eb8740a0991n%40googlegroups.com.


Re: [go-nuts] I'm missing godoc for local packages after converting to modules.

2021-03-28 Thread Kevin Chadwick
I found you had to cd to each directory with a .mod file and run it there.
A global option would be nice to know about.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANNfpqeTbX9xpTXRo5U-LR5J2upKgbigSgtNKKxDHC5by5SQ1w%40mail.gmail.com.


[go-nuts] Re: io/fs adapters?

2021-03-28 Thread Manlio Perillo
For a project I wrote a simplified MapFS, where only the file data needs to 
be specified, as a string:
https://gist.github.com/perillo/45dfe7eb1c87e2d436574cab0d11221c


Manlio

Il giorno domenica 28 marzo 2021 alle 10:06:21 UTC+2 seank...@gmail.com ha 
scritto:

> https://pkg.go.dev/testing/fstest ?
>
> On Sunday, March 28, 2021 at 5:09:12 AM UTC+2 aind...@gmail.com wrote:
>
>> For testing library functions that accept fs.FS, I've been creating mock 
>> FS implementations. However, I'm surprised by the amount of code I've had 
>> to write for something like an FS with a single file in it. See below:
>>
>> *type singleFileFS struct {*
>> *f stringFile*
>> *}*
>>
>> *func (r singleFileFS) Open(name string) (fs.File, error) { return 
>> nil, nil }*
>> *func (r singleFileFS) Stat(name string) (fs.FileInfo, error) { return r, 
>> nil }*
>> *func (r singleFileFS) Name() string  { return 
>> "." }*
>> *func (r singleFileFS) Size() int64   { return -1 
>> }*
>> *func (r singleFileFS) Mode() (m fs.FileMode) { return }*
>> *func (r singleFileFS) ModTime() (t time.Time){ return }*
>> *func (r singleFileFS) IsDir() bool   { return 
>> true }*
>> *func (r singleFileFS) Sys() interface{}  { return 
>> nil }*
>> *func (r singleFileFS) ReadDir(string) ([]fs.DirEntry, error) { return 
>> []fs.DirEntry{r.f}, nil }*
>> *func (r singleFileFS) ReadFile(name string) ([]byte, error)  { return 
>> r.f.body, nil }*
>>
>> *type stringFile struct {*
>> *name string*
>> *body []byte*
>> *}*
>>
>> *func (s stringFile) Stat() (fs.FileInfo, error)   { return s, nil }*
>> *func (s stringFile) Name() string { return s.name 
>>  }*
>> *func (s stringFile) Size() int64  { return 
>> int64(len(s.body)) }*
>> *func (s stringFile) Mode() (m fs.FileMode){ return }*
>> *func (s stringFile) Type() (m fs.FileMode){ return }*
>> *func (s stringFile) ModTime() (t time.Time)   { return }*
>> *func (s stringFile) IsDir() (false bool)  { return }*
>> *func (s stringFile) Sys() (i interface{}) { return }*
>> *func (s stringFile) Read([]byte) (i int, e error) { return }*
>> *func (s stringFile) Close() (err error)   { return }*
>> *func (s stringFile) Info() (fs.FileInfo, error)   { return s, nil }*
>>
>> Does anyone have any tips for creating a smaller version of this kind of 
>> adapter? Or are there any FS adapter libraries that can help with this sort 
>> of thing?
>>
>> Thanks,
>> Akhil
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/67947b8d-863a-49d3-be6c-16ae4983b7c7n%40googlegroups.com.


[go-nuts] Re: How to stream across json-seq RFC-7464

2021-03-28 Thread Brian Candler
> This works, but the downside is that each {...} of bytes has to be pulled 
into memory.  And the functions that is called is already designed to 
receive an io.Reader and parse the VERY large inner blob in an efficient 
manner.

Is the inner blob decoder actually using a json.Decoder, as shown in your 
example func secondDecoder()?  In that case, the simplest and most 
efficient answer is to create a persistent json.Decoder which wraps the 
underlying io.Reader directly, and just keep calling w2.Decode() on each 
call.  It will happily consume the stream, one object at a time.

If that's not possible for some reason, then it sounds like you want to 
break the outer stream at outer object boundaries, i.e. { ... }, without 
fully parsing it.  You can do that with json.RawMessage:
https://play.golang.org/p/BitE6l27160

However, you've still read each object as a stream of bytes into memory, 
and you've still done some of the work of parsing the JSON to find the 
start and end of each object.  You can turn it back into an io.Reader by 
creating a bytes.NewBuffer around it, if that's what the inner parser 
requires.   However if each object is large, and you really need to avoid 
reading it into memory at all, then you'd need some sort of rewindable 
stream.

Another approach is to stop the source generating pretty-printed JSON, and 
make it generate in JSON-Lines  format instead.  It 
sounds like you're unable to change the source, but you might be able to 
un-prettyprint the JSON by using an external tool (perhaps jq can do 
this).  Then I am thinking you could make a custom io.Reader which returns 
data up to a newline, then sends EOF and sends you a fresh io.Reader for 
the next line.

But this is all very complicated, when keeping the inner Decoder around 
from object to object is a simple solution to the problem that you 
described.  Is there some other constraint which prevents you from doing 
this?

On Saturday, 27 March 2021 at 19:42:40 UTC greg.sa...@gmail.com wrote:

> Good afternoon,
>
> For a case where there's a file containing a sequence of hashes (it could 
> be arrays too, as the underlying object type seems irrelevant) as per 
> RFC-7464.  I cannot figure out how to handle this in a memory efficient way 
> that doesn't involve pulling each blob 
>
> I've tried to express this on Go playground here: 
> https://play.golang.org/p/Aqx0gnc39rn
> Note that I'm using exponent-io/jsonpath as the JSON decoder, but 
> certainly that could be swapped for something else.
>
> In essence here is an example of the input bytes:
>
> {
>"elements" : [
>   {
>  "Space" : "YCbCr",
>  "Point" : {
> "Cb" : 0,
> "Y" : 255,
> "Cr" : -10
>  }
>   },
>   {
>  "Point" : {
> "B" : 255,
> "R" : 98,
> "G" : 218
>  },
>  "Space" : "RGB"
>   }
>]
> }
> {
>"elements" : [
>   {
>  "Space" : "YCbCr",
>  "Point" : {
> "Cb" : 3000,
> "Y" : 355,
> "Cr" : -310
>  }
>   },
>   {
>  "Space" : "RGB",
>  "Point" : {
> "B" : 355,
> "G" : 318,
> "R" : 108
>  }
>   }
>]
> }
> {
>"elements" : [
>   {
>  "Space" : "YCbCr",
>  "Point" : {
> "Cr" : -410,
> "Cb" : 400,
> "Y" : 455
>  }
>   },
>   {
>  "Space" : "RGB",
>  "Point" : {
> "B" : 455,
> "R" : 118,
> "G" : 418
>  }
>   }
>]
> }
>
> I can iterate through that with this code:
>
> w := json.NewDecoder(bytes.NewReader(j))
> for w.More() {
> var v interface{}
> w.Decode()
> fmt.Printf("%+v\n", v)
> }
>
> This works, but the downside is that each {...} of bytes has to be pulled 
> into memory.  And the functions that is called is already designed to 
> receive an io.Reader and parse the VERY large inner blob in an efficient 
> manner.
>
> So in principal, this is kinda want I want to do, but maybe I'm looking at 
> it all wrong:
>
>
> w := json.NewDecoder(bytes.NewReader(j))
> for w.More() {
> reader2 :=  //Some io.Reader that represents each of the 3 json-seq 
> blocks
> secondDecoder(reader2)
> }
>
> func secondDecoder(reader io.Reader) {
> w2 := json.NewDecoder(reader)
> var v interface{}
> w2.Decode()
> fmt.Printf("%+v\n", v)
> }
>
> Any ideas on how to solve this problem?
>
> I should note that it is not possible for the input to change in this case 
> as the system that consumes it is not the same one that has been generating 
> it for the past 5 years.
>
> Thanks!
>
> - Greg
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on 

[go-nuts] Re: io/fs adapters?

2021-03-28 Thread Sean Liao
https://pkg.go.dev/testing/fstest ?

On Sunday, March 28, 2021 at 5:09:12 AM UTC+2 aind...@gmail.com wrote:

> For testing library functions that accept fs.FS, I've been creating mock 
> FS implementations. However, I'm surprised by the amount of code I've had 
> to write for something like an FS with a single file in it. See below:
>
> *type singleFileFS struct {*
> *f stringFile*
> *}*
>
> *func (r singleFileFS) Open(name string) (fs.File, error) { return 
> nil, nil }*
> *func (r singleFileFS) Stat(name string) (fs.FileInfo, error) { return r, 
> nil }*
> *func (r singleFileFS) Name() string  { return "." 
> }*
> *func (r singleFileFS) Size() int64   { return -1 
> }*
> *func (r singleFileFS) Mode() (m fs.FileMode) { return }*
> *func (r singleFileFS) ModTime() (t time.Time){ return }*
> *func (r singleFileFS) IsDir() bool   { return 
> true }*
> *func (r singleFileFS) Sys() interface{}  { return nil 
> }*
> *func (r singleFileFS) ReadDir(string) ([]fs.DirEntry, error) { return 
> []fs.DirEntry{r.f}, nil }*
> *func (r singleFileFS) ReadFile(name string) ([]byte, error)  { return 
> r.f.body, nil }*
>
> *type stringFile struct {*
> *name string*
> *body []byte*
> *}*
>
> *func (s stringFile) Stat() (fs.FileInfo, error)   { return s, nil }*
> *func (s stringFile) Name() string { return s.name 
>  }*
> *func (s stringFile) Size() int64  { return 
> int64(len(s.body)) }*
> *func (s stringFile) Mode() (m fs.FileMode){ return }*
> *func (s stringFile) Type() (m fs.FileMode){ return }*
> *func (s stringFile) ModTime() (t time.Time)   { return }*
> *func (s stringFile) IsDir() (false bool)  { return }*
> *func (s stringFile) Sys() (i interface{}) { return }*
> *func (s stringFile) Read([]byte) (i int, e error) { return }*
> *func (s stringFile) Close() (err error)   { return }*
> *func (s stringFile) Info() (fs.FileInfo, error)   { return s, nil }*
>
> Does anyone have any tips for creating a smaller version of this kind of 
> adapter? Or are there any FS adapter libraries that can help with this sort 
> of thing?
>
> Thanks,
> Akhil
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ef40b17e-f8b4-4e9d-8060-ac72b9543360n%40googlegroups.com.