[go-nuts] Re: Handling EOF when using json.NewDecoder() from named pipe
On Tuesday, 17 October 2023 at 14:40:47 UTC+2 Christopher C wrote:
I was thinking partial reads could be an issue
In what regards are "partial reads" less well covered by io.ReadAll
than via json.Decoder?
and the Decoder seemed to do the initial checking for me. Would the
ReadAll() be able to recover from EOF state?
io.ReadAll cannot "recover" from EOF just like an Decoder cannot.
This EOF is smth that happens during _reading_ and both, io.ReadAll
and json.Decoder do the actual read via identical methods.
If your io.Reader allows reading up to EOF and than reading
more (no idea how this is going to happen, but let's assume
your underlying Reader somehow allows this) then it doesn't
matter.
io.ReadAll reads until EOF, forget about "partial reads".
Decoding can be done on the read stuff. I think this is
clearer.
Once your io.Reader signals EOF you have to find some
way to read more (???) or reset the Reader and this has
nothing to do with _how_ you read from that Reader.
V.
On Tuesday, October 17, 2023 at 4:37:58 AM UTC-4 Volker Dobler wrote:
Why do you use a json.Decoder? It seems as reading
everything (io.ReadAll) until EOF and json.Unmarshal'ling
would be a cleaner/simpler solution?
V.
On Tuesday, 17 October 2023 at 09:10:09 UTC+2 Christopher C wrote:
Hello all!
I'm trying to read json objects from a named pipe. The pipe will be
filled intermittently by bash scripts. After the Decode() of the first
object, any more calls to Decode() will return EOF. This seems proper
since the script has completed, but once it errors with EOF, there doesn't
seem to be a way to read any more.
Is there a way to 'reset' the decoder so when another script writes to the
pipe it can process the next object, or should I be doing some pipe length
validation before trying to decode?
Current read code snippet is...
decoder := json.NewDecoder(fpipe)
for {
err := decoder.Decode()
if err != nil {
if err == io.EOF {
// how to reset this?
} else {
logger.Fatal(err)
}
} else {
// send out the msg
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/a418fd99-e618-4b78-a2a8-aff2158f0c70n%40googlegroups.com.
[go-nuts] Re: Handling EOF when using json.NewDecoder() from named pipe
Are you sure that the writer isn't closing the named pipe between writing
JSON objects? If necessary you can check this with strace.
I'm pretty sure you'll get an EOF in the *reader* when the *writer* closes
from their side. You can demonstrate this with the shell:
(In terminal 1)
mkfifo /tmp/fifo
cat /tmp/fifo
(In terminal 2)
cat >/tmp/fifo
abcd
efgh
<< wait a while >>
^D
When you hit ^D in terminal 2, the cat in terminal 1 ends.
The solution (or as you asked, "how to reset this?") is simply to re-open
the fifo for reading.
On Tuesday, 17 October 2023 at 13:40:47 UTC+1 Christopher C wrote:
> I was thinking partial reads could be an issue and the Decoder seemed to
> do the initial checking for me. Would the ReadAll() be able to recover from
> EOF state?
>
> On Tuesday, October 17, 2023 at 4:37:58 AM UTC-4 Volker Dobler wrote:
>
>> Why do you use a json.Decoder? It seems as reading
>> everything (io.ReadAll) until EOF and json.Unmarshal'ling
>> would be a cleaner/simpler solution?
>>
>> V.
>>
>> On Tuesday, 17 October 2023 at 09:10:09 UTC+2 Christopher C wrote:
>>
>>> Hello all!
>>> I'm trying to read json objects from a named pipe. The pipe will be
>>> filled intermittently by bash scripts. After the Decode() of the first
>>> object, any more calls to Decode() will return EOF. This seems proper
>>> since the script has completed, but once it errors with EOF, there doesn't
>>> seem to be a way to read any more.
>>>
>>> Is there a way to 'reset' the decoder so when another script writes to
>>> the pipe it can process the next object, or should I be doing some pipe
>>> length validation before trying to decode?
>>>
>>> Current read code snippet is...
>>>
>>> decoder := json.NewDecoder(fpipe)
>>> for {
>>> err := decoder.Decode()
>>> if err != nil {
>>> if err == io.EOF {
>>> // how to reset this?
>>> } else {
>>> logger.Fatal(err)
>>> }
>>> } else {
>>> // send out the msg
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/75667b50-9eaf-4721-b3a4-75f023be753fn%40googlegroups.com.
[go-nuts] Re: Handling EOF when using json.NewDecoder() from named pipe
I was thinking partial reads could be an issue and the Decoder seemed to do
the initial checking for me. Would the ReadAll() be able to recover from
EOF state?
On Tuesday, October 17, 2023 at 4:37:58 AM UTC-4 Volker Dobler wrote:
> Why do you use a json.Decoder? It seems as reading
> everything (io.ReadAll) until EOF and json.Unmarshal'ling
> would be a cleaner/simpler solution?
>
> V.
>
> On Tuesday, 17 October 2023 at 09:10:09 UTC+2 Christopher C wrote:
>
>> Hello all!
>> I'm trying to read json objects from a named pipe. The pipe will be
>> filled intermittently by bash scripts. After the Decode() of the first
>> object, any more calls to Decode() will return EOF. This seems proper
>> since the script has completed, but once it errors with EOF, there doesn't
>> seem to be a way to read any more.
>>
>> Is there a way to 'reset' the decoder so when another script writes to
>> the pipe it can process the next object, or should I be doing some pipe
>> length validation before trying to decode?
>>
>> Current read code snippet is...
>>
>> decoder := json.NewDecoder(fpipe)
>> for {
>> err := decoder.Decode()
>> if err != nil {
>> if err == io.EOF {
>> // how to reset this?
>> } else {
>> logger.Fatal(err)
>> }
>> } else {
>> // send out the msg
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/8446c446-2113-4c65-bf3c-5a7f84c601dcn%40googlegroups.com.
[go-nuts] Re: Handling EOF when using json.NewDecoder() from named pipe
Why do you use a json.Decoder? It seems as reading
everything (io.ReadAll) until EOF and json.Unmarshal'ling
would be a cleaner/simpler solution?
V.
On Tuesday, 17 October 2023 at 09:10:09 UTC+2 Christopher C wrote:
> Hello all!
> I'm trying to read json objects from a named pipe. The pipe will be
> filled intermittently by bash scripts. After the Decode() of the first
> object, any more calls to Decode() will return EOF. This seems proper
> since the script has completed, but once it errors with EOF, there doesn't
> seem to be a way to read any more.
>
> Is there a way to 'reset' the decoder so when another script writes to the
> pipe it can process the next object, or should I be doing some pipe length
> validation before trying to decode?
>
> Current read code snippet is...
>
> decoder := json.NewDecoder(fpipe)
> for {
> err := decoder.Decode()
> if err != nil {
> if err == io.EOF {
> // how to reset this?
> } else {
> logger.Fatal(err)
> }
> } else {
> // send out the msg
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/0cb1918d-2eb9-41e1-86f0-8cd02e6a0113n%40googlegroups.com.
