Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

Yes, I ended up using this package. Thanks.

On Wed, Jan 22, 2020 at 2:35 AM robfig  wrote:

> You could use json iterator’s Stream type if you prefer to feel better by
> having it wrapped in a library. It does the same thing of course. As the
> name suggests it can be used to incrementally write the json to a writer if
> that’s what you’d like.
> https://godoc.org/github.com/json-iterator/go#Stream
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/202ffb33-2f22-443b-9115-35d1fd93187d%40googlegroups.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0bxZUTyGP8anxmG4HpUvPa-uY%3DC8gCffiem2cNCiUvKRHg%40mail.gmail.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[robfig]

You could use json iterator’s Stream type if you prefer to feel better by 
having it wrapped in a library. It does the same thing of course. As the name 
suggests it can be used to incrementally write the json to a writer if that’s 
what you’d like. 
https://godoc.org/github.com/json-iterator/go#Stream

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/202ffb33-2f22-443b-9115-35d1fd93187d%40googlegroups.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

As responses coming from upstream server, because my application acting as
proxy server and it will not have  visibility of the fields of the
responses coming.
So, think appending the characters to meet JSON format is only way. Am I
right assuming so?

On Mon, Jan 20, 2020 at 4:07 PM Tamás Gulácsi  wrote:

> Please define "better".
>
> For complexity and speed, appending "[", inserting ",", and appending "]"
> at the end is the simplest and fastest solution.
>
> If you want to check for syntax and completeness, then you should
> unmarshal, append, and then marshal again.
> But if not needed, it's just complexity and resource hog.
>
> You can hide it in a function/library, but anyway, any solution would do
> one of the two solutions specified here.
> So it will be either "hacky" or "bloat".
>
> At least when the "hacky" solution is in plain sight, you'll know where to
> check first when syntax errors araise :-)
>
> Tamás
>
> 2020. január 20., hétfő 10:19:26 UTC+1 időpontban pc a következőt írta:
>>
>> Yes, I also need to put '[' at first and ']' at last position. But I was
>> checking if there is any better way to do it. I was even having question
>> how can someone make it better without unmarshalling JSON and remarshalling
>> merged object.
>>
>>
>> On Mon, Jan 20, 2020 at 4:12 AM Tamás Gulácsi  wrote:
>>
>>> Just put the missing [,] chars wher it's required with buf.WriteByte.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/00bb4351-e400-46a6-aec1-05124a966410%40googlegroups.com
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/71b5beb8-31e3-4894-9575-ee3c04514e2a%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0bzaFeUNnNLbiM34B_rVALpD%3DECJcR_nVvhXk1ZLyh6%3DyA%40mail.gmail.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Tamás Gulácsi]

Please define "better".

For complexity and speed, appending "[", inserting ",", and appending "]" 
at the end is the simplest and fastest solution.

If you want to check for syntax and completeness, then you should 
unmarshal, append, and then marshal again.
But if not needed, it's just complexity and resource hog.

You can hide it in a function/library, but anyway, any solution would do 
one of the two solutions specified here.
So it will be either "hacky" or "bloat".

At least when the "hacky" solution is in plain sight, you'll know where to 
check first when syntax errors araise :-)

Tamás

2020. január 20., hétfő 10:19:26 UTC+1 időpontban pc a következőt írta:
>
> Yes, I also need to put '[' at first and ']' at last position. But I was 
> checking if there is any better way to do it. I was even having question 
> how can someone make it better without unmarshalling JSON and remarshalling 
> merged object.
>
>
> On Mon, Jan 20, 2020 at 4:12 AM Tamás Gulácsi  > wrote:
>
>> Just put the missing [,] chars wher it's required with buf.WriteByte.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/00bb4351-e400-46a6-aec1-05124a966410%40googlegroups.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/71b5beb8-31e3-4894-9575-ee3c04514e2a%40googlegroups.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

Yes, I also need to put '[' at first and ']' at last position. But I was
checking if there is any better way to do it. I was even having question
how can someone make it better without unmarshalling JSON and remarshalling
merged object.


On Mon, Jan 20, 2020 at 4:12 AM Tamás Gulácsi  wrote:

> Just put the missing [,] chars wher it's required with buf.WriteByte.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/00bb4351-e400-46a6-aec1-05124a966410%40googlegroups.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0bzvHPbTDKiihuzMNxC40x%3DpNsgZNWh1momYTpDTVvzvfA%40mail.gmail.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Tamás Gulácsi]

Just put the missing [,] chars wher it's required with buf.WriteByte.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/00bb4351-e400-46a6-aec1-05124a966410%40googlegroups.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

Hello,

   I am able to buffer in types.Buffer the multiple response before writing
back to client, I am hit at a problem of JSON formatting the final output.
Example: after first response buf = {...}, after second buf={...}{...}.
Now it will be returned as is, which is not in proper json format. I need
to make this like [{...},{...}]

Any hints to achieve this?

Regards,
Prabhu

On Sat, Jan 18, 2020 at 9:23 PM Prabhu Chawandi 
wrote:

> Thank you for the pointers. I will dig more into them.
>
> Thanks,
> Prabhu
>
> On Sat, Jan 18, 2020 at 12:10 PM Tamás Gulácsi 
> wrote:
>
>> Start simple: use a *bytes.Buffer, and help the GC with using a sync.Pool
>> for those buffers.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/aea5ff9b-db0f-4bb5-abfb-82a2f54f7ae1%40googlegroups.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0bzbddQZyfPtZbjOpDoQ3%3DuUZHMfYUNu4fBPYG3R1XMDvw%40mail.gmail.com.

Re: [go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

Thank you for the pointers. I will dig more into them.

Thanks,
Prabhu

On Sat, Jan 18, 2020 at 12:10 PM Tamás Gulácsi  wrote:

> Start simple: use a *bytes.Buffer, and help the GC with using a sync.Pool
> for those buffers.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/aea5ff9b-db0f-4bb5-abfb-82a2f54f7ae1%40googlegroups.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0by3n5T54LhKq%3Doe5XAF%3DkbU2USY5TCE%3DYMaLF8XAmE0wg%40mail.gmail.com.

[go-nuts] Best way to buffer upstream responses in reverse proxy

[Tamás Gulácsi]

Start simple: use a *bytes.Buffer, and help the GC with using a sync.Pool for 
those buffers.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/aea5ff9b-db0f-4bb5-abfb-82a2f54f7ae1%40googlegroups.com.

[go-nuts] Best way to buffer upstream responses in reverse proxy

[Prabhu Chawandi]

Hello,
 For a single request from a client, I have to make multiple upstream
requests and send a single response back to the client.

I am looking for best way to buffer in the proxy and write all at once,
where GC is not hurting the performance, seamlessly work for multiple
client request simultaneously.

Regards,
pc

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAP3P0bxVKVoxRpSSbcK%2BsG7rdH0rZrqLpqVT5HqmJgYmWRrarw%40mail.gmail.com.