Re: [go-nuts] Re: Handling connection retries on a highly-available service

2019-02-14 Thread marc . zahn
I don't get it. Why are they supposed to be dropped only because the 
logging service is down?

Making a service between P and the logger is an interesting way to go
>
Another service which could be down then as well? Why not a Queue? 

Am Donnerstag, 14. Februar 2019 10:05:40 UTC+1 schrieb Michel Levieux:
>
> Hello everyone, thx for all your interesting answers!
>
> I think the fact that when the logger's down, the requests have to be 
> dropped (not queued, maybe I was not clear enough about that in my first 
> message) restrains our possibilities. Making a service between P and the 
> logger is an interesting way to go. For the moment we have made something 
> quite simple with atomic and some goroutines cooperating to know if the 
> connection is still correct, or to try reconnect when it's not, but I think 
> we will come back on that later.
>
> Le mer. 13 févr. 2019 à 14:54, Dany Xu  > a écrit :
>
>> As discuss above, i think the answer is decoupling the P and logger, 
>> storing the logs when the logger is down.The push and pull pattern would be 
>> better.The p sends all logs and the logger pulls all logs.Just keep a 
>> bigger storage for un-consumed logs.Queue may be is a better way but using 
>> a single storage.
>>
>> 在 2019年2月12日星期二 UTC+8上午12:34:46,Michel Levieux写道:
>>>
>>> Hi guys. I need a little help here.
>>>
>>> I work in a digital marketing company, where we have a program that 
>>> receives a lot of requests every second (counted in thousands) and logs its 
>>> behaviour via a logger that runs on another server. We are currently trying 
>>> to implement a connection-retry system between this program and its logging 
>>> API. What we want is :
>>>
>>> - We have a main program - let's call it P
>>> - P sends logs to the logger in multiple goroutines.
>>> - Sometimes we might need to shut down the logger (for maintenance or 
>>> anything)
>>> - We want P to keep running when the logger's down
>>> - Once the logger's up again, P must Dial it back automatically and 
>>> repair the *bufio.Writer associated with it
>>>
>>> Would you guys know a way not to check each single Read/Write if the 
>>> logger's up?
>>>
>>> Up to here we have thought of using atomic, mutexes and context for 
>>> synchronization, but the issues we face are the following:
>>>
>>> - mutexes create "pending" requests, since there's no way to check if a 
>>> mutex is locked or not
>>> - we're not really sure about the right way to use context for this 
>>> specific case
>>> - we'd like to avoid using atomics as much as possible, notably about 
>>> this quote from the docs : "*Except for special, low-level 
>>> applications, synchronization is better done with channels or the 
>>> facilities of the sync package*"
>>>
>>> In the end, what we're looking for is to reach a minimal checking 
>>> frequency (is connection up? do something, else do nothing), the ideal 
>>> being not to have to check anything.
>>>
>>> Have you guys already faced such problematics in the past? What 
>>> solutions have you come up with?
>>>
>>> Many thx in advance for your help!
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: Handling connection retries on a highly-available service

2019-02-12 Thread marc . zahn

>
> What about queuing
>

I cannot edit my post so a new answer here with more explanation: Just have 
a RabbitMQ or whatever in between and every logging sends a message to it 
while the logging service consumes it. As long as the logging service is 
down the messages remain in the queue.

Your retry approach has another doownside: What if the logging service is 
down for minutes or even hours? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: Handling connection retries on a highly-available service

2019-02-12 Thread marc . zahn
What about Queuing?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: Perf : Multithreaded goroutines files I/O

2018-09-16 Thread marc . zahn
You mean, loading the files in parallel take the ~same time as loading them 
sequentially? It seems that there is somewhere else a bottleneck...

Am Sonntag, 16. September 2018 14:08:50 UTC+2 schrieb Thomas Solignac:
>
> Hello,
>
> I have a loading step, where I have something like 60 files to read and 
> process, as fast as possible.
> I tried loading with goroutines and without, and I get substantially the 
> same process time (38s).
>
> *What is the more idiomatic ? Is Golang designed for concurrent files I/O 
> ?*
>
> Note : One file per goroutine (not multiple concurents I/O on the same 
> file)
>
> Thanks for reading :-)
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.