On Wed, Nov 30, 2016 at 08:13:44PM +0100, Mehdi Abaakouk wrote:
>
>
> Le 2016-11-30 19:11, Nadya Shakhat a écrit :
> > Hi all,
> >
> > This one [1] is used very intensively actually :) It is not merged
> > because
> > we cannot just upgrade python-kafka version because of Monasca. We had
> > som
Le 2016-12-01 01:25, Tony Breeds a écrit :
So right now I think the answer is still no we can't uncap it. To the
best of
my knowledge monasca still suffers a substatial performance decrease
and the
switch from sync to async producers isn't desired for them.
Joe wrote this all up in:
http
On Thu, Dec 01 2016, Mehdi Abaakouk wrote:
> I'm aware of all of that, oslo.messaging patch for the new version is
> ready since 8 months now. And we are still blocked... capping libraries,
> whatever the reason, is very annoying and just freezes people work.
>
> From the API pov python-kafka have
On 12/1/16, 3:11 AM, "Julien Danjou" wrote:
>On Thu, Dec 01 2016, Mehdi Abaakouk wrote:
>
>> I'm aware of all of that, oslo.messaging patch for the new version is
>> ready since 8 months now. And we are still blocked... capping libraries,
>> whatever the reason, is very annoying and just freezes
Keen, Joe wrote:
I¹ll look into testing the newest version of kafka-python and see if it
meets our needs. If it still isn¹t stable and performant enough what are
the available options?
Fix the kafka-python library or fix monasca; those seem to be the
options to me :)
I'd also not like to bl
On Thu, Dec 01 2016, Keen, Joe wrote:
Hi Joe,
[…]
> The message context wrapped around every message is extra overhead we
> don¹t want.
> There¹s no support for batch sends to Kafka.
> There¹s no support for keyed producers.
> The forced auto_commit on consumers isn¹t something we can tolerate
On 12/1/16, 9:41 AM, "Joshua Harlow" wrote:
>Keen, Joe wrote:
>> I¹ll look into testing the newest version of kafka-python and see if it
>> meets our needs. If it still isn¹t stable and performant enough what
>>are
>> the available options?
>
>Fix the kafka-python library or fix monasca; those
Damn, that's supposed to say co-installability...
Lol,
-Josh
Joshua Harlow wrote:
Keen, Joe wrote:
I¹ll look into testing the newest version of kafka-python and see if it
meets our needs. If it still isn¹t stable and performant enough what are
the available options?
Fix the kafka-python lib
Reopen them?
If they weren't fixed, then seems like the right thing to try to do, and
rinse and repeat until actually fixed :-P
Keen, Joe wrote:
> We reported
> the bugs we found to the kafka-python project but they were closed once
> they released a new version.
On 12/1/16, 9:44 AM, "Julien Danjou" wrote:
>On Thu, Dec 01 2016, Keen, Joe wrote:
>
>Hi Joe,
>
>[…]
>
>> The message context wrapped around every message is extra overhead we
>> don¹t want.
>> There¹s no support for batch sends to Kafka.
>> There¹s no support for keyed producers.
>> The forced
On Thu, Dec 01, 2016 at 07:57:37AM +0100, Mehdi Abaakouk wrote:
> I'm aware of all of that, oslo.messaging patch for the new version is
> ready since 8 months now. And we are still blocked... capping libraries,
> whatever the reason, is very annoying and just freezes people work.
And I agree with
On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote:
> Keen, Joe wrote:
> > I¹ll look into testing the newest version of kafka-python and see if it
> > meets our needs. If it still isn¹t stable and performant enough what are
> > the available options?
>
> Fix the kafka-python library or
On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote:
> Unfortunately there’s nothing wrong on the Monasca side so far as we know.
> We test new versions of the kafka-python library outside of Monasca
> before we bother to try integrating a new version. Since 1.0 the
> kafka-python library
On Fri, Dec 02, 2016 at 03:19:26PM +1100, Tony Breeds wrote:
On Thu, Dec 01, 2016 at 07:57:37AM +0100, Mehdi Abaakouk wrote:
I think the solution is pretty simple. Just Fix it. I'm not saying it's easy
but it is *simple*. We're a group of skilled individuals We have several ways
forward which j
On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote:
On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote:
Unfortunately there’s nothing wrong on the Monasca side so far as we know.
We test new versions of the kafka-python library outside of Monasca
before we bother to try integra
On Fri, Dec 02, 2016 at 09:29:56AM +0100, Mehdi Abaakouk wrote:
And my bench seems to confirm the perf issue have been solved:
I have updated my requirement review to require >=1.3.1 [1] to solve
the monasca issue.
[1] https://review.openstack.org/404878
--
Mehdi Abaakouk
mail: sil...@sileht.
On Fri, Dec 02, 2016 at 09:39:41AM +0100, Mehdi Abaakouk wrote:
On Fri, Dec 02, 2016 at 09:29:56AM +0100, Mehdi Abaakouk wrote:
And my bench seems to confirm the perf issue have been solved:
I have updated my requirement review to require >=1.3.1 [1] to solve
the monasca issue.
[1] https://re
Excerpts from Tony Breeds's message of 2016-12-02 15:26:40 +1100:
> On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote:
> > Keen, Joe wrote:
> > > I¹ll look into testing the newest version of kafka-python and see if it
> > > meets our needs. If it still isn¹t stable and performant enoug
On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote:
>On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote:
>>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote:
>>
>>> Unfortunately there¹s nothing wrong on the Monasca side so far as we
>>>know.
>>> We test new versions of the kafka-pyt
On Fri, Dec 02, 2016 at 06:18:39PM +, Keen, Joe wrote:
>
>
> On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote:
>
> >On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote:
> >>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote:
> >>
> >>> Unfortunately there¹s nothing wrong on the Mo
On 12/4/16, 7:36 PM, "Tony Breeds" wrote:
>On Fri, Dec 02, 2016 at 06:18:39PM +, Keen, Joe wrote:
>>
>>
>> On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote:
>>
>> >On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote:
>> >>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote:
>>
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote:
> I wasn’t able to set a test up on Friday and with all the other work I
> have for the next few days I doubt I’ll be able to get to it much before
> Wednesday.
It's Wednesday so can we have an update?
Yours Tony.
signature.asc
Descript
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote:
> I don’t know, yet, that we can. Unless we can find an answer to the
> questions I had above I’m not sure that this new library will be
> performant and durable enough for the use cases Monasca has. I’m fairly
> confident that we can ma
Excerpts from Clint Byrum's message of 2016-12-02 09:44:40 -0800:
> Excerpts from Tony Breeds's message of 2016-12-02 15:26:40 +1100:
> > On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote:
> > > Keen, Joe wrote:
> > > > I¹ll look into testing the newest version of kafka-python and see i
Any progress ?
On Thu, Dec 08, 2016 at 08:32:54AM +1100, Tony Breeds wrote:
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote:
I wasn’t able to set a test up on Friday and with all the other work I
have for the next few days I doubt I’ll be able to get to it much before
Wednesday.
It'
The library final release is really soon, and we are still blocked on
this topic. If this is not solved, we will release one more time an
unusable driver in oslo.messging.
I want to remember that people current uses the kafka driver in
production with 'downstream patches' ready since 1 years to m
Mehdi,
I'd support switching g-r to make oslo.messaging work. period. This is
dragged on way too long.
Thanks,
Dims
On Wed, Jan 11, 2017 at 2:25 AM, Mehdi Abaakouk wrote:
> The library final release is really soon, and we are still blocked on
> this topic. If this is not solved, we will release
Tony, I have some observations on the new client based on a short term
test and a long running test.
For short term use it uses 2x the memory compared to the older client.
The logic that deals with receiving partial messages from Kafka was
completely rewritten in the 1.x series and with logging en
On 17/01/17 06:58 PM, Keen, Joe wrote:
>
> I also see additional problems with the use of the deprecated
> SimpleConsumer and SimpleProducer clients. We really do need to
> investigate migrating to the new async only Producer objects while still
> maintaining the reliability guarantees that Mona
Thanks Joe for all these details, I can see that Monasca is still
not able to switch to the new lib for new very good reasons.
But according your comment on https://review.openstack.org/#/c/420579/ :
I don't think that anyone currently using Monasca wants to accept either
of those options so we
On Wed, Jan 18 2017, Mehdi Abaakouk wrote:
> So, I agree with gordc, perhaps you should stay with the old and
> unsupported lib. And let other to use the supported one.
The best option would be for the Monasca folks to actually participate
upstream to the Kafka driver development and help getting
31 matches
Mail list logo