Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Wed, Jan 18 2017, Mehdi Abaakouk wrote: > So, I agree with gordc, perhaps you should stay with the old and > unsupported lib. And let other to use the supported one. The best option would be for the Monasca folks to actually participate upstream to the Kafka driver development and help getting the perf they want. That would be more constructive than staying at old version and finger-pointing the deficiencies of the new versions. (Maybe they are doing that but I did not see any pointer toward that direction). -- Julien Danjou ;; Free Software hacker ;; https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Thanks Joe for all these details, I can see that Monasca is still
not able to switch to the new lib for new very good reasons.
But according your comment on https://review.openstack.org/#/c/420579/ :
I don't think that anyone currently using Monasca wants to accept either
of those options so we need to find a way to maintain the current data
guarantees while using the async behaviour of the new client library.
That takes time and engineering effort to make that happen. Is there
anyone in the community willing to put in the effort to help build and
test these new features at scale?
Nobody have plan to fix this issues soon.
And about from the same review:
On another topic I'm curious what new features are you looking to get
out of the new library. Is there anything we can do to help you get the
capabilities you want with the existing client?
I don't think asking other projects to use deprecated and unsupported
lib version in new code is good, it's just adding fresh technical debt.
So, I agree with gordc, perhaps you should stay with the old and
unsupported lib. And let other to use the supported one.
On Tue, Jan 17, 2017 at 11:58:25PM +, Keen, Joe wrote:
Tony, I have some observations on the new client based on a short term
test and a long running test.
For short term use it uses 2x the memory compared to the older client.
The logic that deals with receiving partial messages from Kafka was
completely rewritten in the 1.x series and with logging enabled I see
continual warnings about truncated messages. I don’t lose any data
because of this but I haven’t been able to verify if it’s doing more reads
than necessary. I don’t know that either of these problems are really a
sticking point for Monasca but the increase in memory usage is potentially
a problem.
Long term testing showed some additional problems. On a Kafka server that
has been running for a couple weeks I can write data in but the
kafka-python library is no longer able to read data from Kafka. Clients
written in other languages are able to read successfully. Profiling of
the python-kafka client shows that it’s spending all it’s time in a loop
attempting to connect to Kafka:
276150.0860.0000.0860.000 {method 'acquire’ of
'thread.lock' objects}
431520.2500.0000.3850.000 types.py:15(_unpack)
431530.1350.0000.1350.000 {_struct.unpack}
48040/477980.1640.0000.1650.000 {len}
603510.2010.0000.2010.000 {method 'read’ of
'_io.BytesIO' objects}
7389962 23.9850.000 23.9850.000 {method 'keys' of ‘dict'
objects}
738 104.9310.000 395.6540.000 conn.py:560(recv)
738 58.3420.000 100.0050.000
conn.py:722(_requests_timed_out)
738 97.7870.000 167.5680.000 conn.py:588(_recv)
7390071 46.5960.000 46.5960.000 {method 'recv’ of
'_socket.socket' objects}
7390145 23.1510.000 23.1510.000 conn.py:458(connected)
7390266 21.4170.000 21.4170.000 {method 'tell’ of
'_io.BytesIO' objects}
7395664 41.6950.000 41.6950.000 {time.time}
I also see additional problems with the use of the deprecated
SimpleConsumer and SimpleProducer clients. We really do need to
investigate migrating to the new async only Producer objects while still
maintaining the reliability guarantees that Monasca requires.
On 12/13/16, 10:01 PM, "Tony Breeds" wrote:
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote:
I don’t know, yet, that we can. Unless we can find an answer to the
questions I had above I’m not sure that this new library will be
performant and durable enough for the use cases Monasca has. I’m fairly
confident that we can make it work but the performance issues with
previous versions prevented us from even trying to integrate so it will
take us some time. If you need an answer more quickly than a week or
so,
and if anyone in the community is willing, I can walk them through the
testing I’d expect to happen to validate the new library.
Any updates Joe? It's been 10 days and we're running close to Christamas
so
at this rate it'll be next year before we know if this is workable.
Yours Tony.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Mehdi Abaakouk
mail: sil...@sileht.net
irc: sileht
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 17/01/17 06:58 PM, Keen, Joe wrote: > > I also see additional problems with the use of the deprecated > SimpleConsumer and SimpleProducer clients. We really do need to > investigate migrating to the new async only Producer objects while still > maintaining the reliability guarantees that Monasca requires. > is there a reason why you are against bumping Kafka up for OpenStack? it seems Monasca requires 0.9.5 and is content with it. but the oslo team has developed something that works well for the use case for the broader OpenStack ecosystem. this seems like a better solution would be to just allow monasca to stay as is and the openstack requirements to progress rather than get blocked by a service that may or may not be deployed. the same thing was done when we bumped elasticsearch requirements. there are multiple projects using elasticsearch. we didn't have anyone working on it in Ceilometer so rather than block the entire community. we let it proceed and we could catch up later if it was urgent. cheers, -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Tony, I have some observations on the new client based on a short term
test and a long running test.
For short term use it uses 2x the memory compared to the older client.
The logic that deals with receiving partial messages from Kafka was
completely rewritten in the 1.x series and with logging enabled I see
continual warnings about truncated messages. I don’t lose any data
because of this but I haven’t been able to verify if it’s doing more reads
than necessary. I don’t know that either of these problems are really a
sticking point for Monasca but the increase in memory usage is potentially
a problem.
Long term testing showed some additional problems. On a Kafka server that
has been running for a couple weeks I can write data in but the
kafka-python library is no longer able to read data from Kafka. Clients
written in other languages are able to read successfully. Profiling of
the python-kafka client shows that it’s spending all it’s time in a loop
attempting to connect to Kafka:
276150.0860.0000.0860.000 {method 'acquire’ of
'thread.lock' objects}
431520.2500.0000.3850.000 types.py:15(_unpack)
431530.1350.0000.1350.000 {_struct.unpack}
48040/477980.1640.0000.1650.000 {len}
603510.2010.0000.2010.000 {method 'read’ of
'_io.BytesIO' objects}
7389962 23.9850.000 23.9850.000 {method 'keys' of ‘dict'
objects}
738 104.9310.000 395.6540.000 conn.py:560(recv)
738 58.3420.000 100.0050.000
conn.py:722(_requests_timed_out)
738 97.7870.000 167.5680.000 conn.py:588(_recv)
7390071 46.5960.000 46.5960.000 {method 'recv’ of
'_socket.socket' objects}
7390145 23.1510.000 23.1510.000 conn.py:458(connected)
7390266 21.4170.000 21.4170.000 {method 'tell’ of
'_io.BytesIO' objects}
7395664 41.6950.000 41.6950.000 {time.time}
I also see additional problems with the use of the deprecated
SimpleConsumer and SimpleProducer clients. We really do need to
investigate migrating to the new async only Producer objects while still
maintaining the reliability guarantees that Monasca requires.
On 12/13/16, 10:01 PM, "Tony Breeds" wrote:
>On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote:
>
>> I don’t know, yet, that we can. Unless we can find an answer to the
>> questions I had above I’m not sure that this new library will be
>> performant and durable enough for the use cases Monasca has. I’m fairly
>> confident that we can make it work but the performance issues with
>> previous versions prevented us from even trying to integrate so it will
>> take us some time. If you need an answer more quickly than a week or
>>so,
>> and if anyone in the community is willing, I can walk them through the
>> testing I’d expect to happen to validate the new library.
>
>Any updates Joe? It's been 10 days and we're running close to Christamas
>so
>at this rate it'll be next year before we know if this is workable.
>
>Yours Tony.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Mehdi, I'd support switching g-r to make oslo.messaging work. period. This is dragged on way too long. Thanks, Dims On Wed, Jan 11, 2017 at 2:25 AM, Mehdi Abaakouk wrote: > The library final release is really soon, and we are still blocked on > this topic. If this is not solved, we will release one more time an > unusable driver in oslo.messging. > > I want to remember that people current uses the kafka driver in > production with 'downstream patches' ready since 1 years to make it > works. > > We recently remove the kafka dep from oslo.messaging to be able to merge > some of these patches. But we can't untag the experimental flag of > this driver until the dependency issue is solved. > > So what can we do to unblock this situation ? > > > On Fri, Jan 06, 2017 at 02:31:28PM +0100, Mehdi Abaakouk wrote: >> >> Any progress ? >> >> On Thu, Dec 08, 2016 at 08:32:54AM +1100, Tony Breeds wrote: >>> >>> On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote: I wasn’t able to set a test up on Friday and with all the other work I have for the next few days I doubt I’ll be able to get to it much before Wednesday. >>> >>> >>> It's Wednesday so can we have an update? >>> >>> Yours Tony. >> >> >> -- >> Mehdi Abaakouk >> mail: sil...@sileht.net >> >> irc: sileht > > > -- > Mehdi Abaakouk > mail: sil...@sileht.net > ]]irc: sileht > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
The library final release is really soon, and we are still blocked on this topic. If this is not solved, we will release one more time an unusable driver in oslo.messging. I want to remember that people current uses the kafka driver in production with 'downstream patches' ready since 1 years to make it works. We recently remove the kafka dep from oslo.messaging to be able to merge some of these patches. But we can't untag the experimental flag of this driver until the dependency issue is solved. So what can we do to unblock this situation ? On Fri, Jan 06, 2017 at 02:31:28PM +0100, Mehdi Abaakouk wrote: Any progress ? On Thu, Dec 08, 2016 at 08:32:54AM +1100, Tony Breeds wrote: On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote: I wasn’t able to set a test up on Friday and with all the other work I have for the next few days I doubt I’ll be able to get to it much before Wednesday. It's Wednesday so can we have an update? Yours Tony. -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht -- Mehdi Abaakouk mail: sil...@sileht.net ]]irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Any progress ? On Thu, Dec 08, 2016 at 08:32:54AM +1100, Tony Breeds wrote: On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote: I wasn’t able to set a test up on Friday and with all the other work I have for the next few days I doubt I’ll be able to get to it much before Wednesday. It's Wednesday so can we have an update? Yours Tony. -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Excerpts from Clint Byrum's message of 2016-12-02 09:44:40 -0800: > Excerpts from Tony Breeds's message of 2016-12-02 15:26:40 +1100: > > On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote: > > > Keen, Joe wrote: > > > > I¹ll look into testing the newest version of kafka-python and see if it > > > > meets our needs. If it still isn¹t stable and performant enough what > > > > are > > > > the available options? > > > > > > Fix the kafka-python library or fix monasca; those seem to be the options > > > to > > > me :) > > > > Yup, Also worth including fix oslo.messaging to meet monasca's needs. But > > *something* needs fixing. > > > > > I'd also not like to block the rest of the world (from using newer > > > versions > > > of kafka-python) during this as well. But then this may diverge/expand > > > into > > > a discussion we had a few summits ago, about getting rid of > > > co-instability... > > > > lalalalala not listening ;P > > > > Less flippantly, there are a couple of ways to do this but IMO they're not > > in > > the best interest of OpenStack. > > > > 1. vendor/fork python-kafka 0.X > > 2. Stop the proposal-bot from syncing with monasca, thereby allowing it to > > use > > python-kafka 0.X at the expense of co-installability. > > Could there be a (3)? > > 3. change the oslo driver to work with the currently pinned > python-kafka version? Isn't the point that the pinned version, 0.9.5, is out of date? The current version is 1.3.1. Doug > > > > > Fortunately either option is easy to reverse once the underlying issue is > > fixed. > > > > Yours Tony. > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote: > I don’t know, yet, that we can. Unless we can find an answer to the > questions I had above I’m not sure that this new library will be > performant and durable enough for the use cases Monasca has. I’m fairly > confident that we can make it work but the performance issues with > previous versions prevented us from even trying to integrate so it will > take us some time. If you need an answer more quickly than a week or so, > and if anyone in the community is willing, I can walk them through the > testing I’d expect to happen to validate the new library. Any updates Joe? It's been 10 days and we're running close to Christamas so at this rate it'll be next year before we know if this is workable. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Mon, Dec 05, 2016 at 04:03:13AM +, Keen, Joe wrote: > I wasn’t able to set a test up on Friday and with all the other work I > have for the next few days I doubt I’ll be able to get to it much before > Wednesday. It's Wednesday so can we have an update? Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 12/4/16, 7:36 PM, "Tony Breeds" wrote: >On Fri, Dec 02, 2016 at 06:18:39PM +, Keen, Joe wrote: >> >> >> On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote: >> >> >On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote: >> >>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote: >> >> >> >>> Unfortunately there¹s nothing wrong on the Monasca side so far as we >> >>>know. >> >>> We test new versions of the kafka-python library outside of Monasca >> >>> before we bother to try integrating a new version. Since 1.0 the >> >>> kafka-python library has suffered from crashes and memory leaks >>severe >> >>> enough that we¹ve never attempted using it in Monasca itself. We >> >>>reported >> >>> the bugs we found to the kafka-python project but they were closed >>once >> >>> they released a new version. >> >> >> >>So Opening bugs isn't working. What about writing code? >> > >> >The bug https://github.com/dpkp/kafka-python/issues/55 >> > >> >Reopening it would be the right solution here. >> > >> >I can't reproduce the segfault neither and I agree with dpkp, that >>looks >> >like a >> >ujson issue. >> >> >> The bug I had was: https://github.com/dpkp/kafka-python/issues/551 >> >> In the case of that bug ujson was not an issue. The behaviour remained >> even using the standard json library. The primary issue I found with it >> was a memory leak over successive runs of the test script. Eventually >>the >> leak became so bad that the OOM killer killed the process which caused >>the >> segfault I was seeing. The last version I tested was 1.2.1 and it still >> leaked badly. I¹ll need to let the benchmark script run for a while and >> make sure it¹s not still leaking. > >So you write that on Friday so you shoudl know by now if it's leaking >care to >give us an update? I wasn’t able to set a test up on Friday and with all the other work I have for the next few days I doubt I’ll be able to get to it much before Wednesday. > >> >And my bench seems to confirm the perf issue have been solved: >> >(but not in the pointed version...) >> > >> >$ pifpaf run kafka python kafka_test.py >> >kafka-python version: 0.9.5 >> >... >> >fetch size 179200 -> 45681.8728864 messages per second >> >fetch size 204800 -> 47724.3810674 messages per second >> >fetch size 230400 -> 47209.9841092 messages per second >> >fetch size 256000 -> 48340.7719787 messages per second >> >fetch size 281600 -> 49192.9896743 messages per second >> >fetch size 307200 -> 50915.3291133 messages per second >> > >> >$ pifpaf run kafka python kafka_test.py >> >kafka-python version: 1.0.2 >> > >> >fetch size 179200 -> 8546.77931323 messages per second >> >fetch size 204800 -> 9213.30958314 messages per second >> >fetch size 230400 -> 10316.668006 messages per second >> >fetch size 256000 -> 11476.2285269 messages per second >> >fetch size 281600 -> 12353.7254386 messages per second >> >fetch size 307200 -> 13131.2367288 messages per second >> > >> >(1.1.1 and 1.2.5 have also the same issue) >> > >> >$ pifpaf run kafka python kafka_test.py >> >kafka-python version: 1.3.1 >> >fetch size 179200 -> 44636.9371873 messages per second >> >fetch size 204800 -> 44324.7085365 messages per second >> >fetch size 230400 -> 45235.8283208 messages per second >> >fetch size 256000 -> 45793.1044121 messages per second >> >fetch size 281600 -> 44648.6357019 messages per second >> >fetch size 307200 -> 44877.8445987 messages per second >> >fetch size 332800 -> 47166.9176281 messages per second >> >fetch size 358400 -> 47391.0057622 messages per second >> > >> >Looks like it works well now :) >> >> It¹s good that the performance problem has been fixed. The remaining >> issues on the Monasca side are verifying that the batch send method we >> were using in 0.9.5 still works with the new async behaviour, seeing if >> our consumer auto balance still functions or converting to use the Kafka >> internal auto balance in Kafka 0.10, and finding a way to do efficient >> synchronous writes with the new async methods. > >Can you +1 https://review.openstack.org/404878 to make it clear we can >proceed >down this path. I don’t know, yet, that we can. Unless we can find an answer to the questions I had above I’m not sure that this new library will be performant and durable enough for the use cases Monasca has. I’m fairly confident that we can make it work but the performance issues with previous versions prevented us from even trying to integrate so it will take us some time. If you need an answer more quickly than a week or so, and if anyone in the community is willing, I can walk them through the testing I’d expect to happen to validate the new library. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Fri, Dec 02, 2016 at 06:18:39PM +, Keen, Joe wrote: > > > On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote: > > >On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote: > >>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote: > >> > >>> Unfortunately there¹s nothing wrong on the Monasca side so far as we > >>>know. > >>> We test new versions of the kafka-python library outside of Monasca > >>> before we bother to try integrating a new version. Since 1.0 the > >>> kafka-python library has suffered from crashes and memory leaks severe > >>> enough that we¹ve never attempted using it in Monasca itself. We > >>>reported > >>> the bugs we found to the kafka-python project but they were closed once > >>> they released a new version. > >> > >>So Opening bugs isn't working. What about writing code? > > > >The bug https://github.com/dpkp/kafka-python/issues/55 > > > >Reopening it would be the right solution here. > > > >I can't reproduce the segfault neither and I agree with dpkp, that looks > >like a > >ujson issue. > > > The bug I had was: https://github.com/dpkp/kafka-python/issues/551 > > In the case of that bug ujson was not an issue. The behaviour remained > even using the standard json library. The primary issue I found with it > was a memory leak over successive runs of the test script. Eventually the > leak became so bad that the OOM killer killed the process which caused the > segfault I was seeing. The last version I tested was 1.2.1 and it still > leaked badly. I¹ll need to let the benchmark script run for a while and > make sure it¹s not still leaking. So you write that on Friday so you shoudl know by now if it's leaking care to give us an update? > >And my bench seems to confirm the perf issue have been solved: > >(but not in the pointed version...) > > > >$ pifpaf run kafka python kafka_test.py > >kafka-python version: 0.9.5 > >... > >fetch size 179200 -> 45681.8728864 messages per second > >fetch size 204800 -> 47724.3810674 messages per second > >fetch size 230400 -> 47209.9841092 messages per second > >fetch size 256000 -> 48340.7719787 messages per second > >fetch size 281600 -> 49192.9896743 messages per second > >fetch size 307200 -> 50915.3291133 messages per second > > > >$ pifpaf run kafka python kafka_test.py > >kafka-python version: 1.0.2 > > > >fetch size 179200 -> 8546.77931323 messages per second > >fetch size 204800 -> 9213.30958314 messages per second > >fetch size 230400 -> 10316.668006 messages per second > >fetch size 256000 -> 11476.2285269 messages per second > >fetch size 281600 -> 12353.7254386 messages per second > >fetch size 307200 -> 13131.2367288 messages per second > > > >(1.1.1 and 1.2.5 have also the same issue) > > > >$ pifpaf run kafka python kafka_test.py > >kafka-python version: 1.3.1 > >fetch size 179200 -> 44636.9371873 messages per second > >fetch size 204800 -> 44324.7085365 messages per second > >fetch size 230400 -> 45235.8283208 messages per second > >fetch size 256000 -> 45793.1044121 messages per second > >fetch size 281600 -> 44648.6357019 messages per second > >fetch size 307200 -> 44877.8445987 messages per second > >fetch size 332800 -> 47166.9176281 messages per second > >fetch size 358400 -> 47391.0057622 messages per second > > > >Looks like it works well now :) > > It¹s good that the performance problem has been fixed. The remaining > issues on the Monasca side are verifying that the batch send method we > were using in 0.9.5 still works with the new async behaviour, seeing if > our consumer auto balance still functions or converting to use the Kafka > internal auto balance in Kafka 0.10, and finding a way to do efficient > synchronous writes with the new async methods. Can you +1 https://review.openstack.org/404878 to make it clear we can proceed down this path. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 12/2/16, 1:29 AM, "Mehdi Abaakouk" wrote: >On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote: >>On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote: >> >>> Unfortunately there¹s nothing wrong on the Monasca side so far as we >>>know. >>> We test new versions of the kafka-python library outside of Monasca >>> before we bother to try integrating a new version. Since 1.0 the >>> kafka-python library has suffered from crashes and memory leaks severe >>> enough that we¹ve never attempted using it in Monasca itself. We >>>reported >>> the bugs we found to the kafka-python project but they were closed once >>> they released a new version. >> >>So Opening bugs isn't working. What about writing code? > >The bug https://github.com/dpkp/kafka-python/issues/55 > >Reopening it would be the right solution here. > >I can't reproduce the segfault neither and I agree with dpkp, that looks >like a >ujson issue. The bug I had was: https://github.com/dpkp/kafka-python/issues/551 In the case of that bug ujson was not an issue. The behaviour remained even using the standard json library. The primary issue I found with it was a memory leak over successive runs of the test script. Eventually the leak became so bad that the OOM killer killed the process which caused the segfault I was seeing. The last version I tested was 1.2.1 and it still leaked badly. I¹ll need to let the benchmark script run for a while and make sure it¹s not still leaking. > >And my bench seems to confirm the perf issue have been solved: >(but not in the pointed version...) > >$ pifpaf run kafka python kafka_test.py >kafka-python version: 0.9.5 >... >fetch size 179200 -> 45681.8728864 messages per second >fetch size 204800 -> 47724.3810674 messages per second >fetch size 230400 -> 47209.9841092 messages per second >fetch size 256000 -> 48340.7719787 messages per second >fetch size 281600 -> 49192.9896743 messages per second >fetch size 307200 -> 50915.3291133 messages per second > >$ pifpaf run kafka python kafka_test.py >kafka-python version: 1.0.2 > >fetch size 179200 -> 8546.77931323 messages per second >fetch size 204800 -> 9213.30958314 messages per second >fetch size 230400 -> 10316.668006 messages per second >fetch size 256000 -> 11476.2285269 messages per second >fetch size 281600 -> 12353.7254386 messages per second >fetch size 307200 -> 13131.2367288 messages per second > >(1.1.1 and 1.2.5 have also the same issue) > >$ pifpaf run kafka python kafka_test.py >kafka-python version: 1.3.1 >fetch size 179200 -> 44636.9371873 messages per second >fetch size 204800 -> 44324.7085365 messages per second >fetch size 230400 -> 45235.8283208 messages per second >fetch size 256000 -> 45793.1044121 messages per second >fetch size 281600 -> 44648.6357019 messages per second >fetch size 307200 -> 44877.8445987 messages per second >fetch size 332800 -> 47166.9176281 messages per second >fetch size 358400 -> 47391.0057622 messages per second > >Looks like it works well now :) It¹s good that the performance problem has been fixed. The remaining issues on the Monasca side are verifying that the batch send method we were using in 0.9.5 still works with the new async behaviour, seeing if our consumer auto balance still functions or converting to use the Kafka internal auto balance in Kafka 0.10, and finding a way to do efficient synchronous writes with the new async methods. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Excerpts from Tony Breeds's message of 2016-12-02 15:26:40 +1100: > On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote: > > Keen, Joe wrote: > > > I¹ll look into testing the newest version of kafka-python and see if it > > > meets our needs. If it still isn¹t stable and performant enough what are > > > the available options? > > > > Fix the kafka-python library or fix monasca; those seem to be the options to > > me :) > > Yup, Also worth including fix oslo.messaging to meet monasca's needs. But > *something* needs fixing. > > > I'd also not like to block the rest of the world (from using newer versions > > of kafka-python) during this as well. But then this may diverge/expand into > > a discussion we had a few summits ago, about getting rid of > > co-instability... > > lalalalala not listening ;P > > Less flippantly, there are a couple of ways to do this but IMO they're not in > the best interest of OpenStack. > > 1. vendor/fork python-kafka 0.X > 2. Stop the proposal-bot from syncing with monasca, thereby allowing it to use > python-kafka 0.X at the expense of co-installability. Could there be a (3)? 3. change the oslo driver to work with the currently pinned python-kafka version? > > Fortunately either option is easy to reverse once the underlying issue is > fixed. > > Yours Tony. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Fri, Dec 02, 2016 at 09:39:41AM +0100, Mehdi Abaakouk wrote: On Fri, Dec 02, 2016 at 09:29:56AM +0100, Mehdi Abaakouk wrote: And my bench seems to confirm the perf issue have been solved: I have updated my requirement review to require >=1.3.1 [1] to solve the monasca issue. [1] https://review.openstack.org/404878a And this is the update for all projects: https://review.openstack.org/#/q/status:open+branch:master+topic:sileht/kafka-update Nothing should block all of this anymore, except +2/+A :) -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Fri, Dec 02, 2016 at 09:29:56AM +0100, Mehdi Abaakouk wrote: And my bench seems to confirm the perf issue have been solved: I have updated my requirement review to require >=1.3.1 [1] to solve the monasca issue. [1] https://review.openstack.org/404878 -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Fri, Dec 02, 2016 at 03:29:59PM +1100, Tony Breeds wrote: On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote: Unfortunately there’s nothing wrong on the Monasca side so far as we know. We test new versions of the kafka-python library outside of Monasca before we bother to try integrating a new version. Since 1.0 the kafka-python library has suffered from crashes and memory leaks severe enough that we’ve never attempted using it in Monasca itself. We reported the bugs we found to the kafka-python project but they were closed once they released a new version. So Opening bugs isn't working. What about writing code? The bug https://github.com/dpkp/kafka-python/issues/55 Reopening it would be the right solution here. I can't reproduce the segfault neither and I agree with dpkp, that looks like a ujson issue. And my bench seems to confirm the perf issue have been solved: (but not in the pointed version...) $ pifpaf run kafka python kafka_test.py kafka-python version: 0.9.5 ... fetch size 179200 -> 45681.8728864 messages per second fetch size 204800 -> 47724.3810674 messages per second fetch size 230400 -> 47209.9841092 messages per second fetch size 256000 -> 48340.7719787 messages per second fetch size 281600 -> 49192.9896743 messages per second fetch size 307200 -> 50915.3291133 messages per second $ pifpaf run kafka python kafka_test.py kafka-python version: 1.0.2 fetch size 179200 -> 8546.77931323 messages per second fetch size 204800 -> 9213.30958314 messages per second fetch size 230400 -> 10316.668006 messages per second fetch size 256000 -> 11476.2285269 messages per second fetch size 281600 -> 12353.7254386 messages per second fetch size 307200 -> 13131.2367288 messages per second (1.1.1 and 1.2.5 have also the same issue) $ pifpaf run kafka python kafka_test.py kafka-python version: 1.3.1 fetch size 179200 -> 44636.9371873 messages per second fetch size 204800 -> 44324.7085365 messages per second fetch size 230400 -> 45235.8283208 messages per second fetch size 256000 -> 45793.1044121 messages per second fetch size 281600 -> 44648.6357019 messages per second fetch size 307200 -> 44877.8445987 messages per second fetch size 332800 -> 47166.9176281 messages per second fetch size 358400 -> 47391.0057622 messages per second Looks like it works well now :) Just in case I have updated a bit the bench script to ensure it always works for me: --- kafka_test.py.ori 2016-12-02 09:16:10.570677010 +0100 +++ kafka_test.py 2016-12-02 09:06:04.870370438 +0100 @@ -14,7 +14,7 @@ import time import ujson -KAFKA_URL = '92.168.10.6:9092' +KAFKA_URL = '127.0.0.1:9092' KAFKA_GROUP = 'kafka_python_perf' KAFKA_TOPIC = 'raw-events' @@ -24,6 +24,7 @@ def write(): k_client = KafkaClient(KAFKA_URL) +k_client.ensure_topic_exists(KAFKA_TOPIC) p = KeyedProducer(k_client, async=False, req_acks=KeyedProducer.ACK_AFTER_LOCAL_WRITE, -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Fri, Dec 02, 2016 at 03:19:26PM +1100, Tony Breeds wrote: On Thu, Dec 01, 2016 at 07:57:37AM +0100, Mehdi Abaakouk wrote: I think the solution is pretty simple. Just Fix it. I'm not saying it's easy but it is *simple*. We're a group of skilled individuals We have several ways forward which just haven't been done in the last 8 months. Waiting doesn't seem to be a viable course anymore. I agree the solution is *simple*. From my POV this really feels more like a social problem then a technical one. I'd like to remind everyone with a technical stake in this that it's OpenStack First, Project Team Second[1]. The thing that's best for OpenStack is for the oslo.mesaging and monasca teams to come together and find a path forward. It's really great that this thread has started that process! No they are no social issue here, I always tried to fix other projects issues even they are not in my area of expertise when I want to go forward. But I'm realistic, I can only do this only if the invest and the time it requires is not too big. And to fix (by code) that one, this is just too high for me, sorry. Cheers, -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Thu, Dec 01, 2016 at 04:52:52PM +, Keen, Joe wrote: > Unfortunately there’s nothing wrong on the Monasca side so far as we know. > We test new versions of the kafka-python library outside of Monasca > before we bother to try integrating a new version. Since 1.0 the > kafka-python library has suffered from crashes and memory leaks severe > enough that we’ve never attempted using it in Monasca itself. We reported > the bugs we found to the kafka-python project but they were closed once > they released a new version. So Opening bugs isn't working. What about writing code? Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Thu, Dec 01, 2016 at 08:41:54AM -0800, Joshua Harlow wrote: > Keen, Joe wrote: > > I¹ll look into testing the newest version of kafka-python and see if it > > meets our needs. If it still isn¹t stable and performant enough what are > > the available options? > > Fix the kafka-python library or fix monasca; those seem to be the options to > me :) Yup, Also worth including fix oslo.messaging to meet monasca's needs. But *something* needs fixing. > I'd also not like to block the rest of the world (from using newer versions > of kafka-python) during this as well. But then this may diverge/expand into > a discussion we had a few summits ago, about getting rid of > co-instability... lalalalala not listening ;P Less flippantly, there are a couple of ways to do this but IMO they're not in the best interest of OpenStack. 1. vendor/fork python-kafka 0.X 2. Stop the proposal-bot from syncing with monasca, thereby allowing it to use python-kafka 0.X at the expense of co-installability. Fortunately either option is easy to reverse once the underlying issue is fixed. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Thu, Dec 01, 2016 at 07:57:37AM +0100, Mehdi Abaakouk wrote: > I'm aware of all of that, oslo.messaging patch for the new version is > ready since 8 months now. And we are still blocked... capping libraries, > whatever the reason, is very annoying and just freezes people work. And I agree with that statement. Capping isn't the right solution. By the same token uncapping isn't eaither as we know we'll cause breakage. > From the API pov python-kafka haven't break anything, the API is still > here and documented (and deprecated). What monasca raises is performance > issue due to how their uses the library, and on absumption on how it works > internally. Blocking all projects for that looks not fair to me. I understand that your blocked, but blocking the other team isn't the solution either. > As nadya said, now, we have users that that prefers using an unmerged > patch and the new lib instead of using upstream supported > version with the old lib. This is not an acceptable situation to me but > that's just my thought. > > Where is the solution to allow oslo.messaging works blocked since 8 > month to continue ? > > So, I'm waiting for monasca team input now and hope we can move forward. I think the solution is pretty simple. Just Fix it. I'm not saying it's easy but it is *simple*. We're a group of skilled individuals We have several ways forward which just haven't been done in the last 8 months. Waiting doesn't seem to be a viable course anymore. From my POV this really feels more like a social problem then a technical one. I'd like to remind everyone with a technical stake in this that it's OpenStack First, Project Team Second[1]. The thing that's best for OpenStack is for the oslo.mesaging and monasca teams to come together and find a path forward. It's really great that this thread has started that process! Other people are much better placed to define the way forward, clearly fixing python-kafka is the best option which almost certainly means joining that community. Worst case if we really can't find a way forward, monasca can just vendor/fork python-kafka 0.X and we'll all admit defeat. Yours Tony. [1] http://governance.openstack.org/reference/principles.html#openstack-first-project-team-second-company-third signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 12/1/16, 9:44 AM, "Julien Danjou" wrote: >On Thu, Dec 01 2016, Keen, Joe wrote: > >Hi Joe, > >[…] > >> The message context wrapped around every message is extra overhead we >> don¹t want. >> There¹s no support for batch sends to Kafka. >> There¹s no support for keyed producers. >> The forced auto_commit on consumers isn¹t something we can tolerate. >> There is no auto balance of multiple consumers in the same consumer >>group >> on a topic. > >This sounds like interesting features and optimizations to add to >oslo.messaging. Did you already send patches or spec to oslo.messaging >to improve that? What there the issues you encountered to fix those >problems / add those new abilities? > >-- >Julien Danjou >// Free Software hacker >// https://julien.danjou.info I haven’t submitted any patches to oslo.messaging. When I wrote the Kafka interfaces for Monasca I didn’t know oslo.messaging existed. Since then I just haven’t had the time to try merging the two approaches into one library. If anyone else is interested in what we’ve done I can help with that. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Reopen them? If they weren't fixed, then seems like the right thing to try to do, and rinse and repeat until actually fixed :-P Keen, Joe wrote: > We reported > the bugs we found to the kafka-python project but they were closed once > they released a new version. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Damn, that's supposed to say co-installability... Lol, -Josh Joshua Harlow wrote: Keen, Joe wrote: I¹ll look into testing the newest version of kafka-python and see if it meets our needs. If it still isn¹t stable and performant enough what are the available options? Fix the kafka-python library or fix monasca; those seem to be the options to me :) I'd also not like to block the rest of the world (from using newer versions of kafka-python) during this as well. But then this may diverge/expand into a discussion we had a few summits ago, about getting rid of co-instability... -Josh __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 12/1/16, 9:41 AM, "Joshua Harlow" wrote: >Keen, Joe wrote: >> I¹ll look into testing the newest version of kafka-python and see if it >> meets our needs. If it still isn¹t stable and performant enough what >>are >> the available options? > >Fix the kafka-python library or fix monasca; those seem to be the >options to me :) > >I'd also not like to block the rest of the world (from using newer >versions of kafka-python) during this as well. But then this may >diverge/expand into a discussion we had a few summits ago, about getting >rid of co-instability... > >-Josh Unfortunately there’s nothing wrong on the Monasca side so far as we know. We test new versions of the kafka-python library outside of Monasca before we bother to try integrating a new version. Since 1.0 the kafka-python library has suffered from crashes and memory leaks severe enough that we’ve never attempted using it in Monasca itself. We reported the bugs we found to the kafka-python project but they were closed once they released a new version. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Thu, Dec 01 2016, Keen, Joe wrote: Hi Joe, […] > The message context wrapped around every message is extra overhead we > don¹t want. > There¹s no support for batch sends to Kafka. > There¹s no support for keyed producers. > The forced auto_commit on consumers isn¹t something we can tolerate. > There is no auto balance of multiple consumers in the same consumer group > on a topic. This sounds like interesting features and optimizations to add to oslo.messaging. Did you already send patches or spec to oslo.messaging to improve that? What there the issues you encountered to fix those problems / add those new abilities? -- Julien Danjou // Free Software hacker // https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Keen, Joe wrote: I¹ll look into testing the newest version of kafka-python and see if it meets our needs. If it still isn¹t stable and performant enough what are the available options? Fix the kafka-python library or fix monasca; those seem to be the options to me :) I'd also not like to block the rest of the world (from using newer versions of kafka-python) during this as well. But then this may diverge/expand into a discussion we had a few summits ago, about getting rid of co-instability... -Josh __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On 12/1/16, 3:11 AM, "Julien Danjou" wrote: >On Thu, Dec 01 2016, Mehdi Abaakouk wrote: > >> I'm aware of all of that, oslo.messaging patch for the new version is >> ready since 8 months now. And we are still blocked... capping libraries, >> whatever the reason, is very annoying and just freezes people work. >> >> From the API pov python-kafka haven't break anything, the API is still >> here and documented (and deprecated). What monasca raises is performance >> issue due to how their uses the library, and on absumption on how it >>works >> internally. Blocking all projects for that looks not fair to me. >> >> As nadya said, now, we have users that that prefers using an unmerged >> patch and the new lib instead of using upstream supported >> version with the old lib. This is not an acceptable situation to me but >>that's >> just my thought. >> >> Where is the solution to allow oslo.messaging works blocked since 8 >> month to continue ? > >+1 > >And if Monasca is using messaging, I wonder why they don't rely on >oslo.messaging, which would also solve this entire problem in a snap. Julien, I looked at the current oslo.messaging Kafka driver and unfortunately I don¹t think it meets our needs. There are several major pieces that are a problem. The message context wrapped around every message is extra overhead we don¹t want. There¹s no support for batch sends to Kafka. There¹s no support for keyed producers. The forced auto_commit on consumers isn¹t something we can tolerate. There is no auto balance of multiple consumers in the same consumer group on a topic. The current Kafka driver we use in monasca-common has all these features. Without them we can¹t meet the durability and performance levels we need. What sort of performance levels is oslo.messaging tested at? I¹ll look into testing the newest version of kafka-python and see if it meets our needs. If it still isn¹t stable and performant enough what are the available options? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Thu, Dec 01 2016, Mehdi Abaakouk wrote: > I'm aware of all of that, oslo.messaging patch for the new version is > ready since 8 months now. And we are still blocked... capping libraries, > whatever the reason, is very annoying and just freezes people work. > > From the API pov python-kafka haven't break anything, the API is still > here and documented (and deprecated). What monasca raises is performance > issue due to how their uses the library, and on absumption on how it works > internally. Blocking all projects for that looks not fair to me. > > As nadya said, now, we have users that that prefers using an unmerged > patch and the new lib instead of using upstream supported > version with the old lib. This is not an acceptable situation to me but that's > just my thought. > > Where is the solution to allow oslo.messaging works blocked since 8 > month to continue ? +1 And if Monasca is using messaging, I wonder why they don't rely on oslo.messaging, which would also solve this entire problem in a snap. -- Julien Danjou -- Free Software hacker -- https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
Le 2016-12-01 01:25, Tony Breeds a écrit : So right now I think the answer is still no we can't uncap it. To the best of my knowledge monasca still suffers a substatial performance decrease and the switch from sync to async producers isn't desired for them. Joe wrote this all up in: http://lists.openstack.org/pipermail/openstack-dev/2016-June/098420.html Unfortunately this comes down to monasca and olso.messaging getting on the same page with python-kafka. From a requirements POV we really only care about co-installability, which uncappiugn clearly breaks. I'm aware of all of that, oslo.messaging patch for the new version is ready since 8 months now. And we are still blocked... capping libraries, whatever the reason, is very annoying and just freezes people work. From the API pov python-kafka haven't break anything, the API is still here and documented (and deprecated). What monasca raises is performance issue due to how their uses the library, and on absumption on how it works internally. Blocking all projects for that looks not fair to me. As nadya said, now, we have users that that prefers using an unmerged patch and the new lib instead of using upstream supported version with the old lib. This is not an acceptable situation to me but that's just my thought. Where is the solution to allow oslo.messaging works blocked since 8 month to continue ? So, I'm waiting for monasca team input now and hope we can move forward. Cheers, -- Mehdi Abaakouk mail: sil...@sileht.net irc: sileht __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][monasca] Can we uncap python-kafka ?
On Wed, Nov 30, 2016 at 08:13:44PM +0100, Mehdi Abaakouk wrote: > > > Le 2016-11-30 19:11, Nadya Shakhat a écrit : > > Hi all, > > > > This one [1] is used very intensively actually :) It is not merged > > because > > we cannot just upgrade python-kafka version because of Monasca. We had > > some > > talks in their channel, but no results still. I suggest the following: > > 1. On our side we will update this change > > 2. I will ping Monasca guys once more and we will proceed with > > discussions > > in the CR > > > > [1] https://review.openstack.org/#/c/332105/ > > Good news, so I should retitle this thread "Can we uncap python-kafka ?" So right now I think the answer is still no we can't uncap it. To the best of my knowledge monasca still suffers a substatial performance decrease and the switch from sync to async producers isn't desired for them. Joe wrote this all up in: http://lists.openstack.org/pipermail/openstack-dev/2016-June/098420.html Unfortunately this comes down to monasca and olso.messaging getting on the same page with python-kafka. From a requirements POV we really only care about co-installability, which uncappiugn clearly breaks. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
