Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

In order to keep the voting thread on KIP-146 open for at least 72 hours, I'm amending my previous message and I'm starting the voting process on KIP-146 now. The discussion on KIP-146 may continue here for as long as the voting process is open. -Konstantine On Mon, May 8, 2017 at 9:47 AM, Konsta

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Dear all, KIP-146 has been updated to incorporate the changes that were suggested during the discussion so far. You may find the updated version here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-146+-+Classloading+Isolation+in+Connect Since none of the suggestions objected the core of

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Hi Mathieu, I believe you are referring to the connector property "name". This property is a connector config property as opposed to a Connect worker config property. Name is specified per connector instance and not per connector class. The scope of the KIP discussed here (KIP-146) is referring to

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Hi Gwen, as mentioned, Transformations are also included among the modules that will benefit from isolation. Overall such modules are Connectors (Sources and Sinks), Transformations and Converters. Built-in modules such as built-in connectors, transformations and converters will also run in isolat

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thanks again Stephane. Your comments were pretty accurate. A "future work" section was added to account for the suggested enhancements that will follow the initial release of classloading isolation. Specifically regarding versioning, you are right, the corresponding KIP that will describe versioni

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Hi Konstantine, As KIP-151 is currently adding the connector class information to the Kafka Connect REST API, it might be a nice small detail to augment this KIP to add the module name to the same API. T

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Hi, What about Transformations? Are they going to be isolated too? If they are, what do we do with the built-in ones? On Sat, Apr 29, 2017 at 9:16 AM, Konstantine Karantasis < konstant...@confluent.io> wrote: > * Because of KIP number collision, please disregard my previous KIP > announcement an

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

On Wed, May 3, 2017 at 11:17 AM, Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks Ewen. I'm replying inline as well. > > On Tue, May 2, 2017 at 11:24 AM, Ewen Cheslack-Postava > wrote: > > > Thanks for the KIP. > > > > A few responses inline, followed by additional comments. > >

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Glad you find the feedback useful ! Definitely all the ideas should be split in reasonable length KIPs. I just want to make sure the ideas are not lost. I won’t create the subsequent KIPs because I’m not good enough to implement the changes, but happy to keep on providing feedback alongside the

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thank you Stephane, your comments bring interesting and useful subjects to the discussion. I'm adding my replies below Ewen's comments. On Tue, May 2, 2017 at 10:15 PM, Ewen Cheslack-Postava wrote: > On Tue, May 2, 2017 at 10:01 PM, Stephane Maarek < > steph...@simplemachines.com.au> wrote: >

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thanks Ewen. I'm replying inline as well. On Tue, May 2, 2017 at 11:24 AM, Ewen Cheslack-Postava wrote: > Thanks for the KIP. > > A few responses inline, followed by additional comments. > > On Mon, May 1, 2017 at 9:50 PM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Gwen, R

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

On Tue, May 2, 2017 at 10:01 PM, Stephane Maarek < steph...@simplemachines.com.au> wrote: Excellent feedback, Stephane! Thanks for the work, it’s definitely needed! > I’d like to suggest to take it one step further. > > To me, I’d like to see Kafka Connect the same way we have Docker and > Docker

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thanks for the work, it’s definitely needed! I’d like to suggest to take it one step further. To me, I’d like to see Kafka Connect the same way we have Docker and Docker repositories. Here’s how I would envision the flow: - Kafka Connect workers are just workers. They come with no jars whatsoeve

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thanks for the KIP. A few responses inline, followed by additional comments. On Mon, May 1, 2017 at 9:50 PM, Konstantine Karantasis < konstant...@confluent.io> wrote: > Gwen, Randall thank you for your very insightful observations. I'm glad you > find this first draft to be an adequate platform

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Gwen, Randall thank you for your very insightful observations. I'm glad you find this first draft to be an adequate platform for discussion. I'll attempt replying to your comments in order. Gwen, I also debated exactly the same two options: a) interpreting absence of module path as a user's inten

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Very nice work, Konstantine. Conflicting dependencies of connectors is indeed a big issue that makes it hard to manage installed connectors. I do like Gwen's idea about removing the 'module.isolation.enabled' property. However, I would have anticipated always using classpath isolation for *only* t

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Hi Konstantine, Thank you so much for driving this! The connector classpath mess is driving me nuts (or worse, driving me to use Docker). I like the proposal for micro-benchmarks to test the context switching overhead. I have a difficult time figuring out the module.isolation.enabled. Especially

[DISCUSS] KIP-146: Classloading Isolation in Connect

* Because of KIP number collision, please disregard my previous KIP announcement and use this thread for discussion instead * Hi everyone, we aim to address dependency conflicts in Kafka Connect soon by applying class loading isolation. Feel free to take a look at KIP-146 here: *https://cwiki.