Addendum in case my previous email wasn't clear: > So for any given instance of a streams application there will never be both a v1 and v2 alive at the same time
That's right. But the current live instance will be able to tell other instances, via its endpoint setting, whether it wants to be contacted at v1 or at v2. The other instances can't guess that. Think: if an older instance would manually compose the "rest" of an endpoint URI, having only the host and port from the endpoint setting, it might not know that the new instances have a different endpoint suffix, for example). On Fri, Jul 8, 2016 at 6:37 PM, Michael Noll <mich...@confluent.io> wrote: > Damian, > > about the rolling upgrade comment: An instance A will contact another > instance B by the latter's endpoint, right? So if A has no further > information available than B's host and port, then how should instance A > know whether it should call B at /v1/ or at /v2/? I agree that my > suggestion isn't foolproof, but it is afaict better than the host:port > approach. > > > > On Fri, Jul 8, 2016 at 5:15 PM, Damian Guy <damian....@gmail.com> wrote: > >> Michael - i'm ok with changing it to a string. Any one else have a strong >> opinion on this? >> >> FWIW - i don't think it will work fine as is during the rolling upgrade >> scenario as the service that is listening on the port needs to be embedded >> within each instance. So for any given instance of a streams application >> there will never be both a v1 and v2 alive at the same time (unless of >> course the process didn't shutdown properly, but then you have another >> problem...). >> >> On Fri, 8 Jul 2016 at 15:26 Michael Noll <mich...@confluent.io> wrote: >> >> > I have one further comment about `StreamsConfig.USER_ENDPOINT_CONFIG`. >> > >> > I think we should consider to not restricting the value of this setting >> to >> > only `host:port` pairs. By design, this setting is capturing >> user-driven >> > metadata to define an endpoint, so why restrict the creativity or >> > flexibility of our users? I can imagine, for example, that users would >> > like to set values such as `https://host:port/api/rest/v1/` in this >> field >> > (e.g. being able to distinguish between `.../v1/` and `.../v2/` may >> help in >> > scenarios such as rolling upgrades, where, during the upgrade, older >> > instances may need to coexist with newer instances). >> > >> > That said, I don't have a strong opinion here. >> > >> > -Michael >> > >> > >> > >> > On Fri, Jul 8, 2016 at 2:55 PM, Matthias J. Sax <matth...@confluent.io> >> > wrote: >> > >> > > +1 >> > > >> > > On 07/08/2016 11:03 AM, Eno Thereska wrote: >> > > > +1 (non-binding) >> > > > >> > > >> On 7 Jul 2016, at 18:31, Sriram Subramanian <r...@confluent.io> >> wrote: >> > > >> >> > > >> +1 >> > > >> >> > > >> On Thu, Jul 7, 2016 at 9:53 AM, Henry Cai >> <h...@pinterest.com.invalid >> > > >> > > >> wrote: >> > > >> >> > > >>> +1 >> > > >>> >> > > >>> On Thu, Jul 7, 2016 at 6:48 AM, Michael Noll < >> mich...@confluent.io> >> > > wrote: >> > > >>> >> > > >>>> +1 (non-binding) >> > > >>>> >> > > >>>> On Thu, Jul 7, 2016 at 10:24 AM, Damian Guy < >> damian....@gmail.com> >> > > >>> wrote: >> > > >>>> >> > > >>>>> Thanks Henry - we've updated the KIP with an example and the new >> > > config >> > > >>>>> parameter required. FWIW the user doesn't register a listener, >> they >> > > >>>> provide >> > > >>>>> a host:port in config. It is expected they will start a service >> > > running >> > > >>>> on >> > > >>>>> that host:port that they can use to connect to the running >> > > KafkaStreams >> > > >>>>> Instance. >> > > >>>>> >> > > >>>>> Thanks, >> > > >>>>> Damian >> > > >>>>> >> > > >>>>> On Thu, 7 Jul 2016 at 06:06 Henry Cai >> <h...@pinterest.com.invalid> >> > > >>>> wrote: >> > > >>>>> >> > > >>>>>> It wasn't quite clear to me how the user program interacts with >> > the >> > > >>>>>> discovery API, especially on the user supplied listener part, >> how >> > > >>> does >> > > >>>>> the >> > > >>>>>> user program supply that listener to KafkaStreams and how does >> > > >>>>> KafkaStreams >> > > >>>>>> know which port the user listener is running, maybe a more >> > complete >> > > >>>>>> end-to-end example including the steps on registering the user >> > > >>> listener >> > > >>>>> and >> > > >>>>>> whether the user listener needs to be involved with task >> > > >>> reassignment. >> > > >>>>>> >> > > >>>>>> >> > > >>>>>> On Wed, Jul 6, 2016 at 9:13 PM, Guozhang Wang < >> wangg...@gmail.com >> > > >> > > >>>>> wrote: >> > > >>>>>> >> > > >>>>>>> ＋1 >> > > >>>>>>> >> > > >>>>>>> On Wed, Jul 6, 2016 at 12:44 PM, Damian Guy < >> > damian....@gmail.com> >> > > >>>>>> wrote: >> > > >>>>>>> >> > > >>>>>>>> Hi all, >> > > >>>>>>>> >> > > >>>>>>>> I'd like to initiate the voting process for KIP-67 >> > > >>>>>>>> < >> > > >>>>>>>> >> > > >>>>>>> >> > > >>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-67%3A+Queryable+state+for+Kafka+Streams >> > > >>>>>>>>> >> > > >>>>>>>> >> > > >>>>>>>> KAFKA-3909 <https://issues.apache.org/jira/browse/KAFKA-3909 >> > >> > is >> > > >>>> the >> > > >>>>>> top >> > > >>>>>>>> level JIRA for this effort. >> > > >>>>>>>> >> > > >>>>>>>> Initial PRs for Step 1 of the process are: >> > > >>>>>>>> Expose State Store Names < >> > > >>>> https://github.com/apache/kafka/pull/1526> >> > > >>>>>> and >> > > >>>>>>>> Query Local State Stores < >> > > >>>> https://github.com/apache/kafka/pull/1565> >> > > >>>>>>>> >> > > >>>>>>>> Thanks, >> > > >>>>>>>> Damian >> > > >>>>>>>> >> > > >>>>>>> >> > > >>>>>>> >> > > >>>>>>> >> > > >>>>>>> -- >> > > >>>>>>> -- Guozhang >> > > >>>>>>> >> > > >>>>>> >> > > >>>>> >> > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> -- >> > > >>>> Best regards, >> > > >>>> Michael Noll >> > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> *Michael G. Noll | Product Manager | Confluent | +1 >> > > 650.453.5860Download >> > > >>>> Apache Kafka and Confluent Platform: www.confluent.io/download >> > > >>>> <http://www.confluent.io/download>* >> > > >>>> >> > > >>> >> > > > >> > > >> > > >> > >> > >> > -- >> > Best regards, >> > Michael Noll >> > >> > >> > >> > *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860Download >> > Apache Kafka and Confluent Platform: www.confluent.io/download >> > <http://www.confluent.io/download>* >> > >> > > > > -- > Best regards, > Michael Noll > > > > *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860 > <%2B1%20650.453.5860>Download Apache Kafka and Confluent Platform: > www.confluent.io/download <http://www.confluent.io/download>* > -- Best regards, Michael Noll *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860Download Apache Kafka and Confluent Platform: www.confluent.io/download <http://www.confluent.io/download>*
