Hello Guozhang, Thank you for this answer. You've made me realize that David and I are conflating two things which are not the same -- the host information in the consumer group, and the host information in the StreamsMetadata. We found that our interactive queries were being forwarded to the wrong IP addresses, and saw those same IP addresses in the group membership.
I assume the group membership and StreamsMetadata are both updated only in a group rebalance. Because we are using static group membership, they never get updated, even when a group member joins with a new IP address, or with a new `application.server` configuration. But one difference is the host information in the consumer group cannot be configured, and is always an IP address, but we can configure `application.server` with a hostname. Bradley Peterson On 2019/12/09 00:52:37, Guozhang Wang <[email protected]> wrote: > Hello David, > > The host information of the group membership is inferred from the socket > channel's ip address, I'm not certain how EC2 spot instance's > application.server config is used, but if it is not dynamically reflected > on the socket channel's inet address then it would be the case you > observed.. > > It is independent from the StreamsConfig application.server (I presume you > do not meant this streams-level config but still wants to clarify) which is > only used in the subscription information and hence should not be exposed > through the consumer group description. > > Guozhang > > > On Fri, Dec 6, 2019 at 11:09 AM David Garcia <[email protected]> > wrote: > > > I forgot to note that we are using spot instances AND static membership. > > I was able to confirm that setting application.server with the correct > > ip-address seems to result in the correct Host being set...however, upon > > spot-reap/replace the new ip-address does not seem to overwrite the > > previous ip-address in the host-field. Is there a way to make it replace > > the host-info AND use static membership? > > > > > > David Garcia > > Staff Big Data Engineer > > [email protected] > > O: > > M: 512.576.5864 > > Site <https://www.bazaarvoice.com/> | Blog < > > https://www.bazaarvoice.com/blog> | Twitter < > > https://twitter.com/Bazaarvoice> > > > > > > > > On 12/5/19, 1:07 PM, "David Garcia" <[email protected]> wrote: > > > > EXTERNAL: This email originated from outside of Bazaarvoice. Do not > > click any links or open any attachments unless you trust the sender and > > know the content is safe. > > > > > > Hello, my consumers are reporting invalid IP address. When running > > kafka-consumer-groups –describe… I see the following: > > > > > > TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG > > CONSUMER-ID HOST > > CLIENT-ID > > > > Topic1 12 1319627 1319627 > > 0 consumer1 > > /xx.yyy.zzz.xxx(non-existent-address) client-1-StreamThread-4-consumer > > > > Topic2 8 44604 44604 > > 0 consumer2 > > /xx.yyy.zzz.xxy(non-existent-address) client-1-StreamThread-3-consumer > > > > I understand that this hostInfo (i.e. the ip-addresses) are reported > > by the consumers. How can I configure my consumers to report the Host > > information I want? It’s worth noting that my apps are running in AWS > > EC2. Thx for the help! > > > > -David J. Garcia > > > > > > > > -- > -- Guozhang >
