Hi Chad, > I'm wondering if someone can point me to the Kafka streams internal code that reads records for the join? --> You can check StreamThread#pollPhase, where stream thread (main consumer) periodically poll records. And then, it'll process each topology node with these polled records in stream tasks (StreamTask#process).
Hope that helps. Thanks. Luke On Sat, Oct 30, 2021 at 5:42 PM Gilles Philippart <gilles.philipp...@fundingcircle.com.invalid> wrote: > Hi Chad, this talk around 24:00 clearly explains what you’re seeing > https://www.confluent.io/events/kafka-summit-europe-2021/failing-to-cross-the-streams-lessons-learned-the-hard-way/ > < > https://www.confluent.io/events/kafka-summit-europe-2021/failing-to-cross-the-streams-lessons-learned-the-hard-way/ > > > > Gilles > > > On 30 Oct 2021, at 04:02, Chad Preisler <chad.preis...@gmail.com> wrote: > > > > Hello, > > > > I have a stream application that does a KStream to KTable left join. We > > seem to be occasionally missing joins (KTable side is null). > > > > I'm wondering if someone can point me to the Kafka streams internal code > > that reads records for the join? I've poked around the Kafka code base, > but > > there is a lot there. I imagine there is some consumer poll for each side > > of the join, and possibly a background thread for reading the KTable > topic. > > > > I figure there are several possible causes of this issue, and since > nothing > > is really jumping out in my code, I was going to start looking at the > Kafka > > code to see if there is something I can do to fix this. > > > > Thanks, > > Chad > > > -- > > > > > Funding Circle Limited is authorised and regulated by the Financial > Conduct Authority under firm registration number 722513. Funding Circle is > not covered by the Financial Services Compensation Scheme. Registered in > England (Co. No. 06968588) with registered office at 71 Queen Victoria > Street, London EC4V 4AY. >
