Hi All, I have a kafka application which consums events from multiple topics, stores certain fields from the events in local in-memory cache, based on the cached data it enriches some of the events consumed from another topic e.g. say for agent id 10, the first name and last name are cached then if an event with agent id 10 is received on another topic then it enriches the event by appending firstname and lastname and produces the events onto output topic.
The application is designed and developed by someone else and I need to provide persistence to the cached data. Also, there are multiple instances of the service running in a kubernetes cluster so each instance should have access to the persisted cache. Note that the application is not using kafka streams but plain consumer-producer model as described above. What are my choices if i want to make the cache persistent and accessible to all service instances? Can state store be used here without kafka streams? If yes then will the state store be global so it can be accessed by all service instances? If none of above then what are alternatives? Thank you all in advance and looking forward to response from you.
