Use case: - stream data continuously between 2 processes with very low latency
Problem: I have an ArrowStreamWriter running in one process that continuously writes a single integer. I have another process that continuously reads the data using ArrowStreamReader. Everything works fine for a few seconds, but once my reader gets caught up to the writer I start getting exceptions and eventually data loss. Here is the code and sample output - https://gist.github.com/jt70/69780c66ffb531040e97759264bd463a I'm not sure if this is a bug or something I'm doing wrong. Is there a better way to do high performance IPC with Arrow? Thanks for any tips! -Jason
