Hello all, I have to perform a join between two large csv sets that do not fit in ram. I process this two files in batch mode. I also need a side output to catch csv processing errors. So my question is what is the best way to this kind of join operation ? I think I should use a valueState state backend but would it work if my ram is my states goes larger than my RAM ?
Regards. Killian This message contains confidential information and is intended only for the individual(s) addressed in the message. If you are not the named addressee, you should not disseminate, distribute, or copy this e-mail. If you are not the intended recipient, you are notified that disclosing, distributing, or copying this e-mail is strictly prohibited.
