Hello all, I'm looking to batch some ORC files in memory before persisting to cloud storage. As a result, I was interested in utilizing an in-memory VFS to store this data to avoid disk writes for efficiency. JIMFS (from Google) seems to support what I need. The method signature of ORC writer expects a hadoop fs object, which will not support the native Java fs superclass, therefore, JIMFS will not be supported.
My question is -- how can I utilize an in-memory file system to write the row batches using the ORCWriter? Cheers, Ryan
