Re: Null filesystem when using ORC writer

2021-04-16 Thread Ryan Schachte
Ok great, I got everything working now! Not sure why I had such a complicated setup before. I added the overwrite true option and simplified the writer (Without the explicit filesystem and override). Thanks so much! The hardest debug sessions are always the easiest solutions. On Fri, Apr 16, 2021

Re: Null filesystem when using ORC writer

2021-04-16 Thread Ryan Schachte
Hi Pavan, Thanks for getting back to me. I'm giving this a shot to simplify my example to see if I can reproduce. Any chance you could speak to why this error would keep happening? Exception in thread "main" org.apache.hadoop.fs.FileAlreadyExistsException: File already exists: /compact/tester.orc

Re: Null filesystem when using ORC writer

2021-04-16 Thread Pavan Lanka
Hi Ryan, In case you have not checked this might be a good starting point for you. https://orc.apache.org/docs/core-java.html#simple-example When I follow the code you shared, I don’t quite follow why you are creating and passing the L

Null filesystem when using ORC writer

2021-04-16 Thread Ryan Schachte
Hi team. Desperate to understand what my issue is here, hoping someone with better knowledge of the ORC writer and how the local hadoop fs is working. I'm using the LocalFileSystem for my standalone app (compacts ORC files). Logging into the container, I see all my data written to the location I'm