GitHub user aglinxinyuan edited a comment on the discussion: Supporting Directory-Based Dataset Access in UDFs
Can we utilize our existing data processing interface to support this? I.e. use binary tuple as a single file, and use table of binary tuples as a directory. I don't like the idea of bypassing the assumption that data always comes from upstream operators. If we allow any operator to read the data from filesystem, any operator can become a source operator and it's no longer a data flow. GitHub link: https://github.com/apache/texera/discussions/4352#discussioncomment-16484307 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
