Re: Are the Table API Connectors production ready?

2023-03-12 Thread yuxia
Yeah, you're right. We don't provide filtering files with patterns. And actually we had already a jira[1] for it. I was intended to do this in the past, but don't have much time. Anyone who are insterested can take it over. We're happy to help review. [1] https://issues.apache.org/jira/browse

Re: CSV File Sink in Streaming Use Case

2023-03-12 Thread Shammon FY
Hi Chirag CSVBulkWriter implements BulkWriter and no special methods are added in CSVBulkWriter. I think you can use BulkWriter instead of CSVBulkWriter in your application directly. You can have a try, thanks Best, Shammon On Fri, Mar 10, 2023 at 4:05 PM Shammon FY wrote: > >

Re: Are the Table API Connectors production ready?

2023-03-12 Thread Shammon FY
Hi ravi Do you read streaming data with filesystem connector? By default, the file system is bound. If you need to read streaming data, you can look at the configuration `source.monitor.interval` in https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/#directory-wa

Re: Are the Table API Connectors production ready?

2023-03-12 Thread ravi_suryavanshi.yahoo.com via user
Thanks a lot, Yaroslav and Shammon.I want to use the Filesystem Connector.  I tried it works well till it is running. If the job is restarted. It processes all the files again. Could not find the move or delete option after collecting the files. Also, I could not find the filtering using patter