Re: Request to make ContinuousFileReaderOperator public

2024-07-17 Thread Darin Amos
at 1:30 PM Darin Amos wrote: > Hi All! > > I posted on the community slack channel and was referred to this mailing > list. I think it would be helpful if the ContinuousFileReaderOperator was > made a public class and not removed in Flink 2.0 (or to have an equivalent > created).

Re: [jira] [Created] (FLINK-33753) ContinuousFileReaderOperator consume records as mini batch

2023-12-06 Thread Darin Amos
b1470.run(Unknown > Source) > > at > > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) > > at > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) > > at org.apache.flink.runtime.taskmanager.Task.doRun(Task

Re: [jira] [Created] (FLINK-33753) ContinuousFileReaderOperator consume records as mini batch

2023-12-05 Thread Darin Amos
ns/source/ContinuousFileReaderOperator.java#L407-L411> . Cheers Darin On Tue, Dec 5, 2023 at 8:08 PM Darin Amos wrote: > They way I understand this loop is that the ContinuiousFileReaderOperator > only processes records in the background while the operator is idle, i.e. > while it's not

Re: [jira] [Created] (FLINK-33753) ContinuousFileReaderOperator consume records as mini batch

2023-12-05 Thread Darin Amos
r.java#L346 > >, > where it fetches all records from the split, and then only the > MailboxProcessor gets control to check other mail. This loop was introduced > here > < > https://github.com/apache/flink/commit/1a69cb9fce629b0c458f5ea514d9ac8de008687f > > > . > > >

Re: [jira] [Created] (FLINK-33753) ContinuousFileReaderOperator consume records as mini batch

2023-12-05 Thread Darin Amos
I thought for sure this was already the existing behavior with this operator. Does it not check the mailbox executor after every record read? On Tue, Dec 5, 2023 at 6:48 AM Prabhu Joseph (Jira) wrote: > Prabhu Joseph created FLINK-33753: > - > >

Request to make ContinuousFileReaderOperator public

2023-11-21 Thread Darin Amos
! Darin Amos