I have a AvroFileInputOperator which has large no of files to be read when the application starts. While the fire reading goes well at the start, I start getting below warning in log after reading around 50+ files
Is there something to be worried about and is there a way to mitigate it? 2017-05-26 09:10:36,233 INFO fs.AbstractFileInputOperator (AbstractFileInputOperator.java:openFile(796)) - opening file hdfs:///<directory_location>//partition_date=2017-05-05/000000_0_copy_28 2017-05-26 09:11:32,145 WARN internal.DataList (DataList.java:run(743)) - Exceeded allowed memory block allocation by 1 2017-05-26 09:11:58,595 WARN internal.DataList (DataList.java:run(743)) - Exceeded allowed memory block allocation by 1 Regards Vivek -- View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Exceeded-allowed-memory-block-for-AvroFileInputOperator-tp1651.html Sent from the Apache Apex Users list mailing list archive at Nabble.com.
