[ https://issues.apache.org/jira/browse/HUDI-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raymond Xu closed HUDI-1655. ---------------------------- > Support custom date format and fix unsupported exception in > DatePartitionPathSelector > ------------------------------------------------------------------------------------- > > Key: HUDI-1655 > URL: https://issues.apache.org/jira/browse/HUDI-1655 > Project: Apache Hudi > Issue Type: Improvement > Components: DeltaStreamer > Reporter: Raymond Xu > Assignee: Raymond Xu > Priority: Minor > Labels: pull-request-available > Fix For: 0.8.0 > > > Add a config to allow parsing custom date format in > {{DatePartitionPathSelector}}. Currently it assumes date partition string in > the format of {{yyyy-MM-dd}}. > > Also eligibleFiles.sort() throws this exception > {quote}java.lang.UnsupportedOperationException at > java.util.AbstractList.set(AbstractList.java:132) at > java.util.AbstractList$ListItr.set(AbstractList.java:426) at > java.util.List.sort(List.java:482) at > org.apache.hudi.utilities.sources.helpers.DatePartitionPathSelector.getNextFilePathsAndMaxModificationTime(DatePartitionPathSelector.java:141) > at > org.apache.hudi.utilities.sources.ParquetDFSSource.fetchNextBatch(ParquetDFSSource.java:48) > at > org.apache.hudi.utilities.sources.RowSource.fetchNewData(RowSource.java:43) > at org.apache.hudi.utilities.sources.Source.fetchNext(Source.java:75) at > org.apache.hudi.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInRowFormat(SourceFormatAdapter.java:94) > at > org.apache.hudi.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:338) > at > org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:255) > at > org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:587) > {quote} > > {{org.apache.hudi.client.common.HoodieSparkEngineContext#flatMap}} returns a > list that can't be sorted in-place. -- This message was sent by Atlassian Jira (v8.3.4#803005)