[ https://issues.apache.org/jira/browse/HUDI-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan closed HUDI-2001. ------------------------------------- Assignee: sivabalan narayanan Resolution: Invalid > SnapshotRead on MOR table runs into NoSuchMethodError w/ PartitionedFile.init > in MergeOnReadSnapshotRelation > ------------------------------------------------------------------------------------------------------------ > > Key: HUDI-2001 > URL: https://issues.apache.org/jira/browse/HUDI-2001 > Project: Apache Hudi > Issue Type: Bug > Reporter: sivabalan narayanan > Assignee: sivabalan narayanan > Priority: Major > > In latest master, I was trying out hudi test suite job and ran into > NoSuchMethodError. My compiled scala is 2.11 with spark 2.4.4. Run time spark > version is 2.4.7. I tried in EMR. When I do snapshot read with an MOR table, > running into this exception. > Stacktrace: > [https://gist.github.com/nsivabalan/9a276c3c0dc8a2ac93310fedccaa2fea] > Specifically, this happens when ValidateDatasetNode is being used. > ``` > // read from hudi and remove meta columns. > Dataset<Row> hudiDf = session.read().format("hudi").load(hudiPath); > Dataset<Row> trimmedDf = > hudiDf.drop(HoodieRecord.COMMIT_TIME_METADATA_FIELD).drop(HoodieRecord.COMMIT_SEQNO_METADATA_FIELD).drop(HoodieRecord.RECORD_KEY_METADATA_FIELD) > > .drop(HoodieRecord.PARTITION_PATH_METADATA_FIELD).drop(HoodieRecord.FILENAME_METADATA_FIELD); > ``` > -- This message was sent by Atlassian Jira (v8.3.4#803005)