[ https://issues.apache.org/jira/browse/SPARK-16018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated SPARK-16018: ---------------------------------- Assignee: Dhruve Ashar > Shade netty for shuffle to work on YARN > --------------------------------------- > > Key: SPARK-16018 > URL: https://issues.apache.org/jira/browse/SPARK-16018 > Project: Spark > Issue Type: Bug > Components: Shuffle > Affects Versions: 2.0.0 > Reporter: Dhruve Ashar > Assignee: Dhruve Ashar > Priority: Blocker > Fix For: 2.0.0 > > > We replaced LazyFileRegion with netty's DefaultFileRegion. > Because of this (https://issues.apache.org/jira/browse/SPARK-15178), we have > a dependency on netty 4.0.25.Final or >, but hadoop is on 4.0.23.Final, so > the shuffle jar won't load in nodemanager as its trying to find a new version > of the class. > We need to shade netty's classes inorder to load the latest version we need > and not depend on the one loaded by hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org