jerqi commented on issue #352: URL: https://github.com/apache/incubator-uniffle/issues/352#issuecomment-1650891219
> > We don't verify the Hadoop version 3.3. We only verify the Hadoop version 3.2. > > @jerqi thanks !!! have another 2 questions need your help : 1、if hadoop version is 3.3, no matter whether it's correct or not ，the build command should be ./build_distribution.sh --hadoop-profile hadoop3.3 ? > > ``` > 2、want to write shuffle data into hdfs ， my conf is correct ？ > client.conf : > spark.shuffle.manager=org.apache.spark.shuffle.RssShuffleManager > spark.rss.coordinator.quorum=172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999 > spark.rss.storage.type=MEMORY_LOCALFILE_HDFS > spark.rss.remote.storage.path=hdfs://ns1/rss/data > > coordinator.conf: > rss.coordinator.quorum 172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999 > rss.rpc.server.port 19999 > rss.jetty.http.port 19998 > rss.coordinator.server.heartbeat.timeout 30000 > rss.coordinator.app.expired 60000 > rss.coordinator.shuffle.nodes.max 3 > rss.coordinator.exclude.nodes.file.path file:///app/rss-0.7.1/conf/exclude_nodes > > server.conf： > rss.rpc.server.port 20000 > rss.jetty.http.port 20001 > rss.storage.basePath /app/rss-0.7.1/data > rss.storage.type MEMORY_LOCALFILE_HDFS > rss.coordinator.quorum 172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999 > ``` 1. Yes 2. Ok for me. ` rss.coordinator.shuffle.nodes.max` is a little small. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
