smlHao commented on issue #352:
URL:
https://github.com/apache/incubator-uniffle/issues/352#issuecomment-1650888620
> We don't verify the Hadoop version 3.3. We only verify the Hadoop version
3.2.
@jerqi thanks !!! have another 2 questions need your help :
1、if hadoop version is 3.3, no matter whether it's correct or
not ，the build command should be ./build_distribution.sh --hadoop-profile
hadoop3.3 ?
2、want to write shuffle data into hdfs ， my conf is correct ？
client.conf :
spark.shuffle.manager=org.apache.spark.shuffle.RssShuffleManager
spark.rss.coordinator.quorum=172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999
spark.rss.storage.type=MEMORY_LOCALFILE_HDFS
spark.rss.remote.storage.path=hdfs://ns1/rss/data
coordinator.conf:
rss.coordinator.quorum
172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999
rss.rpc.server.port 19999
rss.jetty.http.port 19998
rss.coordinator.server.heartbeat.timeout 30000
rss.coordinator.app.expired 60000
rss.coordinator.shuffle.nodes.max
3
rss.coordinator.exclude.nodes.file.path file:///app/rss-0.7.1/conf/exclude_nodes
server.conf：
rss.rpc.server.port 20000
rss.jetty.http.port 20001
rss.storage.basePath
/app/rss-0.7.1/data
rss.storage.type
MEMORY_LOCALFILE_HDFS
rss.coordinator.quorum
172.100.3.70:19999,172.100.3.71:19999,172.100.3.72:19999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
