Re: HDFS ShortCircuit Read on Mac?
Chris, many thanks for the quick response. I will disable the shortcircuit on my mac for now. :-) Demai On Tue, Sep 8, 2015 at 4:57 PM, Chris Nauroth wrote: > Hello Demai, > > HDFS short-circuit read currently does not work on Mac, due to some > platform differences in handling of domain sockets. The last time I > checked, our Hadoop code was exceeding a maximum path length enforced on > Mac for domain socket paths. I haven't had availability to look at this in > a while, but the prior work is tracked in JIRA issues HDFS-3296 and > HADOOP-11957 if you want to see the current progress. > > --Chris Nauroth > > From: Demai Ni > Reply-To: "user@hadoop.apache.org" > Date: Tuesday, September 8, 2015 at 4:46 PM > To: "user@hadoop.apache.org" > Subject: HDFS ShortCircuit Read on Mac? > > hi, folks, > > wondering anyone has setup HDFS shortcircuit Read on Mac? I installed > hadoop through homebrew on Mac. It is up and running. But I cannot > config "dfs.domain.socket.path" as instructed here: > > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html > Since there is no dn_socket on the mac. > > any pointers are appreciated. > > Demai > >
Re: HDFS ShortCircuit Read on Mac?
Hello Demai, HDFS short-circuit read currently does not work on Mac, due to some platform differences in handling of domain sockets. The last time I checked, our Hadoop code was exceeding a maximum path length enforced on Mac for domain socket paths. I haven't had availability to look at this in a while, but the prior work is tracked in JIRA issues HDFS-3296 and HADOOP-11957 if you want to see the current progress. --Chris Nauroth From: Demai Ni mailto:nid...@gmail.com>> Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" mailto:user@hadoop.apache.org>> Date: Tuesday, September 8, 2015 at 4:46 PM To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" mailto:user@hadoop.apache.org>> Subject: HDFS ShortCircuit Read on Mac? hi, folks, wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html Since there is no dn_socket on the mac. any pointers are appreciated. Demai
HDFS ShortCircuit Read on Mac?
hi, folks, wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html Since there is no dn_socket on the mac. any pointers are appreciated. Demai
