[ https://issues.apache.org/jira/browse/HDFS-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated HDFS-7607: ------------------------------------------ Target Version/s: (was: 2.8.0) Not much going on here for a long time, dropping from 2.8.0. Not putting any target-version either anymore, let's target this depending on when there is patch activity. > Use random rack-local node for webhdfs opens to avoid OOM on DNs > ---------------------------------------------------------------- > > Key: HDFS-7607 > URL: https://issues.apache.org/jira/browse/HDFS-7607 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode, webhdfs > Affects Versions: 2.0.0-alpha > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Priority: Critical > > Webhdfs currently redirects a client to the DN that physically has one of the > replicas. Unlike the hdfs data streamer protocol which can easily handle > hundreds or thousands of connections, jetty has poor performance under heavy > load. Webhdfs clients can easily overwhelm the DNs and likely cause OOMs or > excessive GC. > The NN should redirect the client to a rack-local location to distribute the > webhdfs load across multiple hosts. The rack can then use the lightweight > streamer protocol. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org