Aihua Xu created HIVE-24171: ------------------------------- Summary: Support HDFS reads from observer NameNodes Key: HIVE-24171 URL: https://issues.apache.org/jira/browse/HIVE-24171 Project: Hive Issue Type: New Feature Components: Hive Affects Versions: 3.0.0 Reporter: Aihua Xu Assignee: Aihua Xu
HDFS-12943 introduces the consistent reads from observer NameNodes which can boost the read performance and reduces the overloads on active NameNodes. To take advantage of this feature, the clients are required to make a msync() call after writing the files or before reading the files since observer NameNodes could have the stale data for a small window. Hive needs to make msync() call to HDFS in some places, e.g., 1) after generating the plan files - map.xml and reduce.xml so they can get used later by executors; 2) after the intermediate files are generated so they can get used by later stages or HS2. -- This message was sent by Atlassian Jira (v8.3.4#803005)