Sneha Vijayarajan created HADOOP-17404:
------------------------------------------
Summary: ABFS: Piggyback flush on Append calls for short writes
Key: HADOOP-17404
URL: https://issues.apache.org/jira/browse/HADOOP-17404
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/azure
Affects Versions: 3.3.0
Reporter: Sneha Vijayarajan
Assignee: Sneha Vijayarajan
Fix For: 3.3.1
When Hflush or Hsync APIs are called, a call is made to store backend to commit
the data that was appended.
If the data size written by Hadoop app is small, i.e. data size :
* before any of HFlush/HSync call is made or
* between 2 HFlush/Hsync API calls
is less than write buffer size, 2 separate calls, one for append and another
for flush is made,
Apps that do such small writes eventually end up with almost similar number of
calls for flush and append.
This PR enables Flush to be piggybacked onto append call for such short write
scenarios.
NOTE: The changes is guarded over a config, and is disabled by default until
relevant supported changes is made available on all store production clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
