[
https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=549799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549799
]
ASF GitHub Bot logged work on HADOOP-16948:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 08/Feb/21 19:03
Start Date: 08/Feb/21 19:03
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#1925:
URL: https://github.com/apache/hadoop/pull/1925#discussion_r572291660
##########
File path: hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
##########
@@ -877,6 +877,21 @@ enabled for your Azure Storage account."
The directories can be specified as comma separated values. By default the
value
is "/hbase"
+### <a name="singlewriteroptions"></a> Single Writer Options
+`fs.azure.singlewriter.directories`: Directories for single writer support
+can be specified comma separated in this config. By default, multiple
+clients will be able to write to the same file simultaneously. When writing
+to files contained within the directories specified in this config, the
+client will obtain a lease on the file that will prevent any other clients
+from writing to the file. The lease will be renewed by the client until the
+output stream is closed, after which it will be released. To revoke a client's
+write access for a file, the AzureBlobFilesystem breakLease method may be
+ called.
+
+`fs.azure.lease.threads`: This is the size of the thread pool that will be
+used for lease operations for single writer directories. By default the value
+is 0, so it must be set to at least 1 to support single writer directories.
Review comment:
good writeup
1. is there any validation here, that if a path in the local FS is to be
leased then the executor count must be >1?
2. what if I'm working with >1 FS? Will this configuration be per-fs? Or
does it take a list of paths which can be full URIs to paths in a store? That's
what we ended up doing with s3a authoritative paths0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 549799)
Time Spent: 3h 20m (was: 3h 10m)
> ABFS: Support single writer dirs
> --------------------------------
>
> Key: HADOOP-16948
> URL: https://issues.apache.org/jira/browse/HADOOP-16948
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Priority: Minor
> Labels: abfsactive, pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> This would allow some directories to be configured as single writer
> directories. The ABFS driver would obtain a lease when creating or opening a
> file for writing and would automatically renew the lease and release the
> lease when closing the file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
