This seems like a reasonable and effective use of a feature branch and branch committers to me.
On Tue, May 15, 2018 at 11:34 AM, Steve Loughran <ste...@hortonworks.com> wrote: > Hi > > Chris Douglas I and I've have a proposal for a short-lived feature branch > for the Azure ABFS connector to go into the hadoop-azure package. This will > connect to the new azure storage service, which will ultimately replace the > one used by wasb. It's a big patch and, like all storage connectors, will > inevitably take time to stabilize (i.e: nobody ever get seek() right, even > when we think we have). > > Thomas & Esfandiar will do the coding: they've already done the paperwork. > Chris, myself & anyone else interested can be involved in the review and > testing. > > Comments? > > ------------- > > The initial HADOOP-15407 patch contains a new filesystem client for the > forthcoming Azure ABFS, which is intended to replace Azure WASB as the > Azure storage layer. The patch is large, as it contains the replacement > client, tests, and generated code. > > We propose a feature branch, so the module can be broken into salient, > reviewable chunks. Internal constraints prevented this feature from being > developed in Apache, so we want to ensure that all the code is discussed, > maintainable, and documented by the community before it merges. > > To effect this, we also propose adding two developers as branch > committers: Thomas Marquardt tm...@microsoft.com<mailto:tma > r...@microsoft.com> Esfandiar Manii esma...@microsoft.com<mailto:e > sma...@microsoft.com> > > Beyond normal feature branch activity and merge criteria for FS modules, > we want to add another merge criterion for ABFS. Some of the client APIs > are not GA. It seems reasonable to require that this client works with > public endpoints before it merges to trunk. > > To test the Blob FS driver, Blob FS team (including Esfandiar Manii and > Thomas Marquardt) in Azure Storage will need the MSDN subscription ID(s) > for all reviewers who want to run the tests. The ABFS team will then > whitelist the subscription ID(s) for the Blob FS Preview. At that time, > future storage accounts created will have the Blob FS endpoint, > <accountName>.dfs.core.windows.net<http://dfs.core.windows.net>, which > the Blob FS driver relies on. > > This is a temporary state during the (current) Private Preview and the > early phases of Public Preview. In a few months, the whitelisting will not > be required and anyone will be able to create a storage account with access > to the Blob FS endpoint. > > Thomas and Esfandiar have been active in the Hadoop project working on the > WASB connector (see https://issues.apache.org/jira/browse/HADOOP-14552). > They understand the processes and requirements of the software. Working on > the branch directly will let them bring this significant feature into the > hadoop-azure module without disrupting existing users. >
