[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13455: --- Component/s: documentation > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation, fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345-002.patch, > HADOOP-13455-HADOOP-13345.001.patch, HADOOP-13455-HADOOP-13345.003.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[
https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13455:
---
Attachment: HADOOP-13455-HADOOP-13345.003.patch
Thanks [~fabbri] and [~ste...@apache.org]. I like what you guys are doing here.
I think early doc will help us and incoming contributors.
The default value of {{fs.s3a.s3guard.ddb.table.create}} is false in
{{core-default.xml}}. This makes sense in production by asking users to
explicitly enable table auto-creation to avoid unexpected cost. This, however,
has slight operation burden to users as she has to either enable this config
along with metadata store impl class, or create DynamoDB table in advance. The
latter is not a problem when importing existing data via command line tool
[HADOOP-13650]. Overall, I'm fine with this change.
And we have to remove this config annotation {{@InterfaceAudience.Private}} as
it's becoming public. And to update the javadoc. Moreover I'd suggest we move
this from {{S3Guard}} to {{Constants}}, see v3 patch.
In {{DynamoDBMetadataStore}} it was assumed true originally. The failing unit
test can be fixed by setting this config as true in
{{TestDynamoDBMetadataStore}}, see the v3 patch.
Interestingly, I also found a bug about the config key. Though we should not
create table unless the config {{fs.s3a.s3guard.ddb.table.create}} is set true,
we still have to get the existing table in {{initialize()}}, before any
operation. I will shortly create a separate JIRA for tracking this and link
here.
> S3Guard: Write end user documentation.
> --
>
> Key: HADOOP-13455
> URL: https://issues.apache.org/jira/browse/HADOOP-13455
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Chris Nauroth
>Assignee: Aaron Fabbri
> Attachments: HADOOP-13455-HADOOP-13345-002.patch,
> HADOOP-13455-HADOOP-13345.001.patch, HADOOP-13455-HADOOP-13345.003.patch
>
>
> Write end user documentation that describes S3Guard architecture,
> configuration and usage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13455: Attachment: HADOOP-13455-HADOOP-13345-002.patch I wasn't expecting that :) I'd actually started on some in HADOOP-13899 in the expectation that nobody was going to do it. Nice to see I Was wrong! I've merged in my changes from HADOOP-13899, especially the stuff at the beginning about how this is experimental Added: proofreading, minor formatting changes, a bit on logging (also updating test/resources/log4j.properties to match). FWIW, I'd recommend turning on s3guard in `auth-keys.xml`. That way: no changes for git to accidentally commit. I listed both files, without stating a preference. > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345-002.patch, > HADOOP-13455-HADOOP-13345.001.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13455: Status: Patch Available (was: Open) > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345-002.patch, > HADOOP-13455-HADOOP-13345.001.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13455: Status: Open (was: Patch Available) > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345.001.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Affects Version/s: HADOOP-13345 Status: Patch Available (was: In Progress) > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345.001.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Attachment: HADOOP-13455-HADOOP-13345.001.patch [~steve_l] ask and you shall receive. I took a stab and attached a v1 patch here. > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Aaron Fabbri > Attachments: HADOOP-13455-HADOOP-13345.001.patch > > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13455: --- Priority: Major (was: Minor) > S3Guard: Write end user documentation. > -- > > Key: HADOOP-13455 > URL: https://issues.apache.org/jira/browse/HADOOP-13455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth > > Write end user documentation that describes S3Guard architecture, > configuration and usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
