subject:"\"\\\\\\\[jira\\\\\\\] \\\\\\\[Commented\\\\\\\] \\\\\\\(HADOOP\\\\\\\-15428\\\\\\\) s3guard bucket\\\\\\\-info will create s3guard table if FS is set to do this automatically\""

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-10-11 Thread Steve Loughran (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646822#comment-16646822
 ] 

Steve Loughran commented on HADOOP-15428:
-

linking to HADOOP-14109 which also has some autocreate issues. 

proposed: info, destroy are both non-creating; init and destroy must have a URI 
or FS

> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-04 Thread Gabor Bota (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708785#comment-16708785
 ] 

Gabor Bota commented on HADOOP-15428:
-

submitted patch v001, tested against eu-west-1. 
Issues:
* org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster#testWithMiniCluster 
(known)
* ITestS3GuardToolDynamoDB#testPruneCommandCLI (added to [S3 flakyness 
collecting spreadsheet|
https://docs.google.com/spreadsheets/d/1Z9dkg5yC7Hu7VQ5G2Wz40hG1pb0DhLuLjJ3PZe7WL3Q/edit?usp=sharing)



> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-15428.001.patch
>
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-04 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708912#comment-16708912
 ] 

Hadoop QA commented on HADOOP-15428:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 49s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 55s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m 
35s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
29s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 56s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-15428 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12950566/HADOOP-15428.001.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 2f2333f5f244 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug 
17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ff31313 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15606/testReport/ |
| Max. process+thread count | 309 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/15606/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> -

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-07 Thread Sean Mackrory (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16713101#comment-16713101
 ] 

Sean Mackrory commented on HADOOP-15428:


+1 on the patch, with the caveat that I'm gonna change the usage text from 
"Require S3Guard to be disabled" to "Force S3Guard to be disabled", since this 
does actually proceed with modified behavior instead of failing if the 
conditions aren't met.

> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-15428.001.patch
>
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-10 Thread Hudson (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715829#comment-16715829
 ] 

Hudson commented on HADOOP-15428:
-

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15585 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/15585/])
HADOOP-15428. s3guard bucket-info will create s3guard table if FS is set 
(mackrorysd: rev 3ff8580f2289664a54f369b500ad77f31a07c326)
* (edit) 
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardTool.java
* (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md
* (edit) 
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/AbstractS3GuardToolTestBase.java


> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-15428.001.patch
>
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-11 Thread Gabor Bota (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718503#comment-16718503
 ] 

Gabor Bota commented on HADOOP-15428:
-

What about this as a relnote:

If -unguarded flag passed to `hadoop s3guard bucket-info`, it will proceed with 
S3Guard disabled instead of failing if S3Guard is not already disabled.

> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-15428.001.patch
>
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

2018-12-12 Thread Sean Mackrory (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719117#comment-16719117
 ] 

Sean Mackrory commented on HADOOP-15428:


Yeah that's more direct. Tweaked it a bit and updated. I'm happy with it, so 
I'll resolve. Feel free to discuss if there's more feedback...

> s3guard bucket-info will create s3guard table if FS is set to do this 
> automatically
> ---
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-15428.001.patch
>
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to 
> create a s3guard table on demand, then the DDB table is automatically 
> created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has 
> significant side effects (i.e. it can run up bills)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

[jira] [Commented] (HADOOP-15428) s3guard bucket-info will create s3guard table if FS is set to do this automatically

7 matches

Site Navigation

Mail list logo

Footer information