[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773364#comment-16773364 ] Hadoop QA commented on HADOOP-15567: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 39s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 10s{color} | {color:orange} hadoop-tools/hadoop-azure-datalake: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 33s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 48s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 49m 56s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f | | JIRA Issue | HADOOP-15567 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12929650/HADOOP-15567.001.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 2191af6d1a2d 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / aa3ad36 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/15951/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure-datalake.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/15951/testReport/ | | Max. process+thread count | 328 (vs. ulimit of 1) | | modules | C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/15951/console | | Powered by
[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773306#comment-16773306 ] Íñigo Goiri commented on HADOOP-15567: -- Thanks [~huanbang1993] for the work. Minor comments: * Can we make {{EXPIRY_TIME_DURATION = TimeUnit.DAYS.toMillis(7)}}. * Can you make the constants (EPS and EXPIRY_TIME_DURATION ) final? * Not sure we need EPS, by the way assertEquals has a method for the check with difference: assertEquals(EXPIRY_TIME_DURATION, diff, 500L); * We should make {{assertPathExist()}} a little more friendly and report that the file does not exists in the message. * When expecting exceptions, we should use LambaUtils or exception rules. > Support expiry time in AdlFileSystem > > > Key: HADOOP-15567 > URL: https://issues.apache.org/jira/browse/HADOOP-15567 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Anbang Hu >Priority: Major > Attachments: HADOOP-15567.000.patch, HADOOP-15567.001.patch, > live-test-result.png > > > ADLS supports setting an expiration time for a file. > We can leverage Xattr in FileSystem to set the expiration time. > This could use the same xattr as HDFS-6382 and the interface from HDFS-6525. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527041#comment-16527041 ] genericqa commented on HADOOP-15567: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 29s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 3s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 52s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 59m 17s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-15567 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12929650/HADOOP-15567.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 5336a78b1f4d 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e4d7227 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14834/testReport/ | | Max. process+thread count | 302 (vs. ulimit of 1) | | modules | C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14834/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Support expiry time in AdlFileSystem > > > Key: HADOOP-15567 > URL:
[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526935#comment-16526935 ] genericqa commented on HADOOP-15567: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 10s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color} | {color:red} hadoop-tools_hadoop-azure-datalake generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 39s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 3s{color} | {color:red} hadoop-tools/hadoop-azure-datalake generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 44s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 57m 13s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-tools/hadoop-azure-datalake | | | Found reliance on default encoding in org.apache.hadoop.fs.adl.AdlFileSystem.getXAttr(Path, String):in org.apache.hadoop.fs.adl.AdlFileSystem.getXAttr(Path, String): String.getBytes() At AdlFileSystem.java:[line 1092] | | | Found reliance on default encoding in org.apache.hadoop.fs.adl.AdlFileSystem.setXAttr(Path, String, byte[]):in org.apache.hadoop.fs.adl.AdlFileSystem.setXAttr(Path, String, byte[]): new String(byte[]) At AdlFileSystem.java:[line 1054] | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-15567 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12929638/HADOOP-15567.000.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux a4fa931c79a8 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk /
[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526899#comment-16526899 ] Íñigo Goiri commented on HADOOP-15567: -- [~ste...@apache.org], as you have been working with ADLS and other remote storages, can you take a look? As I mentioned before, we would like to use the same interface as HDFS-6525 which is using user.ttl Xattr. > Support expiry time in AdlFileSystem > > > Key: HADOOP-15567 > URL: https://issues.apache.org/jira/browse/HADOOP-15567 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Anbang Hu >Priority: Major > Attachments: HADOOP-15567.000.patch, live-test-result.png > > > ADLS supports setting an expiration time for a file. > We can leverage Xattr in FileSystem to set the expiration time. > This could use the same xattr as HDFS-6382 and the interface from HDFS-6525. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15567) Support expiry time in AdlFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526894#comment-16526894 ] Anbang Hu commented on HADOOP-15567: [^HADOOP-15567.000.patch] is an initial patch for discussion. Live test result: !live-test-result.png! > Support expiry time in AdlFileSystem > > > Key: HADOOP-15567 > URL: https://issues.apache.org/jira/browse/HADOOP-15567 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Anbang Hu >Priority: Major > Attachments: HADOOP-15567.000.patch, live-test-result.png > > > ADLS supports setting an expiration time for a file. > We can leverage Xattr in FileSystem to set the expiration time. > This could use the same xattr as HDFS-6382 and the interface from HDFS-6525. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org