[GitHub] [hadoop] Apache9 commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
Apache9 commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326955355
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+true
+
+
+
+ test-compile-protoc
Review comment:
Thanks. Will rebase #1496 after you merge this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326955164
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
##
@@ -118,45 +118,21 @@
-org.apache.hadoop
-hadoop-maven-plugins
-
+org.xolstice.maven.plugins
+ protobuf-maven-plugin
+
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-${basedir}/src/main/proto/server
-
-
-${basedir}/src/main/proto
-
- yarn_protos.proto
- yarn_service_protos.proto
- applicationmaster_protocol.proto
- applicationclient_protocol.proto
- containermanagement_protocol.proto
-
server/yarn_server_resourcemanager_service_protos.proto
-
server/resourcemanager_administration_protocol.proto
- application_history_client.proto
- server/application_history_server.proto
- client_SCM_protocol.proto
- server/SCM_Admin_protocol.proto
- yarn_csi_adaptor.proto
- YarnCsiAdaptor.proto
-
-
+
+
+
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
I mean it will compile the proto files under the server directory as we set
the protoc path to ${basedir}/src/main/proto, which is the parent directory of
${basedir}/src/main/proto/server, so we do not need to add extra
'additionalProtoPathElement' any more.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on issue #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on issue #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes URL: https://github.com/apache/hadoop/pull/1494#issuecomment-533966714 Test failures are not related. Most of them are due to "Unable to create native thread" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326953658
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+true
+
+
+
+ test-compile-protoc
Review comment:
Okay, I will update id as per your suggestion, "src-test-compile-protoc" and
merge :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326953058
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
##
@@ -118,45 +118,21 @@
-org.apache.hadoop
-hadoop-maven-plugins
-
+org.xolstice.maven.plugins
+ protobuf-maven-plugin
+
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-${basedir}/src/main/proto/server
-
-
-${basedir}/src/main/proto
-
- yarn_protos.proto
- yarn_service_protos.proto
- applicationmaster_protocol.proto
- applicationclient_protocol.proto
- containermanagement_protocol.proto
-
server/yarn_server_resourcemanager_service_protos.proto
-
server/resourcemanager_administration_protocol.proto
- application_history_client.proto
- server/application_history_server.proto
- client_SCM_protocol.proto
- server/SCM_Admin_protocol.proto
- yarn_csi_adaptor.proto
- YarnCsiAdaptor.proto
-
-
+
+
+
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
I don't think so. It just adds them as arguments to protoc command.
Just checks that it should be a directory. No extra processing done on that.
https://github.com/xolstice/protobuf-maven-plugin/blob/master/src/main/java/org/xolstice/maven/plugin/protobuf/AbstractProtocMojo.java
https://github.com/xolstice/protobuf-maven-plugin/blob/master/src/main/java/org/xolstice/maven/plugin/protobuf/Protoc.java
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326952511
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
##
@@ -150,12 +150,12 @@
protobuf-maven-plugin
${protobuf-maven-plugin.version}
-
com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}
grpc-java
io.grpc:protoc-gen-grpc-java:1.15.1:exe:${os.detected.classifier}
+src-compile-protoc
Review comment:
Yes, follow on would be better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cxorm opened a new pull request #1499: HDDS-1738. Add nullable annotation for OMResponse classes.
cxorm opened a new pull request #1499: HDDS-1738. Add nullable annotation for OMResponse classes. URL: https://github.com/apache/hadoop/pull/1499 Modify classes under hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/ ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15864) Job submitter / executor fail when SBN domain name can not resolved
[
https://issues.apache.org/jira/browse/HADOOP-15864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated HADOOP-15864:
---
Fix Version/s: (was: 3.2.1)
> Job submitter / executor fail when SBN domain name can not resolved
> ---
>
> Key: HADOOP-15864
> URL: https://issues.apache.org/jira/browse/HADOOP-15864
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Critical
> Fix For: 3.0.4, 3.3.0, 3.1.2
>
> Attachments: HADOOP-15864-branch.2.7.001.patch,
> HADOOP-15864-branch.2.7.002.patch, HADOOP-15864.003.patch,
> HADOOP-15864.004.patch, HADOOP-15864.005.patch,
> HADOOP-15864.branch.2.7.004.patch
>
>
> Job submit failure and Task executes failure if Standby NameNode domain name
> can not resolved on HDFS HA with DelegationToken feature.
> This issue is triggered when create {{ConfiguredFailoverProxyProvider}}
> instance which invoke {{HAUtil.cloneDelegationTokenForLogicalUri}} in HA mode
> with Security. Since in HDFS HA mode UGI need include separate token for each
> NameNode in order to dealing with Active-Standby switch, the double tokens'
> content is same of course.
> However when #setTokenService in {{HAUtil.cloneDelegationTokenForLogicalUri}}
> it checks whether the address of NameNode has been resolved or not, if Not,
> throw #IllegalArgumentException upon, then job submitter/ task executor fail.
> HDFS-8068 and HADOOP-12125 try to fix it, but I don't think the two tickets
> resolve completely.
> Another questions many guys consider is why NameNode domain name can not
> resolve? I think there are many scenarios, for instance node replace when
> meet fault, and refresh DNS sometimes. Anyway, Standby NameNode failure
> should not impact Hadoop cluster stability in my opinion.
> a. code ref: org.apache.hadoop.security.SecurityUtil line373-386
> {code:java}
> public static Text buildTokenService(InetSocketAddress addr) {
> String host = null;
> if (useIpForTokenService) {
> if (addr.isUnresolved()) { // host has no ip address
> throw new IllegalArgumentException(
> new UnknownHostException(addr.getHostName())
> );
> }
> host = addr.getAddress().getHostAddress();
> } else {
> host = StringUtils.toLowerCase(addr.getHostName());
> }
> return new Text(host + ":" + addr.getPort());
> }
> {code}
> b.exception log ref:
> {code:xml}
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Couldn't create proxy provider class
> org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:515)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:170)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:761)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:691)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:150)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2713)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2747)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2729)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:385)
> at
> org.apache.hadoop.fs.viewfs.ChRootedFileSystem.(ChRootedFileSystem.java:106)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.getTargetFileSystem(ViewFileSystem.java:178)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.getTargetFileSystem(ViewFileSystem.java:172)
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:303)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:377)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:172)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:172)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2713)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2747)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2729)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:385)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:176)
> at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:665)
> ... 35 more
> Caused by:
[GitHub] [hadoop] timmylicheng commented on a change in pull request #1431: HDDS-1569 Support creating multiple pipelines with same datanode
timmylicheng commented on a change in pull request #1431: HDDS-1569 Support
creating multiple pipelines with same datanode
URL: https://github.com/apache/hadoop/pull/1431#discussion_r326940585
##
File path:
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
##
@@ -81,6 +82,9 @@
@Before
public void setup() throws IOException, AuthenticationException {
OzoneConfiguration conf = new OzoneConfiguration();
+conf.setInt(
+ScmConfigKeys.OZONE_DATANODE_MAX_PIPELINE_ENGAGEMENT,
+1000);
Review comment:
Already updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] timmylicheng commented on a change in pull request #1431: HDDS-1569 Support creating multiple pipelines with same datanode
timmylicheng commented on a change in pull request #1431: HDDS-1569 Support
creating multiple pipelines with same datanode
URL: https://github.com/apache/hadoop/pull/1431#discussion_r326940515
##
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2PipelineMap.java
##
@@ -71,6 +71,10 @@ public synchronized void addPipeline(Pipeline pipeline) {
UUID dnId = details.getUuid();
dn2ObjectMap.computeIfAbsent(dnId, k -> ConcurrentHashMap.newKeySet())
.add(pipeline.getId());
+ dn2ObjectMap.computeIfPresent(dnId, (k, v) -> {
Review comment:
From Java
doc:https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html#computeIfAbsent-K-java.util.function.Function-
, it looks like computeIfAbsent is only adding absent member. Line 74 is
trying to merge a pipelineId into an existed candidate. This happens when one
datanode is assigned to multiple pipelines, which is a classic scenario for
multiraft.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
Apache9 commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326932592
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+true
+
+
+
+ test-compile-protoc
Review comment:
nit: let's align the names? Either we change this to
src-test-compile-protoc, or we change the above to compile-protoc, I know that
this will introduce conflicts with the plugin defined in yarn-csi, but let's do
things cleanly?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326932183
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
Review comment:
I think for most sub modules, if it declares this plugin, it means we want
to execute the compile goal, otherwise just do not declare the plugin. That's
why I mean in the parent pom, we can declare skip=false for the compile
execution. Anyway, can follow your solution. Not a big problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326931345
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+
+
+
+ src-compile-test-protoc
+ generate-test-sources
+
+test-compile
+
+
+
false
+${basedir}/src/test/proto
Review comment:
OK.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326931305
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+
+
+
+ src-compile-test-protoc
Review comment:
Let's align the names? Why for compile it is src-compile-protoc, and then
for test-compile, it is test-compile-protoc, not src-test-compile-protoc? Agree
that here we should put 'test' before 'compile', but let's either both use the
src prefix or not?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326931072
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
##
@@ -150,12 +150,12 @@
protobuf-maven-plugin
${protobuf-maven-plugin.version}
-
com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}
grpc-java
io.grpc:protoc-gen-grpc-java:1.15.1:exe:${os.detected.classifier}
+src-compile-protoc
Review comment:
I think we'd better align this. Otherwise it will be confusing to
developpers, as we define the plugin in the parent pom and reference it here,
but then we just skip all the exections from the parent pom and define a new
compile execution.
Anyway, we can keep it as is in this patch, and file a follow-on issue to
address it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326930922
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
##
@@ -118,45 +118,21 @@
-org.apache.hadoop
-hadoop-maven-plugins
-
+org.xolstice.maven.plugins
+ protobuf-maven-plugin
+
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-${basedir}/src/main/proto/server
-
-
-${basedir}/src/main/proto
-
- yarn_protos.proto
- yarn_service_protos.proto
- applicationmaster_protocol.proto
- applicationclient_protocol.proto
- containermanagement_protocol.proto
-
server/yarn_server_resourcemanager_service_protos.proto
-
server/resourcemanager_administration_protocol.proto
- application_history_client.proto
- server/application_history_server.proto
- client_SCM_protocol.proto
- server/SCM_Admin_protocol.proto
- yarn_csi_adaptor.proto
- YarnCsiAdaptor.proto
-
-
+
+
+
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
I think this is not required? The plugin will scan the directory recursively.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
hadoop-yetus commented on issue #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes URL: https://github.com/apache/hadoop/pull/1494#issuecomment-533926058 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 79 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 70 | Maven dependency ordering for branch | | +1 | mvninstall | 1210 | trunk passed | | +1 | compile | 1050 | trunk passed | | +1 | mvnsite | 278 | trunk passed | | +1 | shadedclient | 3388 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 288 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 205 | the patch passed | | +1 | compile | 1008 | the patch passed | | -1 | cc | 1008 | root generated 2 new + 24 unchanged - 2 fixed = 26 total (was 26) | | +1 | javac | 1008 | the patch passed | | +1 | mvnsite | 280 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 7 | The patch has no ill-formed XML file. | | +1 | shadedclient | 819 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 303 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 24 | hadoop-project in the patch passed. | | +1 | unit | 606 | hadoop-common in the patch passed. | | +1 | unit | 133 | hadoop-hdfs-client in the patch passed. | | -1 | unit | 7225 | hadoop-hdfs in the patch failed. | | +1 | unit | 1716 | hadoop-hdfs-rbf in the patch passed. | | -1 | asflicense | 79 | The patch generated 8 ASF License warnings. | | | | 16374 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks | | | hadoop.hdfs.TestLeaseRecovery2 | | | hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead | | | hadoop.hdfs.server.namenode.TestReencryption | | | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy | | | hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier | | | hadoop.hdfs.TestReplication | | | hadoop.hdfs.TestLease | | | hadoop.hdfs.server.namenode.TestAddStripedBlocks | | | hadoop.hdfs.server.namenode.TestProcessCorruptBlocks | | | hadoop.hdfs.TestReadStripedFileWithDNFailure | | | hadoop.hdfs.TestSafeModeWithStripedFile | | | hadoop.hdfs.server.namenode.TestNameNodeMXBean | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1494 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml cc | | uname | Linux 9a05a3ba85e4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 659c888 | | Default Java | 1.8.0_222 | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/artifact/out/diff-compile-cc-root.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/testReport/ | | asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 2644 (vs. ulimit of 5500) | | modules | C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1494/3/console | | versions | git=2.7.4 maven=3.3.9 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail:
[GitHub] [hadoop] hadoop-yetus commented on issue #1498: HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists
hadoop-yetus commented on issue #1498: HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists URL: https://github.com/apache/hadoop/pull/1498#issuecomment-533913027 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1385 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1067 | trunk passed | | +1 | compile | 31 | trunk passed | | +1 | checkstyle | 25 | trunk passed | | +1 | mvnsite | 34 | trunk passed | | +1 | shadedclient | 750 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 27 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 49 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 29 | the patch passed | | +1 | compile | 25 | the patch passed | | +1 | javac | 24 | the patch passed | | +1 | checkstyle | 18 | the patch passed | | +1 | mvnsite | 28 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 778 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | the patch passed | | +1 | findbugs | 53 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 82 | hadoop-azure in the patch passed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 4542 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1498/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1498 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cf2d08f8b8d7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 659c888 | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1498/1/testReport/ | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1498/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326915036
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
##
@@ -150,12 +150,12 @@
protobuf-maven-plugin
${protobuf-maven-plugin.version}
-
com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}
grpc-java
io.grpc:protoc-gen-grpc-java:1.15.1:exe:${os.detected.classifier}
+src-compile-protoc
Review comment:
This will change the default output directory
"target/generate-sources/protobuf/java" to "target/generate-sources/java"
So, next plugin "build-helper-maven-plugin" in this project may fail.
yarn-csi project was not affected by the protobuf upgrade. So better to keep
it as is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326914634
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+
+
+
+ src-compile-test-protoc
Review comment:
Id of this execution has been changed in #1494 to "test-compile-protoc".
Please update the corresponding Id in applicable changes in this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326914640
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+
+
+
+ src-compile-test-protoc
+ generate-test-sources
+
+test-compile
+
+
+
false
+${basedir}/src/test/proto
Review comment:
for "test-compile" goal src proto files will be referred using
"protoTestSourceRoot". Not "protoSourceRoot".
Mentioned here
https://www.xolstice.org/protobuf-maven-plugin/test-compile-mojo.html#protoTestSourceRoot
This is also updated in #1494. Please rebase.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326914827
##
File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
##
@@ -118,45 +118,21 @@
-org.apache.hadoop
-hadoop-maven-plugins
-
+org.xolstice.maven.plugins
+ protobuf-maven-plugin
+
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-${basedir}/src/main/proto/server
-
-
-${basedir}/src/main/proto
-
- yarn_protos.proto
- yarn_service_protos.proto
- applicationmaster_protocol.proto
- applicationclient_protocol.proto
- containermanagement_protocol.proto
-
server/yarn_server_resourcemanager_service_protos.proto
-
server/resourcemanager_administration_protocol.proto
- application_history_client.proto
- server/application_history_server.proto
- client_SCM_protocol.proto
- server/SCM_Admin_protocol.proto
- yarn_csi_adaptor.proto
- YarnCsiAdaptor.proto
-
-
+
+
+
${basedir}/../../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
`${basedir}/src/main/proto/server` missed from additional paths.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
vinayakumarb commented on a change in pull request #1496: HADOOP-16560. [YARN +
MAPREDUCE] use protobuf-maven-plugin to generat…
URL: https://github.com/apache/hadoop/pull/1496#discussion_r326914809
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,56 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
Review comment:
in #1494 this execution is made to skip by default.
Reason is, every module can specifically enable this execution whenever
required. Anyway every module needs to provide specific dependency information.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16559) [HDFS] use protobuf-maven-plugin to generate protobuf classes
[
https://issues.apache.org/jira/browse/HADOOP-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935388#comment-16935388
]
Vinayakumar B edited comment on HADOOP-16559 at 9/22/19 6:49 PM:
-
{quote}And here, since this is not a trivial change, I suggest we revert
HADOOP-16557 first? Now it is a pain for developpers as we need protobuf 3.7.1
to compile trunk and then 2.5.0 to compile other branches...
{quote}
Need not switch back and forth to build separate branches. protobuf 3.7.1 can
be installed in a separate location (ex: /opt/protobuf-3.7.1) and "protoc.path"
system property pointing to absolute path of 3.7.1 protoc (ex:
-Dprotoc.path=/opt/protobuf-3.7.1/bin/protoc) can be used to build trunk until
we replace protobuf-maven-plugin.
Instead of compiling protobuf from source (which is a very time consuming
task), can download the applicable exe from maven central
[here|https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.7.1/]
directly and provide as "protoc.path". Make sure it exe have executable
permissions.
was (Author: vinayrpet):
{quote}And here, since this is not a trivial change, I suggest we revert
HADOOP-16557 first? Now it is a pain for developpers as we need protobuf 3.7.1
to compile trunk and then 2.5.0 to compile other branches...
{quote}
Need not switch back and forth to build separate branches. protobuf 3.7.1 can
be installed in a separate location (ex: /opt/protobuf-3.7.1) and "protoc.path"
system property pointing to absolute path of 3.7.1 protoc (ex:
-Dprotoc.path=/opt/protobuf-3.7.1/bin/protoc) can be used to build trunk until
we replace protobuf-maven-plugin.
> [HDFS] use protobuf-maven-plugin to generate protobuf classes
> -
>
> Key: HADOOP-16559
> URL: https://issues.apache.org/jira/browse/HADOOP-16559
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Vinayakumar B
>Priority: Major
>
> Use "protoc-maven-plugin" to dynamically download protobuf executable to
> generate protobuf classes from proto file
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16559) [HDFS] use protobuf-maven-plugin to generate protobuf classes
[
https://issues.apache.org/jira/browse/HADOOP-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935388#comment-16935388
]
Vinayakumar B commented on HADOOP-16559:
{quote}And here, since this is not a trivial change, I suggest we revert
HADOOP-16557 first? Now it is a pain for developpers as we need protobuf 3.7.1
to compile trunk and then 2.5.0 to compile other branches...
{quote}
Need not switch back and forth to build separate branches. protobuf 3.7.1 can
be installed in a separate location (ex: /opt/protobuf-3.7.1) and "protoc.path"
system property pointing to absolute path of 3.7.1 protoc (ex:
-Dprotoc.path=/opt/protobuf-3.7.1/bin/protoc) can be used to build trunk until
we replace protobuf-maven-plugin.
> [HDFS] use protobuf-maven-plugin to generate protobuf classes
> -
>
> Key: HADOOP-16559
> URL: https://issues.apache.org/jira/browse/HADOOP-16559
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Vinayakumar B
>Priority: Major
>
> Use "protoc-maven-plugin" to dynamically download protobuf executable to
> generate protobuf classes from proto file
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16578) ABFS: fileSystemExists() should not call container level apis
[ https://issues.apache.org/jira/browse/HADOOP-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935387#comment-16935387 ] Sneha Vijayarajan commented on HADOOP-16578: [~DanielZhou] - Can you please review the PR change. > ABFS: fileSystemExists() should not call container level apis > - > > Key: HADOOP-16578 > URL: https://issues.apache.org/jira/browse/HADOOP-16578 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Da Zhou >Assignee: Sneha Vijayarajan >Priority: Major > Fix For: 3.3.0 > > > ABFS driver should not use container level api "Get Container Properties" as > there is no concept of container in HDFS, and this caused some RBAC check > issue. > Fix: use getFileStatus() to check if the container exists. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16578) ABFS: fileSystemExists() should not call container level apis
[ https://issues.apache.org/jira/browse/HADOOP-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935386#comment-16935386 ] Sneha Vijayarajan commented on HADOOP-16578: PR was tested with a East US account and command line: mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify [INFO] Results: [INFO] [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0 [ERROR] Failures: [ERROR] ITestGetNameSpaceEnabled.testNonXNSAccount:57->Assert.assertFalse:64->Assert.assertTrue:41->Assert.fail:88 Expecting getIsNamespaceEnabled() return false [ERROR] Errors: [ERROR] ITestClientUrlScheme.testClientUrlScheme:85->AbstractAbfsIntegrationTest.getFileSystem:197 » AbfsRestOperation [INFO] [ERROR] Tests run: 382, Failures: 1, Errors: 1, Skipped: 21 [INFO] [WARNING] Tests run: 190, Failures: 0, Errors: 0, Skipped: 23 > ABFS: fileSystemExists() should not call container level apis > - > > Key: HADOOP-16578 > URL: https://issues.apache.org/jira/browse/HADOOP-16578 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Da Zhou >Assignee: Sneha Vijayarajan >Priority: Major > Fix For: 3.3.0 > > > ABFS driver should not use container level api "Get Container Properties" as > there is no concept of container in HDFS, and this caused some RBAC check > issue. > Fix: use getFileStatus() to check if the container exists. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya opened a new pull request #1498: Avoid FileSystem API calls when FileSystem already exists
snvijaya opened a new pull request #1498: Avoid FileSystem API calls when FileSystem already exists URL: https://github.com/apache/hadoop/pull/1498 Even when FileSystem already exists, if the config "fs.azure.createRemoteFileSystemDuringInitialization" is true, a GetFileSystemProperties call is made which enforces container Read RBAC role. This blocks users who have necessary ACLs to proceed using the ABFS Driver. In this PR, RBAC enforcement for GetFileSystemProperties call is avoided in the above flow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326913046
##
File path: hadoop-common-project/hadoop-common/pom.xml
##
@@ -381,73 +381,43 @@
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-version-info
-generate-resources
-
- version-info
-
-
-
-${basedir}/src/main
-
- java/**/*.java
- proto/**/*.proto
-
-
-
+src-compile-protoc
+false
-compile-protoc
+compile-test-protoc
+generate-test-resources
- protoc
+ test-compile
- ${protobuf.version}
- ${protoc.path}
-
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- HAServiceProtocol.proto
- IpcConnectionContext.proto
- ProtocolInfo.proto
- RpcHeader.proto
- ZKFCProtocol.proto
- ProtobufRpcEngine.proto
- Security.proto
- GetUserMappingsProtocol.proto
- TraceAdmin.proto
- RefreshAuthorizationPolicyProtocol.proto
- RefreshUserMappingsProtocol.proto
- RefreshCallQueueProtocol.proto
- GenericRefreshProtocol.proto
- FSProtos.proto
-
-
+
${basedir}/src/test/proto
Review comment:
Yes. Found here
https://www.xolstice.org/protobuf-maven-plugin/test-compile-mojo.html#protoTestSourceRoot
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326912355
##
File path: hadoop-common-project/hadoop-common/pom.xml
##
@@ -381,73 +381,43 @@
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-version-info
-generate-resources
-
- version-info
-
-
-
-${basedir}/src/main
-
- java/**/*.java
- proto/**/*.proto
-
-
-
+src-compile-protoc
+false
-compile-protoc
+compile-test-protoc
+generate-test-resources
- protoc
+ test-compile
- ${protobuf.version}
- ${protoc.path}
-
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- HAServiceProtocol.proto
- IpcConnectionContext.proto
- ProtocolInfo.proto
- RpcHeader.proto
- ZKFCProtocol.proto
- ProtobufRpcEngine.proto
- Security.proto
- GetUserMappingsProtocol.proto
- TraceAdmin.proto
- RefreshAuthorizationPolicyProtocol.proto
- RefreshUserMappingsProtocol.proto
- RefreshCallQueueProtocol.proto
- GenericRefreshProtocol.proto
- FSProtos.proto
-
-
+
${basedir}/src/test/proto
Review comment:
Yes. Found here
https://www.xolstice.org/protobuf-maven-plugin/test-compile-mojo.html#protoTestSourceRoot
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326913010
##
File path: hadoop-hdfs-project/hadoop-hdfs-client/pom.xml
##
@@ -131,36 +131,18 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;>
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- ClientDatanodeProtocol.proto
- ClientNamenodeProtocol.proto
- acl.proto
- xattr.proto
- datatransfer.proto
- hdfs.proto
- encryption.proto
- inotify.proto
- erasurecoding.proto
- ReconfigurationProtocol.proto
-
-
+ false
+
+
+
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
Yes. I agree that keeping proto files in jar may be fine for now.
It would be better to specify explicitly which proto files are required for
successfull proto generation. So I preferred to NOT to include in jars.
> As in the old way, you need make sure that we also depend on hadoop-common
in the dependencies section, otherwise the protoc generating is fine but there
will be compile error...
hadoop-common in dependency section is anyway required, irrespective of
whether proto files are part of jar or not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326912670
##
File path: hadoop-hdfs-project/hadoop-hdfs-client/pom.xml
##
@@ -131,36 +131,18 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;>
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- ClientDatanodeProtocol.proto
- ClientNamenodeProtocol.proto
- acl.proto
- xattr.proto
- datatransfer.proto
- hdfs.proto
- encryption.proto
- inotify.proto
- erasurecoding.proto
- ReconfigurationProtocol.proto
-
-
+ false
+
+
+
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
Yes. I agree that keeping proto files in jar may be fine for now.
It would be better to specify explicitly which proto files are required for
successfull proto generation. So I preferred to NOT to include in jars.
> As in the old way, you need make sure that we also depend on hadoop-common
in the dependencies section, otherwise the protoc generating is fine but there
will be compile error...
hadoop-common in dependency section is anyway required, irrespective of
whether proto files are part of jar or not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326912969
##
File path: hadoop-project/pom.xml
##
@@ -1715,9 +1715,43 @@
frontend-maven-plugin
${frontend-maven-plugin.version}
+
+ org.xolstice.maven.plugins
+ protobuf-maven-plugin
+ ${protobuf-maven-plugin.version}
+ true
+
+
+
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
+
+false
+
+
+
+ src-compile-protoc
+ generate-sources
+
+compile
+
+
+
false
+${basedir}/src/main/proto
+
${project.build.directory}/generated-sources/java
+false
+true
Review comment:
> Do we need this? I suppose this is under the PluginMangement tag, so
unless we declare this plugin explicitly in the pom of a sub module, it will
not be executed?
Yes! you are right. Need to add plugin and execution with "skip=false"
explicitly in submodule to enable this. The reason for keeping in plugin
management is, all other configuration items could be controlled from one place
for all modules. Whatever needs to be specific to submodule, could be
controlled in specific module.
> And maybe we can add the test complie execution here with the skip = true
configuration, and in the sub module which also need to generate test protos
can use skip = false to enable the execution.
Yes you are right. I will do this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326912670
##
File path: hadoop-hdfs-project/hadoop-hdfs-client/pom.xml
##
@@ -131,36 +131,18 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;>
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-compile-protoc
-
- protoc
-
+src-compile-protoc
- ${protobuf.version}
- ${protoc.path}
-
-
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- ClientDatanodeProtocol.proto
- ClientNamenodeProtocol.proto
- acl.proto
- xattr.proto
- datatransfer.proto
- hdfs.proto
- encryption.proto
- inotify.proto
- erasurecoding.proto
- ReconfigurationProtocol.proto
-
-
+ false
+
+
+
${basedir}/../../hadoop-common-project/hadoop-common/src/main/proto
Review comment:
Yes. I agree that keeping proto files in jar may be fine for now.
It would be better to specify explicitly which proto files are required for
successfull proto generation. So I preferred to NOT to include in jars.
> As in the old way, you need make sure that we also depend on hadoop-common
in the dependencies section, otherwise the protoc generating is fine but there
will be compile error...
hadoop-common in dependency section is anyway required, irrespective of
whether proto files are part of jar or not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #1494: HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
vinayakumarb commented on a change in pull request #1494: HADOOP-16558.
[COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes
URL: https://github.com/apache/hadoop/pull/1494#discussion_r326912355
##
File path: hadoop-common-project/hadoop-common/pom.xml
##
@@ -381,73 +381,43 @@
-org.apache.hadoop
-hadoop-maven-plugins
+org.xolstice.maven.plugins
+protobuf-maven-plugin
-version-info
-generate-resources
-
- version-info
-
-
-
-${basedir}/src/main
-
- java/**/*.java
- proto/**/*.proto
-
-
-
+src-compile-protoc
+false
-compile-protoc
+compile-test-protoc
+generate-test-resources
- protoc
+ test-compile
- ${protobuf.version}
- ${protoc.path}
-
-${basedir}/src/main/proto
-
-
-${basedir}/src/main/proto
-
- HAServiceProtocol.proto
- IpcConnectionContext.proto
- ProtocolInfo.proto
- RpcHeader.proto
- ZKFCProtocol.proto
- ProtobufRpcEngine.proto
- Security.proto
- GetUserMappingsProtocol.proto
- TraceAdmin.proto
- RefreshAuthorizationPolicyProtocol.proto
- RefreshUserMappingsProtocol.proto
- RefreshCallQueueProtocol.proto
- GenericRefreshProtocol.proto
- FSProtos.proto
-
-
+
${basedir}/src/test/proto
Review comment:
Yes. Found here
https://www.xolstice.org/protobuf-maven-plugin/test-compile-mojo.html#protoTestSourceRoot
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1481: HADOOP-16587: Made auth endpoints configurable for MSI and refresh token flows
steveloughran commented on a change in pull request #1481: HADOOP-16587: Made
auth endpoints configurable for MSI and refresh token flows
URL: https://github.com/apache/hadoop/pull/1481#discussion_r326909886
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/Utils.java
##
@@ -0,0 +1,29 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.utils;
+
+public final class Utils {
+
+ private Utils() {
+ }
+
+ public static boolean isEmpty(String str) {
Review comment:
normally we use commons-language isEmpty here; this is quietly different in
that it also does trimming.
Can you
* name it as such to avoid confusing people, e.g. "isEmptyAfterTrimming"
* add a test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16577) Build fails as can't retrieve websocket-servlet
[
https://issues.apache.org/jira/browse/HADOOP-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16577:
Labels: build dependencies (was: dependencies)
> Build fails as can't retrieve websocket-servlet
> ---
>
> Key: HADOOP-16577
> URL: https://issues.apache.org/jira/browse/HADOOP-16577
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Erkin Alp Güney
>Priority: Blocker
> Labels: build, dependencies
>
> I encountered this error when building Hadoop:
> Downloading:
> https://repository.apache.org/content/repositories/snapshots/org/eclipse/jetty/websocket/websocket-server/9.3.27.v20190418/websocket-server-9.3.27.v20190418.jar
> Sep 15, 2019 7:54:39 AM
> org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec
> execute
> INFO: I/O exception
> (org.apache.maven.wagon.providers.http.httpclient.NoHttpResponseException)
> caught when processing request to {s}->https://repository.apache.org:443: The
> target server failed to respond
> Sep 15, 2019 7:54:39 AM
> org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec
> execute
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16592) Build fails as can't retrieve websocket-server-impl
[ https://issues.apache.org/jira/browse/HADOOP-16592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16592: Component/s: build > Build fails as can't retrieve websocket-server-impl > --- > > Key: HADOOP-16592 > URL: https://issues.apache.org/jira/browse/HADOOP-16592 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Erkin Alp Güney >Priority: Blocker > > [ERROR] Failed to execute goal on project hadoop-yarn-server-nodemanager: > Could not resolve dependencies for project > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.3.0-SNAPSHOT: The > following artifacts could not be resolved: > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-annotations:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-plus:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-jndi:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:javax-websocket-client-impl:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-client:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-server:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-common:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-api:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-servlet:jar:9.3.27.v20190418: Could not > transfer artifact > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418 > from/to apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): > repository.apache.org: Unknown host repository.apache.org -> [Help 1] > Again, the same as HADOOP-16577, but this time with websocket-server-impl. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16592) Build fails as can't retrieve websocket-server-impl
[ https://issues.apache.org/jira/browse/HADOOP-16592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16592. - Resolution: Works for Me > Build fails as can't retrieve websocket-server-impl > --- > > Key: HADOOP-16592 > URL: https://issues.apache.org/jira/browse/HADOOP-16592 > Project: Hadoop Common > Issue Type: Bug >Reporter: Erkin Alp Güney >Priority: Blocker > > [ERROR] Failed to execute goal on project hadoop-yarn-server-nodemanager: > Could not resolve dependencies for project > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.3.0-SNAPSHOT: The > following artifacts could not be resolved: > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-annotations:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-plus:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-jndi:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:javax-websocket-client-impl:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-client:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-server:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-common:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-api:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-servlet:jar:9.3.27.v20190418: Could not > transfer artifact > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418 > from/to apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): > repository.apache.org: Unknown host repository.apache.org -> [Help 1] > Again, the same as HADOOP-16577, but this time with websocket-server-impl. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16592) Build fails as can't retrieve websocket-server-impl
[ https://issues.apache.org/jira/browse/HADOOP-16592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935367#comment-16935367 ] Steve Loughran commented on HADOOP-16592: - Erkin, I;m going to conclude here that there is something wrong with your build/network setup, "Unknown host repository.apache.org" is a key example of this Closing as a WORKSFORME. I > Build fails as can't retrieve websocket-server-impl > --- > > Key: HADOOP-16592 > URL: https://issues.apache.org/jira/browse/HADOOP-16592 > Project: Hadoop Common > Issue Type: Bug >Reporter: Erkin Alp Güney >Priority: Blocker > > [ERROR] Failed to execute goal on project hadoop-yarn-server-nodemanager: > Could not resolve dependencies for project > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.3.0-SNAPSHOT: The > following artifacts could not be resolved: > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-annotations:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-plus:jar:9.3.27.v20190418, > org.eclipse.jetty:jetty-jndi:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:javax-websocket-client-impl:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-client:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-server:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-common:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-api:jar:9.3.27.v20190418, > org.eclipse.jetty.websocket:websocket-servlet:jar:9.3.27.v20190418: Could not > transfer artifact > org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.3.27.v20190418 > from/to apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): > repository.apache.org: Unknown host repository.apache.org -> [Help 1] > Again, the same as HADOOP-16577, but this time with websocket-server-impl. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1497: HDDS-2001. Update Ratis version to 0.4.0.
hadoop-yetus commented on issue #1497: HDDS-2001. Update Ratis version to 0.4.0. URL: https://github.com/apache/hadoop/pull/1497#issuecomment-533876176 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 157 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ ozone-0.4.1 Compile Tests _ | | 0 | mvndep | 85 | Maven dependency ordering for branch | | +1 | mvninstall | 732 | ozone-0.4.1 passed | | +1 | compile | 444 | ozone-0.4.1 passed | | +1 | checkstyle | 91 | ozone-0.4.1 passed | | +1 | mvnsite | 0 | ozone-0.4.1 passed | | +1 | shadedclient | 1049 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 186 | ozone-0.4.1 passed | | 0 | spotbugs | 438 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 676 | ozone-0.4.1 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 42 | Maven dependency ordering for patch | | +1 | mvninstall | 567 | the patch passed | | +1 | compile | 398 | the patch passed | | +1 | javac | 398 | the patch passed | | +1 | checkstyle | 82 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 3 | The patch has no ill-formed XML file. | | +1 | shadedclient | 648 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 174 | the patch passed | | +1 | findbugs | 643 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 190 | hadoop-hdds in the patch failed. | | -1 | unit | 282 | hadoop-ozone in the patch failed. | | +1 | asflicense | 44 | The patch does not generate ASF License warnings. | | | | 6695 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerCommandHandler | | | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1497/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1497 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 3f9cc7b12697 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | ozone-0.4.1 / 2eb41fb | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1497/1/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1497/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1497/1/testReport/ | | Max. process+thread count | 1150 (vs. ulimit of 5500) | | modules | C: hadoop-hdds hadoop-hdds/container-service hadoop-ozone hadoop-ozone/ozone-manager U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1497/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] nandakumar131 commented on issue #1483: HDDS-2001. Update Ratis version to 0.4.0.
nandakumar131 commented on issue #1483: HDDS-2001. Update Ratis version to 0.4.0. URL: https://github.com/apache/hadoop/pull/1483#issuecomment-533867812 @anuengineer Thanks for the review. There are conflicts while backporting this to ozone-0.4.1, created #1497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] nandakumar131 opened a new pull request #1497: HDDS-2001. Update Ratis version to 0.4.0.
nandakumar131 opened a new pull request #1497: HDDS-2001. Update Ratis version to 0.4.0. URL: https://github.com/apache/hadoop/pull/1497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16560) [YARN + MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes
[ https://issues.apache.org/jira/browse/HADOOP-16560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HADOOP-16560: --- Summary: [YARN + MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes (was: [YARN] use protobuf-maven-plugin to generate protobuf classes) > [YARN + MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes > - > > Key: HADOOP-16560 > URL: https://issues.apache.org/jira/browse/HADOOP-16560 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Duo Zhang >Priority: Major > > Use "protoc-maven-plugin" to dynamically download protobuf executable to > generate protobuf classes from proto file -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 commented on issue #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 commented on issue #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat… URL: https://github.com/apache/hadoop/pull/1496#issuecomment-533856145 The patch here shares some contents with #1494 , will do a rebase after #1494 is merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Apache9 opened a new pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat…
Apache9 opened a new pull request #1496: HADOOP-16560. [YARN + MAPREDUCE] use protobuf-maven-plugin to generat… URL: https://github.com/apache/hadoop/pull/1496 …e protobuf classes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
