Ivy Jenkins jobs have been failing - infra issues

[Jaikiran Pai]

More of a FYI than anything else - the past couple of days or so, we 
haven't had regular successful runs of our various Ivy Jenkins jobs. 
Each run failing with different errors, all pointing back to 
infrastructure issues at this moment. In fact, I see some other Apache 
project builds running into similar issues - as basic as DNS name 
resolution failures of git-wip-us.apache.org on the Jenkins nodes.


None of this is specific to our jobs, so I don't see anything that we 
can do to solve this at the moment, other than maybe raising it with 
Infra team. I haven't had the time to check the Infra team mailing list 
or their JIRA to see if this is already reported by other projects. I 
will see if I can find some time tonight to check that and report it, if 
it hasn't yet been.


-Jaikiran



-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Ivy - Move to symlink creation standard Java API?

[Jaikiran Pai]

I was looking into a JIRA related to symlinking in Ivy and realized that 
(for reasons noted in the docs and the implementation) ourimplementation 
of symlinking relies on launching a process from within the JVM to 
invoke a shell command to create the symlinks. "retrieve" task is the 
only one that deals with symlink creation, from what I can see. 
Furthermore, knowing that this operation is expensive, we even had to 
introduce an additional optionfor that task to allow "mass 
symlinking"[1] so that we launch a single process to create N symlinks 
instead of N processes.


In short, the current implementation of symlinking in Ivy is expensive 
for reasons that have been known. Now that we have moved to Java 7, 
which has a standard APIfor symlink creation[2], I think we should just 
change our internal implementation to use this new standard API.Of 
course, we continue to fail to create symlinks on systems that don't 
support it, just like we do now, except that we let the Java API 
implementation handle those details.


Furthermore, once we move to thisstandard API, I don't think we need the 
"symlinkmass" option onthe retrieve task anymore since the whole purpose 
of it was to avoid launching N processes. So I think we can deprecate 
that option in the upcoming release.


Any thoughts?

[1] https://ant.apache.org/ivy/history/latest-milestone/use/retrieve.html

[2] 
https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#createSymbolicLink(java.nio.file.Path,%20java.nio.file.Path,%20java.nio.file.attribute.FileAttribute...)


-Jaikiran


-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[jaikiran]

Github user jaikiran commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129593699
  
--- Diff: src/java/org/apache/ivy/plugins/report/XmlReportParser.java ---
@@ -71,132 +71,139 @@
 
--- End diff --

This file has a large change which appears to be reordering/restructuring 
the code logic. Given the nature of this diff, it's difficult to review it and 
see if this change introduces any changes in the logic or introduces any 
regressions. Plus this class is one of the very core ones. Also, I'm not too 
sure if the existing test cases cover for this level of change.

Given all this, I'm not confident that these changes in this specific class 
are worth it, at least at the moment. Unless of course, some one else can 
review this specific class and approve it, would you be willing to undo the 
changes to this one class in this PR?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[jaikiran]

Github user jaikiran commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129587968
  
--- Diff: 
src/java/org/apache/ivy/plugins/conflict/LatestCompatibleConflictManager.java 
---
@@ -85,14 +85,12 @@ public LatestCompatibleConflictManager(String name, 
LatestStrategy strategy) {
 if (versionMatcher.isDynamic(mrid)) {
 while (iter.hasNext()) {
 IvyNode other = iter.next();
-if (versionMatcher.isDynamic(other.getResolvedId())) {
+if (versionMatcher.isDynamic(other.getResolvedId())
+|| !versionMatcher.accept(mrid, 
other.getResolvedId())
+&& !handleIncompatibleConflict(parent, conflicts, 
node, other)) {
--- End diff --

This change doesn't look right. The previous logic would have returned 
`null` if the `versionMatcher.isDynamic` returned true. With this change, it 
will require an additional `&&` to be satisfied for the 
`!handlerIncompatibleConflict...`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy issue #53: Suppressed warnings

[jaikiran]

Github user jaikiran commented on the issue:

https://github.com/apache/ant-ivy/pull/53
  
Commit 
https://github.com/apache/ant-ivy/pull/53/commits/432c42ad1ee54ab08ea8b8808086ba19d4880737
 in this PR has now been merged upstream 
https://github.com/apache/ant-ivy/commit/080e96a6cdea2f5fe8727b8249044b89bc54395d


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Ivy - Move to symlink creation standard Java API?

[Nicolas Lalevée]

> Le 26 juil. 2017 à 15:25, Jaikiran Pai  a écrit :
> 
> I was looking into a JIRA related to symlinking in Ivy and realized that (for 
> reasons noted in the docs and the implementation) ourimplementation of 
> symlinking relies on launching a process from within the JVM to invoke a 
> shell command to create the symlinks. "retrieve" task is the only one that 
> deals with symlink creation, from what I can see. Furthermore, knowing that 
> this operation is expensive, we even had to introduce an additional optionfor 
> that task to allow "mass symlinking"[1] so that we launch a single process to 
> create N symlinks instead of N processes.
> 
> In short, the current implementation of symlinking in Ivy is expensive for 
> reasons that have been known. Now that we have moved to Java 7, which has a 
> standard APIfor symlink creation[2], I think we should just change our 
> internal implementation to use this new standard API.Of course, we continue 
> to fail to create symlinks on systems that don't support it, just like we do 
> now, except that we let the Java API implementation handle those details.
> 
> Furthermore, once we move to thisstandard API, I don't think we need the 
> "symlinkmass" option onthe retrieve task anymore since the whole purpose of 
> it was to avoid launching N processes. So I think we can deprecate that 
> option in the upcoming release.
> 
> Any thoughts?

Sounds good to me.

Nicolas


-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Build failed in Jenkins: Ivy #627

[Jaikiran Pai]

To avoid more such mails, I'm going to disable this job for a few hours 
till we get the infra issues sorted out.


-Jaikiran


On 26/07/17 10:44 PM, Apache Jenkins Server wrote:

See 

--
Started by an SCM change
Running as hibou
[EnvInject] - Loading node environment variables.
Building remotely on cassandra15 (cassandra ubuntu) in workspace 

  > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
  > git config remote.origin.url 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git # timeout=10
Cleaning workspace
  > git rev-parse --verify HEAD # timeout=10
Resetting working tree
  > git reset --hard # timeout=10
  > git clean -fdx # timeout=10
  > git submodule foreach --recursive git reset --hard # timeout=10
  > git submodule foreach --recursive git clean -fdx # timeout=10
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git
  > git --version # timeout=10
  > git fetch --tags --progress 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git 
+refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:812)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1079)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1110)
at hudson.scm.SCM.checkout(SCM.java:495)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git 
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: fatal: unable to access 
'https://git-wip-us.apache.org/repos/asf/ant-ivy.git/': Could not resolve host: 
git-wip-us.apache.org

at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1903)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1622)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at ..remote call to cassandra15(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
at hudson.remoting.Channel.call(Channel.java:830)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor864.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy104.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:810)
... 11 more
ERROR: Error fetching remote repo 'origin'
Retrying after 10 seconds
  > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
  > git config remote.origin.url 
https://git-wip-us.apache.org/repos/asf/ant-ivy.git # timeout=10
Cleaning workspace
  > git rev-parse --verify HEAD # timeout=10
Resetting working tree
  >

Re: Ivy Jenkins jobs have been failing - infra issues

[Jaikiran Pai]

https://issues.apache.org/jira/browse/INFRA-14712

-Jaikiran
On 26/07/17 6:36 PM, Jaikiran Pai wrote:
More of a FYI than anything else - the past couple of days or so, we 
haven't had regular successful runs of our various Ivy Jenkins jobs. 
Each run failing with different errors, all pointing back to 
infrastructure issues at this moment. In fact, I see some other Apache 
project builds running into similar issues - as basic as DNS name 
resolution failures of git-wip-us.apache.org on the Jenkins nodes.


None of this is specific to our jobs, so I don't see anything that we 
can do to solve this at the moment, other than maybe raising it with 
Infra team. I haven't had the time to check the Infra team mailing 
list or their JIRA to see if this is already reported by other 
projects. I will see if I can find some time tonight to check that and 
report it, if it hasn't yet been.


-Jaikiran





-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[twogee]

Github user twogee commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129660986
  
--- Diff: src/java/org/apache/ivy/plugins/report/XmlReportParser.java ---
@@ -71,132 +71,139 @@
 
--- End diff --

It's a change from if ... else if ... else if ... else if ... to a switch, 
which changes indentation and messes up git diff. I will rebase and rearrange 
the remaining commits.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[twogee]

Github user twogee commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129663139
  
--- Diff: 
src/java/org/apache/ivy/plugins/conflict/LatestCompatibleConflictManager.java 
---
@@ -85,14 +85,12 @@ public LatestCompatibleConflictManager(String name, 
LatestStrategy strategy) {
 if (versionMatcher.isDynamic(mrid)) {
 while (iter.hasNext()) {
 IvyNode other = iter.next();
-if (versionMatcher.isDynamic(other.getResolvedId())) {
+if (versionMatcher.isDynamic(other.getResolvedId())
+|| !versionMatcher.accept(mrid, 
other.getResolvedId())
+&& !handleIncompatibleConflict(parent, conflicts, 
node, other)) {
--- End diff --

[It still does](http://introcs.cs.princeton.edu/java/11precedence/). 
There's a difference between `&&` and `||` vs `&` and `|`, see "short 
circuiting".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[twogee]

Github user twogee commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129668076
  
--- Diff: src/java/org/apache/ivy/plugins/report/XmlReportParser.java ---
@@ -71,132 +71,139 @@
 
--- End diff --

Here's a "diff -wc"-style diff

[XmlReporteParser.zip](https://github.com/apache/ant-ivy/files/1177761/XmlReporteParser.zip)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy issue #53: Suppressed warnings

[twogee]

Github user twogee commented on the issue:

https://github.com/apache/ant-ivy/pull/53
  
This is strange, all tests still pass for me locally -- could someone 
elucidate what the problem is, please?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy issue #53: Suppressed warnings

[twogee]

Github user twogee commented on the issue:

https://github.com/apache/ant-ivy/pull/53
  
D'uh... getExtends() must respect insertion order -- array straitjacket 
😄 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy issue #53: Suppressed warnings

[twogee]

Github user twogee commented on the issue:

https://github.com/apache/ant-ivy/pull/53
  
I'm sorry for not explaining the point with all the changes around foreach 
loops: they take anything that is iterable rather than an array (which Ivy is 
very fond of API-wise) and by doing so allow for API changes where a collection 
is a more natural choice (vis-à-vis streams in Java 8+). I was first afraid 
that a change of API might be necessary sooner rather than later (because of 
inconsistencies in generics). That is avoided for now, but should the question 
be raised again, this PR tries to lay a groundwork indicating a direction of a 
preferred change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

[GitHub] ant-ivy pull request #53: Suppressed warnings

[jaikiran]

Github user jaikiran commented on a diff in the pull request:

https://github.com/apache/ant-ivy/pull/53#discussion_r129757849
  
--- Diff: 
src/java/org/apache/ivy/plugins/conflict/LatestCompatibleConflictManager.java 
---
@@ -85,14 +85,12 @@ public LatestCompatibleConflictManager(String name, 
LatestStrategy strategy) {
 if (versionMatcher.isDynamic(mrid)) {
 while (iter.hasNext()) {
 IvyNode other = iter.next();
-if (versionMatcher.isDynamic(other.getResolvedId())) {
+if (versionMatcher.isDynamic(other.getResolvedId())
+|| !versionMatcher.accept(mrid, 
other.getResolvedId())
+&& !handleIncompatibleConflict(parent, conflicts, 
node, other)) {
--- End diff --

You are right. Although I was aware of short circuiting, I had a wrong 
semantic in mind about it. I was under the impression, the short circuiting 
happens "smartly" depending on what other operators are present in that 
expression. I read up the JLS (section 15.24)[1] now and as you note, the right 
side of the `||` will be completely ignored (irrespective of what additional 
operators exist in that expression) if the left side returns true. So this 
change of the `if` block is fine.

[1] http://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.24


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org