[GitHub] ant-ivy issue #54: IVY-735 Support timeouts on resolvers
Github user twogee commented on the issue: https://github.com/apache/ant-ivy/pull/54 VsftpRepository has DEFAULT_READ_TIMEOUT and DISCONNECT_COMMAND_TIMEOUT; would that require an additional timeout attribute? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
[GitHub] ant-ivy issue #54: IVY-735 Support timeouts on resolvers
Github user twogee commented on the issue: https://github.com/apache/ant-ivy/pull/54 JSch Session has a single (connection) timeout. There is a GitHub project to mock ssh/sftp server based on Mina at https://github.com/shamsoftware/sham-ssh --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Opportunities for cohesion improvement and refatoring.
Hi everyone. I will prepare the suggestions for changes. Regarding the discussion above, the following classes are the best choices: Javadoc, FTPTask, FileUtils ok? 2017-07-21 12:10 GMT-03:00 Stefan Bodewig: > Yes, using the tool might be a good first indicator. > > I didn't mean to imply you called the tools a complete answer, just > wanted to clarify my own words :-) > > Stefan > > On 2017-07-21, Gintautas Grigelionis wrote: > > > Thanks, Stefan. You're right about the semantics; I did not mean the API > > compatibility analysis to give a complete answer, > > rather a hint about the amount of (potential) breakage. Based on that, a > > decision can be taken whether it can be accepted, > > mitigated or a completely new version is a must. If the latter is the > case, > > then one may decide it's not worth the while because of the cost of > > adoption. > > > Gintas > > > 2017-07-21 11:40 GMT+02:00 Stefan Bodewig : > > >> Hi > > >> thank you, Gintautas. > > >> Yes, using a tool to verify the API hasn't changed will probably > >> help. Over in Commons we run this as a regular part of the release > >> process - it is even more important for things that are meant to be > >> re-usable components, of course. > > >> I'm afraid that won't be enough, though. > > >> Let me pick a silly example > > >> in Project we currently have > > >> public Project createSubProject() { > >> Project subProject = null; > >> try { > >> subProject = (getClass().newInstance()); > >> } catch (final Exception e) { > >> subProject = new Project(); > >> } > >> initSubProject(subProject); > >> return subProject; > >> } > > >> and initSubProject is public as well. > > >> A subclass may override initSubProject and rely on the method being > >> called by createSubProject. If you refactor createSubProject in a way > >> that it no longer calls initSubProject you are going to break the > >> subclass. And I don't think the tools are going to tell you as long as > >> you keep all existing methods. > > >> This is what I meant with "have to ensure you keep the existing who > >> calls which method semantics". > > >> Cheers > > >> Stefan > > > >> On 2017-07-20, Gintautas Grigelionis wrote: > > >>> I looked at Project proposal [1]. > >>> I would suggest running the refactored Ant through japicmp [2] or > revapi > >>> [3] and examining the binary incompatibilities. > > >>> Gintas > > >>> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61305 > >>> [2] http://siom79.github.io/japicmp/ > >>> [3] http://revapi.org/ > > >>> 2017-07-20 16:41 GMT+02:00 Stefan Bodewig : > > Welcome João Paulo > > On 2017-07-20, João Paulo Lemes Machado wrote: > > > I was analyzing the modularization of some classes of Ant, and I > > identified some opportunities for cohesion improvement in the > following > > classes: > > > Javac > > Javadoc > > FTPTask > > FileUtils > > AbstractFileSet > > Similar to what I said about Project in the bugzilla issue you created > all these classes are part of Ant's public API and need to be treated > with care. > > Ant has been around for more than fifteen years and an eco system of > extensions has ground around it. This is something that forces us to > be extra careful with refactoring. > > Of the classes listed above Javadoc and FTPTask are unlikely to have > seen subclasses, Javac may have. FileUtils is unlikely to have seen > subclasses as a lot of its code is inside static methods or is > accessed via a quasi-singleton. > > Still, when refactoring non-static public/protected methods you really > have to ensure you keep the existing who calls which method semantics > that subclasses may rely on. AbstractFileSet is an extremely dangerous > one, as it has certainly seen a lot of extensions outside of our > control. > > > TarEntry > > Is a special case to me. Ant's tar, zip and bzip2 packages have seeded > Commons Compress and from time to time I try to backport changes from > Compress to Ant - usually only the bugfixes. This may become more > difficult when the code bases start to deviate. > > I'd be interested in hearing what kind of changes you'd like to make, > but please take a look at > https://git-wip-us.apache.org/repos/asf?p=commons-compress. > git;a=blob;f=src/main/java/org/apache/commons/compress/ > archivers/tar/TarArchiveEntry.java > ? > > The class has undergone several changes that haven't been reflected > back, maybe it is mostly an issue of backporting those changes? > > Cheers > > Stefan > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail:
[GitHub] ant-ivy issue #54: IVY-735 Support timeouts on resolvers
Github user twogee commented on the issue: https://github.com/apache/ant-ivy/pull/54 The PR contains additional changes to HttpClientHandler, introducing HttpConnectionManagerParams, which is not available in Commons HTTP Client version 2 (over 10 years old and virtually gone). It would be appropriate to remove HttpClientHelper and related classes and methods, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Opportunities for cohesion improvement and refatoring.
Yes, using the tool might be a good first indicator. I didn't mean to imply you called the tools a complete answer, just wanted to clarify my own words :-) Stefan On 2017-07-21, Gintautas Grigelionis wrote: > Thanks, Stefan. You're right about the semantics; I did not mean the API > compatibility analysis to give a complete answer, > rather a hint about the amount of (potential) breakage. Based on that, a > decision can be taken whether it can be accepted, > mitigated or a completely new version is a must. If the latter is the case, > then one may decide it's not worth the while because of the cost of > adoption. > Gintas > 2017-07-21 11:40 GMT+02:00 Stefan Bodewig: >> Hi >> thank you, Gintautas. >> Yes, using a tool to verify the API hasn't changed will probably >> help. Over in Commons we run this as a regular part of the release >> process - it is even more important for things that are meant to be >> re-usable components, of course. >> I'm afraid that won't be enough, though. >> Let me pick a silly example >> in Project we currently have >> public Project createSubProject() { >> Project subProject = null; >> try { >> subProject = (getClass().newInstance()); >> } catch (final Exception e) { >> subProject = new Project(); >> } >> initSubProject(subProject); >> return subProject; >> } >> and initSubProject is public as well. >> A subclass may override initSubProject and rely on the method being >> called by createSubProject. If you refactor createSubProject in a way >> that it no longer calls initSubProject you are going to break the >> subclass. And I don't think the tools are going to tell you as long as >> you keep all existing methods. >> This is what I meant with "have to ensure you keep the existing who >> calls which method semantics". >> Cheers >> Stefan >> On 2017-07-20, Gintautas Grigelionis wrote: >>> I looked at Project proposal [1]. >>> I would suggest running the refactored Ant through japicmp [2] or revapi >>> [3] and examining the binary incompatibilities. >>> Gintas >>> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61305 >>> [2] http://siom79.github.io/japicmp/ >>> [3] http://revapi.org/ >>> 2017-07-20 16:41 GMT+02:00 Stefan Bodewig : Welcome João Paulo On 2017-07-20, João Paulo Lemes Machado wrote: > I was analyzing the modularization of some classes of Ant, and I > identified some opportunities for cohesion improvement in the following > classes: > Javac > Javadoc > FTPTask > FileUtils > AbstractFileSet Similar to what I said about Project in the bugzilla issue you created all these classes are part of Ant's public API and need to be treated with care. Ant has been around for more than fifteen years and an eco system of extensions has ground around it. This is something that forces us to be extra careful with refactoring. Of the classes listed above Javadoc and FTPTask are unlikely to have seen subclasses, Javac may have. FileUtils is unlikely to have seen subclasses as a lot of its code is inside static methods or is accessed via a quasi-singleton. Still, when refactoring non-static public/protected methods you really have to ensure you keep the existing who calls which method semantics that subclasses may rely on. AbstractFileSet is an extremely dangerous one, as it has certainly seen a lot of extensions outside of our control. > TarEntry Is a special case to me. Ant's tar, zip and bzip2 packages have seeded Commons Compress and from time to time I try to backport changes from Compress to Ant - usually only the bugfixes. This may become more difficult when the code bases start to deviate. I'd be interested in hearing what kind of changes you'd like to make, but please take a look at https://git-wip-us.apache.org/repos/asf?p=commons-compress. git;a=blob;f=src/main/java/org/apache/commons/compress/ archivers/tar/TarArchiveEntry.java ? The class has undergone several changes that haven't been reflected back, maybe it is mostly an issue of backporting those changes? Cheers Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org >> - >> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org >> For additional commands, e-mail: dev-h...@ant.apache.org - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Opportunities for cohesion improvement and refatoring.
Thanks, Stefan. You're right about the semantics; I did not mean the API compatibility analysis to give a complete answer, rather a hint about the amount of (potential) breakage. Based on that, a decision can be taken whether it can be accepted, mitigated or a completely new version is a must. If the latter is the case, then one may decide it's not worth the while because of the cost of adoption. Gintas 2017-07-21 11:40 GMT+02:00 Stefan Bodewig: > Hi > > thank you, Gintautas. > > Yes, using a tool to verify the API hasn't changed will probably > help. Over in Commons we run this as a regular part of the release > process - it is even more important for things that are meant to be > re-usable components, of course. > > I'm afraid that won't be enough, though. > > Let me pick a silly example > > in Project we currently have > > public Project createSubProject() { > Project subProject = null; > try { > subProject = (getClass().newInstance()); > } catch (final Exception e) { > subProject = new Project(); > } > initSubProject(subProject); > return subProject; > } > > and initSubProject is public as well. > > A subclass may override initSubProject and rely on the method being > called by createSubProject. If you refactor createSubProject in a way > that it no longer calls initSubProject you are going to break the > subclass. And I don't think the tools are going to tell you as long as > you keep all existing methods. > > This is what I meant with "have to ensure you keep the existing who > calls which method semantics". > > Cheers > > Stefan > > > On 2017-07-20, Gintautas Grigelionis wrote: > > > I looked at Project proposal [1]. > > I would suggest running the refactored Ant through japicmp [2] or revapi > > [3] and examining the binary incompatibilities. > > > Gintas > > > [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61305 > > [2] http://siom79.github.io/japicmp/ > > [3] http://revapi.org/ > > > 2017-07-20 16:41 GMT+02:00 Stefan Bodewig : > > >> Welcome João Paulo > > >> On 2017-07-20, João Paulo Lemes Machado wrote: > > >>> I was analyzing the modularization of some classes of Ant, and I > >>> identified some opportunities for cohesion improvement in the following > >>> classes: > > >>> Javac > >>> Javadoc > >>> FTPTask > >>> FileUtils > >>> AbstractFileSet > > >> Similar to what I said about Project in the bugzilla issue you created > >> all these classes are part of Ant's public API and need to be treated > >> with care. > > >> Ant has been around for more than fifteen years and an eco system of > >> extensions has ground around it. This is something that forces us to > >> be extra careful with refactoring. > > >> Of the classes listed above Javadoc and FTPTask are unlikely to have > >> seen subclasses, Javac may have. FileUtils is unlikely to have seen > >> subclasses as a lot of its code is inside static methods or is > >> accessed via a quasi-singleton. > > >> Still, when refactoring non-static public/protected methods you really > >> have to ensure you keep the existing who calls which method semantics > >> that subclasses may rely on. AbstractFileSet is an extremely dangerous > >> one, as it has certainly seen a lot of extensions outside of our > >> control. > > >>> TarEntry > > >> Is a special case to me. Ant's tar, zip and bzip2 packages have seeded > >> Commons Compress and from time to time I try to backport changes from > >> Compress to Ant - usually only the bugfixes. This may become more > >> difficult when the code bases start to deviate. > > >> I'd be interested in hearing what kind of changes you'd like to make, > >> but please take a look at > >> https://git-wip-us.apache.org/repos/asf?p=commons-compress. > >> git;a=blob;f=src/main/java/org/apache/commons/compress/ > >> archivers/tar/TarArchiveEntry.java > >> ? > > >> The class has undergone several changes that haven't been reflected > >> back, maybe it is mostly an issue of backporting those changes? > > >> Cheers > > >> Stefan > > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > >> For additional commands, e-mail: dev-h...@ant.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > >
Re: Opportunities for cohesion improvement and refatoring.
Hi thank you, Gintautas. Yes, using a tool to verify the API hasn't changed will probably help. Over in Commons we run this as a regular part of the release process - it is even more important for things that are meant to be re-usable components, of course. I'm afraid that won't be enough, though. Let me pick a silly example in Project we currently have public Project createSubProject() { Project subProject = null; try { subProject = (getClass().newInstance()); } catch (final Exception e) { subProject = new Project(); } initSubProject(subProject); return subProject; } and initSubProject is public as well. A subclass may override initSubProject and rely on the method being called by createSubProject. If you refactor createSubProject in a way that it no longer calls initSubProject you are going to break the subclass. And I don't think the tools are going to tell you as long as you keep all existing methods. This is what I meant with "have to ensure you keep the existing who calls which method semantics". Cheers Stefan On 2017-07-20, Gintautas Grigelionis wrote: > I looked at Project proposal [1]. > I would suggest running the refactored Ant through japicmp [2] or revapi > [3] and examining the binary incompatibilities. > Gintas > [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61305 > [2] http://siom79.github.io/japicmp/ > [3] http://revapi.org/ > 2017-07-20 16:41 GMT+02:00 Stefan Bodewig: >> Welcome João Paulo >> On 2017-07-20, João Paulo Lemes Machado wrote: >>> I was analyzing the modularization of some classes of Ant, and I >>> identified some opportunities for cohesion improvement in the following >>> classes: >>> Javac >>> Javadoc >>> FTPTask >>> FileUtils >>> AbstractFileSet >> Similar to what I said about Project in the bugzilla issue you created >> all these classes are part of Ant's public API and need to be treated >> with care. >> Ant has been around for more than fifteen years and an eco system of >> extensions has ground around it. This is something that forces us to >> be extra careful with refactoring. >> Of the classes listed above Javadoc and FTPTask are unlikely to have >> seen subclasses, Javac may have. FileUtils is unlikely to have seen >> subclasses as a lot of its code is inside static methods or is >> accessed via a quasi-singleton. >> Still, when refactoring non-static public/protected methods you really >> have to ensure you keep the existing who calls which method semantics >> that subclasses may rely on. AbstractFileSet is an extremely dangerous >> one, as it has certainly seen a lot of extensions outside of our >> control. >>> TarEntry >> Is a special case to me. Ant's tar, zip and bzip2 packages have seeded >> Commons Compress and from time to time I try to backport changes from >> Compress to Ant - usually only the bugfixes. This may become more >> difficult when the code bases start to deviate. >> I'd be interested in hearing what kind of changes you'd like to make, >> but please take a look at >> https://git-wip-us.apache.org/repos/asf?p=commons-compress. >> git;a=blob;f=src/main/java/org/apache/commons/compress/ >> archivers/tar/TarArchiveEntry.java >> ? >> The class has undergone several changes that haven't been reflected >> back, maybe it is mostly an issue of backporting those changes? >> Cheers >> Stefan >> - >> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org >> For additional commands, e-mail: dev-h...@ant.apache.org - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Proposed changes for IVY-735 - Ability to specify timeouts
The support for this feature is now available. Given that this is kind of a core and relatively big change, I decided to open a PR[1] so that some of us can take a look to make sure the changes make sense. The description in the PR plus the documentation in the commit explains how this works, from a user point of view. Internally, all it does is, pass along the right timeout constraint to the resolvers and use it while dealing with resources. As I note in that PR, I plan to add tests wherever possible to add more coverage to this, in subsequent separate commits. Now coming to one of the suggestions in this mail thread, which said we could have something like a default timeout constraint that's defined for all resolvers: surprisingly (to me), it's currently not possible from what I see in our parsing code. The Ivy documentation of "resolvers"[2] talks about common attributes that are applicable from all resolvers and kind of gives an impression that you can use/set those attributes on the "resolvers" element, but in reality I think what it means is, the attributes are common to all resolvers but need to be set on the individual resolver elements. I in fact looked in the code and even tried out a few things to see if I was missing something, but it appears that whatever attribute you currently set on the resolvers element is completely ignored (it doesn't error out). Actually, this behaviour isn't specific to resolvers element but similar "wrapper" kind of elements from what I see in the code. Anyway, long story short, for now I decided not to add the the support for such a construct at least until we can do a similar thing for all common attributes of resolvers. By the way, although I did actually write a test to verify this behaivour, I may have missed or misunderstood the parsing code so if anyone thinks what I state isn't accurate, feel free to correct me. [1] https://github.com/apache/ant-ivy/pull/54 [2] https://ant.apache.org/ivy/history/latest-milestone/settings/resolvers.html#common -Jaikiran On 18/07/17 2:02 AM, Jan Matèrne (jhm) wrote: Maybe you could set the timeout strategy globally: /ivysettings/resolvers/@timeout-constraint ... Jan -Ursprüngliche Nachricht- Von: jai.forums2...@gmail.com [mailto:jai.forums2...@gmail.com] Gesendet: Montag, 17. Juli 2017 06:56 An: dev@ant.apache.org Betreff: Re: Proposed changes for IVY-735 - Ability to specify timeouts I think, that's actually a good idea. Defining the timeout constraints separately as named constraints and then referencing them would allow reusing these values on whichever resolvers want to: ... -Jaikiran On 14/07/17 10:33 PM, Gintautas Grigelionis wrote: my €.02 : currently, all resolvers only have artifact patterns as child elements, so I'd rather have the timeouts as attributes that might use references, similarly to cache managers or latest strategies. That would imply a global default that may be overridden individually. Gintas 2017-07-14 13:34 GMT+02:00: https://issues.apache.org/jira/browse/IVY-735 is a feature request where the users have asked for relevant timeouts while dealing with downloads. A few weeks back we had a very brief discussion in an unrelated mail where it was proposed that we allow configuring these timeout all the way from ivy settings. I have an initial proposal/attempt to implement this feature. The initial changes are here in my personal repo[1]. It's a work in progress commit which sets up the necessary interfaces and the flow to show what I have in mind. Before proceeding further, I would like some inputs on whether this looks fine. Here's a summary of what the changes are going to be: - Each (dependency) resolver will have the ability to specify "timeout constraints"[2]. - These timeout constraints will be specified while defining the resolver in the ivy settings file. Imagine something like: - Each of the resolver will then use these timeout constraints (if specified) while resolving the module descriptor and downloading the artifacts and dealing with those resources. - The absence of the timeout constraints will let the resolvers behave the way they do currently The changes in the linked commit deprecate some APIs which weren't using timeouts and are replaced by APIs which pass along (an optional) TimeoutConstraints[2]. In summary, the change being proposed is that (dependency) resolvers have the ability to specify these timeout constraints and then use them while dealing with module descriptors and artifacts. One thing I have (to some extent intentionally) left out is the ability to define a global Ivy settings level or "all resolvers" level timeout-constraints. That's because I'm not too sure if it adds much value
[GitHub] ant-ivy pull request #54: IVY-735 Support timeouts on resolvers
GitHub user jaikiran opened a pull request: https://github.com/apache/ant-ivy/pull/54 IVY-735 Support timeouts on resolvers The commit in this PR adds support for specifying connection and read timeouts for resolvers, so that users have control over how the resolvers behave when it comes to slow/unresponsive repositories. This feature has been requested in https://issues.apache.org/jira/browse/IVY-735 This commit introduces the concept of "timeout-constraints" within a Ivy settings file. There can be any number of named timeout constraints, each with (optional) values for connection and read timeouts. These named timeout constraints can then be referred to by the individual resolvers via the `timeoutConstraint` attribute on them. Standard resolvers all have been updated to support this new attribute. An example usage looks something like this: ``` ``` (explanation of what these settings signify are available in the docs here https://github.com/jaikiran/ant-ivy/commit/c0ffb23ae29197e7f47c140deb991139d0688421#diff-e094f56ed4707eb73f8621abe7bfb4e5R54) The commit also contains updates to documentation to include details of this new feature, plus test cases to verify the basic functionality of this feature on some specific resolvers. I need to check a few standard resolvers (like the ssh based ones) to make sure they honour this new timeout semantics too and include tests (if possible) for them. But at this point, this commit should cover the support and the code flow for most of the relevant resolvers like the `URLResolver`. In fact, one of the tests in this commit, actually simulates a HTTP backed repo being down, in a test case to verify the `URLResolver` honours these timeout constraints. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaikiran/ant-ivy ivy-735 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ant-ivy/pull/54.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #54 commit c0ffb23ae29197e7f47c140deb991139d0688421 Author: Jaikiran PaiDate: 2017-07-10T04:50:38Z IVY-735 Support timeouts on resolvers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org