Re: Second MRESOLVER-123
Am 2020-07-11 um 15:52 schrieb Elliotte Rusty Harold: I don't think we can safely or should assume devs use private repo managers, or that those that do improve performance. Why not? This is actually what we are recommending in tickets, SO and our website. Instead of locking, would it be possible to implement some sort of queue system for artifact downloads or use asynchronous futures? Everything is possible as long as someone implements it. The queue would lock until the artifact has been consumed. What difference would that make? The artifact download is already async in terms of that parallel threads with tasks are spawned. I do not intend to put anymore to time to something more sophisticated like a queue for now, I did already invest a lot. This issue has plagued many users. I am open to new issues until someone can implement them in time. Michael On Fri, Jul 10, 2020 at 3:46 AM Michael Osipov wrote: Folks, MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread concurrency in a single JVM while downloading artifacts/metadata. It has been verified to work with several usecases provided by reporters. This approach has one drawback: Since it is a global lock it introduces somwhat of a lock contention. The slower your connection link the longer the lock is held. Ideally, folks use repo managers in private networks to solve this issue. Also, read/write reeentrancy seems to work fine based on the logs provided. At the end I prefer build stability over performance. Please have a look and raise objects if you have any. If you don't hear any I will merge next week and start release process of Resolver 1.5.0. A midterm goal would be to provide a group id based locking which should drastically reduce contention, but no promises here for an ETA. As for the multi JVM safety, Takari local repository failed to provide a working solution, project seems to be abandoned. I have considered to investigate a Redisson-based approach for this. Though, this solely depends on the community how necessary this really is. Michael [1] https://github.com/apache/maven-resolver/pull/65 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Second MRESOLVER-123
I don't think we can safely or should assume devs use private repo managers, or that those that do improve performance. Instead of locking, would it be possible to implement some sort of queue system for artifact downloads or use asynchronous futures? On Fri, Jul 10, 2020 at 3:46 AM Michael Osipov wrote: > > Folks, > > MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread > concurrency in a single JVM while downloading artifacts/metadata. It has > been verified to work with several usecases provided by reporters. This > approach has one drawback: Since it is a global lock it introduces > somwhat of a lock contention. The slower your connection link the longer > the lock is held. Ideally, folks use repo managers in private networks > to solve this issue. Also, read/write reeentrancy seems to work fine > based on the logs provided. At the end I prefer build stability over > performance. > > Please have a look and raise objects if you have any. If you don't hear > any I will merge next week and start release process of Resolver 1.5.0. > > A midterm goal would be to provide a group id based locking which should > drastically reduce contention, but no promises here for an ETA. As for > the multi JVM safety, Takari local repository failed to provide a > working solution, project seems to be abandoned. I have considered to > investigate a Redisson-based approach for this. Though, this solely > depends on the community how necessary this really is. > > Michael > > [1] https://github.com/apache/maven-resolver/pull/65 > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Second MRESOLVER-123
Folks, MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread concurrency in a single JVM while downloading artifacts/metadata. It has been verified to work with several usecases provided by reporters. This approach has one drawback: Since it is a global lock it introduces somwhat of a lock contention. The slower your connection link the longer the lock is held. Ideally, folks use repo managers in private networks to solve this issue. Also, read/write reeentrancy seems to work fine based on the logs provided. At the end I prefer build stability over performance. Please have a look and raise objects if you have any. If you don't hear any I will merge next week and start release process of Resolver 1.5.0. A midterm goal would be to provide a group id based locking which should drastically reduce contention, but no promises here for an ETA. As for the multi JVM safety, Takari local repository failed to provide a working solution, project seems to be abandoned. I have considered to investigate a Redisson-based approach for this. Though, this solely depends on the community how necessary this really is. Michael [1] https://github.com/apache/maven-resolver/pull/65 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
