renechoi commented on PR #4977: URL: https://github.com/apache/zeppelin/pull/4977#issuecomment-3113599171
@ankursaini2006 Thank you for your insights! Regarding ZEPPELIN-6231: Great to hear that these Repository improvements should fix the Kubernetes deployment issue you reported. About the dependency resolution performance issue you raised - this is indeed a critical performance problem. I investigated the code and confirmed that `DependencyResolver.getArtifactsWithDep()` always downloads the entire transitive dependency tree using `system.resolveDependencies()`. Your experience with 20 minutes for downloading trino jdbc jar is a serious concern. Your suggestion to use `resolveArtifact()` instead of `resolveDependencies()` makes sense for cases where only the main artifact is needed. However, this would require careful implementation to avoid runtime ClassNotFoundException issues. @Reamer What do you think about creating a follow-up JIRA ticket for this dependency resolution optimization? We could: 1. Add a configuration option to control transitive dependency resolution 2. Implement a `resolveWithoutTransitive` flag in DependencyResolver 3. Allow users to choose between full dependency tree vs. single artifact download This would be a significant performance improvement while maintaining backward compatibility. Should we track this as a separate enhancement after this PR is merged? Ready for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
