Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
davsclaus merged PR #23129: URL: https://github.com/apache/camel/pull/23129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
davsclaus commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4439806158 > @ammachado can you rebase on main and resolve the upgrade guide conflict please? once done, it is good to merge I did that, the web editor is okay for some tasks like this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
Croway commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4438694456 @ammachado can you rebase on main and resolve the upgrade guide conflict please? once done, it is good to merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
davsclaus commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4437700789 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
davsclaus commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4432211843 So I guess this makes this part a little bit faster - especially when you have custom plugins installed like forage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
Croway commented on code in PR #23129:
URL: https://github.com/apache/camel/pull/23129#discussion_r3225048229
##
dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/common/PluginHelper.java:
##
@@ -179,30 +221,247 @@ public static Map
getActivePlugins(CamelJBangMain main, String r
versionCheck(main, version, firstVersion, command);
}
-Optional plugin = getPlugin(command, version, gav,
repos, main.getOut());
-if (plugin.isPresent()) {
-activePlugins.put(command, plugin.get());
+ResolveResult res = resolvePlugin(properties, command,
version, gav, repos, main.getOut());
+if (res.plugin().isPresent()) {
+activePlugins.put(command, res.plugin().get());
+if (res.cacheWritten()) {
+configDirty = true;
+}
} else {
main.getOut().println("camel-jbang-plugin-" + command + "
not found. Exit");
main.quit(1);
}
}
+if (configDirty) {
+savePluginConfig(config);
+}
}
return activePlugins;
}
public static Optional getPlugin(String name, String
defaultVersion, String gav, String repos, Printer printer) {
+return resolvePlugin(null, name, defaultVersion, gav, repos,
printer).plugin();
+}
+
+/**
+ * Resolves a plugin by trying, in order: the cached metadata in the
plugin entry (fast path with no IO beyond
+ * size+mtime checks), the factory finder (embedded plugin on the JVM
classpath), and finally the Maven downloader.
+ * When the downloader runs, the resolved classpath is captured into the
plugin entry's {@code resolved} block so
+ * subsequent invocations take the fast path.
+ */
+private static ResolveResult resolvePlugin(
+JsonObject entry, String name, String defaultVersion, String gav,
String repos, Printer printer) {
+Optional cached = loadFromCache(entry, defaultVersion, gav,
repos);
+if (cached.isPresent()) {
+return new ResolveResult(cached, false);
+}
+
Optional plugin =
FACTORY_FINDER.newInstance("camel-jbang-plugin-" + name, Plugin.class);
-if (plugin.isEmpty()) {
-final MavenGav mavenGav = dependencyAsMavenGav(gav);
-final String group = extractGroup(mavenGav, "org.apache.camel");
-final String depVersion = extractVersion(mavenGav, defaultVersion);
+if (plugin.isPresent()) {
+return new ResolveResult(plugin, false);
+}
+
+final MavenGav mavenGav = dependencyAsMavenGav(gav);
+final String group = extractGroup(mavenGav, "org.apache.camel");
+final String depVersion = extractVersion(mavenGav, defaultVersion);
+
+DownloadResult dr = downloadPlugin(name, defaultVersion, depVersion,
group, repos, printer);
+boolean cacheWritten = false;
+if (dr.plugin().isPresent() && entry != null && dr.classLoader() !=
null && dr.className() != null) {
+cacheWritten = writeCache(entry, defaultVersion, gav, repos,
dr.className(), dr.classLoader(), name, depVersion);
+}
+return new ResolveResult(dr.plugin(), cacheWritten);
+}
+
+private static Optional loadFromCache(JsonObject entry, String
camelVersion, String gav, String repos) {
+if (entry == null) {
+return Optional.empty();
+}
+JsonObject resolved = entry.getMap("resolved");
+if (resolved == null) {
+return Optional.empty();
+}
+if (!sameCamelVersion(asString(resolved.get("camelVersion")),
camelVersion)) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("gav"))),
normalize(gav))) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("repos"))),
normalize(repos))) {
+return Optional.empty();
+}
+String className = asString(resolved.get("className"));
+if (className == null || className.isBlank()) {
+return Optional.empty();
+}
+Object cpObj = resolved.get("classpath");
+if (!(cpObj instanceof Collection)) {
+return Optional.empty();
+}
+Collection classpath = (Collection) cpObj;
+if (classpath.isEmpty()) {
+return Optional.empty();
+}
+
+List urls = new ArrayList<>(classpath.size());
+for (Object o : classpath) {
+if (!(o instanceof Map)) {
+return Optional.empty();
+}
+Map jar = (Map) o;
+Path p = validateFileEntry(jar);
+if (p == null) {
+return Optional.empty();
+}
+
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
Croway commented on code in PR #23129:
URL: https://github.com/apache/camel/pull/23129#discussion_r3225043169
##
dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/common/PluginHelper.java:
##
@@ -179,30 +221,247 @@ public static Map
getActivePlugins(CamelJBangMain main, String r
versionCheck(main, version, firstVersion, command);
}
-Optional plugin = getPlugin(command, version, gav,
repos, main.getOut());
-if (plugin.isPresent()) {
-activePlugins.put(command, plugin.get());
+ResolveResult res = resolvePlugin(properties, command,
version, gav, repos, main.getOut());
+if (res.plugin().isPresent()) {
+activePlugins.put(command, res.plugin().get());
+if (res.cacheWritten()) {
+configDirty = true;
+}
} else {
main.getOut().println("camel-jbang-plugin-" + command + "
not found. Exit");
main.quit(1);
}
}
+if (configDirty) {
+savePluginConfig(config);
+}
}
return activePlugins;
}
public static Optional getPlugin(String name, String
defaultVersion, String gav, String repos, Printer printer) {
+return resolvePlugin(null, name, defaultVersion, gav, repos,
printer).plugin();
+}
+
+/**
+ * Resolves a plugin by trying, in order: the cached metadata in the
plugin entry (fast path with no IO beyond
+ * size+mtime checks), the factory finder (embedded plugin on the JVM
classpath), and finally the Maven downloader.
+ * When the downloader runs, the resolved classpath is captured into the
plugin entry's {@code resolved} block so
+ * subsequent invocations take the fast path.
+ */
+private static ResolveResult resolvePlugin(
+JsonObject entry, String name, String defaultVersion, String gav,
String repos, Printer printer) {
+Optional cached = loadFromCache(entry, defaultVersion, gav,
repos);
+if (cached.isPresent()) {
+return new ResolveResult(cached, false);
+}
+
Optional plugin =
FACTORY_FINDER.newInstance("camel-jbang-plugin-" + name, Plugin.class);
-if (plugin.isEmpty()) {
-final MavenGav mavenGav = dependencyAsMavenGav(gav);
-final String group = extractGroup(mavenGav, "org.apache.camel");
-final String depVersion = extractVersion(mavenGav, defaultVersion);
+if (plugin.isPresent()) {
+return new ResolveResult(plugin, false);
+}
+
+final MavenGav mavenGav = dependencyAsMavenGav(gav);
+final String group = extractGroup(mavenGav, "org.apache.camel");
+final String depVersion = extractVersion(mavenGav, defaultVersion);
+
+DownloadResult dr = downloadPlugin(name, defaultVersion, depVersion,
group, repos, printer);
+boolean cacheWritten = false;
+if (dr.plugin().isPresent() && entry != null && dr.classLoader() !=
null && dr.className() != null) {
+cacheWritten = writeCache(entry, defaultVersion, gav, repos,
dr.className(), dr.classLoader(), name, depVersion);
+}
+return new ResolveResult(dr.plugin(), cacheWritten);
+}
+
+private static Optional loadFromCache(JsonObject entry, String
camelVersion, String gav, String repos) {
+if (entry == null) {
+return Optional.empty();
+}
+JsonObject resolved = entry.getMap("resolved");
+if (resolved == null) {
+return Optional.empty();
+}
+if (!sameCamelVersion(asString(resolved.get("camelVersion")),
camelVersion)) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("gav"))),
normalize(gav))) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("repos"))),
normalize(repos))) {
+return Optional.empty();
+}
+String className = asString(resolved.get("className"));
+if (className == null || className.isBlank()) {
+return Optional.empty();
+}
+Object cpObj = resolved.get("classpath");
+if (!(cpObj instanceof Collection)) {
+return Optional.empty();
+}
+Collection classpath = (Collection) cpObj;
+if (classpath.isEmpty()) {
+return Optional.empty();
+}
+
+List urls = new ArrayList<>(classpath.size());
+for (Object o : classpath) {
+if (!(o instanceof Map)) {
+return Optional.empty();
+}
+Map jar = (Map) o;
+Path p = validateFileEntry(jar);
+if (p == null) {
+return Optional.empty();
+}
+
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
Croway commented on code in PR #23129:
URL: https://github.com/apache/camel/pull/23129#discussion_r3225041054
##
dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/common/PluginHelper.java:
##
@@ -179,30 +221,247 @@ public static Map
getActivePlugins(CamelJBangMain main, String r
versionCheck(main, version, firstVersion, command);
}
-Optional plugin = getPlugin(command, version, gav,
repos, main.getOut());
-if (plugin.isPresent()) {
-activePlugins.put(command, plugin.get());
+ResolveResult res = resolvePlugin(properties, command,
version, gav, repos, main.getOut());
+if (res.plugin().isPresent()) {
+activePlugins.put(command, res.plugin().get());
+if (res.cacheWritten()) {
+configDirty = true;
+}
} else {
main.getOut().println("camel-jbang-plugin-" + command + "
not found. Exit");
main.quit(1);
}
}
+if (configDirty) {
+savePluginConfig(config);
+}
}
return activePlugins;
}
public static Optional getPlugin(String name, String
defaultVersion, String gav, String repos, Printer printer) {
+return resolvePlugin(null, name, defaultVersion, gav, repos,
printer).plugin();
+}
+
+/**
+ * Resolves a plugin by trying, in order: the cached metadata in the
plugin entry (fast path with no IO beyond
+ * size+mtime checks), the factory finder (embedded plugin on the JVM
classpath), and finally the Maven downloader.
+ * When the downloader runs, the resolved classpath is captured into the
plugin entry's {@code resolved} block so
+ * subsequent invocations take the fast path.
+ */
+private static ResolveResult resolvePlugin(
+JsonObject entry, String name, String defaultVersion, String gav,
String repos, Printer printer) {
+Optional cached = loadFromCache(entry, defaultVersion, gav,
repos);
+if (cached.isPresent()) {
+return new ResolveResult(cached, false);
+}
+
Optional plugin =
FACTORY_FINDER.newInstance("camel-jbang-plugin-" + name, Plugin.class);
-if (plugin.isEmpty()) {
-final MavenGav mavenGav = dependencyAsMavenGav(gav);
-final String group = extractGroup(mavenGav, "org.apache.camel");
-final String depVersion = extractVersion(mavenGav, defaultVersion);
+if (plugin.isPresent()) {
+return new ResolveResult(plugin, false);
+}
+
+final MavenGav mavenGav = dependencyAsMavenGav(gav);
+final String group = extractGroup(mavenGav, "org.apache.camel");
+final String depVersion = extractVersion(mavenGav, defaultVersion);
+
+DownloadResult dr = downloadPlugin(name, defaultVersion, depVersion,
group, repos, printer);
+boolean cacheWritten = false;
+if (dr.plugin().isPresent() && entry != null && dr.classLoader() !=
null && dr.className() != null) {
+cacheWritten = writeCache(entry, defaultVersion, gav, repos,
dr.className(), dr.classLoader(), name, depVersion);
+}
+return new ResolveResult(dr.plugin(), cacheWritten);
+}
+
+private static Optional loadFromCache(JsonObject entry, String
camelVersion, String gav, String repos) {
+if (entry == null) {
+return Optional.empty();
+}
+JsonObject resolved = entry.getMap("resolved");
+if (resolved == null) {
+return Optional.empty();
+}
+if (!sameCamelVersion(asString(resolved.get("camelVersion")),
camelVersion)) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("gav"))),
normalize(gav))) {
+return Optional.empty();
+}
+if (!nullSafeEquals(normalize(asString(resolved.get("repos"))),
normalize(repos))) {
+return Optional.empty();
+}
+String className = asString(resolved.get("className"));
+if (className == null || className.isBlank()) {
+return Optional.empty();
+}
+Object cpObj = resolved.get("classpath");
+if (!(cpObj instanceof Collection)) {
+return Optional.empty();
+}
+Collection classpath = (Collection) cpObj;
+if (classpath.isEmpty()) {
+return Optional.empty();
+}
+
+List urls = new ArrayList<>(classpath.size());
+for (Object o : classpath) {
+if (!(o instanceof Map)) {
+return Optional.empty();
+}
+Map jar = (Map) o;
+Path p = validateFileEntry(jar);
+if (p == null) {
+return Optional.empty();
+}
+
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
github-actions[bot] commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4427738557 :test_tube: **CI tested the following changed modules:** - `docs` - `dsl/camel-jbang/camel-jbang-core` :warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification: - `dsl/camel-jbang/camel-jbang-core`: 1 test(s) disabled on GitHub Actions :bulb: **Manual integration tests recommended:** > You modified `dsl/camel-jbang/camel-jbang-core`. The related integration tests in `dsl/camel-jbang/camel-jbang-it` are excluded from CI. Consider running them manually: > ``` > mvn verify -f dsl/camel-jbang/camel-jbang-it -Djbang-it-test > ``` All tested modules (6 modules) - Camel :: JBang :: Core - Camel :: JBang :: MCP - Camel :: JBang :: Plugin :: Route Parser - Camel :: JBang :: Plugin :: TUI - Camel :: JBang :: Plugin :: Validate - Camel :: Launcher :: Container --- :gear: [View full build and test results](https://github.com/apache/camel/actions/runs/25714518640) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] CAMEL-23335: camel-jbang - Lazy plugin discovery and resolved-classpath cache [camel]
github-actions[bot] commented on PR #23129: URL: https://github.com/apache/camel/pull/23129#issuecomment-4427650096 :star2: Thank you for your contribution to the Apache Camel project! :star2: :robot: CI automation will test this PR automatically. :camel: Apache Camel Committers, please review the following items: * First-time contributors **require MANUAL approval** for the GitHub Actions to run * You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI. * You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR. * Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary. :warning: Be careful when sharing logs. Review their contents before sharing them publicly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
