davsclaus commented on code in PR #29514:
URL: https://github.com/apache/camel/pull/29514#discussion_r4519137354
##########
dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ChaosExplore.java:
##########
@@ -173,11 +214,134 @@ Ask ask(CamelProcess process) {
ask.showTools = showTools;
ask.showStats = true;
ask.verbose = verbose;
- // the exploration runs with the budget of the user: the AI does not
start another one
- ask.hiddenTools = Set.of("chaos_explore_start");
+ // the exploration runs with the budget of the user: the AI does not
start another one, not even with a camel
+ // command, and the chaos tools are all it needs
+ ask.hiddenTools = Set.of("chaos_explore_start", "cli_exec");
return ask;
}
+ /** What the exploration ran with, for the report. */
+ record Run(String integration, String budget, String focus, String model,
Long seed) {
+ }
+
+ /** The files written: the report (markdown, with the journal as JSON next
to it), and a scenario per failure. */
+ record Saved(Path report, Path journal, List<Path> scenarios) {
+ }
+
+ private static Long seed(JsonObject status) {
+ if (status != null && status.get("chaos") instanceof JsonObject chaos
+ && chaos.get("report") instanceof JsonObject report &&
report.get("seed") instanceof Number n) {
+ return n.longValue();
+ }
+ return null;
+ }
+
+ /**
+ * Saves what the exploration found (CAMEL-26788): the report of the AI
with the journal of the experiments as
+ * markdown in reports/, the journal as JSON next to it, and each
experiment that failed as a scenario, so camel run
+ * --chaos or camel cmd chaos-load replays its verdict.
+ *
+ * @param dir the chaos folder of the project, test/chaos, which camel run
and export leave alone
+ */
+ static Saved save(
+ Path dir, LocalDateTime when, Run run, JsonObject exploration,
String report, List<String> journal)
+ throws IOException {
+ String stamp =
when.format(DateTimeFormatter.ofPattern("yyyyMMdd-HHmmss"));
+ String date = when.format(DateTimeFormatter.ofPattern("yyyy-MM-dd
HH:mm"));
+ Path reports = Files.createDirectories(dir.resolve("reports"));
+ List<Path> scenarios = new ArrayList<>();
+ List<Object> experiments = exploration.getCollection("experiments");
+ if (experiments != null) {
+ for (Object o : experiments) {
+ JsonObject e = (JsonObject) o;
+ if (!"FAIL".equals(e.getString("verdict"))) {
Review Comment:
Fixed in f760bccdaec1 on #29519, the next layer, which carries this code and
merges right after this one; that way this PR's CI, nearly done, does not start
again for a guard. All three casts of the journal in `ChaosExplore` use
`instanceof` there.
_Claude Code on behalf of davsclaus_
##########
dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ChaosExplore.java:
##########
@@ -173,11 +214,134 @@ Ask ask(CamelProcess process) {
ask.showTools = showTools;
ask.showStats = true;
ask.verbose = verbose;
- // the exploration runs with the budget of the user: the AI does not
start another one
- ask.hiddenTools = Set.of("chaos_explore_start");
+ // the exploration runs with the budget of the user: the AI does not
start another one, not even with a camel
+ // command, and the chaos tools are all it needs
+ ask.hiddenTools = Set.of("chaos_explore_start", "cli_exec");
return ask;
}
+ /** What the exploration ran with, for the report. */
+ record Run(String integration, String budget, String focus, String model,
Long seed) {
+ }
+
+ /** The files written: the report (markdown, with the journal as JSON next
to it), and a scenario per failure. */
+ record Saved(Path report, Path journal, List<Path> scenarios) {
+ }
+
+ private static Long seed(JsonObject status) {
+ if (status != null && status.get("chaos") instanceof JsonObject chaos
+ && chaos.get("report") instanceof JsonObject report &&
report.get("seed") instanceof Number n) {
+ return n.longValue();
+ }
+ return null;
+ }
+
+ /**
+ * Saves what the exploration found (CAMEL-26788): the report of the AI
with the journal of the experiments as
+ * markdown in reports/, the journal as JSON next to it, and each
experiment that failed as a scenario, so camel run
+ * --chaos or camel cmd chaos-load replays its verdict.
+ *
+ * @param dir the chaos folder of the project, test/chaos, which camel run
and export leave alone
+ */
+ static Saved save(
+ Path dir, LocalDateTime when, Run run, JsonObject exploration,
String report, List<String> journal)
+ throws IOException {
+ String stamp =
when.format(DateTimeFormatter.ofPattern("yyyyMMdd-HHmmss"));
+ String date = when.format(DateTimeFormatter.ofPattern("yyyy-MM-dd
HH:mm"));
+ Path reports = Files.createDirectories(dir.resolve("reports"));
+ List<Path> scenarios = new ArrayList<>();
+ List<Object> experiments = exploration.getCollection("experiments");
+ if (experiments != null) {
+ for (Object o : experiments) {
+ JsonObject e = (JsonObject) o;
+ if (!"FAIL".equals(e.getString("verdict"))) {
+ continue;
+ }
+ Path scenario = dir.resolve("explore-" + stamp + "-" +
e.getIntegerOrDefault("number", 0) + ".chaos");
+ Files.writeString(scenario, scenario(e, run, date, scenario));
+ scenarios.add(scenario);
+ }
+ }
+
+ StringBuilder md = new StringBuilder();
+ md.append("# Chaos exploration:
").append(run.integration()).append("\n\n");
+ md.append(date).append(" · budget ").append(run.budget());
+ if (run.focus() != null && !run.focus().isBlank()) {
+ md.append(" · focus ").append(run.focus());
+ }
+ if (run.model() != null) {
+ md.append(" · ").append(run.model());
+ }
+ if (run.seed() != null) {
+ md.append(" · seed ").append(run.seed());
+ }
+ md.append("\n\n## Report\n\n").append(report != null &&
!report.isBlank()
+ ? report.strip() : "(the AI wrote no report)").append("\n\n##
Experiments\n\n```text\n");
+ journal.forEach(line -> md.append(line).append('\n'));
+ md.append("```\n");
+ if (!scenarios.isEmpty()) {
+ md.append("\n## Scenarios\n\nEach experiment that failed, to
replay its verdict:\n\n");
+ for (Path s : scenarios) {
+ md.append("- `").append(s).append("`: `camel run *
--chaos=").append(s).append("`\n");
+ }
+ }
+ Path markdown = reports.resolve("explore-" + stamp + ".md");
+ Files.writeString(markdown, md.toString());
+
+ JsonObject json = new JsonObject();
+ json.put("integration", run.integration());
+ json.put("budget", run.budget());
+ if (run.focus() != null) {
+ json.put("focus", run.focus());
+ }
+ if (run.model() != null) {
+ json.put("model", run.model());
+ }
+ if (run.seed() != null) {
+ json.put("seed", run.seed());
+ }
+ json.put("report", report);
+ json.put("exploration", exploration);
+ json.put("scenarios", scenarios.stream().map(Path::toString).toList());
+ Path journalFile = reports.resolve("explore-" + stamp + ".json");
+ Files.writeString(journalFile, Jsoner.prettyPrint(json.toJson()));
+ return new Saved(markdown, journalFile, scenarios);
+ }
+
+ /** A failed experiment as a scenario: what it tested and found as
comments, the seed, and its rules. */
+ static String scenario(JsonObject experiment, Run run, String date, Path
file) {
+ StringBuilder sb = new StringBuilder();
+ sb.append("# From the chaos exploration of
").append(run.integration()).append(" on ").append(date)
+ .append(", experiment
").append(experiment.getIntegerOrDefault("number", 0)).append(": ")
+ .append(experiment.getString("verdict")).append('\n');
+ String hypothesis = experiment.getString("hypothesis");
+ if (hypothesis != null) {
+ sb.append("# hypothesis: ").append(hypothesis).append('\n');
+ }
+ List<Object> expectations = experiment.getCollection("expectations");
+ if (expectations != null) {
+ for (Object o : expectations) {
+ JsonObject x = (JsonObject) o;
+ sb.append("# expect ").append(x.getString("expect")).append(":
").append(x.getString("state"))
Review Comment:
Fixed in f760bccdaec1 on #29519, the next layer, which carries this code and
merges right after this one; that way this PR's CI, nearly done, does not start
again for a guard. All three casts of the journal in `ChaosExplore` use
`instanceof` there.
_Claude Code on behalf of davsclaus_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
