ArafatKhan2198 commented on code in PR #10808:
URL: https://github.com/apache/ozone/pull/10808#discussion_r3744314558
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/chatbot/agent/ChatbotAgent.java:
##########
@@ -251,23 +285,31 @@ public String processQuery(String userQuery, String
model, String provider)
/**
* "Step 1" Helper: Talks to the LLM and asks for a JSON object telling us
which API to call.
*/
- private ToolSelection chooseToolsForQuery(String userQuery, String model,
- String provider) throws LLMClient.LLMException,
IOException {
+ private ToolSelection chooseToolsForQuery(String userQuery, String model,
String provider,
+ String historyContext)
+ throws LLMClient.LLMException, IOException {
// --- 1. BUILD THE PROMPT ---
// The system prompt teaches the LLM the Recon API schema and the rules
for picking a tool.
- // The user prompt is just the raw question the user typed.
+ // The user prompt is the current question, optionally preceded by a
fenced,
+ // trimmed conversation-history block (built once by the caller) so the
model
+ // can resolve references.
String systemPrompt = buildToolSelectionPrompt();
- String userPrompt = "User Query: " + userQuery;
+ String userPrompt;
+ if (historyContext.isEmpty()) {
+ userPrompt = "User Query: " + userQuery;
+ } else {
+ userPrompt = historyContext
+ + "\n## CURRENT QUESTION (answer THIS):\n" + userQuery;
+ }
List<ChatMessage> messages = new ArrayList<>();
messages.add(new ChatMessage("system", systemPrompt));
messages.add(new ChatMessage("user", userPrompt));
// --- 2. CONFIGURE GENERATION SETTINGS ---
// Temperature 0.1: very low creativity — we want strict, deterministic
tool selection.
- // max_tokens 8192: allow a large enough reply to fit all tool
descriptions.
- GenParams params = new GenParams(0.1, 8192);
+ GenParams params = new GenParams(0.1, MAX_TOKENS);
Review Comment:
You're right on all counts thanks for catching this.
Yes, the param is applied at model-build time and `maxTokens` is sent
straight to the provider (`applyGenerationParams` → `builder.maxTokens(...)`).
And your Anthropic point is correct: `context-1m-2025-08-07` extends **input**
context, not output, so a value above a model's output cap would be rejected by
the API rather than silently clamped.
Fixed as follows:
- **Configurable now:** replaced the hardcoded constant with
`ozone.recon.chatbot.max.tokens`, read into a `maxTokens` field and used by
both LLM calls.
- **Default lowered to 8192** — provider-safe across OpenAI/Gemini/Anthropic
(within Claude Sonnet's default output cap, so no output-extending beta
required). It can be raised for reasoning models (e.g. gemini-2.5-pro) that
need more room for internal thinking; raising it on Anthropic would require the
appropriate output-extending beta header, which I've noted in the config docs.
Verified end-to-end through the gateway that `claude-sonnet-4-6` and
`claude-opus-4-6` both succeed at the 8192 default (HTTP 200).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
