yrenat opened a new issue, #9120:
URL: https://github.com/apache/texera/issues/9120

   ### What happened?
   
   When a tool call fails, the agent still marks it as `succeeded`.
   
   In `agent-service/src/agent/texera-agent.ts`, `onStepFinish` sets:
   
   ```ts
   isError: !!(tr.output as any)?.error,
   ```
   
   But the tools return errors as a plain string, from `createErrorResult` in 
`tools/tools-utility.ts`:
   
   ```ts
   export function createErrorResult(error: string): string {
     return `[ERROR] ${error}`;
   }
   ```
   
   A string has no `.error` field, so `isError` is always `false`. Then the 
turn history shows every call as `- <tool> (succeeded)`.
   
   Example on `main`: `addOperator` and `modifyOperator` both returned `[ERROR] 
Source operator "op7" not found`. Both were saved with `isError: false`, and 
the next turn showed `- addOperator (succeeded)` and `- modifyOperator 
(succeeded)`. Together with #9115, the agent gets no sign at all that the call 
failed.
   
   Expected: `isError` should be `true` when the tool returns an error, for 
example when the output starts with `[ERROR]`.
   
   **Screenshots**
   
   Step 1, tool call: the result starts with `[ERROR]`.
   
   
![A-step1-toolcall.png](https://raw.githubusercontent.com/yrenat/texera/agent-bug-screenshots/A-step1-toolcall.png)
   
   Step 3, what the agent was given: the same call shows as `- addOperator 
(succeeded)`.
   
   
![A-step3-context.png](https://raw.githubusercontent.com/yrenat/texera/agent-bug-screenshots/A-step3-context.png)
   
   ### How to reproduce?
   
   1. Open an empty workflow, create an agent, and send: `Add a Limit operator 
that keeps 10 rows, and connect its input to the existing operator op7.`
   2. Click "Export ReAct steps as JSON". Step 1's `toolResults[0].output` 
starts with `[ERROR]`, but `isError` is `false`. Step 2's input shows `- 
addOperator (succeeded)`.
   
   ### Version/Branch
   
   1.4.0-incubating-SNAPSHOT (main)
   
   ### Commit Hash (Optional)
   
   57faa6b
   
   ### What browsers are you seeing the problem on?
   
   Chrome
   
   ### Relevant log output
   
   ```shell
   agent-service/src/agent/texera-agent.ts  onStepFinish: isError: !!(tr.output 
as any)?.error
   agent-service/src/agent/tools/tools-utility.ts:30  createErrorResult returns 
a string
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to