GitHub user xintongsong added a comment to the discussion: WIP: Replay-based Per Action State Consistency
Adding to the problem section: Model inference / generation is non-deterministic. Repeating model calls multiple times with identical inputs may result in different outputs. This creates inconsistency, especially when we rely on the model outputs to decide what to do next (reasoning, tool calls, etc.). What should we do with the half performed actions when the model makes a different decision after recovery? I think the best way is to never ask the model to make a decision twice. GitHub link: https://github.com/apache/flink-agents/discussions/108#discussioncomment-14082088 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
