zozo123 opened a new issue, #74194: URL: https://github.com/apache/airflow/issues/74194
### Description Track a Databricks Genie integration delivered in two bounded PRs: a hook plus managed-agent toolset adapter, then a standalone operator with deferrable waiting where supported. ### Use case/motivation A worker-owned agent may need a bounded consultation with a configured Genie space. A standalone DAG task may instead wait for a longer conversation/message operation. These have different resource and recovery requirements but should share authentication and result handling. ### Delivery and acceptance 1. Hook + adapter: use Databricks connections and the actual common.ai BaseManagedAgentToolset extension contract. Expose space/conversation/message identity and structured results/errors; make common.ai optional. Document when generic Databricks MCP is sufficient and what the native API adds. 2. Standalone operator: persist remote conversation/message IDs before downstream publication; reconnect to known operations after retry. Use asynchronous trigger polling and fetch the result on worker resume. 3. Before promising safe submission retry, verify the remote API's idempotency/lookup guarantees. A worker can fail after server acceptance but before saving the ID. If recovery cannot disambiguate this window, report the limitation and require explicit recovery rather than silently creating another conversation. 4. Cover timeout, access denial, remote failure, result pagination/size limits and trigger restart. Keep credentials and large result sets out of serialized triggers/XCom; return bounded metadata or artifact references. 5. Preserve explicit user-supplied conversation ownership; cancellation must not interrupt unrelated work. ### Related issues This is a provider-local proposal, independent of #74191. Reuse existing resumable-job facilities only for Airflow versions that support them; document the supported matrix. Initial deliverable does not include Genie administration APIs or an Airflow HITL bridge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
