GitHub user umitbora55 created a discussion: Extend apache-airflow-providers-informatica with PowerCenter workflow orchestration
I work with Oracle/Informatica/Airflow in production (banking data engineering) and would like to propose extending this provider. **CURRENT STATE** The provider currently covers Informatica EDC (Enterprise Data Catalog) — lineage and catalog metadata only. There's no way to actually orchestrate PowerCenter workflow execution from Airflow. **PROBLEM** Teams running PowerCenter batch pipelines alongside Airflow today have to shell out to pmcmd via BashOperator/SSHOperator: no native hook, no typed status handling, no Airflow-native recovery integration, no XCom-friendly stats surfacing. Every team re-implements this wrapper themselves. **PROPOSAL** Add a PowerCenterHook (wrapping pmcmd, or Informatica's REST API where available) plus operators/sensors covering: 1. Trigger — startworkflow, generating a paramfile from Airflow's own context (execution date, DAG run conf) so PowerCenter workflows can receive Airflow-native runtime parameters. 2. Async status monitoring — a deferrable sensor using getworkflowdetails/gettaskdetails, so long-running ETL workflows don't block a worker slot while waiting. 3. Recovery — recoverworkflow support exposed as a resume-from-failure mode, rather than forcing a full restart of the whole workflow. 4. Observability — getsessionstatistics (row counts, success/failure per session) surfaced via XCom/task logs, so failures are diagnosable without a separate login to Informatica. **FIT** This complements the existing EDC hook rather than overlapping with it — EDC covers metadata/catalog, this covers execution/orchestration. Together they'd give this provider a complete Informatica story. **QUESTIONS** - Is this a direction the maintainers would welcome? - Any existing WIP or design constraints I should know about before starting implementation? - Preferred module/hook naming convention to follow (e.g. relative to how the EDC hook is structured)? Happy to start on an implementation once there's a green light on direction. GitHub link: https://github.com/apache/airflow/discussions/69680 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
