VladaZakharova commented on PR #66342: URL: https://github.com/apache/airflow/pull/66342#issuecomment-4386453986
> I see the value in using Airflow connections for OpenLineage configuration, but I'd suggest expanding the scope beyond just auth tokens for HTTP transport (that are solving a very specific use case). > > Instead of a narrow solution, what if we allow storing the entire OpenLineage config dict (f.e. what can be put in the yaml file, or even just transport config) in an Airflow connection, that we can add to `get_openlineage_config` as yet another config source checked. (It should probably have precedence over yaml file or env vars, but that's something we can decide later) > > This approach would: > > * Work for any transport type, not just HTTP > * Support composite transports (e.g., two HTTP transports with different auth) > * Handle any OL config field, not just auth tokens > * Be more flexible for future use cases > > The current auth-token-only solution would miss users with composite transports or other config needs. Could you expand the scope to cover the full OL config dict? > > cc @mobuchowski hi there! okay, this sounds reasonable, i think it is worth trying should i wait for @mobuchowski response? or i can go with implementation? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
