[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715643#comment-17715643 ] Jing Ge commented on FLINK-6757: NIT: the FLIP URL has been changed: https://cwiki.apache.org/confluence/display/FLINK/FLIP-294%3A+Support+Customized+Job+Meta+Data+Listener > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17709883#comment-17709883 ] Jark Wu commented on FLINK-6757: Hi [~litiliu], [~fangyong] is working on FLIP-294[1], which is designed to support lineage integration. Feel free to join the discussion (not started the discussion on dev ML yet). [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-294%3A+Support+Customized+Job+Status+Listener > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707145#comment-17707145 ] Martijn Visser commented on FLINK-6757: --- [~litiliu] This requires someone creating a FLIP and driving this effort. I don't think there's currently any volunteers who want/have the bandwidth to work on this. > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707087#comment-17707087 ] liting liu commented on FLINK-6757: --- Are we going to do the changes suggested by [~gyfora]? It will help a lot to do lineage integration with other systems (DataHub, Atlas, and so on). > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499958#comment-17499958 ] Ziyoiddin Yusupov commented on FLINK-6757: -- I believe Atlas project is not moving forward. I was checking [OpenLineage |https://github.com/OpenLineage/OpenLineage](open-source) project, similar functionality is being created, and we are looking for changes that was suggested by [~gyfora] > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17474316#comment-17474316 ] HideOnBush commented on FLINK-6757: --- I think this is very meaningful, especially in the construction of real-time data warehouses, through hooks, many manual recording tasks can be solved. and we are already using this function > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401674#comment-17401674 ] Dennis Geisse commented on FLINK-6757: -- Hi, is there any chance this will be pursued further? I see there is some work done by [~gyfora], is this getting merged into Flink? This would be a great improvement to our Data Governance project based on Atlas > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334258#comment-17334258 ] Flink Jira Bot commented on FLINK-6757: --- This issue was marked "stale-assigned" and has not received an update in 7 days. It is now automatically unassigned. If you are still working on it, you can assign it to yourself again. Please also give an update about the status of the work. > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Assignee: Márton Balassi >Priority: Major > Labels: stale-assigned > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325382#comment-17325382 ] Flink Jira Bot commented on FLINK-6757: --- This issue is assigned but has not received an update in 7 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned. > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Till Rohrmann >Assignee: Márton Balassi >Priority: Major > Labels: stale-assigned > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139110#comment-17139110 ] Xiaobin Zheng commented on FLINK-6757: -- Hi [~mbalassi], the integration between Flink and Atlas looks promising. Is there a Jira tracking the executor rework that you mentioned blocking this integration? What are the initial set of entity types that will be extracted from Flink JobListener event beyond Kafka topics? > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: Wish > Components: Connectors / Common >Reporter: Till Rohrmann >Assignee: Márton Balassi >Priority: Major > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-6757) Investigate Apache Atlas integration
[ https://issues.apache.org/jira/browse/FLINK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133130#comment-17133130 ] Márton Balassi commented on FLINK-6757: --- I am happy to take this, in fact we have an implementation and a design doc that we have discussed with the community. This is currently blocked by the Executor rework. https://docs.google.com/document/d/1wSgzPdhcwt-SlNBBqL-Zb7g8fY6bN8JwHEg7GCdsBG8/edit > Investigate Apache Atlas integration > > > Key: FLINK-6757 > URL: https://issues.apache.org/jira/browse/FLINK-6757 > Project: Flink > Issue Type: Wish > Components: Connectors / Common >Reporter: Till Rohrmann >Assignee: Márton Balassi >Priority: Major > > Users asked for an integration of Apache Flink with Apache Atlas. It might be > worthwhile to investigate what is necessary to achieve this task. > References: > http://atlas.incubator.apache.org/StormAtlasHook.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
