On Wed, Aug 16, 2017 at 11:34 AM, Nan Zhu <zhunanmcg...@gmail.com> wrote: > Yes, I know there is such an API, what I don't understand is what I should > pass in the filtering API you mentioned, say we query YARN for every 5 > tickets > > 0: Query and get App A is running > > 4: App A is done > > 5: Query...so what I should fill as filtering parameters at 5 get capture > the changes of App A's state?
You don't query for app state *changes*. You query for the current app state, and compare against what you have, and then you can detect changes that way. The trick is how to filter to get the information you want, so you limit how much data you request from YARN. I'm not aware of any YARN API to query for state changes like that. So even in the individual request case, you'd have to get app A's state, and update the Livy handle if the state has changed from what was previously know. That's most probably why Meisam's PR only filters by app type. If there are further filters than can be applied, then great, but you still need logic in Livy to detect the state changes you want. > If you look at Meisam's PR, they can only filter based on appType > https://github.com/apache/incubator-livy/pull/36/files#diff-a3f879755cfe10a678cc08ddbe60a4d3R75 -- Marcelo
