[ https://issues.apache.org/jira/browse/FLINK-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-2396: ---------------------------------- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Review the datasets of dynamic path and static path in iteration. > ----------------------------------------------------------------- > > Key: FLINK-2396 > URL: https://issues.apache.org/jira/browse/FLINK-2396 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task > Reporter: Chengxiang Li > Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > Currently Flink would cached dataset in static path as it assumes that > dataset stay the same during the iteration, but this assumption does not > always be true. Take sampling for example, the iteration data set is > something like the weight vector of model and there is another training > dataset from which to take a small sample to update the weight vector in each > iteration (e.g. Stochastic Gradient Descent), we expect sampled dataset is > different in each iteration, but Flink would cache the sampled dataset as it > in static path. > We should review how Flink identify dynamic path and static path, and support > add sampled dataset in above example to dynamic path. -- This message was sent by Atlassian Jira (v8.20.1#820001)