> On Jan. 18, 2016, 11:01 a.m., Ajay Yadava wrote: > > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java, > > line 414 > > <https://reviews.apache.org/r/42188/diff/1/?file=1194070#file1194070line414> > > > > Shouldn't this be outside for loop? > > Pallavi Rao wrote: > In this implementation, I'm retrieving instances per state, rather than > retrieve all instances and sort and count in memory. > > Deepak Barr wrote: > states variable is different for different iterations. Should be OK. > > Ajay Yadava wrote: > The ordering can be done in db query, that way all invocations of the > query will result in sorted result and we will not need to make multiple db > calls. That won't work? > > Pallavi Rao wrote: > Yeah.. it will sort, but, I'll still have to iterate over the entire > result set or issue a group command. Also, the result set can be large. If it > is limited to a single state, it gets returned faster. It requires additional > changes to the StateStore layer. Hence, the simplistic approach.
So in this approach we will fire approximately 8 queries(1 corresponding to each state), we will fetch all the columns, convert them to POJOs whereas we need just count. What I am suggesting is just fire 1 query which will group by state and return the count corresponding to each state and this will be several times faster and simpler. Another difference is that for a large range current approach may cause memory issues(and severe performance degradation) as it fetches all columns and all rows whereas the suggested approach is immune to such cases. - Ajay ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42188/#review114982 ----------------------------------------------------------- On Jan. 12, 2016, 10:14 a.m., Pallavi Rao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42188/ > ----------------------------------------------------------- > > (Updated Jan. 12, 2016, 10:14 a.m.) > > > Review request for Falcon. > > > Bugs: FALCON-1742 > https://issues.apache.org/jira/browse/FALCON-1742 > > > Repository: falcon-git > > > Description > ------- > > This was not implemented. > > > Diffs > ----- > > client/src/main/java/org/apache/falcon/resource/EntitySummaryResult.java > 4a885ec > > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java > bffdb0b > unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java a82cf03 > > webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java > b1c8ce0 > > Diff: https://reviews.apache.org/r/42188/diff/ > > > Testing > ------- > > Added IT and did manual testing. > > > Thanks, > > Pallavi Rao > >
