> On Aug. 29, 2013, 7:59 p.m., Alejandro Abdelnur wrote: > > I see several update queries with a 'where a.id = :id' clause. > > > > We could get rid of all those if we use properly JPA. This is something > > I've mentioned before, we are getting a bean from JPA, we copy it to a > > another instance of the same bean we operate, make changes and we then do > > an update. I think we can get rid of a lot of code and get a perf boost and > > let OpenJPA do the magic of updating the right columns (modified properties > > in the bean) by its own. it would be something like: > > > > -JPA executor starts > > WorkflowBean wf = em.find(WorflowBean.class, jobId); > > em.detach(wf); > > -JPA executor ends > > *** use wf as a regular bean modifying its properties > > -JPA executor starts > > em.merge(wf); //it should update only changed properties > > -JPA executor ends > > > > > > > > > > Ryota Egashira wrote: > Thanks for feedback. couple of questions. > - this dynamic-update option is supported in openJPA? (surely supported > in hibernate) if so, is there any config knob (or annotation) to turn it on? > - does em.merge run extra select query to check which column changed > before update? (once detached, I believe em lost track of originally stored > value..)
we are getting a bean from JPA, we copy it to a another instance of the same bean we operate, make changes and we then do an update. - Currently, I see the copy only done in case from Object[] to bean when we do selective column projections. But wherever we use select OBJECT(a), we use the same bean returned and do not create a copy. for eg: WorkflowJobGetJPAExecutor. no em.detach is being done at the moment as em.close() is done at the end of each query and we don't maintain em across queries right now. Virag found that em.merge does a select query every time, compares to see what is the change before updating the object which adds to the latency. Though it is inefficient, our biggest problem was with BLOB columns and em.merge. We had to write a openjpa ValueHandler for blob columns as a byte[] array column cannot be used in a projection in JPQL query (http://stackoverflow.com/questions/17956158/error-query-projections-cannot-include-array-collection-or-map-fields, http://openjpa.208410.n2.nabble.com/Error-Query-projections-cannot-include-array-collection-or-map-fields-td7584606.html). em.merge does not honor the ValueHandler. - Rohini ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13612/#review25757 ----------------------------------------------------------- On Aug. 26, 2013, 11:10 p.m., Ryota Egashira wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/13612/ > ----------------------------------------------------------- > > (Updated Aug. 26, 2013, 11:10 p.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1499 > https://issues.apache.org/jira/browse/OOZIE-1499 > > > Repository: oozie > > > Description > ------- > > https://issues.apache.org/jira/browse/OOZIE-1499 > > this is *WIP*, still rough, just uploading for early review on design. > > > too many file changes in this patch, to help review, describing a bit on > design. > main part is query executor. > > QueryExecutor> > -provides basic API to run query (update query-->executeUpdate(), select > query --> get() or getList()) > -each table (wrk job/action, coord job/action, bundle job/action, sla...) > has it's own implementation of query executor. also there is > BatchQueryExecutor which allow mutiple queries to run in one transaction. > -construct query based on namedQuery name (defined in each Bean), and does > cache for performance enhancement > > How to run query> > currently we instantiate corresponding JPA executor every time, but instead > it's changed to access QueryExecutor and pass namedQuery as argument . > for example, > - jpaService.execute(new > CoordActionUpdateForInputCheckJPAExecutor(action)); > + > CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQuery.UPDATE_COORD_ACTION_FOR_INPUTCHECK, > action); > > > Diffs > ----- > > trunk/core/src/main/java/org/apache/oozie/BundleActionBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/BundleJobBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStatusUpdateXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionMaterializeCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitXCommand.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/command/wf/SuspendXCommand.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BatchQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BulkUpdateDeleteJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BulkUpdateInsertForCoordActionStartJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BulkUpdateInsertForCoordActionStatusJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BulkUpdateInsertJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BundleActionQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BundleActionUpdateJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BundleJobQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/BundleJobUpdateJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionUpdateForInputCheckJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionUpdateForModifiedTimeJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionUpdatePushInputCheckJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionUpdateStatusJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java > PRE-CREATION > trunk/core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/SLAEventQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/SLASummaryQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobQueryExecutor.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java > 1513182 > > trunk/core/src/main/java/org/apache/oozie/service/CoordMaterializeTriggerService.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/service/JPAService.java 1513182 > trunk/core/src/main/java/org/apache/oozie/service/StatusTransitService.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1513182 > trunk/core/src/main/java/org/apache/oozie/store/CoordinatorStore.java > 1513182 > trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1513182 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java > 1513182 > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionErrors.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBatchQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateDeleteJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertForCoordActionStartJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertForCoordActionStatusJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionUpdateForInputCheckJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionUpdateForModifiedTimeJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionUpdateStatusJPAExecutor.java > 1513182 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestSLARegistrationQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestSLASummaryQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowActionQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowJobQueryExecutor.java > PRE-CREATION > > trunk/core/src/test/java/org/apache/oozie/service/TestPauseTransitService.java > 1513182 > trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java > 1513182 > trunk/core/src/test/java/org/apache/oozie/test/XDataTestCase.java 1513182 > > Diff: https://reviews.apache.org/r/13612/diff/ > > > Testing > ------- > > > Thanks, > > Ryota Egashira > >
