[
http://mifosforge.jira.com/browse/MIFOS-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=58027#action_58027
]
johnwoodlock commented on MIFOS-2986:
-------------------------------------
Have had a chat with Vinod so he already knows the generalities. Vinod's aware
that we can work on this together. Also, I can go through any of the stuff
below in more detail (I realise it might be hard to figure out at times).
Even with a reasonable amount of performance related work, this batch job was
quite a risk before the most recent changes were applied to cater for
branch/office level holidays. It's way slower now (10 times possibly) and we
need to get it back down to previous performance. Fully understand that a good
job was done on the recent changes... this is just a scalability issue.
Suggestions
===========
1. Process Holiday by Holiday (earliest holiday first) rather than process all
unapplied holidays for each account.
i.e. get the earliest unapplied holiday, process it, mark it applied, commit...
loop until all done.
Improves restartability and makes it easier to optimise the processing.
2. Method getAccountIdsWithDatesIn: the associated queries should only return
accounts that will be affected by the holiday.
Currently, it will bring back all the accounts between a start date and end
date (which will be fine when processing holidays singly) but we need to
included the affected offices in the query (they are not at the moment) e.g. if
you do a holiday for one branch for next week, it will pick up accounts not in
the branch as well.
Also this brings up the question of whether, for holidays that affect all mfi
offices, it is worth maintaining the link to each office instead of just
linking it to none.
3. Only get the holiday/office holiday information once from the database.
Its only a few records (especially if links are made to generic holidays to all
offices) and can be put in a list/set/array whatever.
4. for each account that is affected, only get the schedules that will be
impacted back. (this has quite a big impact)
typically, only a few schedules will need updating. However, there are for
example an average of 110 customer_schedule's for every customer account at GK.
You don't want to bring all of these back (and come back they do).
I'm attaching the relevant code files for the previous version (I couldn't
easily get it back on git cos I'm not good enough at git yet).
At the end of any tweaking we can give it another once over.
> ApplyHolidayChangesTask batch task needs tweaking now that it handles branch
> level holiday (performance/scalability)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: MIFOS-2986
> URL: http://mifosforge.jira.com/browse/MIFOS-2986
> Project: mifos
> Issue Type: Bug
> Components: Batch jobs
> Affects Versions: Shamim D
> Reporter: johnwoodlock
> Assignee: Vinod John
> Priority: Major
> Fix For: Shamim D
>
>
> Changes to the code to allow the batch job to handle branch level holidays
> have slowed it down a lot.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://mifosforge.jira.com/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
_______________________________________________
Mifos-issues mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mifos-issues
