Hi,
here some considerations being an incremental backup currently implemented:
1. Are the log files flushed before mysqldump is executed?
2. Are the bin log files purged ?
Best Regards
"Amy Bensinger
(Contractor)"
<[EMAIL PROTECTED] To
enfoundation.org> "Developer"
Sent by: <[EMAIL PROTECTED]
mifos-developer-b net>, Biren
[EMAIL PROTECTED] Patnik/Ireland/[EMAIL PROTECTED], "Aliya
ceforge.net Walji"
<[EMAIL PROTECTED]>
cc
02/02/2008 11:44 Van Mittal-Henkle
AM <[EMAIL PROTECTED]>, Mifos Dev
<[EMAIL PROTECTED]>
Subject
Please respond to Re: [Mifos-developer] Database
Developer Growth
<mifos-developer@
lists.sourceforge
.net>
Hi all. corrections below.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Amy
Bensinger (Contractor)
Sent: Friday, February 01, 2008 6:57 PM
To: Biren Patnik; Aliya Walji
Cc: Van Mittal-Henkle; Developer; Mifos Dev
Subject: Re: [Mifos-developer] Database Growth
Hi, Biren. Thanks for that information.
I have been pursing the items further at GK. The database backup
differentials are still 450 larger per day . The actual database size is
48 GB; one month ago it was 43 GB.
GK will continue documenting db size every day.
- Could the problem be in the backup method?
On a different note, in looking at the table list, although the CUSTOMER
table is a large portion of the records, it’s unlikely to grow appreciably
(nor has it since go-live) since GK is not adding new branches and the
number of new members is relatively small (though certainly new loan
accounts are being created).
As to your design questions below, perhaps someone more familiar with the
original design process can comment?
What was the maximum size allowed at design and implementation time for
Mifos DB?
Is 100MB a reasonable growth compared with your initial forecast or is it
too much?
Is it needed to scale the current DB?
Is that the abnormal growth due to application bug?
For you other questions regarding hardware:
1. Hardware where the DB is running: Operating System, CPU, Memory, Disk
Size and their management and free space currently in Grameen Koota.
http://mifos.org/knowledge/deployment-project-pages/grameen-koota/overview#infrastructure-details
2. If other software is running on DB server along with MySQL.
None other than basics like anti-virus.
3. How many concurrent users are currently for Mifos database.
44 branches plus the head office, so likely a maximum of 60 users for now.
This number will increase as soon as branches are added.
4. Volume of critical job activity (max size)
I’m not sure what you mean by this.
Note that we also have some results of server logging (CPU usage, etc that
have been being recorded. Let me know if you would like me to find out if
this data is readily available.
From: Biren Patnik [mailto:[EMAIL PROTECTED]
Sent: Thursday, January 31, 2008 8:59 PM
To: Aliya Walji; Amy Bensinger (Contractor)
Cc: Mifos Dev; Van Mittal-Henkle; Developer; Beth Mazur
Subject: Database Growth
Hi Aliya,
Thanks for your email.
Just a brief summary on the issues related to the database growth problem:
We have been informed in the month of November 2007 from Grameen Koota that
they see an abnormal growth of 450 MB per day
As requested, some data were collected and sent to us few days ago by Aliya
and Amy (thanks to both of them).
A. We analyzed statically these data in the following ways:
We are convinced that only 6 table are contributing at least for 87% of the
total size.
We concentrate our analysis efforts on these tables and we extracted the
data for the week 13-19th Dec 2007,14-20th and 21-27th January 2008
The results are reported in the spreadsheet named comparison.xls
A.1 Findings from comparison.xls
We know that looking only 21 days is not enough, but we got some more
information that are quite different from the starting one.
As you can see in comparison.xls the daily grow seems no more than 50 MB
(quite different from the 450 MB initially communicated to us the week of
November 2007)
50 MB total data length growth per day can be realistic (100MB in total
considering all tables and indexes), but still far from 450 MB per day.
So far, some questions:
What was the maximum size allowed at design and implementation time for
Mifos DB?
Is 100MB a reasonable growth compared with your initial forecast or is it
too much?
Is it needed to scale the current DB?
Is that the abnormal growth due to application bug?
Clearly our analysis was done on data between 13th December 2007 to 28th
January 2008, while the abnormal growth was observered in October 2007:
These are some of the questions which needs to be addressed ..
Is there anything changed between October and December?
Does any system administrator/DBA make any changes to the database during
December?
Has this problem been fixed in the application by anyone during November
,December and January 08
If you still find 450mb database growth per day then what we found out from
the excel sheet is probably not correct?
========================================================================
B. One of our colleagues are defining two different scenarios to stress
MySql Mifos database re-produced on our machine.
The two scenarios are:
bulk entry,
loan account creating
We should be able to run the first stress tests hopefully late next week.
In order to analyze the results we need also to set up correctly our
environment
In order to set up correctly our test environment we need to know:
1. Hardware where the DB is running: Operating System, CPU, Memory, Disk
Size and their management and free space currently in Grameen Koota.
2. If other software is running on DB server along with MySql.
3. How many cuncurrent users are currently for Mifos database.
4. Volume of critical job activity (max size)
Thanks and Kind Regards
Biren Patnaik
Lead Software Developer
IBM Technology Campus
Damestown Industrial Park,Mulhuddart
Dublin 15
Internal Tel: 511211
External Tel : (+353) - 1 - 815-1211
Email: [EMAIL PROTECTED]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
