Re: [Dhis2-devs] DHIS2 Dashboard Apps
Thanks a lot John for those clarification/ideas - agree with most of your points. We would be glad to be part of the process too. Martin On Sat, Apr 28, 2018 at 6:23 PM, John Mukulu wrote: > These are very interesting topics, > > Just weigh in on few of the issues raised. > > 1. Way of widgets identifying themselves, this is actually supposed to be > for all apps, to sort of have unique-app-id that'll unique like android > apps, so names and version can change but it'll stay the same, this is on > jira but not implemented yet, would help tracking and triggering of widgets > and their counterparts apps > > 2. Explore and going full screen limitations, at the moment, there's no > such features, explore simply opens the same widget iframe in separate > page, with scorecard we had to write widget that's actually a fully fledged > widget, so on explore we'd check if it's separate page to give you > impression of full app and impression of just a widget on small page, but > above features would definitely solve this > > 3. Access to global selections, at the moment iframe loaded apps have not > been provided with means of accessing anything outside, because they're > essentially brand new page running as subpage, standards could be put such > that global filters could pass selections to the iframe url of the app and > trigger reload(or abandon iframe idea all together and persue much better > one! Think web components and custom elements) > > 4. Rendering on native mobile apps(android & iOS), because widget are > stand-alone pages but still using utilizing same page sessions cookies and > page header contents, they are presented as a fake new page trying to > impersonate other page, this pause a security dilemma and often forces > nginx to allow such except for browser, but never torelated on native > mobile and hence never working there. > > Most of these concepts are already in past jira issues awaiting priority > to be picked up, but should you have interesting in innovating there, three > major things to solve > > 1. Abandoning iframe in favor of web components (but keeping airframe > simplicity of integration) - solve global filters accesibility and other in > memory info, solve security for android, allow distinction of states(full > or widget) > > 2. Add app property in manifest for appid and make app urls use app ids > with support for accepting more parameters > > 3. Add property in widget for corresponding appids for widgets to allow > explore by reference to apps or prompting install when absent(app id would > guarantee this) > > HISP Tanzania can help further brainstorming and potential solutions, let > me know if you're embarking on this. > > And Sorry for long paragraphs! > > John Francis Mukulu > Software Architect, HISPTZ > http://hisptanzania.org/ > > > On Fri, Apr 27, 2018, 15:53 Martin Van Aken > wrote: > >> Thanks for your answer - any way to specify where the "explore" link >> should lead: >> >> >> this would allow my app to show a small part as an iFrame, then be called >> in full screen on click (like the Pivot tables & chart works in the >> Dashboard right now) >> >> Martin >> >> On Fri, Apr 27, 2018 at 10:47 AM, Edoardo Sabadelli >> wrote: >> >>> Not exactly, the content in the iframe is usable from within the box >>> in the dashboard. >>> There isn't a way of going full screen at the moment, but it can be >>> easily added. >>> >>> On Thu, Apr 26, 2018 at 5:21 PM, Martin Van Aken >>> wrote: >>> > OK, so it will render the "main" page (as refered in the manifest) as >>> an >>> > iframe there, with clicking on it leading to the "full page" app, >>> correct ? >>> > This being said, looks like something I could test easily. >>> > >>> > Martin >>> > >>> > On Thu, Apr 26, 2018 at 11:28 AM, Edoardo Sabadelli >> > >>> > wrote: >>> >> >>> >> Hi Martin, >>> >> >>> >> the dashboard widgets/apps, listed under the Apps section in the >>> >> dashboard item selector, are rendered in the same way as before. >>> >> They are loaded in an iframe in a box added to the dashboard grid. >>> >> This is also to ensure existing apps can still work in the new >>> Dashboards >>> >> app. >>> >> >>> >> As for distinguishing between a full screen and a "widget" app, there >>> >> isn't anything in place as far as I know. >>> >> >>> >> One way is to use a responsive layout in your app, to ensure the >>> >> content fits and is usable in both full screen and the small widget >>> >> box. >>> >> >>> >> I didn't work with widget apps, so hopefully someone in the community >>> >> who has done that can help. >>> >> >>> >> Cheers, >>> >> >>> >> On Thu, Apr 26, 2018 at 10:35 AM, Martin Van Aken >>> >> wrote: >>> >> > Hi! >>> >> > We've been working with DHIS2 Apps for a while now, and found the >>> >> > general >>> >> > experience (as developers) pretty good with the d2 + React combo. >>> >> > Something >>> >> > I could not find info about is how the different kind of apps >>> change, >>> >> > especially the DASHBOARD_WI
[Dhis2-devs] Analytics taking too much time for tracker
Dear Experts I am drawing your attention to one problem and need your kind attention. In one of our DHIS2 instances, 'Central Server' analytics generation taking too much time 12 h, 22 m, 34 s. DB size is 127 GB and have aggregated and trackers as. During analytics run-time the processor utilization and RAM utilization is normal. But from the log it is clear that the trackers are taking too much time though they are not having much data. "EPI Cold Chain" tracker taking near about 9 hour, "Cause of death (MCCOD)"tracker taking 3 hour and "Cervical and Breast Cancer Screening Program" tracker taking 7 hour. What seems to be the reason? Is something wrong with the tracker system? We are using DHIS2 version 2.28 release 533e983. Need immediate help. Regards Muhammad Abdul Hannan Khan Team Leader Support to the National HMIS MIS, Directorate General of Health Services Ministry of Health and Family Welfare T +880-2- 58816459, 58816412 ext 118 F +88 02 58813 875 M+88 01819 239 241 M+88 01534 312 066 E hann...@gmail.com S hannan.khan.dhaka B hannan-tech.blogspot.com L https://bd.linkedin.com/in/hannankhan ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] How do I filter for authorities
Hello! I'm attempting to filter for roles with a specific authority and I'm getting a strange response: Specifically: Collection size must be integer `F_PROGRAM_INDICATOR_PUBLIC_ADD` The URL I'm using is: https://play.dhis2.org/2.29/api/userRoles?fields=name,id,authorities&filter=authorities:eq:F_PROGRAM_INDICATOR_PUBLIC_ADD I haven't tried this on 2.28 or 2.27, currently testing out some of the role changes for 2.29. Does anyone know what might be wrong with my syntax? *Timothy Harding* Sr. Systems Analyst, BAO Systems +1 202-536-1541 | thard...@baosystems.com | http://www.baosystems.com | Skype: hardi...@gmail.com | 2900 K Street, Suite 507, Washington D.C. 20007 ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Analytics taking too much time for tracker
Hi Hannan I recall you had a similar request which I responded to back in Jan 11. Maybe worth re-reading that thread as some of it seems still to be relevant. In particular: 1. The first question, as always, is has this suddenly happened or has something recently changed? Back then there had been some new datasets created by UNICEF which you were suspecting. I recall at the time you said that analytics then was taking 13 hours. So 12 hours is an improvement? Is this a sudden problem, like from a recent system upgrade or some changes to datasets, or just the way it has been for some time .. unacceptably slow. 2. A lot of analytics generation time is taken up on CREATE INDEX. You might try to increase maintenance_work_mem. I think I had suggested back then pushing as far as 4G on your system. 3. You still have some automated creature (username elmisinterop) logging into your system at a crazy rate per second after 1am. As I recall this user is trying to download all the data on your system which is probably not a good thing to do during analytics generation. Whatever it is, you need to disable that. Try 2 and 3 and see where that gets you. Any chance you can also try to get access to some faster disk resource? Hope this helps. Bob On 30 April 2018 at 12:36, Hannan Khan wrote: > Dear Experts > > I am drawing your attention to one problem and need your kind attention. > In one of our DHIS2 instances, 'Central Server' analytics generation taking > too much time 12 h, 22 m, 34 s. DB size is 127 GB and have aggregated and > trackers as. During analytics run-time the processor utilization and RAM > utilization is normal. But from the log it is clear that the trackers are > taking too much time though they are not having much data. "EPI Cold Chain" > tracker taking near about 9 hour, "Cause of death (MCCOD)"tracker taking 3 > hour and "Cervical and Breast Cancer Screening Program" tracker taking 7 > hour. What seems to be the reason? Is something wrong with the tracker > system? > > We are using DHIS2 version 2.28 release 533e983. > > Need immediate help. > > Regards > > Muhammad Abdul Hannan Khan > Team Leader > Support to the National HMIS > MIS, Directorate General of Health Services > Ministry of Health and Family Welfare > > T +880-2- 58816459, 58816412 ext 118 > F +88 02 58813 875 > M+88 01819 239 241 > M+88 01534 312 066 > E hann...@gmail.com > S hannan.khan.dhaka > B hannan-tech.blogspot.com > L https://bd.linkedin.com/in/hannankhan > > > > ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] Data Value Submission Date
Hello Devs, I have looked at the "datavalue" table and discovered there are two columns (created and lastupdated). This I assume are supposed to be be the date (time stamp) of submission and update of the data values. In trying to query the API for the data values, I however have issues with fetching either the created or lastupdated time stamps. Is there any way to fetch these two fields (time stamps) along side the data values? Your prompt response'll be highly appreciated. Regards -- Barnabas AKUMBA *Mobile:* +2348036195778 *Skype:* barnabas.akumba ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] Dimension restrictions for data analytics
Dear All, Has anyone met this scenerio? I noticed some accounts in my instance has some dimensions restricted for data analysis.Whenever i move the dimensions to the Available dimension restrictions for data analytics,when i edit that account again i find the dimensions have gone back to Selected dimension restrictions for data analytics. How can i have the restricted dimensions stay intact to the available dimension side? Any ideas? Stanley ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Data Value Submission Date
Thanks Knut. Regards On Mon, Apr 30, 2018 at 3:20 PM, Knut Staring wrote: > I suppose you mean through the Analytics API, since you can do something > like this: > > https://play.dhis2.org/2.29/api/26/dataValueSets?dataSet= > pBOMPrpg1QX&period=201701&orgUnit=DiszpKrYNg8 > > On Mon, Apr 30, 2018 at 4:08 PM, Barnabas Akumba > wrote: > >> Hello Devs, >> >> I have looked at the "datavalue" table and discovered there are two >> columns (created and lastupdated). This I assume are supposed to be be the >> date (time stamp) of submission and update of the data values. >> >> In trying to query the API for the data values, I however have issues >> with fetching either the created or lastupdated time stamps. Is there any >> way to fetch these two fields (time stamps) along side the data values? >> >> Your prompt response'll be highly appreciated. >> >> Regards >> >> >> -- >> >> Barnabas AKUMBA >> >> *Mobile:* +2348036195778 >> *Skype:* barnabas.akumba >> >> ___ >> Mailing list: https://launchpad.net/~dhis2-devs >> Post to : dhis2-devs@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~dhis2-devs >> More help : https://help.launchpad.net/ListHelp >> >> > > > -- > Knut Staring > > Department of Information, Evidence and Research > World Health Organization, Geneva, Switzerland > Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522 > Skype: knutstar > -- Barnabas AKUMBA *Mobile:* +2348036195778 *Skype:* barnabas.akumba ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Data Value Submission Date
I suppose you mean through the Analytics API, since you can do something like this: https://play.dhis2.org/2.29/api/26/dataValueSets?dataSet=pBOMPrpg1QX&period=201701&orgUnit=DiszpKrYNg8 On Mon, Apr 30, 2018 at 4:08 PM, Barnabas Akumba wrote: > Hello Devs, > > I have looked at the "datavalue" table and discovered there are two > columns (created and lastupdated). This I assume are supposed to be be the > date (time stamp) of submission and update of the data values. > > In trying to query the API for the data values, I however have issues with > fetching either the created or lastupdated time stamps. Is there any way to > fetch these two fields (time stamps) along side the data values? > > Your prompt response'll be highly appreciated. > > Regards > > > -- > > Barnabas AKUMBA > > *Mobile:* +2348036195778 > *Skype:* barnabas.akumba > > ___ > Mailing list: https://launchpad.net/~dhis2-devs > Post to : dhis2-devs@lists.launchpad.net > Unsubscribe : https://launchpad.net/~dhis2-devs > More help : https://help.launchpad.net/ListHelp > > -- Knut Staring Department of Information, Evidence and Research World Health Organization, Geneva, Switzerland Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522 Skype: knutstar ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Analytics taking too much time for tracker
Thanks Bob. I remembered that. 1st step I takes was upgrade postgres to 9.6 and tuning parameters which reduced time from 58 to 13 hour. And after that I removed/update the data types of the aggregated data set what was changed/amended that reduce the time to 8 hour. 1. this time it is gradually increased from 8 hour to 2/13 hour now. This time it seems that tracker contribute to the delays. * INFO 2018-04-30 03:24:56,323 Populated table in 8717.87 seconds: analytics_enrollment_temp_xggeyhhpmkp (AbstractJdbcTableManager.java [taskScheduler-25]) * INFO 2018-04-30 04:01:53,261 Populated table in 10919.99 seconds: analytics_enrollment_temp_knjmf7ttlad (AbstractJdbcTableManager.java [taskScheduler-9]) * INFO 2018-04-30 07:03:46,687 Populated table in 10913.42 seconds: analytics_enrollment_temp_fokn0tj7lmi (AbstractJdbcTableManager.java [taskScheduler-9]) * INFO 2018-04-30 08:18:49,534 Populated table in 26237.16 seconds: analytics_enrollment_temp_oo3ormkzucj (AbstractJdbcTableManager.java [taskScheduler-22]) * INFO 2018-04-30 09:02:22,728 Populated table in 7116.00 seconds: analytics_enrollment_temp_vu0q0uahxxx (AbstractJdbcTableManager.java [taskScheduler-9]) * INFO 2018-04-30 12:22:26,364 Populated table in 32250.03 seconds: analytics_enrollment_temp_swqlgvqvifa (AbstractJdbcTableManager.java [taskScheduler-25]) 2. Yes! Good suggestion. This maintenance work_mem is low compared to these trackers. I am changing now. automated procedures are not causing so much problem. Lets see the progress tomorrow. I will get back to you all. Regards Hannan On Mon, Apr 30, 2018 at 7:48 PM, Bob Jolliffe wrote: > Hi Hannan > > I recall you had a similar request which I responded to back in Jan 11. > Maybe worth re-reading that thread as some of it seems still to be > relevant. In particular: > > 1. The first question, as always, is has this suddenly happened or has > something recently changed? Back then there had been some new datasets > created by UNICEF which you were suspecting. I recall at the time you said > that analytics then was taking 13 hours. So 12 hours is an improvement? > Is this a sudden problem, like from a recent system upgrade or some changes > to datasets, or just the way it has been for some time .. unacceptably > slow. > > 2. A lot of analytics generation time is taken up on CREATE INDEX. You > might try to increase maintenance_work_mem. I think I had suggested back > then pushing as far as 4G on your system. > > 3. You still have some automated creature (username elmisinterop) logging > into your system at a crazy rate per second after 1am. As I recall this > user is trying to download all the data on your system which is probably > not a good thing to do during analytics generation. Whatever it is, you > need to disable that. > > Try 2 and 3 and see where that gets you. > > Any chance you can also try to get access to some faster disk resource? > > Hope this helps. > > Bob > > On 30 April 2018 at 12:36, Hannan Khan wrote: > >> Dear Experts >> >> I am drawing your attention to one problem and need your kind attention. >> In one of our DHIS2 instances, 'Central Server' analytics generation taking >> too much time 12 h, 22 m, 34 s. DB size is 127 GB and have aggregated and >> trackers as. During analytics run-time the processor utilization and RAM >> utilization is normal. But from the log it is clear that the trackers are >> taking too much time though they are not having much data. "EPI Cold Chain" >> tracker taking near about 9 hour, "Cause of death (MCCOD)"tracker taking 3 >> hour and "Cervical and Breast Cancer Screening Program" tracker taking 7 >> hour. What seems to be the reason? Is something wrong with the tracker >> system? >> >> We are using DHIS2 version 2.28 release 533e983. >> >> Need immediate help. >> >> Regards >> >> Muhammad Abdul Hannan Khan >> Team Leader >> Support to the National HMIS >> MIS, Directorate General of Health Services >> Ministry of Health and Family Welfare >> >> T +880-2- 58816459, 58816412 ext 118 >> F +88 02 58813 875 >> M+88 01819 239 241 >> M+88 01534 312 066 >> E hann...@gmail.com >> S hannan.khan.dhaka >> B hannan-tech.blogspot.com >> L https://bd.linkedin.com/in/hannankhan >> >> >> >> > -- Muhammad Abdul Hannan Khan Team Leader Support to the National HMIS MIS, Directorate General of Health Services Ministry of Health and Family Welfare T +880-2- 58816459, 58816412 ext 118 F +88 02 58813 875 M+88 01819 239 241 M+88 01534 312 066 E hann...@gmail.com S hannan.khan.dhaka B hannan-tech.blogspot.com L https://bd.linkedin.com/in/hannankhan ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Analytics taking too much time for tracker
Hannan, Out of interest: how do you measure your DB size? "127GB" does not really say much - is it with or without analytics tables etc. Are you just using pg_database_size? I just ran that on my local instance of the South African "national" HMIS instance - it is 202 GB (NOTE: that includes analytics), and running analytics takes around 1-1.5 hours. So 12-13 hours on your side seems excessive (I presume you are using either high-speed RAID rust platters or else SSD). Regards calle On 30 April 2018 at 19:24, Hannan Khan wrote: > Thanks Bob. > > I remembered that. 1st step I takes was upgrade postgres to 9.6 and tuning > parameters which reduced time from 58 to 13 hour. And after that I > removed/update the data types of the aggregated data set what was > changed/amended that reduce the time to 8 hour. > > 1. this time it is gradually increased from 8 hour to 2/13 hour now. This > time it seems that tracker contribute to the delays. > > * INFO 2018-04-30 03:24:56,323 Populated table in 8717.87 seconds: > analytics_enrollment_temp_xggeyhhpmkp (AbstractJdbcTableManager.java > [taskScheduler-25]) > * INFO 2018-04-30 04:01:53,261 Populated table in 10919.99 seconds: > analytics_enrollment_temp_knjmf7ttlad (AbstractJdbcTableManager.java > [taskScheduler-9]) > * INFO 2018-04-30 07:03:46,687 Populated table in 10913.42 seconds: > analytics_enrollment_temp_fokn0tj7lmi (AbstractJdbcTableManager.java > [taskScheduler-9]) > * INFO 2018-04-30 08:18:49,534 Populated table in 26237.16 seconds: > analytics_enrollment_temp_oo3ormkzucj (AbstractJdbcTableManager.java > [taskScheduler-22]) > * INFO 2018-04-30 09:02:22,728 Populated table in 7116.00 seconds: > analytics_enrollment_temp_vu0q0uahxxx (AbstractJdbcTableManager.java > [taskScheduler-9]) > * INFO 2018-04-30 12:22:26,364 Populated table in 32250.03 seconds: > analytics_enrollment_temp_swqlgvqvifa (AbstractJdbcTableManager.java > [taskScheduler-25]) > > 2. Yes! Good suggestion. This maintenance work_mem is low compared to > these trackers. I am changing now. > > automated procedures are not causing so much problem. > > Lets see the progress tomorrow. I will get back to you all. > > Regards > > Hannan > > > > On Mon, Apr 30, 2018 at 7:48 PM, Bob Jolliffe > wrote: > >> Hi Hannan >> >> I recall you had a similar request which I responded to back in Jan 11. >> Maybe worth re-reading that thread as some of it seems still to be >> relevant. In particular: >> >> 1. The first question, as always, is has this suddenly happened or has >> something recently changed? Back then there had been some new datasets >> created by UNICEF which you were suspecting. I recall at the time you said >> that analytics then was taking 13 hours. So 12 hours is an improvement? >> Is this a sudden problem, like from a recent system upgrade or some changes >> to datasets, or just the way it has been for some time .. unacceptably >> slow. >> >> 2. A lot of analytics generation time is taken up on CREATE INDEX. You >> might try to increase maintenance_work_mem. I think I had suggested back >> then pushing as far as 4G on your system. >> >> 3. You still have some automated creature (username elmisinterop) >> logging into your system at a crazy rate per second after 1am. As I recall >> this user is trying to download all the data on your system which is >> probably not a good thing to do during analytics generation. Whatever it >> is, you need to disable that. >> >> Try 2 and 3 and see where that gets you. >> >> Any chance you can also try to get access to some faster disk resource? >> >> Hope this helps. >> >> Bob >> >> On 30 April 2018 at 12:36, Hannan Khan wrote: >> >>> Dear Experts >>> >>> I am drawing your attention to one problem and need your kind attention. >>> In one of our DHIS2 instances, 'Central Server' analytics generation taking >>> too much time 12 h, 22 m, 34 s. DB size is 127 GB and have aggregated and >>> trackers as. During analytics run-time the processor utilization and RAM >>> utilization is normal. But from the log it is clear that the trackers are >>> taking too much time though they are not having much data. "EPI Cold Chain" >>> tracker taking near about 9 hour, "Cause of death (MCCOD)"tracker taking 3 >>> hour and "Cervical and Breast Cancer Screening Program" tracker taking 7 >>> hour. What seems to be the reason? Is something wrong with the tracker >>> system? >>> >>> We are using DHIS2 version 2.28 release 533e983. >>> >>> Need immediate help. >>> >>> Regards >>> >>> Muhammad Abdul Hannan Khan >>> Team Leader >>> Support to the National HMIS >>> MIS, Directorate General of Health Services >>> Ministry of Health and Family Welfare >>> >>> T +880-2- 58816459, 58816412 ext 118 >>> F +88 02 58813 875 >>> M+88 01819 239 241 >>> M+88 01534 312 066 >>> E hann...@gmail.com >>> S hannan.khan.dhaka >>> B hannan-tech.blogspot.com >>> L https://bd.linkedin.com/in/hannankhan >>> >>> >>> >>> >> > > > -- > Muhammad Abdul Hannan Khan > Team Leader > Support to
Re: [Dhis2-devs] Analytics taking too much time for tracker
Dear Calle Thank you for your information. 127 GB Databases size is including analytics. Yes disks are in RAID 5. Do you have tracker on that instances? Possible to share tomcat and DB tuning parameters? Please suggest. Regards Hannan On Tue, May 1, 2018 at 3:36 AM, Calle Hedberg wrote: > Hannan, > > Out of interest: how do you measure your DB size? "127GB" does not really > say much - is it with or without analytics tables etc. Are you just using > pg_database_size? > > I just ran that on my local instance of the South African "national" HMIS > instance - it is 202 GB (NOTE: that includes analytics), and running > analytics takes around 1-1.5 hours. So 12-13 hours on your side seems > excessive (I presume you are using either high-speed RAID rust platters or > else SSD). > > Regards > calle > > > > On 30 April 2018 at 19:24, Hannan Khan wrote: > >> Thanks Bob. >> >> I remembered that. 1st step I takes was upgrade postgres to 9.6 and >> tuning parameters which reduced time from 58 to 13 hour. And after that I >> removed/update the data types of the aggregated data set what was >> changed/amended that reduce the time to 8 hour. >> >> 1. this time it is gradually increased from 8 hour to 2/13 hour now. This >> time it seems that tracker contribute to the delays. >> >> * INFO 2018-04-30 03:24:56,323 Populated table in 8717.87 seconds: >> analytics_enrollment_temp_xggeyhhpmkp (AbstractJdbcTableManager.java >> [taskScheduler-25]) >> * INFO 2018-04-30 04:01:53,261 Populated table in 10919.99 seconds: >> analytics_enrollment_temp_knjmf7ttlad (AbstractJdbcTableManager.java >> [taskScheduler-9]) >> * INFO 2018-04-30 07:03:46,687 Populated table in 10913.42 seconds: >> analytics_enrollment_temp_fokn0tj7lmi (AbstractJdbcTableManager.java >> [taskScheduler-9]) >> * INFO 2018-04-30 08:18:49,534 Populated table in 26237.16 seconds: >> analytics_enrollment_temp_oo3ormkzucj (AbstractJdbcTableManager.java >> [taskScheduler-22]) >> * INFO 2018-04-30 09:02:22,728 Populated table in 7116.00 seconds: >> analytics_enrollment_temp_vu0q0uahxxx (AbstractJdbcTableManager.java >> [taskScheduler-9]) >> * INFO 2018-04-30 12:22:26,364 Populated table in 32250.03 seconds: >> analytics_enrollment_temp_swqlgvqvifa (AbstractJdbcTableManager.java >> [taskScheduler-25]) >> >> 2. Yes! Good suggestion. This maintenance work_mem is low compared to >> these trackers. I am changing now. >> >> automated procedures are not causing so much problem. >> >> Lets see the progress tomorrow. I will get back to you all. >> >> Regards >> >> Hannan >> >> >> >> On Mon, Apr 30, 2018 at 7:48 PM, Bob Jolliffe >> wrote: >> >>> Hi Hannan >>> >>> I recall you had a similar request which I responded to back in Jan 11. >>> Maybe worth re-reading that thread as some of it seems still to be >>> relevant. In particular: >>> >>> 1. The first question, as always, is has this suddenly happened or has >>> something recently changed? Back then there had been some new datasets >>> created by UNICEF which you were suspecting. I recall at the time you said >>> that analytics then was taking 13 hours. So 12 hours is an improvement? >>> Is this a sudden problem, like from a recent system upgrade or some changes >>> to datasets, or just the way it has been for some time .. unacceptably >>> slow. >>> >>> 2. A lot of analytics generation time is taken up on CREATE INDEX. You >>> might try to increase maintenance_work_mem. I think I had suggested back >>> then pushing as far as 4G on your system. >>> >>> 3. You still have some automated creature (username elmisinterop) >>> logging into your system at a crazy rate per second after 1am. As I recall >>> this user is trying to download all the data on your system which is >>> probably not a good thing to do during analytics generation. Whatever it >>> is, you need to disable that. >>> >>> Try 2 and 3 and see where that gets you. >>> >>> Any chance you can also try to get access to some faster disk resource? >>> >>> Hope this helps. >>> >>> Bob >>> >>> On 30 April 2018 at 12:36, Hannan Khan wrote: >>> Dear Experts I am drawing your attention to one problem and need your kind attention. In one of our DHIS2 instances, 'Central Server' analytics generation taking too much time 12 h, 22 m, 34 s. DB size is 127 GB and have aggregated and trackers as. During analytics run-time the processor utilization and RAM utilization is normal. But from the log it is clear that the trackers are taking too much time though they are not having much data. "EPI Cold Chain" tracker taking near about 9 hour, "Cause of death (MCCOD)"tracker taking 3 hour and "Cervical and Breast Cancer Screening Program" tracker taking 7 hour. What seems to be the reason? Is something wrong with the tracker system? We are using DHIS2 version 2.28 release 533e983. Need immediate help. Regards Muhammad Abdul Hannan Khan Team Leader Support to t
