Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Calle Hedberg 
Morten

Apology accepted - sometimes it's tricky to handle those curve-balls. In
any case, there are some lessons here...

;-)

Regards
Calle

On 12 July 2018 at 13:51, Morten Olav Hansen  wrote:

> Hi Calle
>
> I have only been aware of this bug for a few days (when it hit us in Lao),
> and then we have both release and summer holidays going on. I agree the
> response is not ideal, but its not the easiest time for fixing bug issues.
>
> We are working hard on fixing this now (it might take 1-2 days), and at
> that time we will recommend that everyone updates their 229 and 230
> instances (I will leave that up to Stian when he is ready)
>
> This was a design issue, and they are not that easy to fix (without
> causing havoc internally), but hopefully we have learned from this and we
> won't name domain tables analytics* anymore.
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> Team Integration Lead
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 6:33 PM, Calle Hedberg 
> wrote:
>
>> Hi
>>
>> I'm glad it's being addressed - but I am less happy to hear you are aware
>> of it but haven't said anything to the community...  These type of bugs
>> causes havoc, and waste a lot of time for users affected while they try to
>> figure out what's gone wrong.
>>
>> There are two major lessons here:
>>
>> Firstly, to stick to a standard and manageable naming convention for both
>> tables and fields (and interfaces).
>>
>> Secondly, to inform the user community promptly when you become aware of
>> major, "showstopper", bugs.
>>
>> Best regards
>> calle
>>
>> On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:
>>
>>> Ok, thanks Stian (no need to work on this then Viet)
>>>
>>> (knut, using pg_dump we normally filter away analytic* which means no
>>> period boundaries..)
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:
>>>
 I have started the work on renaming the table. I will update this
 thread as soon as I make progress.

 On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
 wrote:

> Hi Knut
>
> I mean the main issue the thread was about... using maintenance =>
> clear analytic tables, it will delete analytics* which includes the
> analytical boundaries.
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring 
> wrote:
>
>> Hi Morten, sorry but which functionality are you suggesting should
>> not be used? What do you mean by manually deleting?
>>
>> Thanks,
>> Knut
>>
>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
>> wrote:
>>
>>> Hi Calle
>>>
>>> We are aware of this issue (actually it caused a problem with us in
>>> Lao), for now.. I would also say don't use this functionality, its 
>>> better
>>> to manually delete the analytics_* tables if you need it. We will rename
>>> that table soon we hope, and that should fix it (this also causes 
>>> potential
>>> issues with your backups...)
>>>
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg <
>>> calle.hedb...@gmail.com> wrote:
>>>
 Bob,

 No response/action on the JIRA bug report yet - I guess most
 developers are on leave (wonderful summer here in Norway this year).

 Otherwise I agree, the name of that table does not fit the general
 naming convention as far as I can see. It would make more sense to 
 call it
 e.g. "programindicator_periodboundary". The name then provides an
 intuitive description of the content and it sorts together with the 
 group
 of programindicator tables.

 Regards
 calle

 On 12 July 2018 at 08:57, Bob Jolliffe 
 wrote:

> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by
> this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg 
> wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard
> parameter "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or
> similar.
> >
> > The purpose of the parameter was to exclude all analytics tables
> from the
> > dump, since it is significantly faster to restore a dump without
> analytics
> > tables and then run analytics to re-create them (due to the use
> of
> >

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Stian Sandvold 
The convention today is that resource tables are prefixed with an
underscore, while analytics tables start with analytics*. As previously
mentioned in this thread, changing this now will probably cause a lot of
issues for existing scripts, so I think for now we should just avoid using
tables starting with analytics*.

On Thu, Jul 12, 2018 at 3:26 PM, Bob Jolliffe  wrote:

> Agree that might have made sense at the start.  But I think the
> priority now should be to rename that one table so that all the backup
> scripts in the wild get unbroken.  Not to break them even more :-)
>
> On 12 July 2018 at 13:02, Knut Staring  wrote:
> > Would be great if the convention could be that ALL generated tables
> > (including analytics) started with an underscore. This convention is
> sort of
> > already in place, just not applied consistently.
> >
> > Knut
> >
> > On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg 
> > wrote:
> >>
> >> Hi
> >>
> >> I'm glad it's being addressed - but I am less happy to hear you are
> aware
> >> of it but haven't said anything to the community...  These type of bugs
> >> causes havoc, and waste a lot of time for users affected while they try
> to
> >> figure out what's gone wrong.
> >>
> >> There are two major lessons here:
> >>
> >> Firstly, to stick to a standard and manageable naming convention for
> both
> >> tables and fields (and interfaces).
> >>
> >> Secondly, to inform the user community promptly when you become aware of
> >> major, "showstopper", bugs.
> >>
> >> Best regards
> >> calle
> >>
> >> On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:
> >>>
> >>> Ok, thanks Stian (no need to work on this then Viet)
> >>>
> >>> (knut, using pg_dump we normally filter away analytic* which means no
> >>> period boundaries..)
> >>>
> >>> --
> >>> Morten Olav Hansen
> >>> Senior Engineer, DHIS 2
> >>> University of Oslo
> >>> http://www.dhis2.org
> >>>
> >>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold 
> wrote:
> 
>  I have started the work on renaming the table. I will update this
> thread
>  as soon as I make progress.
> 
>  On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen <
> mor...@dhis2.org>
>  wrote:
> >
> > Hi Knut
> >
> > I mean the main issue the thread was about... using maintenance =>
> > clear analytic tables, it will delete analytics* which includes the
> > analytical boundaries.
> >
> >
> > --
> > Morten Olav Hansen
> > Senior Engineer, DHIS 2
> > University of Oslo
> > http://www.dhis2.org
> >
> > On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring 
> wrote:
> >>
> >> Hi Morten, sorry but which functionality are you suggesting should
> not
> >> be used? What do you mean by manually deleting?
> >>
> >> Thanks,
> >> Knut
> >>
> >> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
> >> wrote:
> >>>
> >>> Hi Calle
> >>>
> >>> We are aware of this issue (actually it caused a problem with us in
> >>> Lao), for now.. I would also say don't use this functionality, its
> better to
> >>> manually delete the analytics_* tables if you need it. We will
> rename that
> >>> table soon we hope, and that should fix it (this also causes
> potential
> >>> issues with your backups...)
> >>>
> >>>
> >>> --
> >>> Morten Olav Hansen
> >>> Senior Engineer, DHIS 2
> >>> University of Oslo
> >>> http://www.dhis2.org
> >>>
> >>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg
> >>>  wrote:
> 
>  Bob,
> 
>  No response/action on the JIRA bug report yet - I guess most
>  developers are on leave (wonderful summer here in Norway this
> year).
> 
>  Otherwise I agree, the name of that table does not fit the general
>  naming convention as far as I can see. It would make more sense
> to call it
>  e.g. "programindicator_periodboundary". The name then provides
> an intuitive
>  description of the content and it sorts together with the group of
>  programindicator tables.
> 
>  Regards
>  calle
> 
>  On 12 July 2018 at 08:57, Bob Jolliffe 
>  wrote:
> >
> > Thats nasty alright.  I guess using "-T anlytics_*" instead would
> > help.  But there are so many backup scripts out there broken by
> > this
> > that it will be better to rename the table.
> >
> > On 11 July 2018 at 22:39, Calle Hedberg  >
> > wrote:
> > > Hi
> > >
> > > For as long as I can remember, we have used the standard
> > > parameter "-T
> > > analytics*" when dumping a DHIS2 database into e.g. a backup or
> > > similar.
> > >
> > > The purpose of the parameter was to exclude all analytics
> tables
> > > from the
> > > dump, since it is significantly faster to restore a dump
>

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Bob Jolliffe 
Agree that might have made sense at the start.  But I think the
priority now should be to rename that one table so that all the backup
scripts in the wild get unbroken.  Not to break them even more :-)

On 12 July 2018 at 13:02, Knut Staring  wrote:
> Would be great if the convention could be that ALL generated tables
> (including analytics) started with an underscore. This convention is sort of
> already in place, just not applied consistently.
>
> Knut
>
> On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg 
> wrote:
>>
>> Hi
>>
>> I'm glad it's being addressed - but I am less happy to hear you are aware
>> of it but haven't said anything to the community...  These type of bugs
>> causes havoc, and waste a lot of time for users affected while they try to
>> figure out what's gone wrong.
>>
>> There are two major lessons here:
>>
>> Firstly, to stick to a standard and manageable naming convention for both
>> tables and fields (and interfaces).
>>
>> Secondly, to inform the user community promptly when you become aware of
>> major, "showstopper", bugs.
>>
>> Best regards
>> calle
>>
>> On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:
>>>
>>> Ok, thanks Stian (no need to work on this then Viet)
>>>
>>> (knut, using pg_dump we normally filter away analytic* which means no
>>> period boundaries..)
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:

 I have started the work on renaming the table. I will update this thread
 as soon as I make progress.

 On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
 wrote:
>
> Hi Knut
>
> I mean the main issue the thread was about... using maintenance =>
> clear analytic tables, it will delete analytics* which includes the
> analytical boundaries.
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:
>>
>> Hi Morten, sorry but which functionality are you suggesting should not
>> be used? What do you mean by manually deleting?
>>
>> Thanks,
>> Knut
>>
>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
>> wrote:
>>>
>>> Hi Calle
>>>
>>> We are aware of this issue (actually it caused a problem with us in
>>> Lao), for now.. I would also say don't use this functionality, its 
>>> better to
>>> manually delete the analytics_* tables if you need it. We will rename 
>>> that
>>> table soon we hope, and that should fix it (this also causes potential
>>> issues with your backups...)
>>>
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg
>>>  wrote:

 Bob,

 No response/action on the JIRA bug report yet - I guess most
 developers are on leave (wonderful summer here in Norway this year).

 Otherwise I agree, the name of that table does not fit the general
 naming convention as far as I can see. It would make more sense to 
 call it
 e.g. "programindicator_periodboundary". The name then provides an 
 intuitive
 description of the content and it sorts together with the group of
 programindicator tables.

 Regards
 calle

 On 12 July 2018 at 08:57, Bob Jolliffe 
 wrote:
>
> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by
> this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg 
> wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard
> > parameter "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or
> > similar.
> >
> > The purpose of the parameter was to exclude all analytics tables
> > from the
> > dump, since it is significantly faster to restore a dump without
> > analytics
> > tables and then run analytics to re-create them (due to the use
> > of
> > multi-threading), compared to dumping and restoring a database
> > instance with
> > all the analytics table (restore is NOT using multi-threading).
> >
> > For some reason, in 2.29 a new table that stores periodboundary
> > data for
> > Program Indicators was called "analyticsperiodboundary" - which
> > means the
> > standard pg_dump parameter will leave that table behind together
> > with all
> > other "analytics*" tables.
> >

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Knut Staring 
Would be great if the convention could be that ALL generated tables
(including analytics) started with an underscore. This convention is sort
of already in place, just not applied consistently.

Knut

On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg 
wrote:

> Hi
>
> I'm glad it's being addressed - but I am less happy to hear you are aware
> of it but haven't said anything to the community...  These type of bugs
> causes havoc, and waste a lot of time for users affected while they try to
> figure out what's gone wrong.
>
> There are two major lessons here:
>
> Firstly, to stick to a standard and manageable naming convention for both
> tables and fields (and interfaces).
>
> Secondly, to inform the user community promptly when you become aware of
> major, "showstopper", bugs.
>
> Best regards
> calle
>
> On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:
>
>> Ok, thanks Stian (no need to work on this then Viet)
>>
>> (knut, using pg_dump we normally filter away analytic* which means no
>> period boundaries..)
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:
>>
>>> I have started the work on renaming the table. I will update this thread
>>> as soon as I make progress.
>>>
>>> On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
>>> wrote:
>>>
 Hi Knut

 I mean the main issue the thread was about... using maintenance =>
 clear analytic tables, it will delete analytics* which includes the
 analytical boundaries.


 --
 Morten Olav Hansen
 Senior Engineer, DHIS 2
 University of Oslo
 http://www.dhis2.org

 On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:

> Hi Morten, sorry but which functionality are you suggesting should not
> be used? What do you mean by manually deleting?
>
> Thanks,
> Knut
>
> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
> wrote:
>
>> Hi Calle
>>
>> We are aware of this issue (actually it caused a problem with us in
>> Lao), for now.. I would also say don't use this functionality, its better
>> to manually delete the analytics_* tables if you need it. We will rename
>> that table soon we hope, and that should fix it (this also causes 
>> potential
>> issues with your backups...)
>>
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg <
>> calle.hedb...@gmail.com> wrote:
>>
>>> Bob,
>>>
>>> No response/action on the JIRA bug report yet - I guess most
>>> developers are on leave (wonderful summer here in Norway this year).
>>>
>>> Otherwise I agree, the name of that table does not fit the general
>>> naming convention as far as I can see. It would make more sense to call 
>>> it
>>> e.g. "programindicator_periodboundary". The name then provides an 
>>> intuitive
>>> description of the content and it sorts together with the group of
>>> programindicator tables.
>>>
>>> Regards
>>> calle
>>>
>>> On 12 July 2018 at 08:57, Bob Jolliffe 
>>> wrote:
>>>
 Thats nasty alright.  I guess using "-T anlytics_*" instead would
 help.  But there are so many backup scripts out there broken by this
 that it will be better to rename the table.

 On 11 July 2018 at 22:39, Calle Hedberg 
 wrote:
 > Hi
 >
 > For as long as I can remember, we have used the standard
 parameter "-T
 > analytics*" when dumping a DHIS2 database into e.g. a backup or
 similar.
 >
 > The purpose of the parameter was to exclude all analytics tables
 from the
 > dump, since it is significantly faster to restore a dump without
 analytics
 > tables and then run analytics to re-create them (due to the use of
 > multi-threading), compared to dumping and restoring a database
 instance with
 > all the analytics table (restore is NOT using multi-threading).
 >
 > For some reason, in 2.29 a new table that stores periodboundary
 data for
 > Program Indicators was called "analyticsperiodboundary" - which
 means the
 > standard pg_dump parameter will leave that table behind together
 with all
 > other "analytics*" tables.
 >
 > Furthermore, the routine called "Clear Analytics Tables" found
 under Data
 > Administration -> Maintenance is as before deleting all tables
 named
 > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
 > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
 >
 > Which will crash your system in the sense that you won't see any
 program

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Morten Olav Hansen 
Hi Calle

I have only been aware of this bug for a few days (when it hit us in Lao),
and then we have both release and summer holidays going on. I agree the
response is not ideal, but its not the easiest time for fixing bug issues.

We are working hard on fixing this now (it might take 1-2 days), and at
that time we will recommend that everyone updates their 229 and 230
instances (I will leave that up to Stian when he is ready)

This was a design issue, and they are not that easy to fix (without causing
havoc internally), but hopefully we have learned from this and we won't
name domain tables analytics* anymore.

-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
Team Integration Lead
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:33 PM, Calle Hedberg 
wrote:

> Hi
>
> I'm glad it's being addressed - but I am less happy to hear you are aware
> of it but haven't said anything to the community...  These type of bugs
> causes havoc, and waste a lot of time for users affected while they try to
> figure out what's gone wrong.
>
> There are two major lessons here:
>
> Firstly, to stick to a standard and manageable naming convention for both
> tables and fields (and interfaces).
>
> Secondly, to inform the user community promptly when you become aware of
> major, "showstopper", bugs.
>
> Best regards
> calle
>
> On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:
>
>> Ok, thanks Stian (no need to work on this then Viet)
>>
>> (knut, using pg_dump we normally filter away analytic* which means no
>> period boundaries..)
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:
>>
>>> I have started the work on renaming the table. I will update this thread
>>> as soon as I make progress.
>>>
>>> On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
>>> wrote:
>>>
 Hi Knut

 I mean the main issue the thread was about... using maintenance =>
 clear analytic tables, it will delete analytics* which includes the
 analytical boundaries.


 --
 Morten Olav Hansen
 Senior Engineer, DHIS 2
 University of Oslo
 http://www.dhis2.org

 On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:

> Hi Morten, sorry but which functionality are you suggesting should not
> be used? What do you mean by manually deleting?
>
> Thanks,
> Knut
>
> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
> wrote:
>
>> Hi Calle
>>
>> We are aware of this issue (actually it caused a problem with us in
>> Lao), for now.. I would also say don't use this functionality, its better
>> to manually delete the analytics_* tables if you need it. We will rename
>> that table soon we hope, and that should fix it (this also causes 
>> potential
>> issues with your backups...)
>>
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg <
>> calle.hedb...@gmail.com> wrote:
>>
>>> Bob,
>>>
>>> No response/action on the JIRA bug report yet - I guess most
>>> developers are on leave (wonderful summer here in Norway this year).
>>>
>>> Otherwise I agree, the name of that table does not fit the general
>>> naming convention as far as I can see. It would make more sense to call 
>>> it
>>> e.g. "programindicator_periodboundary". The name then provides an
>>> intuitive description of the content and it sorts together with the 
>>> group
>>> of programindicator tables.
>>>
>>> Regards
>>> calle
>>>
>>> On 12 July 2018 at 08:57, Bob Jolliffe 
>>> wrote:
>>>
 Thats nasty alright.  I guess using "-T anlytics_*" instead would
 help.  But there are so many backup scripts out there broken by this
 that it will be better to rename the table.

 On 11 July 2018 at 22:39, Calle Hedberg 
 wrote:
 > Hi
 >
 > For as long as I can remember, we have used the standard
 parameter "-T
 > analytics*" when dumping a DHIS2 database into e.g. a backup or
 similar.
 >
 > The purpose of the parameter was to exclude all analytics tables
 from the
 > dump, since it is significantly faster to restore a dump without
 analytics
 > tables and then run analytics to re-create them (due to the use of
 > multi-threading), compared to dumping and restoring a database
 instance with
 > all the analytics table (restore is NOT using multi-threading).
 >
 > For some reason, in 2.29 a new table that stores periodboundary
 data for
 > Program Indicators was called "analyticsperiodboundary" - which
 means the
 > standard pg_dump parameter

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Calle Hedberg 
Hi

I'm glad it's being addressed - but I am less happy to hear you are aware
of it but haven't said anything to the community...  These type of bugs
causes havoc, and waste a lot of time for users affected while they try to
figure out what's gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both
tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of
major, "showstopper", bugs.

Best regards
calle

On 12 July 2018 at 13:15, Morten Olav Hansen  wrote:

> Ok, thanks Stian (no need to work on this then Viet)
>
> (knut, using pg_dump we normally filter away analytic* which means no
> period boundaries..)
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:
>
>> I have started the work on renaming the table. I will update this thread
>> as soon as I make progress.
>>
>> On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
>> wrote:
>>
>>> Hi Knut
>>>
>>> I mean the main issue the thread was about... using maintenance => clear
>>> analytic tables, it will delete analytics* which includes the analytical
>>> boundaries.
>>>
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:
>>>
 Hi Morten, sorry but which functionality are you suggesting should not
 be used? What do you mean by manually deleting?

 Thanks,
 Knut

 On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
 wrote:

> Hi Calle
>
> We are aware of this issue (actually it caused a problem with us in
> Lao), for now.. I would also say don't use this functionality, its better
> to manually delete the analytics_* tables if you need it. We will rename
> that table soon we hope, and that should fix it (this also causes 
> potential
> issues with your backups...)
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg <
> calle.hedb...@gmail.com> wrote:
>
>> Bob,
>>
>> No response/action on the JIRA bug report yet - I guess most
>> developers are on leave (wonderful summer here in Norway this year).
>>
>> Otherwise I agree, the name of that table does not fit the general
>> naming convention as far as I can see. It would make more sense to call 
>> it
>> e.g. "programindicator_periodboundary". The name then provides an
>> intuitive description of the content and it sorts together with the group
>> of programindicator tables.
>>
>> Regards
>> calle
>>
>> On 12 July 2018 at 08:57, Bob Jolliffe  wrote:
>>
>>> Thats nasty alright.  I guess using "-T anlytics_*" instead would
>>> help.  But there are so many backup scripts out there broken by this
>>> that it will be better to rename the table.
>>>
>>> On 11 July 2018 at 22:39, Calle Hedberg 
>>> wrote:
>>> > Hi
>>> >
>>> > For as long as I can remember, we have used the standard parameter
>>> "-T
>>> > analytics*" when dumping a DHIS2 database into e.g. a backup or
>>> similar.
>>> >
>>> > The purpose of the parameter was to exclude all analytics tables
>>> from the
>>> > dump, since it is significantly faster to restore a dump without
>>> analytics
>>> > tables and then run analytics to re-create them (due to the use of
>>> > multi-threading), compared to dumping and restoring a database
>>> instance with
>>> > all the analytics table (restore is NOT using multi-threading).
>>> >
>>> > For some reason, in 2.29 a new table that stores periodboundary
>>> data for
>>> > Program Indicators was called "analyticsperiodboundary" - which
>>> means the
>>> > standard pg_dump parameter will leave that table behind together
>>> with all
>>> > other "analytics*" tables.
>>> >
>>> > Furthermore, the routine called "Clear Analytics Tables" found
>>> under Data
>>> > Administration -> Maintenance is as before deleting all tables
>>> named
>>> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
>>> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>>> >
>>> > Which will crash your system in the sense that you won't see any
>>> program
>>> > indicator data in dashboards etc.
>>> >
>>> > The "analyticsperiodboundary" table will be re-created and
>>> re-populated with
>>> > DEFAULT (boundless) Program Indicator Period boundaries when you
>>> re-start
>>> > the system (it's part of the TableAlteror routine during startup),
>>> but
>>> > - you have to re-start the system
>>> > - you will lose any non-default boundary

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Morten Olav Hansen 
Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no
period boundaries..)

-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold  wrote:

> I have started the work on renaming the table. I will update this thread
> as soon as I make progress.
>
> On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
> wrote:
>
>> Hi Knut
>>
>> I mean the main issue the thread was about... using maintenance => clear
>> analytic tables, it will delete analytics* which includes the analytical
>> boundaries.
>>
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:
>>
>>> Hi Morten, sorry but which functionality are you suggesting should not
>>> be used? What do you mean by manually deleting?
>>>
>>> Thanks,
>>> Knut
>>>
>>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
>>> wrote:
>>>
 Hi Calle

 We are aware of this issue (actually it caused a problem with us in
 Lao), for now.. I would also say don't use this functionality, its better
 to manually delete the analytics_* tables if you need it. We will rename
 that table soon we hope, and that should fix it (this also causes potential
 issues with your backups...)


 --
 Morten Olav Hansen
 Senior Engineer, DHIS 2
 University of Oslo
 http://www.dhis2.org

 On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg >>> > wrote:

> Bob,
>
> No response/action on the JIRA bug report yet - I guess most
> developers are on leave (wonderful summer here in Norway this year).
>
> Otherwise I agree, the name of that table does not fit the general
> naming convention as far as I can see. It would make more sense to call it
> e.g. "programindicator_periodboundary". The name then provides an
> intuitive description of the content and it sorts together with the group
> of programindicator tables.
>
> Regards
> calle
>
> On 12 July 2018 at 08:57, Bob Jolliffe  wrote:
>
>> Thats nasty alright.  I guess using "-T anlytics_*" instead would
>> help.  But there are so many backup scripts out there broken by this
>> that it will be better to rename the table.
>>
>> On 11 July 2018 at 22:39, Calle Hedberg 
>> wrote:
>> > Hi
>> >
>> > For as long as I can remember, we have used the standard parameter
>> "-T
>> > analytics*" when dumping a DHIS2 database into e.g. a backup or
>> similar.
>> >
>> > The purpose of the parameter was to exclude all analytics tables
>> from the
>> > dump, since it is significantly faster to restore a dump without
>> analytics
>> > tables and then run analytics to re-create them (due to the use of
>> > multi-threading), compared to dumping and restoring a database
>> instance with
>> > all the analytics table (restore is NOT using multi-threading).
>> >
>> > For some reason, in 2.29 a new table that stores periodboundary
>> data for
>> > Program Indicators was called "analyticsperiodboundary" - which
>> means the
>> > standard pg_dump parameter will leave that table behind together
>> with all
>> > other "analytics*" tables.
>> >
>> > Furthermore, the routine called "Clear Analytics Tables" found
>> under Data
>> > Administration -> Maintenance is as before deleting all tables named
>> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
>> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>> >
>> > Which will crash your system in the sense that you won't see any
>> program
>> > indicator data in dashboards etc.
>> >
>> > The "analyticsperiodboundary" table will be re-created and
>> re-populated with
>> > DEFAULT (boundless) Program Indicator Period boundaries when you
>> re-start
>> > the system (it's part of the TableAlteror routine during startup),
>> but
>> > - you have to re-start the system
>> > - you will lose any non-default boundary settings used for any
>> program
>> > indicator.
>> >
>> > This has also been reported as a high-priority bug on JIRA
>> (DHIS2-4260).
>> >
>> > Regards
>> > Calle
>> >
>> > ***
>> >
>> > Calle Hedberg
>> >
>> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>> 
>> >
>> > Tel/fax (home): +27-21-685-6472
>> >
>> > Cell: +27-82-853-5352
>> >
>> > Iridium SatPhone: +8816-315-19119
>> >
>> > Email: calle.hedb...@gmail.com
>> >
>> > Skype: calle_hedberg
>> >
>> > ***
>> >
>> >
>> >

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Stian Sandvold 
I have started the work on renaming the table. I will update this thread as
soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen 
wrote:

> Hi Knut
>
> I mean the main issue the thread was about... using maintenance => clear
> analytic tables, it will delete analytics* which includes the analytical
> boundaries.
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:
>
>> Hi Morten, sorry but which functionality are you suggesting should not be
>> used? What do you mean by manually deleting?
>>
>> Thanks,
>> Knut
>>
>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
>> wrote:
>>
>>> Hi Calle
>>>
>>> We are aware of this issue (actually it caused a problem with us in
>>> Lao), for now.. I would also say don't use this functionality, its better
>>> to manually delete the analytics_* tables if you need it. We will rename
>>> that table soon we hope, and that should fix it (this also causes potential
>>> issues with your backups...)
>>>
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg 
>>> wrote:
>>>
 Bob,

 No response/action on the JIRA bug report yet - I guess most developers
 are on leave (wonderful summer here in Norway this year).

 Otherwise I agree, the name of that table does not fit the general
 naming convention as far as I can see. It would make more sense to call it
 e.g. "programindicator_periodboundary". The name then provides an
 intuitive description of the content and it sorts together with the group
 of programindicator tables.

 Regards
 calle

 On 12 July 2018 at 08:57, Bob Jolliffe  wrote:

> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg 
> wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard parameter
> "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or
> similar.
> >
> > The purpose of the parameter was to exclude all analytics tables
> from the
> > dump, since it is significantly faster to restore a dump without
> analytics
> > tables and then run analytics to re-create them (due to the use of
> > multi-threading), compared to dumping and restoring a database
> instance with
> > all the analytics table (restore is NOT using multi-threading).
> >
> > For some reason, in 2.29 a new table that stores periodboundary data
> for
> > Program Indicators was called "analyticsperiodboundary" - which
> means the
> > standard pg_dump parameter will leave that table behind together
> with all
> > other "analytics*" tables.
> >
> > Furthermore, the routine called "Clear Analytics Tables" found under
> Data
> > Administration -> Maintenance is as before deleting all tables named
> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
> >
> > Which will crash your system in the sense that you won't see any
> program
> > indicator data in dashboards etc.
> >
> > The "analyticsperiodboundary" table will be re-created and
> re-populated with
> > DEFAULT (boundless) Program Indicator Period boundaries when you
> re-start
> > the system (it's part of the TableAlteror routine during startup),
> but
> > - you have to re-start the system
> > - you will lose any non-default boundary settings used for any
> program
> > indicator.
> >
> > This has also been reported as a high-priority bug on JIRA
> (DHIS2-4260).
> >
> > Regards
> > Calle
> >
> > ***
> >
> > Calle Hedberg
> >
> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> 
> >
> > Tel/fax (home): +27-21-685-6472
> >
> > Cell: +27-82-853-5352
> >
> > Iridium SatPhone: +8816-315-19119
> >
> > Email: calle.hedb...@gmail.com
> >
> > Skype: calle_hedberg
> >
> > ***
> >
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to : dhis2-devs@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>



 --

 ***

 Calle Hedberg

 46D Alma Road, 7700 Rosebank, SOUTH

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Knut Staring 
Right... I assume you could trigger that also through the API in order to
continue to have backup scripts without manual intervention?

On Thu, Jul 12, 2018, 4:47 PM Morten Olav Hansen  wrote:

> Hi Knut
>
> I mean the main issue the thread was about... using maintenance => clear
> analytic tables, it will delete analytics* which includes the analytical
> boundaries.
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:
>
>> Hi Morten, sorry but which functionality are you suggesting should not be
>> used? What do you mean by manually deleting?
>>
>> Thanks,
>> Knut
>>
>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen 
>> wrote:
>>
>>> Hi Calle
>>>
>>> We are aware of this issue (actually it caused a problem with us in
>>> Lao), for now.. I would also say don't use this functionality, its better
>>> to manually delete the analytics_* tables if you need it. We will rename
>>> that table soon we hope, and that should fix it (this also causes potential
>>> issues with your backups...)
>>>
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg 
>>> wrote:
>>>
 Bob,

 No response/action on the JIRA bug report yet - I guess most developers
 are on leave (wonderful summer here in Norway this year).

 Otherwise I agree, the name of that table does not fit the general
 naming convention as far as I can see. It would make more sense to call it
 e.g. "programindicator_periodboundary". The name then provides an intuitive
 description of the content and it sorts together with the group of
 programindicator tables.

 Regards
 calle

 On 12 July 2018 at 08:57, Bob Jolliffe  wrote:

> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg 
> wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard parameter
> "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or
> similar.
> >
> > The purpose of the parameter was to exclude all analytics tables
> from the
> > dump, since it is significantly faster to restore a dump without
> analytics
> > tables and then run analytics to re-create them (due to the use of
> > multi-threading), compared to dumping and restoring a database
> instance with
> > all the analytics table (restore is NOT using multi-threading).
> >
> > For some reason, in 2.29 a new table that stores periodboundary data
> for
> > Program Indicators was called "analyticsperiodboundary" - which
> means the
> > standard pg_dump parameter will leave that table behind together
> with all
> > other "analytics*" tables.
> >
> > Furthermore, the routine called "Clear Analytics Tables" found under
> Data
> > Administration -> Maintenance is as before deleting all tables named
> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
> >
> > Which will crash your system in the sense that you won't see any
> program
> > indicator data in dashboards etc.
> >
> > The "analyticsperiodboundary" table will be re-created and
> re-populated with
> > DEFAULT (boundless) Program Indicator Period boundaries when you
> re-start
> > the system (it's part of the TableAlteror routine during startup),
> but
> > - you have to re-start the system
> > - you will lose any non-default boundary settings used for any
> program
> > indicator.
> >
> > This has also been reported as a high-priority bug on JIRA
> (DHIS2-4260).
> >
> > Regards
> > Calle
> >
> > ***
> >
> > Calle Hedberg
> >
> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> 
> >
> > Tel/fax (home): +27-21-685-6472
> >
> > Cell: +27-82-853-5352
> >
> > Iridium SatPhone: +8816-315-19119
> >
> > Email: calle.hedb...@gmail.com
> >
> > Skype: calle_hedberg
> >
> > ***
> >
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to : dhis2-devs@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>



 --

 ***

 Calle Hedberg

 46D

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Morten Olav Hansen 
Hi Knut

I mean the main issue the thread was about... using maintenance => clear
analytic tables, it will delete analytics* which includes the analytical
boundaries.


-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring  wrote:

> Hi Morten, sorry but which functionality are you suggesting should not be
> used? What do you mean by manually deleting?
>
> Thanks,
> Knut
>
> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen  wrote:
>
>> Hi Calle
>>
>> We are aware of this issue (actually it caused a problem with us in Lao),
>> for now.. I would also say don't use this functionality, its better to
>> manually delete the analytics_* tables if you need it. We will rename that
>> table soon we hope, and that should fix it (this also causes potential
>> issues with your backups...)
>>
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg 
>> wrote:
>>
>>> Bob,
>>>
>>> No response/action on the JIRA bug report yet - I guess most developers
>>> are on leave (wonderful summer here in Norway this year).
>>>
>>> Otherwise I agree, the name of that table does not fit the general
>>> naming convention as far as I can see. It would make more sense to call it
>>> e.g. "programindicator_periodboundary". The name then provides an
>>> intuitive description of the content and it sorts together with the group
>>> of programindicator tables.
>>>
>>> Regards
>>> calle
>>>
>>> On 12 July 2018 at 08:57, Bob Jolliffe  wrote:
>>>
 Thats nasty alright.  I guess using "-T anlytics_*" instead would
 help.  But there are so many backup scripts out there broken by this
 that it will be better to rename the table.

 On 11 July 2018 at 22:39, Calle Hedberg 
 wrote:
 > Hi
 >
 > For as long as I can remember, we have used the standard parameter "-T
 > analytics*" when dumping a DHIS2 database into e.g. a backup or
 similar.
 >
 > The purpose of the parameter was to exclude all analytics tables from
 the
 > dump, since it is significantly faster to restore a dump without
 analytics
 > tables and then run analytics to re-create them (due to the use of
 > multi-threading), compared to dumping and restoring a database
 instance with
 > all the analytics table (restore is NOT using multi-threading).
 >
 > For some reason, in 2.29 a new table that stores periodboundary data
 for
 > Program Indicators was called "analyticsperiodboundary" - which means
 the
 > standard pg_dump parameter will leave that table behind together with
 all
 > other "analytics*" tables.
 >
 > Furthermore, the routine called "Clear Analytics Tables" found under
 Data
 > Administration -> Maintenance is as before deleting all tables named
 > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
 > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
 >
 > Which will crash your system in the sense that you won't see any
 program
 > indicator data in dashboards etc.
 >
 > The "analyticsperiodboundary" table will be re-created and
 re-populated with
 > DEFAULT (boundless) Program Indicator Period boundaries when you
 re-start
 > the system (it's part of the TableAlteror routine during startup), but
 > - you have to re-start the system
 > - you will lose any non-default boundary settings used for any program
 > indicator.
 >
 > This has also been reported as a high-priority bug on JIRA
 (DHIS2-4260).
 >
 > Regards
 > Calle
 >
 > ***
 >
 > Calle Hedberg
 >
 > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
 
 >
 > Tel/fax (home): +27-21-685-6472
 >
 > Cell: +27-82-853-5352
 >
 > Iridium SatPhone: +8816-315-19119
 >
 > Email: calle.hedb...@gmail.com
 >
 > Skype: calle_hedberg
 >
 > ***
 >
 >
 >
 > ___
 > Mailing list: https://launchpad.net/~dhis2-devs
 > Post to : dhis2-devs@lists.launchpad.net
 > Unsubscribe : https://launchpad.net/~dhis2-devs
 > More help   : https://help.launchpad.net/ListHelp
 >

>>>
>>>
>>>
>>> --
>>>
>>> ***
>>>
>>> Calle Hedberg
>>>
>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>> 
>>>
>>> Tel/fax (home): +27-21-685-6472
>>>
>>> Cell: +27-82-853-5352
>>>
>>> Iridium SatPhone: +8816-315-19119
>>>
>>> Email: calle.hedb...@gmail.com
>>>
>>> Skype: calle_hedberg
>>>
>>> ***
>>>
>>>
>>>

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Knut Staring 
Hi Morten, sorry but which functionality are you suggesting should not be
used? What do you mean by manually deleting?

Thanks,
Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen  wrote:

> Hi Calle
>
> We are aware of this issue (actually it caused a problem with us in Lao),
> for now.. I would also say don't use this functionality, its better to
> manually delete the analytics_* tables if you need it. We will rename that
> table soon we hope, and that should fix it (this also causes potential
> issues with your backups...)
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg 
> wrote:
>
>> Bob,
>>
>> No response/action on the JIRA bug report yet - I guess most developers
>> are on leave (wonderful summer here in Norway this year).
>>
>> Otherwise I agree, the name of that table does not fit the general naming
>> convention as far as I can see. It would make more sense to call it e.g.
>> "programindicator_periodboundary". The name then provides an intuitive
>> description of the content and it sorts together with the group of
>> programindicator tables.
>>
>> Regards
>> calle
>>
>> On 12 July 2018 at 08:57, Bob Jolliffe  wrote:
>>
>>> Thats nasty alright.  I guess using "-T anlytics_*" instead would
>>> help.  But there are so many backup scripts out there broken by this
>>> that it will be better to rename the table.
>>>
>>> On 11 July 2018 at 22:39, Calle Hedberg  wrote:
>>> > Hi
>>> >
>>> > For as long as I can remember, we have used the standard parameter "-T
>>> > analytics*" when dumping a DHIS2 database into e.g. a backup or
>>> similar.
>>> >
>>> > The purpose of the parameter was to exclude all analytics tables from
>>> the
>>> > dump, since it is significantly faster to restore a dump without
>>> analytics
>>> > tables and then run analytics to re-create them (due to the use of
>>> > multi-threading), compared to dumping and restoring a database
>>> instance with
>>> > all the analytics table (restore is NOT using multi-threading).
>>> >
>>> > For some reason, in 2.29 a new table that stores periodboundary data
>>> for
>>> > Program Indicators was called "analyticsperiodboundary" - which means
>>> the
>>> > standard pg_dump parameter will leave that table behind together with
>>> all
>>> > other "analytics*" tables.
>>> >
>>> > Furthermore, the routine called "Clear Analytics Tables" found under
>>> Data
>>> > Administration -> Maintenance is as before deleting all tables named
>>> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
>>> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>>> >
>>> > Which will crash your system in the sense that you won't see any
>>> program
>>> > indicator data in dashboards etc.
>>> >
>>> > The "analyticsperiodboundary" table will be re-created and
>>> re-populated with
>>> > DEFAULT (boundless) Program Indicator Period boundaries when you
>>> re-start
>>> > the system (it's part of the TableAlteror routine during startup), but
>>> > - you have to re-start the system
>>> > - you will lose any non-default boundary settings used for any program
>>> > indicator.
>>> >
>>> > This has also been reported as a high-priority bug on JIRA
>>> (DHIS2-4260).
>>> >
>>> > Regards
>>> > Calle
>>> >
>>> > ***
>>> >
>>> > Calle Hedberg
>>> >
>>> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>> >
>>> > Tel/fax (home): +27-21-685-6472
>>> >
>>> > Cell: +27-82-853-5352
>>> >
>>> > Iridium SatPhone: +8816-315-19119
>>> >
>>> > Email: calle.hedb...@gmail.com
>>> >
>>> > Skype: calle_hedberg
>>> >
>>> > ***
>>> >
>>> >
>>> >
>>> > ___
>>> > Mailing list: https://launchpad.net/~dhis2-devs
>>> > Post to : dhis2-devs@lists.launchpad.net
>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>> > More help   : https://help.launchpad.net/ListHelp
>>> >
>>>
>>
>>
>>
>> --
>>
>> ***
>>
>> Calle Hedberg
>>
>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>
>> Tel/fax (home): +27-21-685-6472
>>
>> Cell: +27-82-853-5352
>>
>> Iridium SatPhone: +8816-315-19119
>>
>> Email: calle.hedb...@gmail.com
>>
>> Skype: calle_hedberg
>>
>> ***
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to :

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Morten Olav Hansen 
Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao),
for now.. I would also say don't use this functionality, its better to
manually delete the analytics_* tables if you need it. We will rename that
table soon we hope, and that should fix it (this also causes potential
issues with your backups...)


-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg 
wrote:

> Bob,
>
> No response/action on the JIRA bug report yet - I guess most developers
> are on leave (wonderful summer here in Norway this year).
>
> Otherwise I agree, the name of that table does not fit the general naming
> convention as far as I can see. It would make more sense to call it e.g.
> "programindicator_periodboundary". The name then provides an intuitive
> description of the content and it sorts together with the group of
> programindicator tables.
>
> Regards
> calle
>
> On 12 July 2018 at 08:57, Bob Jolliffe  wrote:
>
>> Thats nasty alright.  I guess using "-T anlytics_*" instead would
>> help.  But there are so many backup scripts out there broken by this
>> that it will be better to rename the table.
>>
>> On 11 July 2018 at 22:39, Calle Hedberg  wrote:
>> > Hi
>> >
>> > For as long as I can remember, we have used the standard parameter "-T
>> > analytics*" when dumping a DHIS2 database into e.g. a backup or similar.
>> >
>> > The purpose of the parameter was to exclude all analytics tables from
>> the
>> > dump, since it is significantly faster to restore a dump without
>> analytics
>> > tables and then run analytics to re-create them (due to the use of
>> > multi-threading), compared to dumping and restoring a database instance
>> with
>> > all the analytics table (restore is NOT using multi-threading).
>> >
>> > For some reason, in 2.29 a new table that stores periodboundary data for
>> > Program Indicators was called "analyticsperiodboundary" - which means
>> the
>> > standard pg_dump parameter will leave that table behind together with
>> all
>> > other "analytics*" tables.
>> >
>> > Furthermore, the routine called "Clear Analytics Tables" found under
>> Data
>> > Administration -> Maintenance is as before deleting all tables named
>> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
>> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>> >
>> > Which will crash your system in the sense that you won't see any program
>> > indicator data in dashboards etc.
>> >
>> > The "analyticsperiodboundary" table will be re-created and re-populated
>> with
>> > DEFAULT (boundless) Program Indicator Period boundaries when you
>> re-start
>> > the system (it's part of the TableAlteror routine during startup), but
>> > - you have to re-start the system
>> > - you will lose any non-default boundary settings used for any program
>> > indicator.
>> >
>> > This has also been reported as a high-priority bug on JIRA (DHIS2-4260).
>> >
>> > Regards
>> > Calle
>> >
>> > ***
>> >
>> > Calle Hedberg
>> >
>> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>> >
>> > Tel/fax (home): +27-21-685-6472
>> >
>> > Cell: +27-82-853-5352
>> >
>> > Iridium SatPhone: +8816-315-19119
>> >
>> > Email: calle.hedb...@gmail.com
>> >
>> > Skype: calle_hedberg
>> >
>> > ***
>> >
>> >
>> >
>> > ___
>> > Mailing list: https://launchpad.net/~dhis2-devs
>> > Post to : dhis2-devs@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>>
>
>
>
> --
>
> ***
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19119
>
> Email: calle.hedb...@gmail.com
>
> Skype: calle_hedberg
>
> ***
>
>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Calle Hedberg 
Bob,

No response/action on the JIRA bug report yet - I guess most developers are
on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming
convention as far as I can see. It would make more sense to call it e.g.
"programindicator_periodboundary". The name then provides an intuitive
description of the content and it sorts together with the group of
programindicator tables.

Regards
calle

On 12 July 2018 at 08:57, Bob Jolliffe  wrote:

> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg  wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard parameter "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or similar.
> >
> > The purpose of the parameter was to exclude all analytics tables from the
> > dump, since it is significantly faster to restore a dump without
> analytics
> > tables and then run analytics to re-create them (due to the use of
> > multi-threading), compared to dumping and restoring a database instance
> with
> > all the analytics table (restore is NOT using multi-threading).
> >
> > For some reason, in 2.29 a new table that stores periodboundary data for
> > Program Indicators was called "analyticsperiodboundary" - which means the
> > standard pg_dump parameter will leave that table behind together with all
> > other "analytics*" tables.
> >
> > Furthermore, the routine called "Clear Analytics Tables" found under Data
> > Administration -> Maintenance is as before deleting all tables named
> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
> >
> > Which will crash your system in the sense that you won't see any program
> > indicator data in dashboards etc.
> >
> > The "analyticsperiodboundary" table will be re-created and re-populated
> with
> > DEFAULT (boundless) Program Indicator Period boundaries when you re-start
> > the system (it's part of the TableAlteror routine during startup), but
> > - you have to re-start the system
> > - you will lose any non-default boundary settings used for any program
> > indicator.
> >
> > This has also been reported as a high-priority bug on JIRA (DHIS2-4260).
> >
> > Regards
> > Calle
> >
> > ***
> >
> > Calle Hedberg
> >
> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> >
> > Tel/fax (home): +27-21-685-6472
> >
> > Cell: +27-82-853-5352
> >
> > Iridium SatPhone: +8816-315-19119
> >
> > Email: calle.hedb...@gmail.com
> >
> > Skype: calle_hedberg
> >
> > ***
> >
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to : dhis2-devs@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>



-- 

***

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedb...@gmail.com

Skype: calle_hedberg

***
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Knut Staring 
It would have been good if all the "dispensable" tables could rather have
started with an underscore (though that again will of course break extant
scripts).

Knut

On Thu, Jul 12, 2018, 1:27 PM Bob Jolliffe  wrote:

> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> help.  But there are so many backup scripts out there broken by this
> that it will be better to rename the table.
>
> On 11 July 2018 at 22:39, Calle Hedberg  wrote:
> > Hi
> >
> > For as long as I can remember, we have used the standard parameter "-T
> > analytics*" when dumping a DHIS2 database into e.g. a backup or similar.
> >
> > The purpose of the parameter was to exclude all analytics tables from the
> > dump, since it is significantly faster to restore a dump without
> analytics
> > tables and then run analytics to re-create them (due to the use of
> > multi-threading), compared to dumping and restoring a database instance
> with
> > all the analytics table (restore is NOT using multi-threading).
> >
> > For some reason, in 2.29 a new table that stores periodboundary data for
> > Program Indicators was called "analyticsperiodboundary" - which means the
> > standard pg_dump parameter will leave that table behind together with all
> > other "analytics*" tables.
> >
> > Furthermore, the routine called "Clear Analytics Tables" found under Data
> > Administration -> Maintenance is as before deleting all tables named
> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
> >
> > Which will crash your system in the sense that you won't see any program
> > indicator data in dashboards etc.
> >
> > The "analyticsperiodboundary" table will be re-created and re-populated
> with
> > DEFAULT (boundless) Program Indicator Period boundaries when you re-start
> > the system (it's part of the TableAlteror routine during startup), but
> > - you have to re-start the system
> > - you will lose any non-default boundary settings used for any program
> > indicator.
> >
> > This has also been reported as a high-priority bug on JIRA (DHIS2-4260).
> >
> > Regards
> > Calle
> >
> > ***
> >
> > Calle Hedberg
> >
> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> >
> > Tel/fax (home): +27-21-685-6472
> >
> > Cell: +27-82-853-5352
> >
> > Iridium SatPhone: +8816-315-19119
> >
> > Email: calle.hedb...@gmail.com
> >
> > Skype: calle_hedberg
> >
> > ***
> >
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to : dhis2-devs@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-12 Thread Bob Jolliffe 
Thats nasty alright.  I guess using "-T anlytics_*" instead would
help.  But there are so many backup scripts out there broken by this
that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg  wrote:
> Hi
>
> For as long as I can remember, we have used the standard parameter "-T
> analytics*" when dumping a DHIS2 database into e.g. a backup or similar.
>
> The purpose of the parameter was to exclude all analytics tables from the
> dump, since it is significantly faster to restore a dump without analytics
> tables and then run analytics to re-create them (due to the use of
> multi-threading), compared to dumping and restoring a database instance with
> all the analytics table (restore is NOT using multi-threading).
>
> For some reason, in 2.29 a new table that stores periodboundary data for
> Program Indicators was called "analyticsperiodboundary" - which means the
> standard pg_dump parameter will leave that table behind together with all
> other "analytics*" tables.
>
> Furthermore, the routine called "Clear Analytics Tables" found under Data
> Administration -> Maintenance is as before deleting all tables named
> Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>
> Which will crash your system in the sense that you won't see any program
> indicator data in dashboards etc.
>
> The "analyticsperiodboundary" table will be re-created and re-populated with
> DEFAULT (boundless) Program Indicator Period boundaries when you re-start
> the system (it's part of the TableAlteror routine during startup), but
> - you have to re-start the system
> - you will lose any non-default boundary settings used for any program
> indicator.
>
> This has also been reported as a high-priority bug on JIRA (DHIS2-4260).
>
> Regards
> Calle
>
> ***
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19119
>
> Email: calle.hedb...@gmail.com
>
> Skype: calle_hedberg
>
> ***
>
>
>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

[Dhis2-devs] 2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

2018-07-11 Thread Calle Hedberg 
Hi

For as long as I can remember, we have used the standard parameter "-T
analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the
dump, since it is significantly faster to restore a dump without analytics
tables and then run analytics to re-create them (due to the use of
multi-threading), compared to dumping and restoring a database instance
with all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for
Program Indicators was called "analyticsperiodboundary" - which means the
standard pg_dump parameter will leave that table behind together with all
other "analytics*" tables.

Furthermore, the routine called "Clear Analytics Tables" found under Data
Administration -> Maintenance is as before deleting all tables named
Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won't see any program
indicator data in dashboards etc.

The "analyticsperiodboundary" table will be re-created and re-populated
with DEFAULT (boundless) Program Indicator Period boundaries when you
re-start the system (it's part of the TableAlteror routine during startup),
but
- you have to re-start the system
- you will lose any non-default boundary settings used for any program
indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards
Calle

***

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedb...@gmail.com

Skype: calle_hedberg

***
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp