Halvdan, Firstly, I have just done a test import of the za_CTRY.gml file using the latest 2.20 trunk - and IT WORKS. I just hope using 2.20 for importing those files and then reverting back to our customised 2.19 won't mess up anything (NOTE: for the time being, we have to use a customised version of 2.19 due to the inclusion of 1.4 conversion code, the new database Synchronisation Manager enabling synching between many different DHIS2 instances, and the small piece of code enabling vertical cursor movement when capturing daily data in the custom "pivot" data entry form).
Secondly, I do get some warning messages when importing - not sure if they are relevant/expected (see tomcat log extract below). I really appreciate the effort you've put in on this, especially during those hot mid-summer days in Oslo :-) Regards Calle Tomcat log: 04-Jul-2015 01:33:26.270 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 78200 ms * WARN 2015-07-04 01:33:39,997 Authentication event AuthenticationSuccessEvent: Calle_Hedberg; details: org.springframework.security.web.authentication.WebAuthenticationDetails@fffbcba8: RemoteIpAddress: 0 0-exec-5]) * WARN 2015-07-04 01:33:39,998 Authentication event SessionFixationProtectionEvent: Calle_Hedberg; details: org.springframework.security.web.authentication.WebAuthenticationDetails@fffbcba8: RemoteIpAddres -8080-exec-5]) * WARN 2015-07-04 01:33:39,999 Authentication event InteractiveAuthenticationSuccessEvent: Calle_Hedberg; details: org.springframework.security.web.authentication.WebAuthenticationDetails@fffbcba8: RemoteI ttp-nio-8080-exec-5]) Warning: org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser: Property ' http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized. Compiler warnings: WARNING: 'org.apache.xerces.jaxp.SAXParserImpl: Property ' http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.' Warning: org.apache.xerces.parsers.SAXParser: Feature ' http://javax.xml.XMLConstants/feature/secure-processing' is not recognized. Warning: org.apache.xerces.parsers.SAXParser: Property ' http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized. Warning: org.apache.xerces.parsers.SAXParser: Property ' http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized. * INFO 2015-07-04 01:34:39,988 User 'Calle_Hedberg' started import at Sat Jul 04 01:34:39 CAT 2015 (DefaultImportService.java [taskScheduler-1]) * INFO 2015-07-04 01:34:39,988 [Level: INFO, category: METADATA_IMPORT, time: Sat Jul 04 01:34:39 CAT 2015, message: Importing meta-data] (InMemoryNotifier.java [taskScheduler-1]) * INFO 2015-07-04 01:34:39,998 Building object-bridge maps (preheatCache: true). (DefaultObjectBridge.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,235 Building object-bridge maps took 3.24 seconds. (DefaultObjectBridge.java [taskScheduler-1]) * WARN 2015-07-04 01:34:43,235 Can not find getter for 'oAuth2Clients'. (DefaultImportService.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,235 [Level: INFO, category: METADATA_IMPORT, time: Sat Jul 04 01:34:43 CAT 2015, message: Importing 1 OrganisationUnits] (InMemoryNotifier.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,245 Deleted objects associated with object of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,245 Deleted objects associated with object of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,245 Deleted objects associated with object of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1]) * WARN 2015-07-04 01:34:43,265 Can not find getter for 'programRuleVariables'. (DefaultImportService.java [taskScheduler-1]) * WARN 2015-07-04 01:34:43,265 Can not find getter for 'programRules'. (DefaultImportService.java [taskScheduler-1]) * WARN 2015-07-04 01:34:43,265 Can not find getter for 'programRuleActions'. (DefaultImportService.java [taskScheduler-1]) * INFO 2015-07-04 01:34:43,275 [Level: INFO, category: METADATA_IMPORT, time: Sat Jul 04 01:34:43 CAT 2015, message: Import done. Completed in 3.28 seconds.] (InMemoryNotifier.java [taskScheduler-1]) On 4 July 2015 at 01:12, Halvdan Grelland <halvda...@gmail.com> wrote: > Oh, and for the record: the stack overflow was caused by cirular calls to > toString between AttributeValue and Attribute, respectively. > > 2015-07-04 0:56 GMT+02:00 Halvdan Grelland <halvda...@gmail.com>: > >> Yes, that is the plan, ultimately. >> It's a pretty major change though, making it non-trivial to port the fix >> (it's more of a refactor). I'll look into it after the weekend (or during >> the weekend if the sun stops shining in Oslo). >> >> I did make (and backport) some changes which fix the stack overflow >> issue, however. In practice this means that GML import should work for you >> on the latest 2.19, but any attribute values will unfortunately be deleted >> in the process (!), or you might even experience the importer halting if >> there are attribute values for the orgunits. Not ideal, in other words, and >> something you should be aware of. >> >> Halvdan >> >> 2015-07-03 18:55 GMT+02:00 Calle Hedberg <calle.hedb...@gmail.com>: >> >>> Halvdan, >>> >>> I see you have committed a significant fix to 2.20 trunk - which I hope >>> will also fix my GML import issue. >>> >>> Will you port this fix to 2.19 also, please? >>> >>> Regards >>> Calle >>> >>> On 1 July 2015 at 15:01, Halvdan Grelland <halvda...@gmail.com> wrote: >>> >>>> Ok, so I've looked further into this. The only issue I am able to >>>> consistently reproduce happens only in the GML import process. The stack >>>> trace can be found here >>>> <https://gist.github.com/grelland/45acadfbe475422889ec> >>>> Note that this might be unrelated to the looping/recursion issue., >>>> >>>> This does not happen if I pre-stage the meta data payload (ie write it >>>> to a file and import using the metadata importer). My suspicion that the >>>> conflict occurs because of the clear/re-add of attributevalues in the >>>> orgunit.mergeWith() method (which I'm using to merge geodata updates into >>>> the orgunit), which happens within a transactional scope. I'm gonna have to >>>> experiment further to find a solution. Any suggestions are much >>>> appreciated. >>>> >>>> >>>> 2015-07-01 14:39 GMT+02:00 Bob Jolliffe <bobjolli...@gmail.com>: >>>> >>>>> Morten this is a long shot, but do you think there is any possibility >>>>> the removal of xerces exclusions in 2.19 might be causing these >>>>> strange unpredictable problems? Weird StackOverflowErrors and Xerces >>>>> problems seem to go hand in hand - google the two terms :-( I guess >>>>> the nub is that it becomes unpredictable which xercesimpl the >>>>> classloader uses. >>>>> >>>>> The original reason for adding the xercesImpl exclusions was that it >>>>> seemed to make the java xml parsing more stable. I am not sure how it >>>>> easy it would be test this as you'd have to build without openid4java >>>>> .. which is the library which caused us to remove the exclusions. >>>>> >>>>> On 1 July 2015 at 10:16, Morten Olav Hansen <morte...@gmail.com> >>>>> wrote: >>>>> > >>>>> > On Wed, Jul 1, 2015 at 4:12 PM, Bob Jolliffe <bobjolli...@gmail.com> >>>>> wrote: >>>>> >> >>>>> >> Having said that, this stack overflow business might carry some >>>>> clue. >>>>> >> @Morten you mention elsewhere in this thread "It seems to be >>>>> looping in >>>>> >> Attribute.toString()". Are you sure it is looping or is there >>>>> something >>>>> >> recursive going on here? (note that xslt with very large nodesets >>>>> could >>>>> >> indeed also cause recursion problems) >>>>> > >>>>> > >>>>> > Well, it reminded me of an earlier issue we had, but that have since >>>>> been >>>>> > fixed. >>>>> > >>>>> > As I can import the sample XML file from Halvdan without any issues >>>>> in 2.19 >>>>> > / 2.20 (but he could not...) I might not be reproducing it properly, >>>>> so I'm >>>>> > waiting for input from him. >>>>> > >>>>> > -- >>>>> > Morten >>>>> >>>>> _______________________________________________ >>>>> Mailing list: https://launchpad.net/~dhis2-devs >>>>> Post to : dhis2-devs@lists.launchpad.net >>>>> Unsubscribe : https://launchpad.net/~dhis2-devs >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~dhis2-devs >>>> Post to : dhis2-devs@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~dhis2-devs >>>> More help : https://help.launchpad.net/ListHelp >>>> >>>> >>> >>> >>> -- >>> >>> ******************************************* >>> >>> Calle Hedberg >>> >>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA >>> >>> Tel/fax (home): +27-21-685-6472 >>> >>> Cell: +27-82-853-5352 >>> >>> Iridium SatPhone: +8816-315-19274 >>> >>> Email: calle.hedb...@gmail.com >>> >>> Skype: calle_hedberg >>> >>> ******************************************* >>> >>> >> > -- ******************************************* Calle Hedberg 46D Alma Road, 7700 Rosebank, SOUTH AFRICA Tel/fax (home): +27-21-685-6472 Cell: +27-82-853-5352 Iridium SatPhone: +8816-315-19274 Email: calle.hedb...@gmail.com Skype: calle_hedberg *******************************************
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
