Hi, Slight adjustment to this message. There is a flaw in the example below. The risk of sending untested code..
It turns out that you cannot override systemStart and systemEnd this way. MarkLogic silently ignores those values in the metadata, hence the reason it slipped through unnoticed. To make this work, make sure LSQT has been enabled on your temporal collection. For instance with: xquery version "1.0-ml"; import module namespace temporal = "http://marklogic.com/xdmp/temporal"; at "/MarkLogic/temporal.xqy"; temporal:set-use-lsqt("uni-temporal", true()) It will initialize LSQT to 1601-01-01T00:00:00Z. After that, you have to use temporal:statement-set-system-time to override the systemStart. Try avoid enabling LSQT automation until you have loaded all historic data, MarkLogic will not allow you to set the system-time before LSQT. The following MLCP transform worked at my end without LSQT automation enabled: xquery version "1.0-ml"; module namespace example = "http://marklogic.com/example";; import module namespace temporal = "http://marklogic.com/xdmp/temporal"; at "/MarkLogic/temporal.xqy"; declare default function namespace "http://www.w3.org/2005/xpath-functions";; (::) declare option xdmp:mapping "false"; declare function example:transform( $content as map:map, $context as map:map ) as map:map* { (: grab uri and content from $content :) let $uri := map:get($content, "uri") let $doc := map:get($content, "value") (: grab other argument values passed in via cmd-line from $context :) let $transform_param := map:get($context, "transform_param") let $collections := map:get($context, "collections") let $permissions := map:get($context, "permissions") let $quality := head((map:get($context, "quality"), 0)) let $temporalCollection := map:get($context, "temporalCollection") (: determine the start you want to apply :) let $systemStart := current-dateTime() - xs:yearMonthDuration("P1Y") (: or an expression to grab the value from $doc or $transform_param :) (: apply systemStart to all subsequent temporal inserts :) let $_ := temporal:statement-set-system-time($systemStart) (: do a temporal inserts as desired :) let $_ := temporal:document-insert( $temporalCollection, $uri, $doc, map:new(( map:entry("collections", $collections), map:entry("permissions", $permissions), map:entry("quality", $quality) )) ) (: return empty-sequence to let MLCP know it doesn't need to insert docs itself :) return () }; Cheers, Geert From: <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of Geert Josten <geert.jos...@marklogic.com<mailto:geert.jos...@marklogic.com>> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Date: Saturday, February 24, 2018 at 4:49 PM To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Subject: Re: [MarkLogic Dev General] Importing temporal documents with MLCP Hi Hans, Sorry for being late with this reply, hopefully it is still useful to you. This was a non-trivial question though, so I had to poke around in the docs to verify various things.. I think it is possible with MLCP. You can override the document-insert mechanism of MLCP using an MLCP transform. It would roughly look like: xquery version "1.0-ml"; module namespace example = "http://marklogic.com/example";; import module namespace temporal = "http://marklogic.com/xdmp/temporal"; at "/MarkLogic/temporal.xqy"; declare function example:transform( $content as map:map, $context as map:map ) as map:map* { (: grab uri and content from $content :) let $uri := map:get($content, "uri") let $doc := map:get($content, "value") (: grab other argument values passed in via cmd-line from $context :) let $transform_param := map:get($context, "transform_param") let $collections := map:get($context, "collections") let $permissions := map:get($context, "permissions") let $quality := map:get($context, "quality") let $temporalCollection := map:get($context, "temporalCollection") (: determine the start/end you want to apply :) let $systemStart := current-dateTime() (: or an expression to grab the value from $doc or $transform_param :) let $systemEnd := $temporal:MAX_TIME (: or an expression to grab the value from $doc or $transform_param :) (: do a temporal insert as desired :) let $_ := temporal:document-insert( $temporalCollection, $uri, $doc, map:new(( map:entry("collections", $collections), map:entry("permissions", $permissions), map:entry("quality", $quality), map:entry("metadata", map:new(( map:entry("systemStart", $systemStart), map:entry("systemEnd", $systemEnd) ))) )) ) (: return empty-sequence to let MLCP know it doesn't need to insert docs itself :) return () }; Note: I haven’t actually tested above code, but I have overridden doc-insert in an MLCP transform before, and am pretty sure the temporal-insert should work like this too. Regarding alternatives, you could take a look at doing REST calls to /v1/documents (multipart if possible), or at using DMSDK for large volume bulk loading with transforms. Cheers, Geert From: <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of Hans Hübner <hans.hueb...@lambdawerk.com<mailto:hans.hueb...@lambdawerk.com>> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Date: Friday, February 9, 2018 at 6:27 AM To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] Importing temporal documents with MLCP Hello, when importing documents into a temporal collection with MLCP, is there a way to override the system time stamp? We need to import large amounts of historic data into a database and want to set the system time to the time when those documents were created. We anticipate needing the valid axis as well, so we do not want to abuse the valid axis to represent the document creation time. If MLCP cannot do it, what would be a good alternative to bulk insert documents with a specified system timestamp? Any pointers would be appreciated. Thanks, Hans -- LambdaWerk GmbH Oranienburger Straße 87/89 10178 Berlin Phone: +49 30 555 7335 0 Fax: +49 30 555 7335 99 HRB 169991 B Amtsgericht Charlottenburg USt-ID: DE301399951 Geschäftsführer: Hans Hübner http://lambdawerk.com/
_______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
