Hi Markus, I set my keep-alive timeout to 0 seconds - coz' I've read somewhere in the documentation to set it to zero when I've been experiencing this error 'Error parsing HTTP headers: Premature EOF, partial header line read: '' before. Now, this error doesn't appear anymore, but there's that 'Session has been closed' error.
Thanks, Matt On Fri, Oct 16, 2015 at 9:33 AM, <[email protected]> wrote: > Send General mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://developer.marklogic.com/mailman/listinfo/general > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of General digest..." > > > Today's Topics: > > 1. Re: MLCP content transformation between database during > ingestion not working (Markus Flatscher) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 15 Oct 2015 21:33:57 -0400 > From: Markus Flatscher <[email protected]> > Subject: Re: [MarkLogic Dev General] MLCP content transformation > between database during ingestion not working > To: MarkLogic Developer Discussion <[email protected]> > Message-ID: > <CAGgfSV4SRwo3= > [email protected]> > Content-Type: text/plain; charset="utf-8" > > Matet, > > what is the keep-alive setting on the XDBC app server you're pointed at? > > Markus > > On Thu, Oct 15, 2015 at 9:15 PM, Matet Layosa <[email protected]> wrote: > > > Hi Geert, > > > > Yes, I execute my transform module installation in my -output_host > server. > > > > I've also finally made the transformation code running with my expected > > result. But there's still one thing that I notice during ingestion - > there > > are error messages such like: > > ERROR contentpump.LocalJobRunner: Error committing task: > > java.lang.IllegalStateException: Session has been closed > > > > and also: > > > > Cannot obtain connection: Cannot assign requested address > > > > I'm not sure if it is a problem regarding my network connection to server > > or if I have misconfiguration on Admin Server > > > > Thanks, > > Matt > > > > > > On Thu, Oct 15, 2015 at 6:08 PM, < > [email protected]> > > wrote: > > > >> Send General mailing list submissions to > >> [email protected] > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> http://developer.marklogic.com/mailman/listinfo/general > >> or, via email, send a message with subject or body 'help' to > >> [email protected] > >> > >> You can reach the person managing the list at > >> [email protected] > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of General digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Re: MLCP content transformation between database during > >> ingestion not working (Geert Josten) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Thu, 15 Oct 2015 10:08:33 +0000 > >> From: Geert Josten <[email protected]> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation > >> between database during ingestion not working > >> To: MarkLogic Developer Discussion <[email protected]> > >> Message-ID: <d2454716.b2622%[email protected]> > >> Content-Type: text/plain; charset="us-ascii" > >> > >> Hi Matt, > >> > >> Sorry, I mentioned `-copy_collections` before, but I really meant > >> `-collection_filter` instead. You are correct about a,b,c,d. Not sure > what > >> you mean exactly with e, but the transform should be loaded in the > modules > >> database that is associated with the output host:port app-server. And > it is > >> hence evaluated on that host.. > >> > >> Cheers, > >> Geert > >> > >> From: <[email protected]<mailto: > >> [email protected]>> on behalf of Matet Layosa < > >> [email protected]<mailto:[email protected]>> > >> Reply-To: MarkLogic Developer Discussion < > [email protected] > >> <mailto:[email protected]>> > >> Date: Thursday, October 15, 2015 at 3:49 AM > >> To: "[email protected]<mailto: > >> [email protected]>" <[email protected] > >> <mailto:[email protected]>> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation between > >> database during ingestion not working > >> > >> Hi Geert, > >> > >> Okay, now things seem to make sense for me here with how transform > works. > >> But I'd like to clarify things on mlcp options. This is what I think I > >> understand base on the documentation: > >> > >> a.) -copy_collections = its default value is "true" so I did not need > >> to specify it. > >> b.) -collection_filter = I used this to exports only documents in the > >> specified collection/s > >> c.) -output_host/username/password = it is the server details where > >> the output result of the transform will be save > >> d.) -input_host/username/password = server details as a source of > >> documents to copy > >> > >> e.) while I run the custom transform load into the output database in my > >> query console. > >> > >> *I'm not sure with the last part (e.) if that's really where it must be > >> loaded :) > >> > >> Anyway, I'll modify the code for transformation just as what you > suggest. > >> :) > >> > >> > >> Thanks! > >> Matt > >> > >> > >> Maria Theresa R. Layosa > >> Advanced Science and Technology Institute > >> UP Technology Park Complex, C.P. Garcia Avenue, > >> Diliman, Quezon City, Philippines 1101 > >> Email: [email protected]<mailto:[email protected]> ; > >> [email protected]<mailto:[email protected]> ; > >> > >> On Wed, Oct 14, 2015 at 6:14 PM, < > [email protected] > >> <mailto:[email protected]>> wrote: > >> Send General mailing list submissions to > >> [email protected]<mailto: > >> [email protected]> > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> http://developer.marklogic.com/mailman/listinfo/general > >> or, via email, send a message with subject or body 'help' to > >> [email protected]<mailto: > >> [email protected]> > >> > >> You can reach the person managing the list at > >> [email protected]<mailto: > >> [email protected]> > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of General digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Re: MLCP content transformation between database during > >> ingestion not working (Geert Josten) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Wed, 14 Oct 2015 10:14:12 +0000 > >> From: Geert Josten <[email protected]<mailto: > >> [email protected]>> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation > >> between database during ingestion not working > >> To: MarkLogic Developer Discussion <[email protected] > >> <mailto:[email protected]>> > >> Message-ID: <d243f591.b1a3a%[email protected]<mailto: > >> d243f591.b1a3a%[email protected]>> > >> Content-Type: text/plain; charset="windows-1252" > >> > >> Hi Matt, > >> > >> MLCP transforms still work on a per-document basis. You can either > >> squeeze down the input by using -copy_collections, or add an if > statement > >> around the part of the transform function that does the actual job. > >> > >> I just realize that the transform code is executed against the > >> destination. That means you are trying to read collection(?fadperson?) > >> against the destination database. That will return an empty sequence if > >> your destination is empty, causing the entire transform to return empty > >> sequence as well. That explains why you were not seeing any result > document > >> at all. My suggestions still stand though, they should get you going.. > >> > >> Cheers > >> > >> From: <[email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>>> on behalf of Matet Layosa < > >> [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>>> > >> Reply-To: MarkLogic Developer Discussion < > [email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected] > >>> > >> Date: Wednesday, October 14, 2015 at 11:22 AM > >> To: "[email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>>" < > >> [email protected]<mailto:[email protected] > >> ><mailto:[email protected]<mailto: > >> [email protected]>>> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation between > >> database during ingestion not working > >> > >> Hi Geert, > >> > >> I'd like to give first a quick background on what is inside my 'loaded' > >> database. It only have 1 forest, has 8 collections inside the forest. > The > >> reason why I'm using collections here is because I have 2 collections on > >> the same namespace: "http://marklogic.com/ofis/fad"; . And for this > >> particular module, I just need to transform the documents in "fadperson" > >> collection. So I'm thinking if I only simply use the map:get($content, > >> "value"), it will read the whole database which will take up time to > load. > >> Or am I thinking the wrong way? > >> > >> > >> Thanks, > >> Matt > >> > >> > >> On Wed, Oct 14, 2015 at 5:08 PM, < > [email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>>> wrote: > >> Send General mailing list submissions to > >> [email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>> > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> http://developer.marklogic.com/mailman/listinfo/general > >> or, via email, send a message with subject or body 'help' to > >> [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>> > >> > >> You can reach the person managing the list at > >> [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>> > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of General digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Re: MLCP content transformation between database during > >> ingestion not working (Geert Josten) > >> 2. Re: MLCP content transformation between database during > >> ingestion not working (Matet Layosa) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Wed, 14 Oct 2015 08:01:45 +0000 > >> From: Geert Josten <[email protected]<mailto: > >> [email protected]><mailto:[email protected]<mailto: > >> [email protected]>>> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation > >> between database during ingestion not working > >> To: MarkLogic Developer Discussion <[email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected] > >>> > >> Message-ID: <d243d747.b1916%[email protected]<mailto: > >> d243d747.b1916%[email protected]><mailto: > >> d243d747.b1916%[email protected]<mailto: > >> d243d747.b1916%[email protected]>>> > >> Content-Type: text/plain; charset="windows-1252" > >> > >> Hi Matt, > >> > >> MLCP will already call the transform for each document separately, so > you > >> don?t need the for loops. Use something like let $p := map:get($content, > >> "value") instead. I also think you don?t really need to rewrite the uri, > >> but maybe you prefer different uri?s as well. > >> > >> About what is actually going wrong: for each document MLCP will be > >> processing fadperson docs 1 to 100, and updating the $content map:map > with > >> ultimately the uri and contents of only the last fadperson result. You > will > >> end up with just one new file in the final database if I read this > >> correctly.. > >> > >> Cheers, > >> Geert > >> > >> PS: collection() already returns documents, do you don?t need to use > >> doc(base-uri()) on its output either.. > >> > >> From: <[email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>><mailto: > >> [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>>>> on behalf of Matet Layosa < > >> [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>><mailto: > >> [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>>>> > >> Reply-To: MarkLogic Developer Discussion < > [email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected] > >> >><mailto:[email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>>>> > >> Date: Wednesday, October 14, 2015 at 9:01 AM > >> To: "[email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>><mailto: > >> [email protected]<mailto:[email protected] > >> ><mailto:[email protected]<mailto: > >> [email protected]>>>" <[email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected] > >> >><mailto:[email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>>>> > >> > >> Subject: [MarkLogic Dev General] MLCP content transformation between > >> database during ingestion not working > >> > >> Hi there! I am just new to Marklogic/Xquery and is currently working on > a > >> project where I need to transform the files from an existing database > with > >> millions of xml documents, change each files' document structure while > >> ingesting it to another database. > >> So far, I have this transform module: > >> > >> xquery version "1.0-ml"; > >> module namespace master-person = " > http://marklogic.com/ofis/masterperson > >> "; > >> declare namespace fad = "http://marklogic.com/ofis/fad";; > >> > >> declare variable $person := /fad:fadperson; > >> > >> declare function master-person:transform( > >> $content as map:map, > >> $context as map:map > >> ) as map:map* > >> { > >> for $docs in fn:collection("fadperson") [1 to 100] > >> for $p in fn:doc(fn:base-uri($docs))/fad:fadperson > >> (: let $map := map:map() :) > >> let $id := $p/fad:id/fn:string() > >> let $profile := > >> <person> > >> <id>{$p/fad:id/fn:string()}</id> > >> <name> > >> <firstname>{$p/fad:first_name/fn:string()}</firstname> > >> <lastname>{$p/fad:last_name/fn:string()}</lastname> > >> <middlename>{$p/fad:last_name/fn:string()}</middlename> > >> > >> <nameextension>{$p/fad:name_extension/fn:string()}</nameextension> > >> <othername>{$p/fad:other_name/fn:string()}</othername> > >> </name> > >> <gender>{$p/fad:gender/fn:string()}</gender> > >> <civilstatus>{$p/fad:civil_status/fn:string()}</civilstatus> > >> <birthdate>{$p/fad:birthdate/fn:string()}</birthdate> > >> <birthplace>{$p/fad:birthplace/fn:string()}</birthplace> > >> </person> > >> return > >> ( > >> map:put($content, "uri", fn:concat("/fadperson/", $id, ".xml")) > >> , > >> map:put($content, "value", document { $profile }) > >> , > >> $content > >> ) > >> > >> }; > >> > >> I also have installed this in my current XDBC server: > >> xquery version "1.0-ml"; > >> > >> > xdmp:document-load("/home/jema/Marklogic/apps/data-transform/transform-person.xqy", > >> <options xmlns="xdmp:document-load"> > >> <uri>/data/transform-person.xqy</uri> > >> <repair>none</repair> > >> <permissions>{xdmp:default-permissions()}</permissions> > >> </options>) > >> > >> Then I execute this on terminal: > >> mlcp.sh copy -mode local -input_host localhost -input_port 8004 > >> -input_username fad-admin -input_password f4d123 -output_host localhost > >> -output_port 8041 -output_username fad-admin -input_password f4d123 > >> -collection_filter 'fadperson' -transform_module > /data/transform-person.xqy > >> -transform_namespace "http://marklogic.com/ofis/masterperson"; > >> > >> the input_port: 8004 = contains the files for transformation; > >> output_port:8041 = where the transformed files must be ingested > >> > >> It doesn't output anything. Neither an error is showing up. It reached > >> 100% complete yet no files loaded to the database. > >> What am I missing here? Is my transform module code not creating a > >> document? > >> > >> > >> Thanks! > >> > >> Matt > >> [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>><mailto: > >> [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>>> > >> > >> > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> URL: > >> > http://developer.marklogic.com/pipermail/general/attachments/20151014/b4a27c6f/attachment-0001.html > >> > >> ------------------------------ > >> > >> Message: 2 > >> Date: Wed, 14 Oct 2015 17:08:51 +0800 > >> From: Matet Layosa <[email protected]<mailto:[email protected] > >> ><mailto:[email protected]<mailto:[email protected]>>> > >> Subject: Re: [MarkLogic Dev General] MLCP content transformation > >> between database during ingestion not working > >> To: [email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>> > >> Message-ID: > >> < > >> cadpmupsl-ns3clnkm5sffh1h+begsrunw60+uctwtvbkvet...@mail.gmail.com > >> <mailto: > >> cadpmupsl-ns3clnkm5sffh1h%2bbegsrunw60%[email protected] > >> ><mailto: > >> cadpmupsl-ns3clnkm5sffh1h%2bbegsrunw60%[email protected] > >> <mailto: > >> > cadpmupsl-ns3clnkm5sffh1h%252bbegsrunw60%[email protected] > >> >>> > >> Content-Type: text/plain; charset="utf-8" > >> > >> No problem. They're just a dummy username-password accounts. :) > >> > >> Hope to hear solution to this problem soon. I'm even thinking of using > the > >> REST-API transform service but it's not working also. > >> > >> Thanks, > >> Matt > >> > >> > >> On Wed, Oct 14, 2015 at 3:40 PM, < > [email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>>> > >> wrote: > >> > >> > Send General mailing list submissions to > >> > [email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>> > >> > > >> > To subscribe or unsubscribe via the World Wide Web, visit > >> > http://developer.marklogic.com/mailman/listinfo/general > >> > or, via email, send a message with subject or body 'help' to > >> > [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>> > >> > > >> > You can reach the person managing the list at > >> > [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>> > >> > > >> > When replying, please edit your Subject line so it is more specific > >> > than "Re: Contents of General digest..." > >> > > >> > > >> > Today's Topics: > >> > > >> > 1. MLCP content transformation between database during > >> ingestion > >> > not working (Matet Layosa) > >> > 2. Re: MLCP content transformation between database during > >> > ingestion not working (Jeff) > >> > > >> > > >> > ---------------------------------------------------------------------- > >> > > >> > Message: 1 > >> > Date: Wed, 14 Oct 2015 15:01:13 +0800 > >> > From: Matet Layosa <[email protected]<mailto:[email protected] > >> ><mailto:[email protected]<mailto:[email protected]>>> > >> > Subject: [MarkLogic Dev General] MLCP content transformation between > >> > database during ingestion not working > >> > To: [email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>> > >> > Message-ID: > >> > <CADpmUPt= > >> > [email protected]<mailto: > >> [email protected]><mailto: > >> [email protected]<mailto: > >> [email protected]>>> > >> > >> > Content-Type: text/plain; charset="utf-8" > >> > > >> > Hi there! I am just new to Marklogic/Xquery and is currently working > on > >> a > >> > project where I need to transform the files from an existing database > >> with > >> > millions of xml documents, change each files' document structure while > >> > ingesting it to another database. > >> > So far, I have this transform module: > >> > > >> > xquery version "1.0-ml"; > >> > > >> > module namespace master-person = " > >> http://marklogic.com/ofis/masterperson";; > >> > > >> > declare namespace fad = "http://marklogic.com/ofis/fad";; > >> > > >> > > >> > declare variable $person := /fad:fadperson; > >> > > >> > > >> > declare function master-person:transform( > >> > > >> > $content as map:map, > >> > > >> > $context as map:map > >> > > >> > ) as map:map* > >> > > >> > { > >> > > >> > for $docs in fn:collection("fadperson") [1 to 100] > >> > > >> > for $p in fn:doc(fn:base-uri($docs))/fad:fadperson > >> > > >> > (: let $map := map:map() :) > >> > > >> > let $id := $p/fad:id/fn:string() > >> > > >> > let $profile := > >> > > >> > <person> > >> > > >> > <id>{$p/fad:id/fn:string()}</id> > >> > > >> > <name> > >> > > >> > <firstname>{$p/fad:first_name/fn:string()}</firstname> > >> > > >> > <lastname>{$p/fad:last_name/fn:string()}</lastname> > >> > > >> > <middlename>{$p/fad:last_name/fn:string()}</middlename> > >> > > >> > > >> > <nameextension>{$p/fad:name_extension/fn:string()}</nameextension> > >> > > >> > <othername>{$p/fad:other_name/fn:string()}</othername> > >> > > >> > </name> > >> > > >> > > >> > <gender>{$p/fad:gender/fn:string()}</gender> > >> > > >> > > <civilstatus>{$p/fad:civil_status/fn:string()}</civilstatus> > >> > > >> > <birthdate>{$p/fad:birthdate/fn:string()}</birthdate> > >> > > >> > <birthplace>{$p/fad:birthplace/fn:string()}</birthplace> > >> > > >> > </person> > >> > > >> > return > >> > > >> > ( > >> > > >> > map:put($content, "uri", fn:concat("/fadperson/", $id, ".xml")) > >> > > >> > , > >> > > >> > map:put($content, "value", document { $profile }) > >> > > >> > , > >> > > >> > $content > >> > > >> > ) > >> > > >> > > >> > > >> > }; > >> > > >> > > >> > I also have installed this in my current XDBC server: > >> > > >> > xquery version "1.0-ml"; > >> > > >> > > >> > xdmp:document-load("/home/jema/Marklogic/apps/data-transform/transform-person.xqy", > >> > <options xmlns="xdmp:document-load"> > >> > <uri>/data/transform-person.xqy</uri> > >> > <repair>none</repair> > >> > <permissions>{xdmp:default-permissions()}</permissions> > >> > </options>) > >> > > >> > > >> > Then I execute this on terminal: > >> > mlcp.sh copy -mode local -input_host localhost -input_port 8004 > >> > -input_username fad-admin -input_password f4d123 -output_host > localhost > >> > -output_port 8041 -output_username fad-admin -input_password f4d123 > >> > -collection_filter > >> > 'fadperson' -transform_module /data/transform-person.xqy > >> > -transform_namespace "http://marklogic.com/ofis/masterperson"; > >> > > >> > the input_port: 8004 = contains the files for transformation; > >> > output_port:8041 = where the transformed files must be ingested > >> > > >> > It doesn't output anything. Neither an error is showing up. It reached > >> 100% > >> > complete yet no files loaded to the database. > >> > What am I missing here? Is my transform module code not creating a > >> > document? > >> > > >> > > >> > Thanks! > >> > > >> > Matt > >> > [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>> > >> > -------------- next part -------------- > >> > An HTML attachment was scrubbed... > >> > URL: > >> > > >> > http://developer.marklogic.com/pipermail/general/attachments/20151014/26dedb1f/attachment-0001.html > >> > > >> > ------------------------------ > >> > > >> > Message: 2 > >> > Date: Wed, 14 Oct 2015 15:40:36 +0800 > >> > From: Jeff <[email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>>> > >> > Subject: Re: [MarkLogic Dev General] MLCP content transformation > >> > between database during ingestion not working > >> > To: MarkLogic Developer Discussion <[email protected] > >> <mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected] > >>> > >> > Message-ID: > >> > < > >> > cajmd6t+xmt6ofseozfngoqj5xckavygxvk_hlzaxm29dt9z...@mail.gmail.com > >> <mailto: > >> cajmd6t%[email protected] > >> ><mailto: > >> cajmd6t%[email protected] > >> <mailto: > >> cajmd6t%[email protected] > >>> > >> > Content-Type: text/plain; charset="utf-8" > >> > > >> > Hi Matt, > >> > > >> > It might not be wise as practice to include username and password data > >> in > >> > your posts. Nevertheless, for sure there will be people around here > who > >> > could provide you with an answer to your question. :) > >> > > >> > Best regards, > >> > Jeff > >> > > >> > On Wed, Oct 14, 2015 at 3:01 PM, Matet Layosa <[email protected] > >> <mailto:[email protected]><mailto:[email protected]<mailto: > >> [email protected]>>> wrote: > >> > > >> > > Hi there! I am just new to Marklogic/Xquery and is currently working > >> on a > >> > > project where I need to transform the files from an existing > database > >> > with > >> > > millions of xml documents, change each files' document structure > while > >> > > ingesting it to another database. > >> > > So far, I have this transform module: > >> > > > >> > > xquery version "1.0-ml"; > >> > > > >> > > module namespace master-person = " > >> http://marklogic.com/ofis/masterperson > >> > "; > >> > > > >> > > declare namespace fad = "http://marklogic.com/ofis/fad";; > >> > > > >> > > > >> > > declare variable $person := /fad:fadperson; > >> > > > >> > > > >> > > declare function master-person:transform( > >> > > > >> > > $content as map:map, > >> > > > >> > > $context as map:map > >> > > > >> > > ) as map:map* > >> > > > >> > > { > >> > > > >> > > for $docs in fn:collection("fadperson") [1 to 100] > >> > > > >> > > for $p in fn:doc(fn:base-uri($docs))/fad:fadperson > >> > > > >> > > (: let $map := map:map() :) > >> > > > >> > > let $id := $p/fad:id/fn:string() > >> > > > >> > > let $profile := > >> > > > >> > > <person> > >> > > > >> > > <id>{$p/fad:id/fn:string()}</id> > >> > > > >> > > <name> > >> > > > >> > > <firstname>{$p/fad:first_name/fn:string()}</firstname> > >> > > > >> > > <lastname>{$p/fad:last_name/fn:string()}</lastname> > >> > > > >> > > > <middlename>{$p/fad:last_name/fn:string()}</middlename> > >> > > > >> > > > >> > > <nameextension>{$p/fad:name_extension/fn:string()}</nameextension> > >> > > > >> > > <othername>{$p/fad:other_name/fn:string()}</othername> > >> > > > >> > > </name> > >> > > > >> > > > >> > > <gender>{$p/fad:gender/fn:string()}</gender> > >> > > > >> > > > >> <civilstatus>{$p/fad:civil_status/fn:string()}</civilstatus> > >> > > > >> > > <birthdate>{$p/fad:birthdate/fn:string()}</birthdate> > >> > > > >> > > <birthplace>{$p/fad:birthplace/fn:string()}</birthplace> > >> > > > >> > > </person> > >> > > > >> > > return > >> > > > >> > > ( > >> > > > >> > > map:put($content, "uri", fn:concat("/fadperson/", $id, > ".xml")) > >> > > > >> > > , > >> > > > >> > > map:put($content, "value", document { $profile }) > >> > > > >> > > , > >> > > > >> > > $content > >> > > > >> > > ) > >> > > > >> > > > >> > > > >> > > }; > >> > > > >> > > > >> > > I also have installed this in my current XDBC server: > >> > > > >> > > xquery version "1.0-ml"; > >> > > > >> > > > >> > > >> > xdmp:document-load("/home/jema/Marklogic/apps/data-transform/transform-person.xqy", > >> > > <options xmlns="xdmp:document-load"> > >> > > <uri>/data/transform-person.xqy</uri> > >> > > <repair>none</repair> > >> > > <permissions>{xdmp:default-permissions()}</permissions> > >> > > </options>) > >> > > > >> > > > >> > > Then I execute this on terminal: > >> > > mlcp.sh copy -mode local -input_host localhost -input_port 8004 > >> > > -input_username fad-admin -input_password f4d123 -output_host > >> localhost > >> > > -output_port 8041 -output_username fad-admin -input_password f4d123 > >> > -collection_filter > >> > > 'fadperson' -transform_module /data/transform-person.xqy > >> > > -transform_namespace "http://marklogic.com/ofis/masterperson"; > >> > > > >> > > the input_port: 8004 = contains the files for transformation; > >> > > output_port:8041 = where the transformed files must be ingested > >> > > > >> > > It doesn't output anything. Neither an error is showing up. It > reached > >> > > 100% complete yet no files loaded to the database. > >> > > What am I missing here? Is my transform module code not creating a > >> > > document? > >> > > > >> > > > >> > > Thanks! > >> > > > >> > > Matt > >> > > [email protected]<mailto:[email protected]><mailto: > >> [email protected]<mailto:[email protected]>> > >> > > > >> > > > >> > > > >> > > _______________________________________________ > >> > > General mailing list > >> > > [email protected]<mailto: > >> [email protected]><mailto:[email protected] > >> <mailto:[email protected]>> > >> > > Manage your subscription at: > >> > > http://developer.marklogic.com/mailman/listinfo/general > >> > > > >> > > > >> > -------------- next part -------------- > >> > An HTML attachment was scrubbed... > >> > URL: > >> > > >> > http://developer.marklogic.com/pipermail/general/attachments/20151014/b71f538b/attachment.html > >> > > >> > ------------------------------ > >> > > >> > _______________________________________________ > >> > General mailing list > >> > [email protected]<mailto: > [email protected] > >> ><mailto:[email protected]<mailto: > >> [email protected]>> > >> > Manage your subscription at: > >> > http://developer.marklogic.com/mailman/listinfo/general > >> > > >> > > >> > End of General Digest, Vol 136, Issue 31 > >> > **************************************** > >> > > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> URL: > >> > http://developer.marklogic.com/pipermail/general/attachments/20151014/a7ef7d6a/attachment.html > >> > >> ------------------------------ > >> > >> _______________________________________________ > >> General mailing list > >> [email protected]<mailto:[email protected] > >> ><mailto:[email protected]<mailto: > >> [email protected]>> > >> Manage your subscription at: > >> http://developer.marklogic.com/mailman/listinfo/general > >> > >> > >> End of General Digest, Vol 136, Issue 32 > >> **************************************** > >> > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> URL: > >> > http://developer.marklogic.com/pipermail/general/attachments/20151014/d98426fb/attachment.html > >> > >> ------------------------------ > >> > >> _______________________________________________ > >> General mailing list > >> [email protected]<mailto:[email protected]> > >> Manage your subscription at: > >> http://developer.marklogic.com/mailman/listinfo/general > >> > >> > >> End of General Digest, Vol 136, Issue 35 > >> **************************************** > >> > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> URL: > >> > http://developer.marklogic.com/pipermail/general/attachments/20151015/c8a0e9a6/attachment.html > >> > >> ------------------------------ > >> > >> _______________________________________________ > >> General mailing list > >> [email protected] > >> Manage your subscription at: > >> http://developer.marklogic.com/mailman/listinfo/general > >> > >> > >> End of General Digest, Vol 136, Issue 38 > >> **************************************** > >> > > > > > > _______________________________________________ > > General mailing list > > [email protected] > > Manage your subscription at: > > http://developer.marklogic.com/mailman/listinfo/general > > > > > > > -- > Markus Flatscher > Consultant | Avalon Consulting, LLC > [email protected] > > LinkedIn: http://www.linkedin.com/company/avalon-consulting-llc > Google+: http://www.google.com/+AvalonConsultingLLC > Twitter: https://twitter.com/avalonconsult > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://developer.marklogic.com/pipermail/general/attachments/20151015/ec77c884/attachment.html > > ------------------------------ > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > > End of General Digest, Vol 136, Issue 42 > **************************************** >
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
