Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
I see a couple problems here.
Your trail.xml doesn't have a root node, and so it isn't valid XML.
--
xdmp:node-delete — from the
documentation<http://docs.marklogic.com/xdmp:node-delete>:
Deletes a node from the database. On-the-fly constructed nodes cannot be
deleted.
With an MLCP transform, you're working with an in-memory node. All your case
"element(old_location)" section needs is "return ()".
--
Your typeswitch will need to handle the root node, once you have one.
--
Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel>
Technical Community Manager
MarkLogic Corporation<http://www.marklogic.com/>
http://developer.marklogic.com/
From: "Khan, Zishan"
mailto:zishan.k...@capgemini.com>>
Date: Wednesday, August 31, 2016 at 12:00 PM
To: Dave Cassel mailto:dave.cas...@marklogic.com>>
Subject: RE: [MarkLogic Dev General] #CGO#How to ingest data of selected
columns from CSV using MLCP & how to define and use primary key to see the log
data (failed data in terms of Bad file)
Hi Dave,
I used the transform function as
xquery version "1.0-ml";
module namespace example = "http://marklogic.com/example";;
declare function local:change($node)
{
typeswitch($node)
case element(old_location) return
xdmp:node-delete($node as node()) as empty-sequence()
default return $node
};
Wanted to upload the file trial.xml as
zishanBangaloreHyderabad
Here I want to skip
Runne mlcp and got below error.. please look at this and help me in resolving
the error…
C:\Users\zikhan>mlcp import -mode local -host localhost -port 8055 -username adm
in -password admin -input_file_path D:\Users\zikhan\Desktop\trial_fol -transform
_module /example/mlcp-trialtransform.xqy -transform_namespace "http://marklogic.
com/example"
16/08/31 21:24:09 INFO contentpump.LocalJobRunner: Content type is set to MIXED.
The format of the inserted documents will be determined by the MIME type spe
cification configured on MarkLogic Server.
16/08/31 21:24:10 INFO contentpump.FileAndDirectoryInputFormat: Total input path
s to process : 1
16/08/31 21:24:11 ERROR mapreduce.ContentWriter: XDMP-PARSE: (err:XPST0003) xdmp
:unquote("zishanBangaloremailto:dave.cas...@marklogic.com]
Sent: Wednesday, August 31, 2016 7:39 PM
To: Khan, Zishan; MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected
columns from CSV using MLCP & how to define and use primary key to see the log
data (failed data in terms of Bad file)
The code for your MLCP transform will receive an XML document. Your transform
code will then do a recursive descent transformation (see link I provided
earlier). When it gets to a field you don't want to include, return the empty
sequence.
--
Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel>
Technical Community Manager
MarkLogic Corporation<http://www.marklogic.com/>
http://developer.marklogic.com/
From: "Khan, Zishan"
mailto:zishan.k...@capgemini.com>>
Date: Tuesday, August 30, 2016 at 7:46 AM
To: MarkLogic Developer Discussion
mailto:general@developer.marklogic.com>>, Dave
Cassel mailto:dave.cas...@marklogic.com>>
Subject: RE: [MarkLogic Dev General] #CGO#How to ingest data of selected
columns from CSV using MLCP & how to define and use primary key to see the log
data (failed data in terms of Bad file)
Thanks Dave,
I got the ideas how to use transformation modules but I didn’t find any modules
related to skipping the columns as in the MarkLogic material it was only a
brief concept to use transformation on input data.
Please guide me by putting your valuable inputs so that I could stair up step
by step on MarkLogic ladder.
Your efforts would be priceless and much appreciable, hope your input will work
as a backbone for my solution.
Regards,
Zishan Khan / Capgemini
Associate Consultant | FSGBU | Insight and Data
From:general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy Taylor
Sent: Tuesday, August 30, 2016 4:04 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected
columns from CSV using MLCP & how to define and use primary key to see the log
data (failed data in terms of Bad file)
Hey Dave,
Tim Taylor from the alliances team here. Subscribed from my personal email.
Any thoughts on whether an mlcp transform on the inbound side versus load as
is and run a CORB job to clean up afterwards would perform better?
Tim
Sent from my iPhone
On Aug 29, 2016, at 2:10 PM, Dave Cassel
mailto:dave.cas...@marklogic.com>> wrote:
You can write an MLCP
transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should
get
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
Another benefit to staging and then transforming in two separate steps is that your transformation might require data that hasn't yet been loaded, for example to denormalize values from related documents that may or may not have already arrived. Justin -- Justin Makeig Director, Product Management MarkLogic jmak...@marklogic.com<mailto:jmak...@marklogic.com> On Aug 31, 2016, at 7:06 AM, Dave Cassel mailto:dave.cas...@marklogic.com>> wrote: Hi Tim, If you know at ingest time how you want to transform data, then from a performance point of view I think it's better to do it with the MLCP transform. Doing so means writing each fragment just once. There are some trade-offs: Advantages of an mlcp transform: * the data only need to be written once, instead of written and then updated. Doing the latter will result in deleted fragments, requiring merges to clean up. * Once the data are in the database, they are fully ready for use — no need to segment freshly loaded data from data that is ready for use Advantages of load-as-is followed by a CORB job: * works if you don't know how you want to format the data — load, play, revise, repeat * protection against errors in the transform: if your MLCP has a fatal error that affects only some documents, the whole batch will fail to get inserted. Good error handling can prevent this, but you may still need to account for not-fully-transformed documents. Dave. -- Dave Cassel<http://davidcassel.net/>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of Timothy Taylor mailto:timmy...@gmail.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 6:34 PM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hey Dave, Tim Taylor from the alliances team here. Subscribed from my personal email. Any thoughts on whether an mlcp transform on the inbound side versus load as is and run a CORB job to clean up afterwards would perform better? Tim Sent from my iPhone On Aug 29, 2016, at 2:10 PM, Dave Cassel mailto:dave.cas...@marklogic.com>> wrote: You can write an MLCP transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should get the individual XML documents as input and your output can structure them however you want. This post on recursive descent<http://developer.marklogic.com/blog/xquery-recursive-descent> should help, too — you'll use that in your transform. -- Dave Cassel<http://davidcassel.net/>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 8:12 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
Hi Tim, If you know at ingest time how you want to transform data, then from a performance point of view I think it's better to do it with the MLCP transform. Doing so means writing each fragment just once. There are some trade-offs: Advantages of an mlcp transform: * the data only need to be written once, instead of written and then updated. Doing the latter will result in deleted fragments, requiring merges to clean up. * Once the data are in the database, they are fully ready for use — no need to segment freshly loaded data from data that is ready for use Advantages of load-as-is followed by a CORB job: * works if you don't know how you want to format the data — load, play, revise, repeat * protection against errors in the transform: if your MLCP has a fatal error that affects only some documents, the whole batch will fail to get inserted. Good error handling can prevent this, but you may still need to account for not-fully-transformed documents. Dave. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of Timothy Taylor mailto:timmy...@gmail.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 6:34 PM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hey Dave, Tim Taylor from the alliances team here. Subscribed from my personal email. Any thoughts on whether an mlcp transform on the inbound side versus load as is and run a CORB job to clean up afterwards would perform better? Tim Sent from my iPhone On Aug 29, 2016, at 2:10 PM, Dave Cassel mailto:dave.cas...@marklogic.com>> wrote: You can write an MLCP transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should get the individual XML documents as input and your output can structure them however you want. This post on recursive descent<http://developer.marklogic.com/blog/xquery-recursive-descent> should help, too — you'll use that in your transform. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 8:12 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation dependent 1 ABC X2 2 DEF Y1 (output is in default XML format, here this is just to understand) 3 GHI X0 4 ABC Z2 Q.2) Suppose input data corresponding to id = 4 is failed. Failed data : 4 ABC
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
The code for your MLCP transform will receive an XML document. Your transform code will then do a recursive descent transformation (see link I provided earlier). When it gets to a field you don't want to include, return the empty sequence. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Date: Tuesday, August 30, 2016 at 7:46 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>>, Dave Cassel mailto:dave.cas...@marklogic.com>> Subject: RE: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Thanks Dave, I got the ideas how to use transformation modules but I didn’t find any modules related to skipping the columns as in the MarkLogic material it was only a brief concept to use transformation on input data. Please guide me by putting your valuable inputs so that I could stair up step by step on MarkLogic ladder. Your efforts would be priceless and much appreciable, hope your input will work as a backbone for my solution. Regards, Zishan Khan / Capgemini Associate Consultant | FSGBU | Insight and Data From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy Taylor Sent: Tuesday, August 30, 2016 4:04 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hey Dave, Tim Taylor from the alliances team here. Subscribed from my personal email. Any thoughts on whether an mlcp transform on the inbound side versus load as is and run a CORB job to clean up afterwards would perform better? Tim Sent from my iPhone On Aug 29, 2016, at 2:10 PM, Dave Cassel mailto:dave.cas...@marklogic.com>> wrote: You can write an MLCP transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should get the individual XML documents as input and your output can structure them however you want. This post on recursive descent<http://developer.marklogic.com/blog/xquery-recursive-descent> should help, too — you'll use that in your transform. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 8:12 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation dependent 1 ABC X2 2 DEF Y1 (output is in default XML format, here this is just to understand) 3 GHI X0 4 ABC
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
Thanks Dave, I got the ideas how to use transformation modules but I didn’t find any modules related to skipping the columns as in the MarkLogic material it was only a brief concept to use transformation on input data. Please guide me by putting your valuable inputs so that I could stair up step by step on MarkLogic ladder. Your efforts would be priceless and much appreciable, hope your input will work as a backbone for my solution. Regards, Zishan Khan / Capgemini Associate Consultant | FSGBU | Insight and Data From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy Taylor Sent: Tuesday, August 30, 2016 4:04 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hey Dave, Tim Taylor from the alliances team here. Subscribed from my personal email. Any thoughts on whether an mlcp transform on the inbound side versus load as is and run a CORB job to clean up afterwards would perform better? Tim Sent from my iPhone On Aug 29, 2016, at 2:10 PM, Dave Cassel mailto:dave.cas...@marklogic.com>> wrote: You can write an MLCP transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should get the individual XML documents as input and your output can structure them however you want. This post on recursive descent<http://developer.marklogic.com/blog/xquery-recursive-descent> should help, too — you'll use that in your transform. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 8:12 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation dependent 1 ABC X2 2 DEF Y1 (output is in default XML format, here this is just to understand) 3 GHI X0 4 ABC Z2 Q.2) Suppose input data corresponding to id = 4 is failed. Failed data : 4 ABC Z2 (say) How to see this data using id as primary key (Even I don’t know how to mention primary key in MarkLogic) using MLCP or any other means. Your lead would be much more appreciated if I will get the solution of respected solution with required supporting stuffs. Thanks & Regards, Zishan Khan / Capgemini Associate Consultant | FSGBU | Insight and Data This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this messag
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
Hey Dave, Tim Taylor from the alliances team here. Subscribed from my personal email. Any thoughts on whether an mlcp transform on the inbound side versus load as is and run a CORB job to clean up afterwards would perform better? Tim Sent from my iPhone > On Aug 29, 2016, at 2:10 PM, Dave Cassel wrote: > > You can write an MLCP transform. That should get the individual XML documents > as input and your output can structure them however you want. This post on > recursive descent should help, too — you'll use that in your transform. > > -- > Dave Cassel, @dmcassel > Technical Community Manager > MarkLogic Corporation > http://developer.marklogic.com/ > > > From: on behalf of "Khan, Zishan" > > Reply-To: MarkLogic Developer Discussion > Date: Monday, August 29, 2016 at 8:12 AM > To: MarkLogic Developer Discussion > Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns > from CSV using MLCP & how to define and use primary key to see the log data > (failed data in terms of Bad file) > > Hi Folk, > > Being a newbie in MarkLogic, I need your serious response to come out of > faced challenges. > The moment when I was ingesting CSV Structured data into MarkLogic as-is to > get XML output data in our database using MLCP, I am ok with normal ingestion > using MLCP in any file format but I got stuck in finding the solution of > below mentioned problem > > 1. How to ingest data corresponding to selected columns only in > MarkLogic using MLCP or by any means. > 2. How to define and use Primary key to check the logs ( say failed data > ). > > For the ease of understanding I am exploring my doubts with below example. > > Input : > > idemp_name salarydesignation mobile_no > dependent > > 1 ABC 3000 X > 442 > 2 DEF 4000 Y > 221 (input is in CSV format) > 3 GHI 3000 X > 330 > 4 ABC 8000 Z > 992 > > Q.1) > > Output : > > idemp_name designation dependent > > 1 ABC X2 > > 2 DEF Y1 > (output is in default XML > format, here this is just to understand) > 3 GHI X0 > 4 ABC Z2 > > Q.2) > > Suppose input data corresponding to id = 4 is failed. > > Failed data : > > 4 ABC Z2 > (say) > > How to see this data using id as primary key (Even I don’t know how to > mention primary key in MarkLogic) using MLCP or any other means. > > Your lead would be much more appreciated if I will get the solution of > respected solution with required supporting stuffs. > > > Thanks & Regards, > Zishan Khan / Capgemini > Associate Consultant | FSGBU | Insight and Data > This message contains information that may be privileged or confidential and > is the property of the Capgemini Group. It is intended only for the person to > whom it is addressed. If you are not the intended recipient, you are not > authorized to read, print, retain, copy, disseminate, distribute, or use this > message or any part thereof. If you receive this message in error, please > notify the sender immediately and delete all copies of this message. > > ___ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
You can write an MLCP transform<http://docs.marklogic.com/guide/mlcp/import#id_82518>. That should get the individual XML documents as input and your output can structure them however you want. This post on recursive descent<http://developer.marklogic.com/blog/xquery-recursive-descent> should help, too — you'll use that in your transform. -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: mailto:general-boun...@developer.marklogic.com>> on behalf of "Khan, Zishan" mailto:zishan.k...@capgemini.com>> Reply-To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Date: Monday, August 29, 2016 at 8:12 AM To: MarkLogic Developer Discussion mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file) Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation dependent 1 ABC X2 2 DEF Y1 (output is in default XML format, here this is just to understand) 3 GHI X0 4 ABC Z2 Q.2) Suppose input data corresponding to id = 4 is failed. Failed data : 4 ABC Z2 (say) How to see this data using id as primary key (Even I don’t know how to mention primary key in MarkLogic) using MLCP or any other means. Your lead would be much more appreciated if I will get the solution of respected solution with required supporting stuffs. Thanks & Regards, Zishan Khan / Capgemini Associate Consultant | FSGBU | Insight and Data This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] #CGO#How to ingest data of selected columns from CSV using MLCP & how to define and use primary key to see the log data (failed data in terms of Bad file)
Hi Folk, Being a newbie in MarkLogic, I need your serious response to come out of faced challenges. The moment when I was ingesting CSV Structured data into MarkLogic as-is to get XML output data in our database using MLCP, I am ok with normal ingestion using MLCP in any file format but I got stuck in finding the solution of below mentioned problem 1. How to ingest data corresponding to selected columns only in MarkLogic using MLCP or by any means. 2. How to define and use Primary key to check the logs ( say failed data ). For the ease of understanding I am exploring my doubts with below example. Input : idemp_name salarydesignation mobile_no dependent 1 ABC 3000 X 442 2 DEF 4000 Y 221 (input is in CSV format) 3 GHI 3000 X 330 4 ABC 8000 Z 992 Q.1) Output : idemp_name designation dependent 1 ABC X2 2 DEF Y1 (output is in default XML format, here this is just to understand) 3 GHI X0 4 ABC Z2 Q.2) Suppose input data corresponding to id = 4 is failed. Failed data : 4 ABC Z2 (say) How to see this data using id as primary key (Even I don't know how to mention primary key in MarkLogic) using MLCP or any other means. Your lead would be much more appreciated if I will get the solution of respected solution with required supporting stuffs. Thanks & Regards, Zishan Khan / Capgemini Associate Consultant | FSGBU | Insight and Data This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
