Damon, I am porting an application which makes a heavy use of map-reduce to MarkLogic. My blockages at this moment: 1. How to reproduce map-reduce in MarkLogic? I might take a different route to process in a functional manner and can avoid map-reduce altogether. 2. How to create multiple worker threads to categorize data into multiple categories at the same time? 3. Find out how many such workers I can have running concurrently.
Thanks, Ankur On Jul 5, 2011, at 2:37 PM, Damon Feldman wrote: > Ankur, > > MarkLogic implements some big data algorithms transparently, in that if you > call some functions or use XPath etc. it will transparently divide > data-oriented work among the Nodes in a cluster and re-combine the results > into a single final result. > > Other bulk data operations can be performed using MarkLogic-specific tools > such as CoRB or CPF, and a Hadoop connector has been discussed for an > upcoming release of MarkLogic. > > Can you be more specific about the nature of the algorithm you want to run, > and what the specific big data challenges are with your algorithms? > > Yours, > Damon > ________________________________________ > From: [email protected] > [[email protected]] On Behalf Of Ankur Patwa > [[email protected]] > Sent: Monday, July 04, 2011 12:54 PM > To: [email protected] > Subject: [MarkLogic Dev General] Big Data Strategies > > All, > > What is the best way to implement big data algorithms with xQuery and > MarkLogic? > > Can you please guide me to some resource on map-reduce/pregel using MarkLogic? > > Regards, > Ankur > > > NOTICE OF CONFIDENTIALITY: This electronic message, including attachments, is > for the sole use of the named recipient and may contain confidential or > privileged information protected by State of Tennessee and Federal > regulations. Any unauthorized review, use, disclosure, copying or > distribution is strictly prohibited. If you are not the intended recipient > or have received this communication in error please contact the sender or > email [email protected] and destroy all copies of the original message. > Thank you. > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general NOTICE OF CONFIDENTIALITY: This electronic message, including attachments, is for the sole use of the named recipient and may contain confidential or privileged information protected by State of Tennessee and Federal regulations. Any unauthorized review, use, disclosure, copying or distribution is strictly prohibited. If you are not the intended recipient or have received this communication in error please contact the sender or email [email protected] and destroy all copies of the original message. Thank you. _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
