The problem is that BulkNtripsInputTool actually tries to parse each line of input independently. This lets it handle large and arbitrarily split N-Triples files (it only needs one line at a time), but means it won't work with any input file that has structure spanning multiple lines (such as a Turtle namespace declaration that's used throughout the file).
You could try RdfFileInputTool (in the same package), which reads and parses entire files. This will only work if an individual file is small enough to be read in completely (though it can at least handle multiple files in parallel). The format can be specified using the "rdf.format" configuration value, e.g. "hadoop jar [...] -Drdf.format=Turtle", so there should be no need to modify the source. -Jesse -----Original Message----- From: Aaron D. Mihalik [mailto:[email protected]] Sent: Wednesday, June 22, 2016 10:52 AM To: [email protected] Subject: Re: MR bulk loading turtle files You should be all set. But be careful with file splits. You can't split a turtle file at an arbitrary line (like you can with an NTriple file). --Aaron On Wed, Jun 22, 2016 at 10:40 AM Ly, Kiet <[email protected]> wrote: > I look at the MR bulk loading tool, BulkNtripsInputTool, which support > N-Triples format. My guess I can clone BulkNtripsInputTool and change > format = RDFFormat.TURTLE. Are there anything else I need to change? > > Confidentiality Notice:: This email, including attachments, may > include non-public, proprietary, confidential or legally privileged > information. > If you are not an intended recipient or an authorized agent of an > intended recipient, you are hereby notified that any dissemination, > distribution or copying of the information contained in or transmitted > with this e-mail is unauthorized and strictly prohibited. If you have > received this email in error, please notify the sender by replying to > this message and permanently delete this e-mail, its attachments, and > any copies of it immediately. You should not retain, copy or use this > e-mail or any attachment for any purpose, nor disclose all or any part of the > contents to any other person. > Thank you. >
