All,
Am I doing things the best way?
I have a pile of data that I need to load into Rya. I must first convert
it into RDF, then do the load. I am using map/reduce because I have a lot
of data.
I have an hdfs directory full of RDF in NTRIPLE format.
I have a mapper like this:
protected void map(LongWritable key, RyaStatementWritable value, Context ctx)
{
// RyaStatementWritable gives me a RyaStatement like this:
RyaStatement ryaStatement = value.getRyaStatement();
// At this point I find myself having to convert the
// RyaStatement into an OpenRDF Statement like this:
Sail ryaSail = RyaSailFactory.getInstance(conf);
ValueFactory vf = ryaSail.getValueFactory();
Statement stmt = vf.createStatement(vf.createURI(sS), vf.createURI(sP), vf
.createURI(sO));
ctx.write(null, stmt);
}
In my reducer, I use AccumuloLoadStatements to lood Rya like this:
protected void reduce(NullWritable key, Iterable<Statement> stmts,
Reducer<NullWritable, Statement, NullWritable, NullWritable>.Context ctx)
throws IOException, InterruptedException {
super.reduce(key, stmts, ctx);
AccumuloLoadStatements load = ...omitted for brevity...
try {
load.loadStatements(instance, stmts);
} catch (RyaClientException e) {
log.error("", e);
}
}
Thanks
--
There are ways and there are ways,
Geoffry Roberts
