On 13/06/11 16:55, Paolo Castagna wrote:
Andy Seaborne wrote:
On 26/05/11 15:37, Laurent Pellegrino wrote:
Hi all,
I am using FmtUtils.stringForNode(...) from ARQ to encode a Node to a
String. Now, I have to perform the reverse operation: from the String
I want to create the Node. Is there a class and method to do that from
the ARQ library?
It seems that NodecLib.decode(...) do the trick but it is in the TDB
library and I am not sure that it works with any output from
FmtUtils.stringForNode(...)?
Kind Regards,
Laurent
There are ways to reverse the process - too many in fact.
Simple: SSE.parseNode: String -> Node
It uses a javacc parser so the overall efficiency isn't ideal.
But RIOT is in the process of reworking I/O for efficiency; the input
side is the area that is most finished. The tokenizer will do what you
want.
What's missing in RIOT is Node to stream writing without using
FmtUtils -- this is OutputLangUtils which is unfinished. FmtUtils
creates intermediate strings, when the output could be straight to a
stream, avoiding a copy and the temporary object allocation.
The Tokenizer is:
interface Tokenizer extends Iterator<Token>
and see org.openjena.riot.tokens.TokenizerFactory
especially if you have a sequence of them to parse ... like a TSV
file. But you will have to manage newlines as to the tokenizer they
are whitespace like anything else.
There is some stuff in my scratch area for streams of tuples of RDF
terms and variables:
https://svn.apache.org/repos/asf/incubator/jena/Scratch/AFS/trunk/src/riot/io/
TokenInputStream and TokenOutputStream might be useful.
Until TSV, a tuple of terms is a number of RDF terms, terminated by a
DOT (not newline).
This could be useful to JENA-44, JENA-45 and JENA-69
Hi,
I am looking at the code to serialize bindings (in relation to JENA-44
and JENA-45) and I would like to use as much as I can what is already
available in RIOT (and/or help to add what's missing, once I understand
what is the right thing to do).
I am having a few problems with blank nodes.
This is a snipped of code which explains my problem:
1 Node node1 = Node_Blank.createAnon();
2 String str = NodeFmtLib.serialize(node1);
3 Tokenizer tokenizer = TokenizerFactory.makeTokenizerString(str);
4 Token token = tokenizer.next();
5 Node node2 = token.asNode();
6 assertEquals(node1, node2);
I have two different problems.
In the case the blank node id starts with a digit, the assertion at
line 6 fails with, for example: "expected:<1c7b85b4:13089a0cb42:-7fff>
but was:<1c7b85b4>".
If the blank node id is a negative number (i.e. it starts with a '-'),
I have a RiotParserException: "org.openjena.riot.RiotParseException:
[line: 1, col: 3 ] Blank node label does not start with alphabetic or
_ :-" from TokenizerText.java line 1067.
Setting onlySafeBNodeLabels to true might help.
Because TDB does not use the tokenizer for decode, the raw path may be
buggy.
See OutputLangUtils because that has the prospect of streaming.
We may need to switch OutputStream but there is OutStreamUTF8 if UTF-8
encoding by std Java is costly.
What I am trying to do is to rewrite the BindingSerializer in the patch
for JENA-44. These are the signatures of the two methods I am implementing:
public void serialize(Binding b, DataOutputStream out) throws IOException
public Binding deserialize(DataInputStream in) throws IOException
What's wrong with TokenOutputStream which even does some buffering.
Binding -> Nodes (you're only writing the RDF term values), beware of
missingbindings. See the TSV output format that Laurent has been
looking at.
DataOutputStream can only write 16bit lengths for strings - so you use
write(byte[]) and much of the point of DataOutputStream is lost. Seems
better to be to use our own internal interface and map to whatever
mechanism is most appropriate. testing the round-tripping between
TokenOutputStream and TokenInputStream being then done.
At the moment, I am assuming all the bindings written in the same file have
the same variables and I am writing them only once at the beginning of the
file and after that I am serializing binding values only:
for (Var var : vars) {
Node node = b.get(var);
byte[] buf = NodeFmtLib.serialize(node).getBytes("UTF-8");
whether this is faster that converting to UTF-8 duirectly into the
stream will need testing but it's a point optimization. For now, it's
the design that matters.
out.writeInt(buf.length);
out.write(buf);
}
Should I try to use OutputLangUtils instead? And Writer(s) instead of
DataOutputStream(s)?
Thanks,
Paolo
I'm keen that we create a single solid I/O layer so it can teste and
optimized then shared amongst all the code doing I/O related things.
Nodec is an interface specializes attempt to ByteBuffers for file, not
stream I/O. File I/O can be random access.
Andy
