Github user afs commented on the issue:
https://github.com/apache/jena/pull/233
@ajs6f, have you run this?
Andy wrote:
> I'm looking for 2 things : what informs the core framework and what are
new capabilities for systems/modules.
> This started with our discussions about binaries and binary protocols so
that would be a good starting place.
@dick-twocows Sorry I haven't commented on this - it's big.
I understood this to be for discussion, not contribution directly at the
moment. My comments/questions above still stand.
What is "mosaic"? What is "mirage"?
At a guess, "thrift" is thrift machinery, with it's own encoding and the
paging. Is paging necessary? What about buffered streaming?
"mosaic" is the dataset machinery over thrift (but "thrift" has at least
one dependency on "mosaic").
What's "mirage"'s role?
"spark" (and the dependencies on spark and Elephas in the POM) is not used.
There are quite a lot of TODO items :-)
Can we converge on one Thrift/binary protocol? At least reuse the current
RDF encoding.
I haven't seen anything yet that needs to be in jena-arq. It would be
better to have more modules, not include code in the core (in not the jena-core
sense) system.
For resource and practical reasons, the project can't take on a large,
complex system that it can't provide long term support, maintenance and
enhancement. Hence what are the requirements for changes to jena-arq etc to
support these (it's more than one thing?) either as independent projects, with
their own lifecycle, or as separte modules?
General comment : TDB2 has a proper, independent 2-phase commit which will
make distributed transactions easier. I'm not seeing a prepare phase in Mosaic.
@ajs6f see `StreamRDFTriHexTable`.
@dick-twocows Does it make a difference to stream to the internal
data-structures rather than the `DatasetGraph` interface?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
