Andy Seaborne wrote:
On 10/05/11 12:07, Paolo Castagna wrote:
Hi Andy,
I reverted the changes. More details below.
Andy Seaborne wrote:
On 10/05/11 11:15, Paolo Castagna wrote:
Hi Andy
Andy Seaborne wrote:
Fuseki does not currently compile and some other things have changed
without warning in such a way a build would affect a user:
I would prefer that systems are always in a state whereby they can be
built and released unless they is good reason and notification.
Yes, I agree.
1/ Depends on a non-existent TDB 0.8.11-SNAPSHOT
Sorry, I didn't spot this immediately (since I had TDB 0.8.11-SNAPSHOT
in my
local Maven repository).
Will publishing a TDB 0.8.11-SNAPSHOT fix the problem?
But there are a mix of outstanding patches for ARQ and TDB. What state
is Fuseki in?
If Fuseki depends on a SNAPSHOT (either ARQ or TDB) is in the same state
as ARQ-x-SNAPSHOT or TDB-x-SNAPSHOT.
Which is? They have some sort of dependency on LARQ but I don't know
what it is.
SNAPSHOT == in development.
ARQ (includes the old/legacy LARQ) but it does not depend on LARQ.
LARQ depends on ARQ, instead. This is to avoid cyclic dependencies.
We use reflection to wire the new LARQ into ARQ (if present in the
classpath).
TDB does not depend on LARQ.
If someone wants to use the new LARQ (the one distributed as separate
module) they need to have their project depending on ARQ and on LARQ
(and, at the moment, they will need to exclude dependency on Lucene
v2.3.1 from ARQ).
To release it, it must not depend on SNAPSHOTs so, either a dependency
on a SNAPSHOT is decreased to the latest stable release or a release of
the dependent module is necessary first.
When is the next Fuseki release planned?
Bug fix releases are not "planned". It is, to me, rather important the
system is buildable and releasable at all times possible.
Yes. I agree.
People do use development builds out of SVN. We even ask people to test
thing by doing that.
Yes.
However, at release time Fuseki will need to depend only on non
SNAPSHOT artifacts though.
2/ Introduces a dependency on talis-oss-snapshots Maven repository.
I'll remove this one.
Is it possible to have a SNAPSHOT for LARQ on the Jena's repo-dev?
Yes - I have to do it because the file permissions are set up right
and I've never got round to fixing it.
It would be useful to have a LARQ SNAPSHOT on jena-dev, this way we can
try it or depend on it (from SNAPSHOT artifacts) without introducing
dependencies on other repositories.
I still need to check if it is possible (and how) for incubating projects
to use: https://repository.apache.org/content/repositories/snapshots/
All committers should be able to publish SNAPSHOTs or a CI systems
should/could do this for us (every commit or daily).
But this should not matter
It removes the dependency on talis-oss-snapshots Maven repository.
3/ Updating the dependencies does not get a working development
system.
Could you expand on this?
I updated Fuseki - it does not compile even after updating the
dependencies.
What was your problem?
Was it from Maven or from Eclipse?
It was compiling for me, both from the command line and Eclipse.
I was able to package a new Fuseki artifacts, run it and test LARQ was
functioning correctly. Indeed, this is what I did before committing
any change.
But, as I said, I had TDB 0.8.11-SNAPSHOT (and an updated
ARQ-2.8.9-SNAPSHOT)
in my local Maven repo and therefore I did not see the problem.
The artifact in the Talis repo is not called 0.8.11-SNAPSHOT.
We use the talis-snapshots repo to publish SNAPSHOTs with uncommitted
patches we want to test and allow/invite other people to test.
The artifacts there without a JIRA issue number and without SNAPSHOT
are not a good idea (and indeed a mistake, fixed now).
It also uses LARQ and seems to force Lucene 3 - isn't that a one-way
migration isn't it?
LARQ in ARQ depends on Lucene v2.3.1
LARQ as separate module depends on Lucene v3.1.0.
Yes - but what is the migration plan for LARQ?
Is the next release of ARQ going to have just separate LARQ or a
mixture of internal and external modules? (seems kind of complicated).
When does this happen?
To minimize problems and changes, I propose to release ARQ as it is
with the old legacy LARQ depending on Lucene 2.3.1 included.
However, I propose to commit the changes discussed in JENA-63 so that
it is possible and easy to enable and use LARQ (old or new from Fuseki
or other systems using TDB).
We could add the new LARQ (as separate module) to Fuseki.
In order to do this, Fuseki needs to depend on LARQ as well as on a
version of ARQ which includes the necessary changes to the
DatasetAssembler.
This will make easy for users to enable and use LARQ with Fuseki.
They'll just need to add a line to their assembler config file.
Does this seem reasonable to you?
Having LARQ is Fuseki is really valuable IMHO.
Using the new LARQ keeps Lucene indexes up-to-date with TDB and it avoids
duplicates. This, IMHO, is another really valuable thing for LARQ users.
Not disagreeing but there seems to be no migration plan or documentation.
If we leave the old/legacy LARQ in ARQ as it is, no migration plan is necessary.
People can decide which one to use.
Documentation: yes, necessary but there are no actual changes from an API or
use point of view. For developers, I'll document how to add a dependency on
LARQ artifacts if they want to use the new/separate module.
Why don't we just switch to LARQ now?
I'd prefer to add LARQ to Fuseki first, so that we can make it easier for
people to try it out, they can report us feedback and if it's ok we can
then remove the old LARQ from ARQ.
Leaving LARQ in ARQ minimize risk and changes and it offers a fall back
to people.
If it's not automatic (Lucene3 can read Lucen2 indexes?), then we need
to document how the user can do an upgrade.
The new LARQ adds a new field to the Lucene index (i.e. "hash") to support
unindex/deletes, see [1], therefore a reindex (using the usual larqbuild is
necessary).
Paolo
[1]
https://jena.svn.sourceforge.net/svnroot/jena/LARQ/trunk/src/main/java/org/apache/jena/larq/LARQ.java
Andy
Paolo
Some methods have been deprecated and then removed from Lucene v3.x,
therefore it is not possible to have a system using Lucene v3.x which
allows for a drop-in replacement if someone needs to an older version
of Lucene.
I can revert the changes, but having LARQ available in Fuseki is really
valuable for me (and I suspect many other Fuseki users).
See above on migration plan. Basically, what is going on? Suppose a
serious bug comes up in ARQ? TDB? Fuseki? can it be fixed and released
now?
If a serious bug comes up in ARQ, it can be fixed a a new bug fix release
can be published for ARQ. ARQ currently compiles and all the tests pass.
If a serious bug comes up in TDB, same as above. Fix => bug fix release.
TDB currently compiles and all tests pass.
For Fuseki, since it depends on ARQ and TDB, there are two different
situations:
1. Fuseki depends on ARQ or TDB SNAPSHOTs
2. Fuseki depends on ARQ or TDB latest releases
In case of 1. if Fuseki needs the changes in ARQ or TDB SNAPSHOTs, a
release
of Fuseki must be preceded by a release of ARQ or TDB.
Paolo
Andy
