Hi All,
Modules
The modularizing the DSpace codebase proposal is going to require project
restructuring. The benefits to us all is that unused modules (in my case
LNI, SWORD, JSPUI) can be disabled/dropped from my view of the project
all-together. And then on the positives, new developments in DSpace that I'm
interested in using become easier or just more evident on how to enable
them.
Adding a new module to DSpace is as easy as editing a pom.
https://github.com/DSpace/DSpace/blob/master/dspace/pom.xml#L502
I should be able to just simply edit that and add an entry for the REST API,
or for WebMVC and when the project gets rebuilt, it has a new module
attached.
Project Restructuring
There is a minimum necessary amount of project rearrangement needed for this
pick-and-choose modularization mechanism to work smoothly. Mainly the
current [dspace-source]/dspace directory has to be off on its own in the
code repository. It shouldn't have a parent pom, and dspace-jspui,
dspace-xmlui, etc shouldn't be its "neighbors". JSPUI, and XMLUI should
become pluggable modules that one enables to have as the web interface to
their DSpace repository.
Note: I'm still fuzzy on the implementation specifics, especially with
regards to where dspace-api goes. But the [dspace-source]/dspace project is
highly dependent on a lot of Java classes that are banded together to give
us our data model, and actions available to perform on objects. I'll punt on
dspace-api.
With the restructuring, there becomes several new entries to the modules
directory in SVN. As we essentially move everything there. For each one of
these modules or projects, we all know who is particularly knowledgeable
about them, and we all know our own personal knowledge of them.
Here's my rough analysis of the refactorable projects, and my best guess as
to who is/has been active on them.
JSPUI - We all cut our teeth here, and several people are still maintaining
their customizations.
XMLUI - Experts include Scott Phillips, but lots of us are active with this
daily.
Discovery - Experts on this are Kevin Van de Velde and the rest of @mire.
Additionally NZ is making use of it.
LNI - Robert Tansley??, Nobody?
OAI - Kim and Robin have been active lately
Stats/Solr - @mire, Peter, Stuart
SWORD - Stuart, NZ, others
Things that only existed as modules, and are mostly just a person's "pet"
project:
REST - Bojan. There are a few other people who have dug in.
WebMVC - Graham, Peter
DSpace-Project-With-Funny-Name - Mark Diggory
...and many others, @Needs Documentation
Either way, everyone who's been approved as a committer is still an approved
committer. However, some people don't touch certain projects because they're
not familiar with them, and they'd prefer to have a more knowledgeable
person look their proposed changes to it over first. People who are not
marked as being a "DSpace Committer", though they are active and have
contributions are typically limited in access to the code through Jira.
So, the reason why many people have volunteered Git, is that it seems like a
natural fit. However, I'll suggest the same medicine that was given to me,
and remind that its a tool, and should not be confused with what is
actually necessary to pull off the migration/restructuring. Maybe its just
that restructuring the project to suit maven and modules has a cost, so
would migrating to Git, so it also seems like an opportune moment. Note that
even if no projects migrated to Git, the module restructuring would work
just fine. However, depending on how much time you have, I could fill your
ear off on why I think Git+GitHub is fantastic. For now I think that we
would all be satisfied with a blanket statement that projects don't need to
live in SVN (or Git) for that matter to be connected as a module. A
commercial module could live in a black box. Another module could be local
file system. The base agreement is that free/open modules will have code
that is freely available and visible through multiple means. (Download a
zip, visit the web interface, check out the code from repo.). You can also
contribute improvements to them.
Another thought is that the module restructuring should still make it easy
for a developer to go from 0 to 60 as easily as possible. (For those on
metric, 0 to 100). So you can start with a minimal setup of dspace, that
fetches everything from magic maven. And with a few changes to your pom, and
a few svn/git checkouts of source to your filesystem, your next rebuild
could have every imaginable DSpace module installed on your system. So that
you the developer of module-X who suspects that a bug in a higher up project
is causing you problems can quick make a change to the code you got for
module/ dspace-api / org.dspace.content.Community.findAllTop, to see if that
fixes what is happening in your module.
The meaning of "asynchronous" perhaps is the red herring. But if Stuart is
busy chugging away at SWORD, and finishes Sword 2.0, and I'm still on DSpace
1.7. I could edit a few lines in my pom to make it pull the latest module.
Either through having version ranges that say, sorry, sticking to sword 1.x
[1.0.0 TO 1.9.9], or more or less open ended sword x.y [0.0 TO 9.9].
Re-running maven, should have it pick up the latest sword release from the
maven repos. If you understand how xmlui's language i18n tags are versioned,
then this should make sense to you. Other projects such as LNI that haven't
had a new version released in some time would still be on 1.0 for several
years. However, there is nothing forbidding another developer from jumping
in and implementing AtomPub or CMIS, on top of the existing WebDAV
functionality there. Then LNI has its asynchronous bump to 2.0, which has no
implications for other modules so long as everything plays by the rules.
Peter Dietz
On Thu, May 5, 2011 at 5:48 PM, Tim Donohue <tdono...@duraspace.org> wrote:
> Mark,
>
> I feel like you and I may actually be coming at this from two different
> directions. But I'm not sure, primarily because I'm not sure I fully
> understand some of your ideas (see below for more)
>
> On 5/5/2011 3:43 PM, Mark Diggory wrote:
>
> >> Robin can correct me if I'm wrong, but I feel Robin's questions
> >> were more related to *policy decisions* from the Committers group.
> >> For example, his first question could have been reworded as: "What
> >> should be our *policy* around who should have commit rights to
> >> various modules?".
> >
> > Commit rights should be given to those that want to work on the
> > module and improve it. It would be good to have a module "lead" that
> > managed dealing with such rights. In either svn or git cases.
>
> This is something I see as a "committer policy" issue, and something we
> have never actually discussed as a group. Should Commit rights be given
> to anyone who wants to work on any module (including 'core' ones) to
> improve it? Should we only assign one "lead" for each Module, or should
> every 'core' module fall under "Committer Team" control?
>
> These are honest questions. The Committers as a whole have never
> discussed these sorts of policy changes. Obviously, we *don't* allow
> anyone who wants to work on XMLUI or JSPUI (as examples) to just start
> committing code. These policies can always be changed, but we need to
> discuss these changes before jumping to conclusions.
>
> >> To be honest, I worry that jumping directly into technology
> >> implementation ideas (git/github, etc.) only muddles things a bit.
> >> I don't mean any offense with this suggestion, I'm just noting that
> >> I don't think we're even all on the "same page" yet -- which is
> >> where we need to be before we can make decisions on *how* we want
> >> to move forward.
> >
> > I don't really have any questions about this, its relatively clear
> > that gatekeeping cripples participation and we should try to avoid it
> > as to not raise the bar to participation too high.
>
> I'm all for changing some of the 'gatekeeping' to make it easier for
> participation. But, we do have to realize one of the main purposes of
> "gatekeeping" is to ensure that each DSpace release is well vetted,
> tested and approved by a "trusted" group. This is what our community of
> 1000+ institutions expects from the Committers Group -- we are here to
> ensure we are providing a consistently stable, well vetted & useful
> product.
>
> We can definitely change where the "gatekeeping" occurs, but I think
> there will always need to be some gates in place in order to properly
> 'vet' the "trustworthy" modules from the "not-so-trustworthy" ones (or
> the ones that may be highly unstable). We also need to ensure the final
> "packaged" DSpace product is vetted & well tested itself, so there are
> additional "gates" that still need to be in place to ensure unstable or
> unapproved code isn't in that "packaged" release.
>
> >
> >> To put this another way, as a group, we first need to determine
> >> "what it is we are trying to achieve" (policy-wise and at a 'higher
> >> level'), and then we can look at "how best to achieve that" (via
> >> technology, whether it is git/github or an SVN reorganization or
> >> further modularization or whatever).
> >
> > I'm more for defacto than dejour in this case. The modules workspace
> > allows for non-dspace/trunk commiters to participate in dspace
> > development directly by participating in supporting projects and
> > addons.
>
> But, the reality here is that we have several Committers who are
> actively *questioning* whether the "modules workspace" is also causing
> some core source code to be "invisible" (or at least harder to find).
>
> So, I don't know that I agree the "modules workspace" is truly a
> 'defacto' standard -- it's more like a few people have gone that route,
> and there's a few others who now are popping up with concerns that
> haven't been answered.
>
> I will state that I *completely* agree that we should have an "open
> development workspace" for anyone (committers or non-committers) to
> create their own Maven projects & DSpace plugins. But, I think that is
> *NOT* the same as what the "modules workspace" currently is. The
> "modules workspace" is currently an unfortunate jumble of "stable,
> controlled" modules, unstable/experimental modules, third-party
> developed modules, and outdated/obsolete modules.
>
> I admit, I'm still hazy on whether you are suggesting that we should
> remove the Committer boundaries/gatekeeping on *all* Maven projects, or
> just on a sub-set of those projects. Obviously, there's a big
> difference there. So, I'd be curious to hear which "boundaries" you
> would see as remaining, and which would "go away" based on your
> suggestions.
>
> >> If there was one thing that came out of our discussion at
> >> yesterday's developers meeting, it's that we definitely don't all
> >> seem to have a common understanding of what it is we want to
> >> achieve (in regards to asynchronous releases or modularization,
> >> etc).
> >
> > We want to attain a build process that allows for addons to exist
> > independently of the release cycle. thats always been the case.
>
> I think we found out yesterday that several people are not sure if we
> actually want that. We had several "undecided" votes (more 'undecided'
> than 'decided' votes in fact).
>
> So, I think we need to take a step back and actually reanalyze what we
> mean by "we want a build process that allows for addons to exist
> independently of the release cycle". I don't think we all have the same
> understanding of what that statement means.
>
> > My concern with the common "assumption" that is made in the commiters
> > group that all code that is part of the release should be in
> > tdspace-trunk has gotten rather antiquated. It is quite difficult to
> > catch changes to code that shouldn't have happened in parts of dspace
> > that should be considered critical/core when the din of commit
> > activity in the trunk gets deafening. The point of modularization is
> > to compartmentalize that "din" so that we can make sure changes
> > shouldn't happen in important parts of DSpace that impact backward
> > compatibility.
> >
> > This said, in modularizing, I'm not suggesting its a "free for all"
> > commit wherever the hell you want. Its actually delegating
> > management of specific portions of the codebase to "expert groups"
> > which anyone can join and participate in. For instance I'm not going
> > to monitor much of anything going on in JSPUI project directly
> > because its are outside the scope on my use/need. This said, if
> > someone comes through with JSPUI development that is going to impact
> > org.dspace.content as well... I do want to know about and act in a
> > peer review role in such cases. It would be best id such changes were
> > compartmentalized under a subproject to ease monitoring... (again,
> > another feature more easily configured in GitHub than in our current
> > svn).
>
> So, who would form these "expert groups" you are talking about? Are you
> suggesting splitting up the Committers into subgroups? Are you
> suggesting that any developer can join any "expert group" if they are
> interested in helping to develop on that module?
>
> Mark, I think your last paragraph here gets to the heart of my confusion
> around your idea. I don't quite understand your suggestions around
> "expert groups" (instead, I had thought you were in fact suggesting a
> near "free for all"). How would one get "chosen" for these "expert
> groups"? I'm also not sure where the "Committers" fit into all this (are
> we the "overseeing" group? are we just here to package up releases? are
> we essentially "split up" and we all join the "expert groups" we are
> most interested in?).
>
> Definitely interested in better understanding your suggestions here.
>
> Thanks,
>
> - Tim
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today. Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> Dspace-devel mailing list
> Dspace-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-devel
>
------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
