I've started to winnow some of this thread. As not every bullet point received a response, I've removed those under the assumption there's nothing further to discuss on them.
On 2012-02-04 21:57 , David Karger wrote: > On 2/4/2012 8:15 PM, Ryan Lee wrote: >> On 2012-02-03 11:42 , David Karger wrote: >>> OK, I'm going to try to unpack a few different arguments. >> I'll respond inline below, but I should note that there are several >> other things going on in this thread that have wider repercussions >> within this community than whether this proposed release goes forward. >> >> First and foremost is closing all back doors into the privilege of >> becoming a code committer on the project. While I go on more about this >> under your point 4 and others, I'd like to hear your response. I've >> brought it up in every message I've sent and am bringing it up again. I >> do very much believe this general policy will have little to no impact >> on your research goals and methods, but if you think differently, that >> will have to be addressed. > > I am comfortable with any community-determined process for admitting > core committers. To date there hasn't been one, so I've been left to > make decisions on my own. Great. I believe there will be more on this topic in the future. >> The second, as I think we've teased out in these discussions, is how >> best to develop and present new extension level features to users. It's >> becoming clear to me that most extensions should be released on their >> own schedule from the core of Exhibit. To date, the ones colocated with >> Exhibit have been in lockstep. Perhaps there is a subset of these that >> should continue to be held in sync with core, but there may be some that >> we've got that could be cut loose to follow their own development flow. >> This suggests a bit of work is needed for extensions to identify which >> version of core they're meant to work with. > This discussion is highlighting a worrying tension between the developer > community and the user community. I hope we can figure out a way to > release this tension. You misread my analogy to medical research; it > was not trying to contrast maintenance releases from experimental > releases, but rather was trying to contrast work aimed at future benefit > (careful gatekeeping to build a robust development community) from work > aimed at addressing immediate needs of current users. My apologies for misreading it. I'm not sure careful gatekeeping makes sense as a contrast to addressing immediate needs outside of this particular, odd moment in time. I'd prefer to start looking to intended patterns as guidance. I also wouldn't classify releasing new features that change the core as necessarily addressing the immediate needs of current users. It's a bit much to claim releasing a feature some have asked for as equal to fixing long standing bugs that negatively affect many existing users in terms of meeting needs. There should be a process for moving work done in the developer community into the core for the user community. If we're going to continue disagreeing on terms, then we also disagree on what constitutes immediacy and needs. > When I saw David's earliest version of Exhibit, an aspect that really > inspired me was the idea that here is something that non-developer, HTML > novice can use. And that is where I have continued to focus my energy. > I worry about any changes that will impact such naive users of exhibit. > It's fine to tell a developer "well, take this core from this site, then > you can combine with this version of this extension from that site (but > not another version), and if you want this functionality I'm sure that > there's an extension for that somewhere else if you can find it". But > that's a complete non-starter for naive users. I think there's a > noticeable gap be telling such people "put type='application/xml' in > your link tag" and "to import xml you need to include such and such > script in your document". I foresee us heading in the direction of > jquery-ui, which has its own web application that lets you build and > download to host your own custom script bundle. Fine for developers, > but a disaster for naive users. A process is intended to move those features that are best suited for everybody into core so that, at some point, the barriers get reduced to an absolute minimum. Those who can keep up with the necessary moving parts of material under active development are also those who don't need extra handholding. If the features are good, everybody should eventually have full access to them. By avoiding putting undercooked features into releases, you do what you can to avoid getting stuck in a miasma of simultaneously trying to support and back out released problems. That's no guarantee, to be sure, but it helps. Your prediction is hard to see coming true given we don't aim for it. You don't accidentally fall into that kind of architecture (and jQuery UI is quite purposefully modular, for good or ill). Nevertheless, there's no reason to assume every version of an extension is fully compatible with every version of core. Some mechanics around that are going to be necessary as part of the code base maturing. >>> 1. It seems we disagree on whether or not the release I am proposing is >>> a fork. I think this is based on a difference of opinion about whether >>> a fork is defined by an _action_ or by an _intention_. I do not believe >>> this is a fork, because there is no intention (and, I believe, no >>> effect) to divide community effort. I am fully supportive of the >>> completion of and ultimate migration to E3. >> It's good to hear your intentions restated in this form; nevertheless, >> forking *is* an action. Well-intentioned forks still have all the >> consequences of forking code. It appears we are still at odds on what >> it means. >> >> While I've mostly avoided discussing the specifics of this proposed >> release to avoid ratholes on debating the relative merits of each, I >> will pick on two to illustrate my point. You created ex:getter and >> ex:parser for importers, a design problem I also noted and resolved in >> 3.0 - completely differently (via an API). There might be a way to >> automatically resolve it when the two are joined, or it might be a >> nightmare that forces every importer developer to implement in such a >> way as to answer to both. You never intended it, but there is a >> potential mire to bog down in if both are out there as official >> solutions. > As a minor point, I am suspicious of anything that is solved "via an > API" since (given my interests) that suggests it will not be accessible > to naive users. The API is for those writing importers. Maybe someone should take a moment to see where the differences actually are, but to step around that, the fact that this engenders discussion is a mark against making a release with it included in full. >> I would also pick on the current implementation of logging. It took the >> easy way out and is as a result way out of line. It should not be in >> core. It should instead be made into an extension users can load off of >> a csail.mit.edu server since it already is opt-in and requires editing >> the HTML. The Exhibit parameters introduced with logging should not >> exist in core. I am rather certain it is difficult and ugly to do what >> I just suggested in Exhibit 2. It should be trivial in Exhibit 3. Put >> a hold on introducing your research team's logging to a release; it >> definitely doesn't belong there. (I haven't been tracking this closely, >> but you may also want to warn Exhibit authors using logging to warn >> their own users that their activity is being recorded). > Here I have to disagree on technical grounds. I can see absolutely no > way to accomplish logging without tinkering core. We're trying to > record actions taken in the exhibit core. To do that from outside core > would require a script that, after loading core, redefines every single > logged function inside core to first log and then call the in-core > function. This seems incredibly fragile and will break any time > somebody changes core. Like I said, ugly in Exhibit 2, likely trivial in Exhibit 3. https://github.com/zepheira/exhibit3/wiki/Scripted-Event-API If there are events missing, do suggest additional useful ones. As I have stated, I don't believe logging code belongs in core at all, in any version. > To the separate concern of adding exhibit parameters, I am comfortable > changing the way we _activate_ the logging functionality, replacing the > current parameter setting with a script, e.g. <script > src="http://csail/logger.js";> which would turn on the logging. That's certainly a step in a more agreeable direction, but presumably the response to being activated is still present in core; this would push it to a different level but is still problematic in terms of presenting an interface that requires future commitment. And it still leaves the logging code there. > To the question of warning users, note that we chose an opt-in mechanism > because we considered it very much the role of the web author to decide > whether to log. Subsequently, the web author has their own decision to > make about whether to give the page's visitors a choice about > logging---however, this is exactly the same decision as they make when > they decide to put the google analytics urchin into their pages, and I > don't think _anyone_ bothers to ask permission of that from their > users. The logger collects exactly the same kind of data as those > analytics tools; in fact if exhibit was server based and generated a new > page for each interface action, we could just use google analytics to > gather the same data. Logging every interaction with Exhibit is pretty different from analyzing visitor traffic (which, though obvious to anybody who's seen a web server log, is still spelled out in P3P as a privacy disclosure). Drilling into this topic further seems tangential. Again, my interest was not in debating any specific feature. My point is that these impose fork-like burdens on Exhibit 3 that would not exist if they weren't included in a release, regardless of how the release is positioned. >>> 2. You object that "making experimental commitments" may not coincide >>> with "making a great release". This may be true, but remember that >>> there is no intention to ever make another "great release" of E2. >>> Instead, we expect the next great release to be E3. With E2 abandoned, >>> there seems no downside to releasing some experimental functionality >>> on it. >> What I was saying too backhandedly was that experimental commits >> probably make for a terrible point release, not that I think Exhibit 2 >> now needs a great release to have a release. >> >> It's rather important to point out that to date, Exhibit releases have >> been of the type that suggest if a feature is in, it's in it to stay. >> That's quite contrary to experimenting. Experiments are allowed to fail >> in order to provide interesting information. Releases of the type >> Exhibit has undergone have not seemed to me to be intended for >> experimenting, where failed features get removed. Releasing web-based >> services might carry some extra caveats about going too experimental, >> too. >> >> I should note on the side that I do believe in releasing early and often >> and unknowingly adding bugs and making mistakes to further a project - >> as long as there is appropriate gatekeeping in place. And maybe some >> test harnesses. > I agree that past releases have been more reflective of future > commitment. However, as I said, we have all made quite clear that E3 is > the future of exhibit. But that doesn't absolve the apparent future commitment implicit in an Exhibit 2 release for Exhibit 3. >>> 3. You're right that I believe this code should be released because it >>> is there. Of course this release could happen in different ways. I >>> suppose I could put a copy of my code at >>> http://people.csail.mit.edu/karger/exhibit-api.js , but really why is >>> this any better? The code still _exists_. Isn't it just as much a fork >>> that way? On the other hand, the option of deleting my changes seems >>> damaging, since I think they make the tool better and thus help our >>> current users---see my point 7 below. >> There's a rather substantive difference (of thousands of existing users) >> between api.simile-widgets.org and people.csail.mit.edu/karger. >> Everybody is already free to "fork" hosting in the sense you describe, >> I'm not sure it pertains to this discussion. What you're proposing is >> an *official* fork release. > If we can resolve this whole debate by serving the release from > simile-widgets.org/exhibit-mit , or by reactivating simile.mit.edu and > hosting the new release there, I'm all for it. Of course, I'll > recommend everyone switch to it, since I think it's better. And we'll > probably need a separate copy of the documentation wiki, to describe the > features that aren't in the official release. And a separate web-site > to which I can direct people who are interest in exhibit and may want to > use the added features. This seems much more forky to me than anything > we've yet discussed. I would rather avoid a fork regardless of its degree of fork-iness. There are ways forward that don't requiring forking of any kind. >>> 4. "How has the liverpool group participated in the community?" By >>> creating a pretty cool extension. We might wish that they would >>> participate more in the discussion group, but I'd rather appreciate >>> their contribution than feel bad about what they haven't done. >> That's a false dichotomy. One of the points I'll keep drumming at you >> is that people who aren't yet part of this community can't be let in to >> make commits without first entering into it through the front door, >> otherwise it doesn't work as a community. That fact doesn't diminish >> their eagerness to contribute or the usefulness of their work; it means >> that work never should have come in the back door through you (or any of >> us). Officially releasing a non/back door participant's work is not a >> good signal to the rest of the community, the way letting unknown >> committers commit to trunk is not a good signal - it's just a lot bigger >> of a signal. >> >> In the future when newcomers approach any of us (or, I suppose, are >> approached by us) regarding improving Exhibit, I hope our line is >> something like, "Excellent, join the list, fork on GitHub, and send in >> your pull requests" - just like everybody else. > So this helps me understand another issue to tease apart. Liverpool > produced an *extension* and never touched the exhibit core. In fact, > all I was thinking about when I provided commit access was that this > simplified the *hosting* of the extension on simile-widgets.org. I > understand that this had *development authority* implications I wasn't > considering. I'd be perfectly happy to resolve them by moving the > extension to a different subversion repository. However, in the > interest of keeping things simple for those naive users, I would still > want to *serve* the extension from simile-widgets.org. simile-widgets.org carries more to it than simply hosting. That code committed to the master branch is also going to be served at the canonical URL seems to be a consequence of ease of deployment. I'm not really convinced hosting static JavaScript is a so big a barrier to developers, though perhaps there's a reasonable way to formulate Exhibit lab hosting on simile-widgets independent of revision control. Given you have to add a script tag to use the extensions, at least, I don't see how serving from simile-widgets makes things all that more simple for authors. > And why simile-widgets.org instead of csail.mit.edu/karger ? > Resources. I'm happy to leverage the fact the MIT libraries are > maintaining a well-run, high capacity server that can deliver our > scripts. Kenzie may disapprove of my taking advantage of MIT libraries > this way, but as long as they are willing to do the management, I don't > have to find the resources to do it. >>> 6. I think the really big question is whether, at some point, we should >>> switch over simile-widgets.org/exhibit/api to use the 2.3 version >>> instead of 2.2. Obviously, 2.2 will still exist at >>> api.simile-widget.org/exhibit/2.2.0 , so it isn't a matter of >>> eliminating something people depends on. But a switchover would mean a >>> "forced upgrade" for anyone who isn't paying attention to their >>> exhibits. Is this a good thing or a bad thing? Thinking about the >>> changes, we have (i) maintenance that should rescue people from bugs >>> (and painter failures) and (ii) new features that they won't see at all >>> if they don't choose to use them. On the downside, we have the risk >>> that I have introduced new bugs, or incompatibilities with certain >>> customizations people have already made. That would obviously be bad, >>> which is why I want to take the time to let people test trunk >>> voluntarily before we do a switchover. I believe that at a certain >>> point, we will have enough evidence that 2.3 is "safe" to make the >>> (known) benefits outweigh the (unknown) risks. >> You understate the impact of new features, not all of which are outside >> of core. The code is there whether one particular user puts it into >> play or not. The fact that the code is there binds you to its >> existence, and you have to wrestle with the difficult questions of how >> to deal with its existence in the future. >> >> For an outside example, HTML5 gets to wrestle with XHTML/XML namespaces. >> It's not at all an easy matter to resolve, because lots of people ended >> up adopting namespaces, and HTML5 isn't XML. Some people are simply >> never going to leave XHTML because of it. The current solution is that >> a very few "xmlns:xx" attributes exist, for which the prefix "xmlns:" is >> meaningless and just part of the name. I'd like to avoid that kind of >> thing. > And some people may never leave exhibit 2 because they are using some > feature only it has. But in the meantime they have a feature they can > use. Again, this comes back to the medicine metaphor. If I can provide > this needed feature to users now, when it's impossible to provide in E3, > then I'm helping them *now*. That is a different but valuable > contribution distinct from things that might help them *later*. I very much see your point that not sacrificing helping users now for a fuzzy future is a good thing to do. The problem is that I don't agree that everything you intend to release is an unmitigated help to users or to other developers. >>> 8. Another way to interpret your question is as worrying about the >>> consequences if many exhibit users adopt the new features like data >>> import or data editing. I think this would be great! It would provide >>> a very strong signal that these things are important to incorporate into >>> E3. Perhaps this is where you worry about leaving users "stranded" in >>> the move to E3---that they will have become dependent on those features >>> and won't be able to move. But again, I don't buy the argument that we >>> should withhold something from our current users just because we might >>> not be able to provide it in the future. >> Not really. You clearly don't need to make a release to make these >> things available to people as they're already testing them. Your >> argument cuts both ways; if it isn't in the release, it can still be >> provided for users to try out. My argument was never to completely >> shutter that work - it was to avoid conflating it with a release by >> carving it out of one. > OK, it seems that if I just leave everything as it is, with people > needing the new features linking to trunk.simile-widgets.org , then > everything is fine? This is a bit terse for me to parse what you mean. It sounds to me like it's about not making a Exhibit 2 release after all; was that what you meant? -- You received this message because you are subscribed to the Google Groups "SIMILE Widgets" group. To post to this group, send email to simile-widgets@googlegroups.com. To unsubscribe from this group, send email to simile-widgets+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/simile-widgets?hl=en.
