On 12/18/05, Robert J. Chassell <[EMAIL PROTECTED]> wrote: > > ...... > Jurors should evaluate an article in several ways. An encyclopedia > entry, for example, requires one evaluation for accuracy and another > for style. Thus, an inaccurate article that claims the earth is flat > might show great style. > > The evaluation should be on a scale of one through five, with five the > best. Moreover, since people pay attention to the words that frame an > option, the scale should be: > > 5 -- excellent > much better than expected > > 4 -- good > better than expected > > 3 -- normal > expected > > 2 -- bad > worse than expected > > 1 -- terrible > much worse than expected > > A minimum number of jurors must judge before the results are > published. Otherwise, a few people will effect many. As a beginning, > I suggest a minimum of twelve. However, as I said above, editors > might want to look at infrequently judged items. > > Administrators must choose when (or whether) to ask editors to edit a > modifiable posting. As a beginning, I suggest a threshold in which a > majority of posters say that accuracy or style is less than normal. > (This is in addition to editing that ordinary people may do. Editors > are, to some extent, `insiders', which means they have duties as well > rights.) > > Every one needs to see a profile of judgements. In some situations, > the profile will bimodal. For example, I suspect that in 2003 in the > United States, we would have seen a bimodal distribution of accuracy > judgements about an article that claimed that in 2002 Saddam Hussein > and the Iraqi government were giving chemical, biological, or nuclear > weapons to Al Qaeda. On the other hand, I expect that every accuracy > judgement about a claim that the earth is flat would tell us that the > article is at level 1, and is much less accurate than expected. > (Style judgements might be different; the article would be wrong but > might be well written.) > > At the same time, many entries will be judged the same way, so an > average should be posted, too. > > > Jurors might also tell us their confidence about their judgements. > > For example, regarding accuracy, many might be confident that Michael > Faraday was not born in the 17th century since he worked in the 19th > century, but be not so certain whether he was born in the latter 18th > century (as he was) or early 19th. > > As a practical matter, I expect that jurors will be more confident in > style judgements than accuracy judgements. > > A confidence scale could use five levels: > > 5 -- entirely certain > > 4 -- strongly certain, but some doubt > > 3 -- moderately certain > > 2 -- somewhat uncertain, but a little confidence > > 1 -- completely uncertain > > To show certainty, an accuracy or style profile would have to be three > dimensional. Each level of accuracy or style displays its own > confidence scale. > > A high-resolution computer or printed output can do this readily. The > output is an image. It is harder to convey the equivalent information > on a low-resolution display, as with Lynx, or with an audio output for > the blind, such as Emacspeak. > > (Since car drivers should keep their eyes on the road, audio is > becoming more and more important. Consequently, every design must > consider it as one of several different kinds of output.) > > Is it true that in common law countries randomly selected jurors serve > both on grand and petit juries? > > What more should I add? (I am thinking of adding this, or part of > this, to `Choice and Constraint'. See > > http://www.rattlesnake.com/notions/Choice-and-Constraint.html > > for the HTML, the same directory for the other output formats) > > -- > Robert J. Chassell
Some interesting ideas here, Rob. Looks to me like you are trying to do several things here- you want multidemensional judgments (2, if I understand you); one for general quality (including comprehensiveness, formatting, and prose quality), and the other for accuracy. It would be neat if someone were to write a plugin for Wikipedia's wiki software (MediaWiki) which displayed a little Cartesian 2-d graph, where x could be general quality and y accuracy (or vice versa), and all one had to do was mouse-click in the right location, and your vote would be registered. ~Maru _______________________________________________ http://www.mccmedia.com/mailman/listinfo/brin-l
