*Call for Papers*
******************************************************************
2012 AAAI Fall Symposium on
Machine Aggregation of Human Judgment
November 2-4, 2012 Arlington, VA, USA
URL:http://magg.c4i.gmu.edu/
Organizers:
Kathryn B. Laskey, Wei Sun, John Irvine,
Dirk B. Warnaar, H. Van Dyke Parunak
******************************************************************
The AAAI 2012 Fall Symposium on Machine Aggregation of Human Judgment
focuses on combining human and machine inference. For unique events and
data-poor problems, there is no substitute for human judgment. Even for
data-rich problems, human input is needed to account for contextual
factors. For example, textual analysis is data rich, but context and
semantics often make automated parsing unusable. However, humans are
notorious for underestimating the uncertainty in their forecasts and
even the most expert judgments exhibit well-known cognitive biases. The
challenge is therefore to aggregate expert judgment such that it
compensates for the human deficiencies.
There are fundamental theoretical reasons to expect aggregated estimates
to out-perform individual forecasts. These theoretical results are borne
out by a robust empirical literature demonstrating the superiority of
opinion pools and prediction markets over individual forecasts, and of
ensemble forecasts over those of top models. While weighted forecasts
are theoretically optimal, among human experts unweighted forecasts have
been hard to beat.
This symposium focuses on methods with the potential to come closer to
the theoretical optimum. While a number of methods have shown promise
individually, there is potential for significant advancement from
combining them into structured, efficient, repeatable elicitation and
aggregation protocols. Benefits of improved aggregation methods include
substantial increases in the quality and reliability of expert
judgements, removing misunderstanding, illuminating context dependence
of forecasts, and reducing overconfidence and motivational bias in
forecasts. On the other hand, there's some skepticism that statistical
models can outperform experts most of the time. Machine reasoning lacks
the context to know when the models no longer apply, or in cases like
natural language, simply lack sufficient context to be reliable in
open-world or novel problems. This symposium considers powerful hybrid
techniques using humans to help aggregate computer models.
A broad range of researchers in the AI community and other application
fields such as econometrics, sociology, political science, and
intelligence analysis will find this symposium interesting and useful.
Bringing these disciplines together to the venue also greatly
facilitates the research endeavors.
*Format*
The symposium will accept a number of regular papers (6-8 pages), and
short papers/extended abstracts (2 pages). In addition to oral
presentations, we intend to provide several poster sessions for more
interactions. Further, invited talks by leading researchers in the
fields and/or domain experts will be arranged. We will also reserve
substantial time for questions and discussions after talks.
*Topics include but not limited to the following*
* Reasoning under uncertainty
* Ensembles & aggregation
* Information fusion
* Crowdsourcing techniques & applications
* Information elicitation & presentation
* Performance evaluation: scalability and accuracy
* Prediction markets
* Collective intelligence
*Submission*
Submission should be done through EasyChair submission site. Authors,
who do not have accounts on EasyChair, will be directed to create a new
account before they can make submission.
*Important Dates*
* Friday, May 25, 2012, 11:59pm Eastern Time -- Papers/abstracts due.
* Friday, June 22, 2012 -- Author notification of acceptance/rejection.
* Friday, September 7, 2012 - Camera-ready paper due.
* Friday November 2 - Sunday, November 4, 2012 - Symposium at
Arlington, VA.
**
*Organizing Committee*
Kathryn B. Laskey, Wei Sun (George Mason University), John Irvine
(Draper Laboratory), Dirk B. Warnaar (Applied Research Associates,
Inc.), H. Van Dyke Parunak (Jacobs Technology Inc.)
*Supplementary Website*
For more information about the symposium, and for submission guidelines
and links, please visit the supplementary website (http://magg.c4i.gmu.edu).
_______________________________________________
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
