Mark, I did not say that "theory should trump data." When theory should trump data is a very complex question.
I don't mind reading the book you suggested eventually but I have a long list of other stuff to read that seems to have higher priority. I don't believe there exists a complete, well-defined set of rules for evaluating scientific evidence in real-world cases, sorry. If you want to say "there is a complete set of rules, but there is no complete set of rules for how to apply these rules" -- well, I still doubt it, but it seems like a less absurd contention. But in that case, so what? In that case the rules don't actually tell you how to evaluate scientific evidence. In bioinformatics, it seems to me that evaluating complex datasets gets into tricky questions of applied statistics, on which expert biostatisticians don't always agree (and they write papers about their arguments). Clearly this is not a pursuit for dumbheads ;-p ... Perhaps you would classify this as a dispute about "how to apply the rules" and not about the rules themselves? I don't really understand the distinction you're drawing there... ben g On Sun, Oct 19, 2008 at 7:15 PM, Mark Waser <[EMAIL PROTECTED]> wrote: > >> It is really not true that there is a set of simple rules adequate to > tell people how to evaluate scientific results effectively. > > Get the book and then speak from a position of knowledge by telling > me something that you believe it is missing. When I cite a specific example > that you can go and verify or disprove, it is not an opinion but a valid > data point (and your perception of my vehemence and/or confidence and your > personal reaction to it are totally irrelevant). The fact that you can > make a statement like this from a position of total ignorance when I cite a > specific example is a clear example of not following basic scientific > principles. You can be insulted all you like but that is not what a good > scientist would do on a good day -- it is simply lazy and bad science. > > >> As often occurs, there may be rules that tell you how to handle 80% of > cases (or whatever), but then the remainder of the cases are harder and > require actual judgment. > Is it that the rules don't have 100% coverage or is that it isn't always > clear how to appropriately apply the rules and that is where the questions > come in? There is a huge difference between the two cases -- and your > statement "no one knows what the rules are" argues for the former not the > latter. I'd be more than willing to accept the latter -- but the former is > an embarrassment. Do you really mean to contend the former? > > >> It is possible I inaccurately remembered an anecdote from Feynman's > book, but that's irrelevant to my point. > No, you accurately remembered the anecdote. As I recall, Feynman was > expressing frustration at the slowness of the process -- particularly > because no one would consider his hypothesis enough to perform the > experiments necessary to determine whether the point was an outlier or not. > Not performing the experiment was an unfortunate choice of trade-offs (since > I'm sure that they were doing something else that they deemed more likely to > produce worthwhile results) but accepting his theory without first proving > that the outlier was indeed an outlier (regardless of his > "intelligence") would have been far worse and directly contrary to the > scientific method. > > >>>> Using that story as an example shows that you don't understand how to > properly run a scientific evaluative process. > >> Wow, that is quite an insult. So you're calling me an incompetent in my > profession now. > > It depends. Are you going to continue promoting something as inexcusable > as saying that theory should trump data (because of the source of the > theory)? I was quite clear that I was criticizing a very specific action. > Are you going to continue to defend that improper action? > > And why don't we keep this on the level of scientific debate rather than > arguing insults and vehemence and confidence? That's not particularly good > science either. > > > ----- Original Message ----- > *From:* Ben Goertzel <[EMAIL PROTECTED]> > *To:* agi@v2.listbox.com > *Sent:* Sunday, October 19, 2008 6:31 PM > *Subject:* Re: AW: AW: [agi] Re: Defining AGI > > > Sorry Mark, but I'm not going to accept your opinion on this just because > you express it with vehemence and confidence. > > I didn't argue much previously when you told me I didn't understand > engineering ... because, although I've worked with a lot of engineers, I > haven't been one. > > But, I grew up around scientists, I've trained scientists, and I am > currently (among other things) working as a scientist. > > It is really not true that there is a set of simple rules adequate to tell > people how to evaluate scientific results effectively. As often occurs, > there may be rules that tell you how to handle 80% of cases (or whatever), > but then the remainder of the cases are harder and require actual judgment. > > This is, by the way, the case with essentially every complex human process > that people have sought to cover via "expert rules." The rules cover many > cases ... but as one seeks to extend them to cover all relevant cases, one > winds up adding more and more and more specialized rules... > > It is possible I inaccurately remembered an anecdote from Feynman's book, > but that's irrelevant to my point. > > *** > Using that story as an example shows that you don't understand how to > properly run a scientific evaluative process. > *** > > Wow, that is quite an insult. So you're calling me an incompetent in my > profession now. > > I don't have particularly "thin skin", but I have to say that I'm getting > really tired of being attacked and insulted on this email list. > > -- Ben G > > > On Sun, Oct 19, 2008 at 6:18 PM, Mark Waser <[EMAIL PROTECTED]> wrote: > >> >> Whether a stupid person can do good scientific evaluation "if taught >> the rules" is a badly-formed question, because no one knows what the rules >> are. They are learned via experience just as much as by explicit teaching >> Wow! I'm sorry but that is a very scary, incorrect opinion. There's a >> really good book called "The Game of Science" by McCain and Segal that >> clearly explains all of the rules. I'll get you a copy. >> >> I understand that most "scientists" aren't trained properly -- but that is >> no reason to continue the problem by claiming that they can't be trained >> properly. >> >> You make my point with your explanation of your example of biology >> referees. And the Feynman example, if it is the story that I've heard >> before, was actually an example of good science in action because the >> outlier was eventually overruled AFTER ENOUGH GOOD DATA WAS COLLECTED to >> prove that the outlier was truly an outlier and not just a mere >> inconvenience to someone's theory. Feynman's exceptional intelligence >> allowed him to discover a possibility that might have been correct if the >> point was an outlier, but good scientific evaluation relies on data, data, >> and more data. Using that story as an example shows that you don't >> understand how to properly run a scientific evaluative process. >> >> >> ----- Original Message ----- >> *From:* Ben Goertzel <[EMAIL PROTECTED]> >> *To:* agi@v2.listbox.com >> *Sent:* Sunday, October 19, 2008 6:07 PM >> *Subject:* Re: AW: AW: [agi] Re: Defining AGI >> >> >> Whether a stupid person can do good scientific evaluation "if taught the >> rules" is a badly-formed question, because no one knows what the rules >> are. They are learned via experience just as much as by explicit teaching >> >> Furthermore, as anyone who has submitted a lot of science papers to >> journals knows, even smart scientists can be horrendously bad at scientific >> evaluation. I've had some really good bioscience papers rejected from >> journals, by presumably intelligent referees, for extremely bad reasons (and >> these papers were eventually published in good journals). >> >> Evaluating research is not much easier than doing it. When is someone's >> supposed test of statistical validity really the right test? Too many >> biology referees just look for the magic number of p<.05 rather than >> understanding what test actually underlies that number, because they don't >> know the math or don't know how to connect the math to the experiment in a >> contextually appropriate way. >> >> As another example: When should a data point be considered an outlier >> (meaning: probably due to equipment error or some other quirk) rather than a >> genuine part of the data? Tricky. I recall Feynman noting that he was held >> back in making a breakthrough discovery for some time, because of an outlier >> on someone else's published data table, which turned out to be spurious but >> had been accepted as valid by the community. In this case, Feyman's >> exceptional intelligence allowed him to carry out scientific evaluation more >> effectively than other, intelligent but less-so-than-him, had done... >> >> -- Ben G >> >> On Sun, Oct 19, 2008 at 6:00 PM, Mark Waser <[EMAIL PROTECTED]> wrote: >> >>> Actually, I should have drawn a distinction . . . . there is a major >>> difference between performing discovery as a scientist and evaluating data >>> as a scientist. I was referring to the latter (which is similar to >>> understanding Einstein) as opposed to the former (which is being Einstein). >>> You clearly are referring to the creative act of discovery (Programming is >>> also a discovery operation). >>> >>> So let me rephrase my statement -- Can a stupid person do good scientific >>> evaluation if taught the rules and willing to abide by them? Why or why >>> not? >>> >>> ----- Original Message ----- >>> *From:* Ben Goertzel <[EMAIL PROTECTED]> >>> *To:* agi@v2.listbox.com >>> *Sent:* Sunday, October 19, 2008 5:52 PM >>> *Subject:* Re: AW: AW: [agi] Re: Defining AGI >>> >>> >>> Mark, >>> >>> It is not the case that I have merely lectured rather than taught. I've >>> lectured (math, CS, psychology and futurology) at university, it's true ... >>> but I've also done extensive one-on-one math tutoring with students at >>> various levels ... and I've also taught small groups of kids aged 7-12, >>> hands-on (math & programming), and I've taught retirees various skills >>> (mostly computer related). >>> >>> Why can't a stupid person do good science? Doing science in reality >>> seems to require a whole bunch of implicit, hard-to-verbalize knowledge that >>> stupid people just don't seem to be capable of learning. A stupid person >>> can possibly be trained to be a good lab assistant, in some areas of science >>> but not others (it depends on how flaky and how complex the lab technology >>> involved is in that area). But, being a scientist involves a lot of >>> judgment, a lot of heuristic, uncertain reasoning drawing on a wide variety >>> of knowledge. >>> >>> Could a stupid person learn to be a good scientist given, say, a thousand >>> years of training? Maybe. But I doubt it, because by the time they had >>> moved on to learning the second half of what they need to know, they would >>> have already forgotten the first half ;-p >>> >>> You work in software engineering -- do you think a stupid person could be >>> trained to be a really good programmer? Again, I very much doubt it ... >>> though they could be (and increasingly are ;-p) trained to do routine >>> programming tasks. >>> >>> Inevitably, in either of these cases, the person will encounter some >>> situation not directly covered in their training, and will need to >>> intelligently analogize to their experience, and will fail at this because >>> they are not very intelligent... >>> >>> -- Ben G >>> >>> On Sun, Oct 19, 2008 at 5:43 PM, Mark Waser <[EMAIL PROTECTED]>wrote: >>> >>>> Funny, Ben. >>>> >>>> So . . . . could you clearly state why science can't be done by anyone >>>> willing to simply follow the recipe? >>>> >>>> Is it really anything other than the fact that they are stopped by their >>>> unconscious beliefs and biases? If so, what? >>>> >>>> Instead of a snide comment, defend your opinion with facts, >>>> explanations, and examples of what it really is. >>>> >>>> I can give you all sorts of examples where someone is capable of doing >>>> something "by the numbers" until they are told that they can't. >>>> >>>> What do you believe is so difficult about science other than overcoming >>>> the sub/unconscious? >>>> >>>> Your statement is obviously spoken by someone who has lectured as >>>> opposed to taught. >>>> >>>> ----- Original Message ----- >>>> *From:* Ben Goertzel <[EMAIL PROTECTED]> >>>> *To:* agi@v2.listbox.com >>>> *Sent:* Sunday, October 19, 2008 5:26 PM >>>> *Subject:* Re: AW: AW: [agi] Re: Defining AGI >>>> >>>> >>>> >>>> >>>>> >>> >>>>> *Any* human who can understand language beyond a certain point (say, >>>>> that of >>>>> >>>>> a slightly sub-average human IQ) can easily be taught to be a good >>>>> scientist >>>>> >>>>> if they are willing to play along. Science is a rote process that can >>>>> be >>>>> learned and executed by anyone -- as long as their beliefs and biases >>>>> don't >>>>> get in the way. >>>>> << >>>>> >>>> >>>> This is obviously spoken by someone who has never been a professional >>>> teacher ;-p >>>> >>>> ben g >>>> ------------------------------ >>>> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >>>> <https://www.listbox.com/member/archive/rss/303/> | >>>> Modify<https://www.listbox.com/member/?&;>Your Subscription >>>> <http://www.listbox.com> >>>> >>>> ------------------------------ >>>> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >>>> <https://www.listbox.com/member/archive/rss/303/> | >>>> Modify<https://www.listbox.com/member/?&;>Your Subscription >>>> <http://www.listbox.com> >>>> >>> >>> >>> >>> -- >>> Ben Goertzel, PhD >>> CEO, Novamente LLC and Biomind LLC >>> Director of Research, SIAI >>> [EMAIL PROTECTED] >>> >>> "Nothing will ever be attempted if all possible objections must be first >>> overcome " - Dr Samuel Johnson >>> >>> >>> ------------------------------ >>> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >>> <https://www.listbox.com/member/archive/rss/303/> | >>> Modify<https://www.listbox.com/member/?&;>Your Subscription >>> <http://www.listbox.com> >>> >>> ------------------------------ >>> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >>> <https://www.listbox.com/member/archive/rss/303/> | >>> Modify<https://www.listbox.com/member/?&;>Your Subscription >>> <http://www.listbox.com> >>> >> >> >> >> -- >> Ben Goertzel, PhD >> CEO, Novamente LLC and Biomind LLC >> Director of Research, SIAI >> [EMAIL PROTECTED] >> >> "Nothing will ever be attempted if all possible objections must be first >> overcome " - Dr Samuel Johnson >> >> >> ------------------------------ >> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com> >> >> ------------------------------ >> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com> >> > > > > -- > Ben Goertzel, PhD > CEO, Novamente LLC and Biomind LLC > Director of Research, SIAI > [EMAIL PROTECTED] > > "Nothing will ever be attempted if all possible objections must be first > overcome " - Dr Samuel Johnson > > > ------------------------------ > *agi* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com> > > ------------------------------ > *agi* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com> > -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "Nothing will ever be attempted if all possible objections must be first overcome " - Dr Samuel Johnson ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=117534816-b15a34 Powered by Listbox: http://www.listbox.com
