Re: [Sugar-devel] Assessment in Karma

2009-08-30 Thread NoiseEHC 
Okay, a little holiday happened with me and then had to read through 
100+ olpc emails and then realized that there is a spam filter on my 
email which blocked sugar emails since july so it took some time to 
catch up but here I am... :)


Martin Langhoff wrote:

2009/8/19 NoiseEHC :
  

 - Automatic assessment is snake oil, Bryan is well intentioned but
deeply wrong. See the earlier email at
http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html
  

Or you are wrong.



I may well be wrong, but to explore that you will have to talk about
what I am stating :-)
  

So I was wrong too... :)

We could rephrase it as

 - Computer-based automatic assessment / grading is only passably
accurate for a tiny *tiny* subset of relevant human skills.
  
I am 98% sure that automatic grading (if you mean assessment == grading 
here) is absolutely not possible at all. (And it is a different issue 
that as far as I know grading in class 1-4 is totally pointless and 
harms children.) Most of the problems are that those tests which would 
in theory differentiate children in a 1-5 grade scale are high stake 
tests which are inherently unreliable.

 - However, it's very spectacular, and people are drawn to it... so
much that they are drawn to it even when it *clearly does not work*
for the skill being tested. It's so easy (for the teacher) and so
flashy, that people use it regardless of whether it works.

I say this after 9 years of work in the field -- I have seen
interactive SCORM objects, Moodle quizzes & lessons, HotPotatoes
activities, LAMS assessments, lots of other standalone assessment
tools. Have worked with teachers, watching their use.

What did I see? See the 2 points above.

There is a 3rd part... because these tools are cool, easy to use, they
do a lot of damage. In large part because they replace the "I don't
know how my students are doing" with "hey, I have all these scores are
numbers... nevermind they are inaccurate and only cover about 3% of
what these kids should know".

So an inaccurate view of a tiny slice of the skillset -- but hey, we
have a number representing what this kid knows! Let's use it! The link
that follows is from the Asttle project, which I was briefly involved
in several years ago:

  http://www.tki.org.nz/r/governance/consider/steps/analyse_e.php

Even worse, the Asttle project promotes the idea that you can get a
form of 'dashboard' of what kids know. Looks like an airplane
dashboard, lots of dials, full of inaccurate data _about a tiny
subset_ of what matters.

  
I can feel your pain (especially seeing the Asttle project) so thanks 
for the warning!

What more interesting is that there is some research in Hungary [1] about
the prerequisites of learning certain skills which are based on each other.



Sure, that is interesting. Now how amenable to computer-grading are
those skills? What automated computer tests can assess them with a
decent accuracy?
  
It is not about computer-grading at all!!! It is about well tested 
low-stake tests (children must reach 80-90%) which can reliably measure 
whether a child for example reached finishing or optimal level in 
reading (I am talking only about basic skills). If a child does not 
reach those levels then no matter how constructivist your education 
policy is in the higher grades (4-8) because this children simply will 
not be able to gain any information from books. Currently teachers can 
only deduce missing basic skills from the simple fact that the children 
fail biology or geography tests (because they cannot read reliably) and 
nobody tests reading skills continuously. The computerized assessment 
cannot be worst than that can it?
  

... Automatizing at
least some of those tests are probably the biggest thing since sliced bread
in education in my humble opinion

Are any of those tests "automatizable"? With what accuracy? If it
turns out some can be computer assessed... _how do we keep the
non-automatizable tests in the map_? Teachers forget them
*immediately*.

  
If the school decides to run this little research then I will let you 
know about the "automatizabledness" of the relevant tests, I promise! :)
And I am sorry but I will not be able to solve those other social 
problems (non-automatizable tests) with computers that is sure...


___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-20 Thread Martin Langhoff 
On Fri, Aug 21, 2009 at 3:58 AM, Bryan Berry wrote:
> I agree that automatic assessment is no magic cure-all but it does free
> teachers from a lot of drudgery in grading worksheets.

I understand your point, and respect your good intentions. I worry --
quite a bit -- about the outcome however...

> Teachers should be grading student essays not arithmetic exercises or
> vocabulary exercises.

What I worry is that once we automated arithmetic exercises, they'll
focus on that... as you say

> We especially need automatic assessment for contexts where teachers
> don't have time to grade homework, like Nepal, India, Pakistan, etc.

So they don't have time for either. We automate one, and the fact that
we provide easy to get, easy to use grades takes over. They still
don't have time for essays.

[ The sad thing I find is that they *will* find time to make pretty
graphs of the paltry numbers they get. The graphs make the teacher
look good and in control. ]

> I think that Karma is approaching from a much different vantage point
> than teachers in the developed world do. We are not looking to capture
> "excellence" but rather diagnose if kids are having trouble with basic
> skills and give kids instant feedback rather than make them wait a week
> to get their graded homework back, if it ever comes back.

John Hattie, in pretty developed NZ, has done a lot of work on that
exact track ("early diagnosis of kids falling behind on basics" and
"instant feedback"). Hence Asttle.

Maybe I am a luddite and it'll happen anyway. Hmmm.



m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-20 Thread Bryan Berry 
On Thu, 2009-08-20 at 09:57 +0200, Martin Langhoff wrote:
> 2009/8/19 NoiseEHC :
> >
> >>  - Automatic assessment is snake oil, Bryan is well intentioned but
> >> deeply wrong. See the earlier email at
> >> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html

I agree that automatic assessment is no magic cure-all but it does free
teachers from a lot of drudgery in grading worksheets.

Teachers should be grading student essays not arithmetic exercises or
vocabulary exercises.

We especially need automatic assessment for contexts where teachers
don't have time to grade homework, like Nepal, India, Pakistan, etc.

I think that Karma is approaching from a much different vantage point
than teachers in the developed world do. We are not looking to capture
"excellence" but rather diagnose if kids are having trouble with basic
skills and give kids instant feedback rather than make them wait a week
to get their graded homework back, if it ever comes back.

-- 
Bryan W. Berry
Technology Director
OLE Nepal, http://www.olenepal.org

___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-20 Thread Martin Langhoff 
2009/8/19 NoiseEHC :
>
>>  - Automatic assessment is snake oil, Bryan is well intentioned but
>> deeply wrong. See the earlier email at
>> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html
>
> Or you are wrong.

I may well be wrong, but to explore that you will have to talk about
what I am stating :-)

We could rephrase it as

 - Computer-based automatic assessment / grading is only passably
accurate for a tiny *tiny* subset of relevant human skills.

 - However, it's very spectacular, and people are drawn to it... so
much that they are drawn to it even when it *clearly does not work*
for the skill being tested. It's so easy (for the teacher) and so
flashy, that people use it regardless of whether it works.

I say this after 9 years of work in the field -- I have seen
interactive SCORM objects, Moodle quizzes & lessons, HotPotatoes
activities, LAMS assessments, lots of other standalone assessment
tools. Have worked with teachers, watching their use.

What did I see? See the 2 points above.

There is a 3rd part... because these tools are cool, easy to use, they
do a lot of damage. In large part because they replace the "I don't
know how my students are doing" with "hey, I have all these scores are
numbers... nevermind they are inaccurate and only cover about 3% of
what these kids should know".

So an inaccurate view of a tiny slice of the skillset -- but hey, we
have a number representing what this kid knows! Let's use it! The link
that follows is from the Asttle project, which I was briefly involved
in several years ago:

  http://www.tki.org.nz/r/governance/consider/steps/analyse_e.php

Even worse, the Asttle project promotes the idea that you can get a
form of 'dashboard' of what kids know. Looks like an airplane
dashboard, lots of dials, full of inaccurate data _about a tiny
subset_ of what matters.

> What more interesting is that there is some research in Hungary [1] about
> the prerequisites of learning certain skills which are based on each other.

Sure, that is interesting. Now how amenable to computer-grading are
those skills? What automated computer tests can assess them with a
decent accuracy?

> ... Automatizing at
> least some of those tests are probably the biggest thing since sliced bread
> in education in my humble opinion.

Are any of those tests "automatizable"? With what accuracy? If it
turns out some can be computer assessed... _how do we keep the
non-automatizable tests in the map_? Teachers forget them
*immediately*.

cheers,



m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-19 Thread NoiseEHC 

>  - Automatic assessment is snake oil, Bryan is well intentioned but
> deeply wrong. See the earlier email at
> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html
>   
Or you are wrong. Since you did not define exactly what you have meant 
by assessment in your message, I assume that you were talking about 
something like automatic grading or creativity assessment versus 
something mechanical and mind-numbingly trivial. (Any other assumption 
makes your message meaningless in my eyes but correct me if I am 
wrong...)  Now I will not talk about that you have just presented us a 
false dichotomy since it is not too interesting (and can be that I just 
have misunderstand and misrepresented what you have been said).

What more interesting is that there is some research in Hungary [1] 
about the prerequisites of learning certain skills which are based on 
each other. (So for example reading fast - whole word reading - requires 
to be able to read words letter by letter which requires reading out 
certain letter combinations which requires knowing the letters which 
requires differentiating vowels and so on... Since I do not know these 
things in English I did not try to be precise in this example.) The 
point of this research is that it is clear now that if a child does not 
reach a certain threshold in those skills then he cannot progress in 
other skills (these are the critical skills).
Here lies the bane of the Hungarian education system that children go to 
school at the age of 6 and they can be +-2 year apart in mental and 
physical development. So some children are like 8 year olds and some are 
like 4 years olds. Unfortunately this mental/physical difference is 
usually totally different across subjects so for example if a child is 
totally lame in writing (because he cannot control his small muscles 
adequately) but is an ace in math then it is normal and repeating the 
class is not an option. In Hungary (and probably a lot of other 
countries) the problem is that teaching these basic skills goes 
according to the national curriculum and there are no tests which 
measure them so if a child is left behind then game over for him. For 
example if he cannot reach optimal or finishing level in reading then he 
will not learn from books normally since he cannot read fast enough. 
Because teaching of reading skills stops at grade 3 or so these children 
leave the elementary school at advanced, beginner or sometimes 
preparatory level (40% of people in Hungary!!!) and they grow into 
functionally illiterate adults who cannot learn anything from books or 
read newspapers (or watch films with subtitles so they cannot learn 
languages in that way).
Now there are some good news that there exist tests which can measure 
the level in those basic skills and these are low stake tests. Usually 
optimal level means >90% and finishing level is >80% (in some skills 
these are around 80% and 60% respectively). The problem is that 
measuring those takes a LOT of time and energy and no teacher wants to 
do that. Automatizing at least some of those tests are probably the 
biggest thing since sliced bread in education in my humble opinion.

So it will not help teachers giving a mark at all because you know ALL 
the children should pass those tests sooner or later. Of course it will 
not measure creativity either. What it should do is to measure some 
mechanical and mind-numbingly trivial skills what are not measured 
without computers because it is mind-numbingly boring for a teacher. Now 
the questions is: which basic skills can be measured with a computer and 
which cannot? Calling assessment snake oil because it cannot give marks 
automatically is at least strange to me...

[1]
http://www.staff.u-szeged.hu/~nagyjozs/#res

ps:
As the subject becomes more advanced it becomes much more painfully 
obvious that my native language is not English. I am sorry for that.

ps2:
Of course this is not the most important aspect of OLPC but giving these 
assessment tools to children in Ghana probably cannot hurt can it?

___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-18 Thread Christoph Derndorfer 
On Tue, Aug 18, 2009 at 5:17 PM, Martin Langhoff
wrote:

> 2009/7/29 Christoph Derndorfer :
> > As previously mentioned by Bryan in his "Automated Assessment is the
> Killer
> > App" blog post
> > (
> http://karmaproject.wordpress.com/2009/07/06/assessment-is-the-killer-app/
> )
> > student assessment is an important component of Karma.
> ...
> > Now I was wondering whether anyone here had specific suggestions on how
> to
> > address this or pointers to how other e-learning solutions (regardless of
> > whether stand-alone applications or Web based) solve this interesting
> > challenge.
>
> Yup - from moodle-land... my original notes are in this email from
> earlier this year
> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html
>
> Super short recap, with bonus links...
>
>  - Recommended: look at what SCORM objects store in the LMS backend
> (the "datamodel")  -- grab the API docs and look for cmi.score.raw
> cmi.progress_measure -- the full listing is here, but it is missing
> the full discussion of the meaning of each value:
>
> http://www.scorm.com/scorm-explained/technical-scorm/run-time/run-time-reference/
>
>  - Recommended: look at Moodle's Grades API
> http://docs.moodle.org/en/Grades
>
> Both APIs have been designed trying to represent many ways of grading
> and scoring. I would try to fit with one of those rather than spend
> precious years revisiting mistakes :-)


Thanks a lot for the link and all the information, especially since I had
missed that discussion back in June...


>  - Automatic assessment is snake oil, Bryan is well intentioned but
> deeply wrong. See the earlier email at
> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html


Well, here I think it really depends on the specific subject matter. Primary
school Maths for example is an area where I definitely believe that
automatic assessment can be very useful. Learning about the rivers, cities,
provinces and countries in your proximity is another one (which is basically
what "Conozco Uruguay" is doing). Biology with learning about the parts of a
flower or a human ear is another example that comes to mind.

What I'm basically thinking is that a lot of the areas of knowledge that I
as a kid (I went to a Montessori primary school) explored with the help of
tools such as LÜK (http://www.luek.de/EN/index_english.html) can be nicely
implemented in a digital form. While back then it was the pattern on the
back of the LÜK pieces that told me whether my solution was correct these
days this can be done by a computer.

Just my 2 cents,
Christoph

-- 
Christoph Derndorfer
co-editor, olpcnews
url: www.olpcnews.com
e-mail: christ...@olpcnews.com
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-18 Thread Martin Langhoff 
2009/7/29 Christoph Derndorfer :
> As previously mentioned by Bryan in his "Automated Assessment is the Killer
> App" blog post
> (http://karmaproject.wordpress.com/2009/07/06/assessment-is-the-killer-app/)
> student assessment is an important component of Karma.
...
> Now I was wondering whether anyone here had specific suggestions on how to
> address this or pointers to how other e-learning solutions (regardless of
> whether stand-alone applications or Web based) solve this interesting
> challenge.

Yup - from moodle-land... my original notes are in this email from
earlier this year
http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html

Super short recap, with bonus links...

 - Recommended: look at what SCORM objects store in the LMS backend
(the "datamodel")  -- grab the API docs and look for cmi.score.raw
cmi.progress_measure -- the full listing is here, but it is missing
the full discussion of the meaning of each value:
http://www.scorm.com/scorm-explained/technical-scorm/run-time/run-time-reference/

 - Recommended: look at Moodle's Grades API http://docs.moodle.org/en/Grades

Both APIs have been designed trying to represent many ways of grading
and scoring. I would try to fit with one of those rather than spend
precious years revisiting mistakes :-)

 - Automatic assessment is snake oil, Bryan is well intentioned but
deeply wrong. See the earlier email at
http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html

cheers,




m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-08-18 Thread Christoph Derndorfer 
On Thu, Jul 30, 2009 at 4:46 PM, Christoph Derndorfer <
christoph.derndor...@gmail.com> wrote:

> On Wed, Jul 29, 2009 at 6:28 PM, Bryan Berry  wrote:
>>
>> ChristophD, we really need to talk to sunil and kamana about assessment
>> as they probably have a lot of good ideas. Well, actually they created
>> all the assessments for EPaath but we need to abstract their work to
>> something broadly usable.
>
>
> Yes, we definitely need to do that. (Though unfortunately Kamana will be
> gone all August.)
>
> I've also started collected some very brief notes and thoughts at
> http://wiki.sugarlabs.org/go/Karma/Assessment
>

I've updated the Wiki entry with some information on what I consider to be
the three different types or levels of assessment implementation that we
need to consider.

Let me know what you think.

Christoph

-- 
Christoph Derndorfer
co-editor, olpcnews
url: www.olpcnews.com
e-mail: christ...@olpcnews.com
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-30 Thread Bryan Berry 
On Thu, 2009-07-30 at 16:46 +0545, Christoph Derndorfer wrote:
> On Wed, Jul 29, 2009 at 6:28 PM, Bryan Berry 
> wrote:
> ChristophD, we really need to talk to sunil and kamana about
> assessment
> as they probably have a lot of good ideas. Well, actually they
> created
> all the assessments for EPaath but we need to abstract their
> work to
> something broadly usable.
> 
> 
> Yes, we definitely need to do that. (Though unfortunately Kamana will
> be gone all August.)

Kamana will probably be at home during that time. You can probably visit
her at her home near Patan and discuss it w/ her.

> 
> I've also started collected some very brief notes and thoughts
> at http://wiki.sugarlabs.org/go/Karma/Assessment
>  
> > ps:
> > I intend to use Karma for interactive curriculum development
> for
> > reasons Bryan Berry talked about a lot.
> 
> 
> is it time to for us to open a mailing list specific to
> karma?  like  sugar-ka...@l.s.o ?
> 
> 
> Nope, personally I think this would be premature. Let's keep the
> discussion here on sugar-devel as far as the more technical aspects
> are concerned, maybe use IAEP for talking about broader educational
> issues and increase our post frequency on the Karma blog so people who
> don't want to follow the discussions on the mailing lists can also
> find out what's going on.

I think u are right. We should stick to the existing lists until people
complain.

> Christoph
> 
> 
> -- 
> Christoph Derndorfer
> co-editor, olpcnews
> url: www.olpcnews.com
> e-mail: christ...@olpcnews.com
-- 
Bryan W. Berry
Technology Director
OLE Nepal, http://www.olenepal.org

___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-30 Thread Tomeu Vizoso 
2009/7/30 Christoph Derndorfer :
> On Wed, Jul 29, 2009 at 6:28 PM, Bryan Berry  wrote:
>>
>> ChristophD, we really need to talk to sunil and kamana about assessment
>> as they probably have a lot of good ideas. Well, actually they created
>> all the assessments for EPaath but we need to abstract their work to
>> something broadly usable.
>
> Yes, we definitely need to do that. (Though unfortunately Kamana will be
> gone all August.)
> I've also started collected some very brief notes and thoughts
> at http://wiki.sugarlabs.org/go/Karma/Assessment
>
>>
>> > ps:
>> > I intend to use Karma for interactive curriculum development for
>> > reasons Bryan Berry talked about a lot.
>>
>> is it time to for us to open a mailing list specific to karma?  like
>>  sugar-ka...@l.s.o ?
>
> Nope, personally I think this would be premature. Let's keep the discussion
> here on sugar-devel as far as the more technical aspects are concerned,
> maybe use IAEP for talking about broader educational issues and increase our
> post frequency on the Karma blog so people who don't want to follow the
> discussions on the mailing lists can also find out what's going on.

+1

I feel sorry when I hear that there's some group somewhere around the
world doing Sugar stuff and not using the Sugar channels for their
work. This means that the people here cannot help them even if we
would like to.

Regards,

Tomeu

> Christoph
> --
> Christoph Derndorfer
> co-editor, olpcnews
> url: www.olpcnews.com
> e-mail: christ...@olpcnews.com
>
> ___
> Sugar-devel mailing list
> Sugar-devel@lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/sugar-devel
>
>
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-30 Thread Christoph Derndorfer 
On Wed, Jul 29, 2009 at 6:28 PM, Bryan Berry  wrote:
>
> ChristophD, we really need to talk to sunil and kamana about assessment
> as they probably have a lot of good ideas. Well, actually they created
> all the assessments for EPaath but we need to abstract their work to
> something broadly usable.


Yes, we definitely need to do that. (Though unfortunately Kamana will be
gone all August.)

I've also started collected some very brief notes and thoughts at
http://wiki.sugarlabs.org/go/Karma/Assessment


> > ps: > I intend to use Karma for interactive curriculum development for
> > reasons Bryan Berry talked about a lot.
>
> is it time to for us to open a mailing list specific to karma?  like
>  sugar-ka...@l.s.o ?


Nope, personally I think this would be premature. Let's keep the discussion
here on sugar-devel as far as the more technical aspects are concerned,
maybe use IAEP for talking about broader educational issues and increase our
post frequency on the Karma blog so people who don't want to follow the
discussions on the mailing lists can also find out what's going on.

Christoph

-- 
Christoph Derndorfer
co-editor, olpcnews
url: www.olpcnews.com
e-mail: christ...@olpcnews.com
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-29 Thread Gary C Martin 

On 29 Jul 2009, at 06:40, Christoph Derndorfer wrote:

> As previously mentioned by Bryan in his "Automated Assessment is the  
> Killer App" blog post 
> (http://karmaproject.wordpress.com/2009/07/06/assessment-is-the-killer-app/ 
> ) student assessment is an important component of Karma.
>
> While toying around with the lesson UI I realized that the  
> assessment area is still very much an empty space (which kinda  
> reminds me of the group view in Sugar;-)

Feedback most welcome (or else you may get something you don't  
like)! ;-b

http://wiki.sugarlabs.org/go/Design_Team/Proposals/Groups

Regards,
--Gary

> as we haven't really discussed how it should look, what kind of data  
> will be kept (which is of course heavily dependent on the type of  
> lesson), how the data will be stored, etc. To keep the efforts  
> required to do that in check I think that it might make sense to  
> start by offering a handful of templates - both on the UI and  
> storage backend side of things - to fit the most common use-cases in  
> terms of different lesson types (e.g.inserted words into sentences,  
> doing calculations, etc.)
>
> Now I was wondering whether anyone here had specific suggestions on  
> how to address this or pointers to how other e-learning solutions  
> (regardless of whether stand-alone applications or Web based) solve  
> this interesting challenge.
>
> Thanks in advance,
> Christoph
>
> -- 
> Christoph Derndorfer
> co-editor, olpcnews
> url: www.olpcnews.com
> e-mail: christ...@olpcnews.com
> ___
> Sugar-devel mailing list
> Sugar-devel@lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/sugar-devel

___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-29 Thread NoiseEHC 



ps:
I intend to use Karma for interactive curriculum development for
reasons Bryan Berry talked about a lot.



Awesome! How can we involve you more? is it time to for us to open a
mailing list specific to karma?  like  sugar-ka...@l.s.o ?

u can find christoph, myself, and the ole nepal team on #olenepal or
#sugar Mon-Fri  03:00 -- 11:00 GMT (day time hours in Nepal)
  
The whole point of this project is that this school developed a 
curriculum that works. The plan is to use the inevitable transition to 
digital learning materials to push this working curriculum onto other 
schools. However it is not my project but the research school's project 
so until they decide that they start this project I cannot do anything 
education related since I am just a coder. I will talk with the 
headmaster in August next time so then I will be able to tell you 
something more concrete.


What I see is that in order to win the curriculum battle the following 
things must be met:
1. It has to be cheaper than paper books so parents will want it. As it 
seems the XO is already cheaper (200$/4years = 50$/year what is already 
spent on paper books).
2. It has to be wanted by children. The XO 1.5 (or maybe the ChromeOS or 
Android based smartbooks, we will see next year) will be able to run 
flash/youtube/web so it is OK.
3. It has to be wanted by teachers. So it has to ease teaching. Like 
automatic assessment.
4. It has to be developed on the cheap. Like reusing the skill set of 
thousands of existing web developers.
So it seems to me that it will not be much different than what you are 
doing in Nepal so we will have the same problems what you had and will 
need the same tools what you use...



___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-29 Thread Bryan Berry 
On Wed, 2009-07-29 at 12:01 +0200, NoiseEHC wrote:
> 1. Karma could measure the time it takes to finish a task in a
> compound exercise. If it is statistically relevant then it could give
> some hint to the teacher which parts of the curriculum are hard for
> the given child. If I will be able to persuade the headmaster then
> there will be research (hard data) which hopefully will find out
> whether this measurement is meaningful or pointless.

But also, the children should speed up over time doing certain times of
exercises.  For example, we found that kids were counting on their
fingers instead of inside their heads during an adding game until we put
a timer on it. 

I think measuring how long something takes is useful in a variety of
circumstances.

great point

ChristophD, we really need to talk to sunil and kamana about assessment
as they probably have a lot of good ideas. Well, actually they created
all the assessments for EPaath but we need to abstract their work to
something broadly usable.


> 2. There is a VERY HARD to develop exercise model where the child can
> follow multiple paths to the solution. In this case the possible
> choices the child can follow makes a graph with most of the leaves
> ending with "does not compute". If this exercise model proves to be
> useful (which will be determined in the mentioned research) then Karma
> should store the way the child solved the exercise and not only the
> end result. So what I wanted to say is that it is possible that the
> wrong steps can give more informations to the teacher about the
> knowledge of the child than the fact whether the child was able to
> solve the exercise or not.
> Because the 2. point can be a little bit hard to understand, here is
> an example:
> The problem is this: "If we used A amount of paint to fully paint a
> little picture with sides B and C, how much paint do we need to fully
> paint a room with sides D, E and height F?"
> This is an exercise with 3 steps. The child has to calculate the area
> of the picture (P=BxC) and the walls' (W=(D+E)*F*2) and has to
> calculate A/P*W.
> Now the graph looks something like this: from the beginning the child
> can calculate either area or can do something like B*E which "does not
> compute". If he has P and W then he can do either A/P or W/P then
> either A/P*W or W/P*A respectively. In every step the program should
> graphically show what the child just have calculated (what can be
> impossible if he uses D*E*F*B for example).
> 
> Now I do not insist that you develop this 2. point since it is a HUGE
> task (and needs some research to know whether it really help children
> understand complex relationships) but moving Karma into this direction
> (just a little bit) would help a lot if the headmaster would decide to
> do this research. In this case we will translate the resulting
> exercises to English, probably you will be interested in that.

Sounds like a hard but worthwhile task



> ps:
> I intend to use Karma for interactive curriculum development for
> reasons Bryan Berry talked about a lot.

Awesome! How can we involve you more? is it time to for us to open a
mailing list specific to karma?  like  sugar-ka...@l.s.o ?

u can find christoph, myself, and the ole nepal team on #olenepal or
#sugar Mon-Fri  03:00 -- 11:00 GMT (day time hours in Nepal)


> ps2:
> By research I mean "try out with kids while somebody from the
> University measures the outcome".



-- 
Bryan W. Berry
Technology Director
OLE Nepal, http://www.olenepal.org

___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

Re: [Sugar-devel] Assessment in Karma

2009-07-29 Thread NoiseEHC 
In June I was talking with the headmaster of a school which does 
education research. There are two things from that discussion that seems 
relevant for this question:


1. Karma could measure the time it takes to finish a task in a compound 
exercise. If it is statistically relevant then it could give some hint 
to the teacher which parts of the curriculum are hard for the given 
child. If I will be able to persuade the headmaster then there will be 
research (hard data) which hopefully will find out whether this 
measurement is meaningful or pointless.


2. There is a VERY HARD to develop exercise model where the child can 
follow multiple paths to the solution. In this case the possible choices 
the child can follow makes a graph with most of the leaves ending with 
"does not compute". If this exercise model proves to be useful (which 
will be determined in the mentioned research) then Karma should store 
the way the child solved the exercise and not only the end result. So 
what I wanted to say is that it is possible that the wrong steps can 
give more informations to the teacher about the knowledge of the child 
than the fact whether the child was able to solve the exercise or not.
Because the 2. point can be a little bit hard to understand, here is an 
example:
The problem is this: "If we used A amount of paint to fully paint a 
little picture with sides B and C, how much paint do we need to fully 
paint a room with sides D, E and height F?"
This is an exercise with 3 steps. The child has to calculate the area of 
the picture (P=BxC) and the walls' (W=(D+E)*F*2) and has to calculate A/P*W.
Now the graph looks something like this: from the beginning the child 
can calculate either area or can do something like B*E which "does not 
compute". If he has P and W then he can do either A/P or W/P then either 
A/P*W or W/P*A respectively. In every step the program should 
graphically show what the child just have calculated (what can be 
impossible if he uses D*E*F*B for example).


Now I do not insist that you develop this 2. point since it is a HUGE 
task (and needs some research to know whether it really help children 
understand complex relationships) but moving Karma into this direction 
(just a little bit) would help a lot if the headmaster would decide to 
do this research. In this case we will translate the resulting exercises 
to English, probably you will be interested in that.


ps:
I intend to use Karma for interactive curriculum development for reasons 
Bryan Berry talked about a lot.


ps2:
By research I mean "try out with kids while somebody from the University 
measures the outcome".



Christoph Derndorfer wrote:
As previously mentioned by Bryan in his "Automated Assessment is the 
Killer App" blog post 
(http://karmaproject.wordpress.com/2009/07/06/assessment-is-the-killer-app/) 
student assessment is an important component of Karma.


While toying around with the lesson UI I realized that the assessment 
area is still very much an empty space (which kinda reminds me of the 
group view in Sugar;-) as we haven't really discussed how it should 
look, what kind of data will be kept (which is of course heavily 
dependent on the type of lesson), how the data will be stored, etc. To 
keep the efforts required to do that in check I think that it might 
make sense to start by offering a handful of templates - both on the 
UI and storage backend side of things - to fit the most common 
use-cases in terms of different lesson types (e.g.inserted words into 
sentences, doing calculations, etc.)


Now I was wondering whether anyone here had specific suggestions on 
how to address this or pointers to how other e-learning solutions 
(regardless of whether stand-alone applications or Web based) solve 
this interesting challenge.


Thanks in advance,
Christoph

--
Christoph Derndorfer
co-editor, olpcnews
url: www.olpcnews.com 
e-mail: christ...@olpcnews.com 


___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel
  


___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

[Sugar-devel] Assessment in Karma

2009-07-28 Thread Christoph Derndorfer 
As previously mentioned by Bryan in his "Automated Assessment is the Killer
App" blog post (
http://karmaproject.wordpress.com/2009/07/06/assessment-is-the-killer-app/)
student assessment is an important component of Karma.
While toying around with the lesson UI I realized that the assessment area
is still very much an empty space (which kinda reminds me of the group view
in Sugar;-) as we haven't really discussed how it should look, what kind of
data will be kept (which is of course heavily dependent on the type of
lesson), how the data will be stored, etc. To keep the efforts required to
do that in check I think that it might make sense to start by offering a
handful of templates - both on the UI and storage backend side of things -
to fit the most common use-cases in terms of different lesson
types (e.g.inserted words into sentences, doing calculations, etc.)

Now I was wondering whether anyone here had specific suggestions on how to
address this or pointers to how other e-learning solutions (regardless of
whether stand-alone applications or Web based) solve this interesting
challenge.

Thanks in advance,
Christoph

-- 
Christoph Derndorfer
co-editor, olpcnews
url: www.olpcnews.com
e-mail: christ...@olpcnews.com
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel