***
 And anyway why would your pure experience-based learning approach be any less likely to lead to subtly but dangerously warped ethical systems? The trainers could make errors and a Novamente's self-learning could be skewed by the limits of its experience and the modelling it observes.   
***
 
I don't have a good argument on this point, just an intuition, based on the fact that generally speaking in narrow AI, inductive learning based rules based on a very broad range of experience, are much more robust than expert-encoded rules.  The key is a broad range of experience, otherwise inductive learning can indeed lead to rules that are "overfit" to their training situations and don't generalize well to fundamentally novel situations.
 
A key form of ethical testing will be to give the system situations very different from those it has ever seen in the past, and see how well it generalizes its ethical knowledge to these domains.
 
***
 For example, why couldn't someone outside your group get a copy of a Novamente and just strip out the learned data and then retrain the new copy of Novamente themselves?   
***
 
That danger exists no matter what method you use.  If you use expert rules for ethics, someone could steal a copy and fill it up with expert rules for evil instead...
 
 
 ***
 Getting back to your basic preferred concept of using expience-based learning to build a Novamente's ethical beliefs - this means that every Novamente has to start as a tabula rasa and in effect learn all the lessons of evolution all by itself.   
***
 
If there need to be multiple Novamentes (not clear -- one might be enough), they could be produced through "cloning" rather than raising each one from scratch.
 
The engineering/teaching of ethics in an AI system is pretty different from its evolution in natural systems...
 
-- Ben G

Reply via email to