Hi.
Here are some additional thoughts on OpenCog+PPL, which I didn't include in
the first message
*OpenCog + PPL*
One of the ideas/tasks was to make an OpenCog a “better Church”.
*1. **Why is it possible*
Universal probabilistic programming languages (PPLs) utilize sampling-based
approaches to infer posterior probabilities. They do not “reason” about
tasks.
Consider a number of examples.
1.1)
(rejection-query
(define n (random-integer 1000000))
n
(= n 10))
This program says that *n* is a random integer number from 0 to 999999, and
it tries to estimate P(*n*|*n*=10). It will take quite long also it is
obvious that P(*n*=10|*n*=10)=1. We can imagine an (extended) Pattern
Matcher easily deducing *n*=10 and checking it fits [0, 999999].
1.2)
(rejection-query
(define n (random-integer 10))
(define m (+ n 5))
n
(= m 0))
Here, rejection-query will hang forever, since the condition cannot be met.
Again, the absence of the answer can be easily deduced (given some
properties of +).
1.3) Einstein's (zebra) puzzle can be easily represented in Church
(define where list-index)
(define what list-ref)
(define (is? xs x ys y)
(eq? (what ys (where xs x)) y))
(define (neighbour? x y)
(or (= x (+ y 1))
(= x (- y 1))))
(rejection-query
(define colors (shuffle '(Yellow Blue Red Ivory Green)))
(define nationalities (shuffle '(Norwegian Ukrainian Englishman Spaniard
Japanese)))
(define drinks (shuffle '(Water Tea Milk OrangeJuice Coffee)))
(define smokes (shuffle '(Kools Chesterfield OldGold LuckyStrike
Parliament)))
(define pets (shuffle '(Fox Horse Snails Dog Zebra)))
(what nationalities (where drinks 'Water))
(and (is? nationalities 'Englishman colors 'Red)
(is? nationalities 'Spaniard pets 'Dog)
(is? drinks 'Coffee colors 'Green)
(is? nationalities 'Ukrainian drinks 'Tea)
(= (where colors 'Green) (+ (where colors 'Ivory) 1))
(is? smokes 'OldGold pets 'Snails)
(is? smokes 'Kools colors 'Yellow)
(= (where drinks 'Milk) 2)
(= (where nationalities 'Norwegian) 0)
(neighbour? (where smokes 'Chesterfield) (where pets 'Fox))
(neighbour? (where smokes 'Kools) (where pets 'Horse))
(is? smokes 'LuckyStrike drinks 'OrangeJuice)
(is? nationalities 'Japanese smokes 'Parliament)
(neighbour? (where nationalities 'Norwegian) (where colors 'Blue))
)
)
Again, blind search performed by PPLs is too inefficient here. OpenCog can
solve this problem quite efficiently, and we can imagine that an equivalent
probabilistic program is written in Atomese and URE deductively infers the
answer starting from constraints without actually sampling random variables.
We can see that from PPLs perspective, OpenCog can be used to make
inference over probabilistic programs much more efficient (at least, in
some cases). Why not write these programs in Atomese directly? For what
reason do we need a PPL metaphor?
*2. **What is missing*
Consider the following example.
2.1)
(rejection-query
(define x (gaussian 0 1))
(define y (gaussian 0 1))
x
(= (+ x y) 1))
It will actually not work in Church, because the strict condition will not
be satisfied (but it can be modified with the use of soft equality). URE
(given necessary axioms) can easily infer *y*=1–*x*. However, it will not
be able to ground variables. Actually, these are not variables to be
grounded using number nodes, but rather values should be assigned to them.
Also, we shouldn’t just sample *x* and then calculate *y*=1–*x*, because
different values of *y* have different prior probability. Thus, if we want
to estimate posterior probabilities, we should add specific mechanisms of
taking prior probabilities of inferred values into account. Of course, we
will also need some basic random distributions.
2.2) Fitting a polynomial with an unknown degree.
(define (calc-poly x ws)
(if (null? ws) 0
(+ (car ws) (* x (calc-poly x (cdr ws))))))
(define (calc-poly-noise x ws sigma)
(+ (calc-poly x ws) (gaussian 0 sigma)))
(define (generate xs ws sigma)
(map (lambda (x) (calc-poly-noise x ws sigma)) xs))
(define (sum-dif2 xs ys)
(if (null? xs) 0
(+ (+ (* (- (car xs) (car ys)) (- (car xs) (car ys))))
(sum-dif2 (cdr xs) (cdr ys)))))
(define (samples xs ys)
(mh-query 10 10
(define degree (sample-integer 4))
(define ws (repeat (+ 1 degree) (lambda () (gaussian 0 3))))
(define sigma (gamma 1 2))
degree
(< (sum-dif2 (generate xs ws sigma) ys) 0.5)))
(define xs '(0 1 2 3))
(define ys (generate xs '(0.1 1 2) 0.01))
(hist (samples xs ys) "degree")
This program actually works and finds a correct degree of the polynomial
using Bayesian Occam razor that is available “for free” in PPLs. Here,
Atomese appears to be rather useless (constraints cannot be deductively
propagated/pattern-matched, and there is no set of atoms to which variables
can be grounded). Small modifications of Pattern Matcher might be enough
(e.g. introducing RandomVariableNode and sampling its value from a given
distribution instead of systematically enumerating all possible
groundings), but there still remain some problems (e.g. with
marginalization, stochastic recursion, the inefficiency of blind guess in
comparison with the metaheuristic search, etc.).
-- Alexey
--
You received this message because you are subscribed to the Google Groups
"opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to opencog+unsubscr...@googlegroups.com.
To post to this group, send email to opencog@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit
https://groups.google.com/d/msgid/opencog/CABpRrhx1GXntp%2BBo5T1LSH3rGdOgv%2Bw2JVAYhZkn1YU%2BEkWZbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
