There was a paper indicating 4 to 16 demultiplexing occurring in the brain.
That is also a sort of step function (4 binary inputs light up one of 16
outputs.) If I do 1 to 2 demultiplexing in single layer nets I get great
generalization ability but rather slow training. If I do 4 to 16
de
Recently I recognized more clearly that step activation functions in single
layer neural networks give the best performance in terms of learning speed and
separation of similar inputs.
I use the signof function:
fn(x) = 1, x>=0
fn(x) =-1, x<0
Or a soft version:
fn(x) = sqr(x), x>=0
fn(x)
