Step right up, step right up, Confabulation Theory...

Was just reading a couple of Hecht-Nielsen's papers on Confabulation Theory (available at http://inc2.ucsd.edu/addedpages/techreports.html : "Confabulation Theory Synopis" (short, in 05.01.pdf), and "A Theory of Cortex" (long, in 04.04.pdf).

I was indirectly prompted to look up Hecht-Nielsen's stuff by a email from Ross Gayler who pointed me to Sahlgren's paper on random indexing, which had a reference to a 1994 Hecht-Nielsen paper:

Hecht-Nielsen (1994), … demonstrated that there are many more nearly orthogonal than truly orthogonal directions in a high-dimensional space.

It's undeniable that Hecht-Nielsen has chutzpa. The synopsis paper was somewhat tantalizing, but I found myself wondering "where's the beef?" Seems solidly in the tradition of scruffy AI -- thinking up a useful mechanism to solve an informally stated problem, rather than mathematically defining a problem and then deriving a way of solving it. He claims this theoretical architecture can do everything, but doesn't really specify how to do anything. I'd like to see some concrete examples of performing an interesting computation.

His profile at UCSD starts with:

An authority on neural networks, he introduced the first comprehensive theory of the mammalian cerebral cortex and thalamus in 2002.

Also left me wondering about what exactly is "Confabulation theory". He claims it is the principle that we choose the H that maximises p(E | H). Says this is different from Maximum A-Posteriori reasoning. So, what's the difference? He takes into account priors over different possible H's? So then it's Bayesian reasoning? If so, why not just say so? Bayesian reasoning is so widely known now that I would call it a jargonistic offence of the first order to readers to not state this if this is so.

"Confabulation Theory: A Synopsis" (TR#0501) cites "A Theory of Thalamocortex" for details (Chapter 4 of "Computational Models for Neuroscience" , ed Robert Hecht-Nielsen & Thomas McKenna, Springer, 2003). This chapter starts in wonderfully Hecht-Nielsenly way:

This chapter presents the first comprehensive high-level theory of
the information processing function of mammalian cortex and
thalamus; ... READER WARNING: The content of this chapter is
complicated and almost entirely novel and unfamiliar. ...

Although this chapter doesn't use the term "Confabulation Theory", it does contain "the principle of mammalian induction", which I think is the same thing. This is the reasoning rule that, when faced with pieces of evidence E1 through E_n, and possible hypothesis, H_1 through H_k, chooses the hypothesis H_i that maximizes min over j P(E_j | H_i). So, this is different to both Bayesian reasoning (which takes the joint probability of all evidence into account, as well as priors on the hypothesis) and MAP reasoning (which just takes the joint probability of all evidence into account). As Hecht-Nielsen says, this proposed reasoning rule could generate lots of testable hypothesis. At first glance, it does provide a better model of the common (incorrect) conclusion that people arrive at in the case of a positive outcome of a test for a rare disease (most people, including doctors, fail to take the priors into account and estimate a erroneously high probability of actually having the disease.) However, I'd be sort of surprised if this reasoning rule is novel.

Some other comments about confabulation theory:

interesting excerpts from a psychological work on confabulation; uncritical general article about Hecht-Nielsen's theory.

Personally, I suspect confabulation is far more central to human cognition (memory and reasoning) than most people think. So, maybe Hecht-Nielsen is onto something, or maybe it's just good marketing.

Random Indexing

Ross Gayler pointed out two interesting statements in An Introduction to Random Indexing (Magnus Sahlgren):

• “Hecht-Nielsen (1994), … demonstrated that there are many more nearly orthogonal than truly orthogonal directions in a high-dimensional space.” (Hecht-Nielsen, R.; Context vectors; general purpose approximate meaning representations self-organized from raw data. In Zurada, J. M.; R. J. Marks II; C. J. Robinson; Computational intelligence: imitating life. IEEE Press, 1994.)
  
• “the Johnson-Lindenstrauss lemma (Johnson & Lindenstrauss, 1984) – that states that if we project points in a vector space into a randomly selected subspace of sufficiently high dimensionality, the distances between the points are approximately preserved.” (Johnson, W. B. & J. Lindenstrauss; Extensions to Lipshitz mapping into Hilbert space. Contemporary mathematics. 26, 1984.)
  

Both these are very powerful properties of vector space representations, and these are particularly nice statemnts of these properties.