One of the subjects I have passion about is the development of Intelligence, be it looking at how a baby learns, studying about Machine Learning (either new or old papers), psychology or neuroscience I like to keep up to date

An important, but often overlooked subject is the way of encoding data is done and manipulated as input for an Artificial Neural Network, even if there are many ways of encoding the input, none of them tackle the subject in a way that can be used universally for all types of input, and they mostly depend on data pre-processing to get the set of possible inputs restricting the values and domains in which the pre-trained networks can be used.

In the spirit of having a fresh view I started from the begining, and studied the subject in depth mainly starting in the neuroscience field from many research papers and mainly based on the following two books:

  • Neural Engineering (Eliasmith and Anderson- 2003)
  • How to Build a Brain (Eliasmith - 2013)

Then did some studies and experiments on the subject which now I’m going to be releasing in a way that my time allows (which is in a blog post form). The project will be develped in this repository.

While this studies goals were only personal and dedicated to understanding more on how the brain works encoding and decoding information, it gave me some ideas to test on how to encode information as input for Artificial Neural Networks, which led to more studies into the field of encoding and decoding.

This page lists only the first part of the study on how information can be encoded, combined and extracted, without proposing any new encoding mechanism. The Encoding propositions will be added in future posts.

This document will be updated when each new page is ready and the links added to the following list

  1. An Introduction to Scalar Representations in Neural Models . This shows the basic concepts on which the rest of the studies are based.
  2. Sparse Distributed Representations of Scalar Values on Non Spiking Networks TBD
  3. Combining concepts, Binding and Unbinding Vectors