Introduction

1.1 Inspiration from Neuroscience

Neurons

McCulloch-Pitts Neuron Model

$$ n_{i}(t+1) = \Theta\left[\sum_{j}w_{ij}n_{j}(t)-\mu_{i}\right],\quad \Theta(x) = \begin{cases} 1, & x \geq 0 \\ 0, & \text{otherwise} \end{cases} $$

$w_{ij}$: the strength of the synapse connecting neuron $j$ to neuron $i$.

graded response: respond to their input in a continuous way

asynchronous updating(异步更新)

$$ n_{i}:= g\left( \sum_{j}w_{ij}n_{j}-\mu_{i} \right) $$

$n_{i}$: the state or activation of unit $i$.

$g(x)$: (nonlinear) activation function, gain function, transfer function, or squashing function

neuron $\rightarrow$ unit(processing element, neurodes), synapse(突触) $\rightarrow$ weight(权重).

Parallel Processing

数量的巨大以允许错误与噪声.

1.2 History

associative content-addressable memory(联想内容寻址记忆), CAM