Theoretical neuroscience has experienced explosive growth over the past 20 years. In addition to bringing new researchers into the field with backgrounds in physics, mathematics, computer science, and engineering, theoretical approaches have helped to introduce new ideas and shape directions of neuroscience research. This review presents some of the developments that have occurred and the lessons they have taught us.
理论神经科学在过去 20 年经历了爆炸式的增长。除了将具有物理学、数学、计算机科学和工程学背景的新研究人员带入这一领域外，理论方法还帮助引入了新思想，并塑造了神经科学研究的方向。本综述将介绍其中的一些发展及其给我们带来的启示。
Introduction
Twenty years ago, when Neuron got its start, theoretical neuroscience was experiencing a start of its own. Of course, there were important theoretical contributions to neuroscience long before 1988, most notably: the development of what we now call the integrate-and-fire model by Lapicque in 1907; the modeling of the action potential by Hodgkin and Huxley, a brilliant theoretical offshoot of their experimental work; the development of dendritic and axonal cable theory by Wilfred Rall; and the broad insights of David Marr. Nevertheless, over the past 20 years, theoretical neuroscience has changed from a field practiced by a few multitalented experimentalists and dedicated theorists (Jack Cowan, Steven Grossberg, John Rinzel, and Terry Sejnowski being early examples) sparsely scattered around the world to an integral component of virtually every scientific meeting and major department. Something has changed. How did this happen, and what impact has it had?
20 年前，当 神经 刚刚起步时，理论神经科学也正经历着自己的起步。当然，在 1988 年之前，神经科学领域已经有了重要的理论贡献，最著名的包括：Lapicque 在 1907 年提出的我们现在称之为积分-发射模型的发展；Hodgkin 和 Huxley 对动作电位的建模，这是他们实验工作的一个出色的理论分支；Wilfred Rall 对树突和轴突电缆理论的发展；以及 David Marr 的广泛见解。然而，在过去的 20 年里，理论神经科学已经从一个由少数多才多艺的实验家和专注的理论家（如 Jack Cowan、Steven Grossberg、John Rinzel 和 Terry Sejnowski 等早期代表）在世界各地零星分布的领域，发展成为几乎每个科学会议和主要部门的一个不可或缺的组成部分。某些事情发生了变化。这是如何发生的？它产生了什么影响？
Two developments in the mid-1980s set the stage for the rapid expansion of theoretical neuroscience. One was the popularization of the backpropagation algorithm for training artificial neural networks (Rumelhart and McClelland, 1986). This greatly expanded the range of tasks that artificial neural networks could perform and led to a number of people entering neural network research. Around the same time, Amit, Gutfreund, and Sompolinsky (Amit et al., 1985) showed how a memory model proposed by Hopfield (1982) could be analyzed using methods of statistical physics originally designed for spin glasses. The sheer beauty of this calculation drew a large batch of physicists into the field. These new immigrants entered with high confidence-to-knowledge ratios that, hopefully, have been reduced through large growth in the denominators and more modest adjustments of the numerators.
1980 年代中期的两项发展为理论神经科学的快速扩展奠定了基础。其中一项是反向传播算法在训练人工神经网络中的普及（Rumelhart 和 McClelland，1986）。这大大扩展了人工神经网络可以执行的任务范围，并吸引了许多人进入神经网络研究。与此同时，Amit、Gutfreund 和 Sompolinsky（Amit 等人，1985）展示了如何使用最初为自旋玻璃设计的统计物理方法来分析 Hopfield（1982）提出的记忆模型。这一计算的纯粹美感吸引了一大批物理学家进入该领域。这些新移民带着高信心与知识比率进入，希望通过分母的大幅增长和分子的较小调整来降低这一比率。
What has a theoretical component brought to the field of neuroscience? Neuroscience has always had models (how would it be possible to contemplate experimental results in such complex systems without a model in one’s head?), but prior to the invasion of the theorists, these were often word models. There are several advantages of expressing a model in equations rather than words. Equations force a model to be precise, complete, and self-consistent, and they allow its full implications to be worked out. It is not difficult to find word models in the conclusions sections of older neuroscience papers that sound reasonable but, when expressed as mathematical models, turn out to be inconsistent and unworkable. Mathematical formulation of a model forces it to be self-consistent and, although self-consistency is not necessarily truth, self-inconsistency is certainly falsehood.
理论成分为神经科学领域带来了什么？神经科学一直有模型（在如此复杂的系统中，如果没有头脑中的模型，如何考虑实验结果？），但在理论家的入侵之前，这些模型往往是文字模型。用方程而不是文字来表达模型有几个优点。方程迫使模型变得精确、完整和自洽，并允许其全部含义得以实现。在较早的神经科学论文的结论部分，不难找到听起来合理但用数学模型表达时却被证明是不一致且不可行的文字模型。对模型进行数学表述迫使其自洽，尽管自洽不一定是真理，但自相矛盾无疑是假话。
A skillful theoretician can formulate, explore, and often reject models at a pace that no experimental program can match. This is a major role of theory—to generate and vet ideas prior to full experimental testing. Having active theoretical contributors in the field allows us collectively to contemplate a vastly greater number of solutions to the many problems we face in neuroscience. Both theorists and experimentalists generate and test ideas, but due to the more rapid turnover time in mathematical and computational compared to experimental analyses, theorists can act as initial filters of ideas prior to experimental investigation. In this regard, it is the theorist’s job to develop, test, frequently reject, and sometimes promote new ideas.
一位熟练的理论家可以以实验计划无法匹配的速度来制定、探索并经常否定模型。这是理论的一个主要作用——在全面实验测试之前激发和审查思想。让该领域有活跃的理论贡献者，使我们能够集体思考我们在神经科学中面临的许多问题的更多解决方案。理论家和实验家都在生成和测试想法，但由于数学和计算分析相比实验分析具有更快的周转时间，理论家可以在实验调查之前作为想法的初步过滤器。在这方面，理论家的工作是开发、测试、经常否定，有时推广新想法。
Theoretical neuroscience is sometimes criticized for not making enough predictions. This is part of a pre-versus-post debate about the field that has nothing to do with synapses. Although there are notable examples of predictions made by theorists and later verified by experimentalists in neuroscience, examples of postdictions are far more numerous and often more interesting. To apply prediction as the ultimate test of a theory is a distortion of history. Many of the most celebrated moments in quantitative science—the gravitational basis of the shape of planetary orbits, the quantum basis of the spectrum of the hydrogen atom, and the relativistic origin of the precession of the orbit of Mercury—involved postdictions of known and well-characterized phenomena. In neuroscience especially, experimentalists have gotten a big head start. There is nothing wrong with a model that ‘‘postdicts’’ previously known phenomena. The key test of the value of a theory is not necessarily whether it predicts something new, but whether it makes postdictions that generalize to other systems and provide valuable new ways of thinking.
理论神经科学有时因未能做出足够的预测而受到批评。这是关于该领域的前后辩论的一部分，与突触无关。尽管有一些著名的例子是理论家提出的预测后来被神经科学实验家验证，但事后解释的例子要多得多，而且往往更有趣。将预测作为检验理论的最终标准是对历史的扭曲。定量科学中许多最著名的时刻——行星轨道形状的引力基础、氢原子光谱的量子基础以及水星轨道进动的相对论起源——都涉及对已知且特征明确现象的事后解释。尤其是在神经科学中，实验家已经取得了很大的领先。一个“事后解释”先前已知现象的模型没有任何问题。检验理论价值的关键不一定是它是否预测了新事物，而是它是否做出了可以推广到其他系统并提供有价值的新思维方式的事后解释。
The development of a theoretical component to neuroscience research has had significant educational impact across the biological sciences. The Sloan-Swartz initiative, for example, has supported almost 80 researchers who successfully transitioned from other fields to faculty positions in neuroscience. Jim Bower and Christof Koch set up the computational neuroscience course at Woods Hole, a summer course that is still educating people with backgrounds in both the biological and physical sciences and that has been copied in courses around the world. Biology used to be a refuge for students fleeing mathematics, but now many life sciences students have a solid knowledge of basic mathematics and computer programming, and those that don’t at least feel guilty about it. A number of developments have led to this shift, the rise of theoretical neuroscience certainly being one of them.
神经科学研究中理论成分的发展对生物科学各领域产生了重大教育影响。例如，Sloan-Swartz 计划支持了近 80 名成功从其他领域转型为神经科学教职的研究人员。Jim Bower 和 Christof Koch 在伍兹霍尔开设了计算神经科学课程，这是一门夏季课程，至今仍在教育具有生物和物理科学背景的人，并且已被世界各地的课程所借鉴。生物学曾是逃避数学的学生的避难所，但现在许多生命科学学生对基础数学和计算机编程有扎实的了解，而那些没有的人至少对此感到内疚。许多发展促成了这一转变，理论神经科学的兴起无疑是其中之一。
The following sections provide a sparse sampling of theoretical developments that have occurred over the past 20 years and discuss some of the things they have taught us. The presentation is idiosyncratic, with some developments presented in a different context than when they first appeared and perhaps from what their creators intended, and many important achievements ignored entirely. The focus is on lessons learned from a subset of the theoretical advances over the past 20 years.
以下各节提供了过去 20 年中发生的一些理论发展的稀疏采样，并讨论了它们教给我们的一些东西。该介绍具有特异性，一些发展以与其首次出现时不同的背景呈现，可能与其创造者的意图不同，许多重要成就则完全被忽略。重点是过去 20 年部分理论进展中学到的经验教训。
Basic Principles
Many researchers have sought basic principles to help guide us through the complexities of neural circuits and cognition. Examples, discussed in the following paragraphs, are efficient coding, Bayesian inference, generative models, causality, and what I call the positivity of the neural code. The last of these, perhaps more accurately termed a basic constraint, is consider in a bit more detail to highlight a number of different developments.
许多研究人员一直在寻找基本原则，以帮助我们应对神经回路和认知的复杂性。以下段落中讨论的例子包括高效编码、贝叶斯推断、生成模型、因果关系以及我称之为神经编码的积极性。最后一个，也许更准确地说是一个基本约束，将被稍微详细地考虑，以突出一些不同的发展。
The efficient coding hypothesis, formulated by Horace Barlow (Barlow, 2001), postulates that sensory systems are adapted to convey information about natural stimuli faithfully using a minimal amount of activity. This idea has been used to account for the receptive field properties of neurons in the retina (Atick and Redlich, 1990) and lateral geniculate nucleus (Dong and Atick, 1995), in primary visual cortex (Olshausen and Field, 1996; Bell and Sejnowski, 1997), in the fly visual system (van Hateren, 1997; Niven et al., 2007), and in the auditory system (Lewicki, 2002). An offshoot of this work has been a better understanding of the statistical properties of natural stimuli, such as visual scenes (Field, 1987; Tolhurst et al., 1992; Ruderman and Bialek, 1994; van der Schaaf and van Hateren, 1996).
高效编码假说由 Horace Barlow 提出（Barlow，2001），假设感觉系统适应于使用最少的活动忠实地传递有关自然刺激的信息。这个想法被用来解释视网膜（Atick 和 Redlich，1990）和外侧膝状体核（Dong 和 Atick，1995）、初级视觉皮层（Olshausen 和 Field，1996；Bell 和 Sejnowski，1997）、苍蝇视觉系统（van Hateren，1997；Niven 等人，2007）以及听觉系统（Lewicki，2002）中神经元的感受野特性。这项工作的一个分支是对自然刺激的统计特性有了更好的理解，例如视觉场景（Field，1987；Tolhurst 等人，1992；Ruderman 和 Bialek，1994；van der Schaaf 和 van Hateren，1996）。
The application of Bayesian inference to both neural systems and the behaviors they generate is another “first principles” approach to neuroscience. As an example of Bayesian inference, suppose that our faith in a certain statement $S$ being true is characterized by a probability $P(S)$. Now, imagine that we make an observation $O$ that occurs with probability $P(O|S)$ if S is true and with probability $P(O)$ whether or not $S$ is true. Bayesian inference says that our belief in the veracity of the statement $S$ after this observation should be $P(S)P(O|S)/P(O)$. The key point here is that our belief in $S$ upon observation of $O$ should change by an amount that depends on the how much more likely the truth of $S$ makes $O$, quantified by the ratio $P(O|S)/P(O)$. Suppose that $S$ is the statement that a pair of dice is loaded, and $O$ is the observation of several consecutive rolls of double 6. This observation clearly increases the probability that $S$ is true and, according to Bayesian inference, it should do so to the extent that the loaded dice hypothesis makes those rolls more likely. Bayesian inference thus provides a principle for quantifying the effect that evidence should have on belief (specifically how new evidence and prior expectation should be combined), and it can be used to derive optimal ways that multiple forms of evidence should be weighed in making a decision. On the behavioral level, Bayesian inference has been applied to quantify how subjects combine multiple sources of information in proportion to their reliability (Ernst and Banks, 2002) and to account for aspects of motor control (Kording and Wolpert, 2006) and perception (Knill and Richards, 1996; Stocker and Simoncelli, 2006). In addition, proposals have been made for how such computations might be performed at the neuronal level (Ma et al., 2006).
将贝叶斯推断应用于神经系统及其产生的行为是神经科学的另一种“第一性原理”方法。作为贝叶斯推断的一个例子，假设我们对某个陈述 $S$ 为真的命题以概率 $P(S)$ 来表征。现在，假设我们进行了一次观察 $O$，如果 $S$ 为真，则该观察以概率 $P(O|S)$ 发生，而无论 $S$ 是否为真，该观察都以概率 $P(O)$ 发生。贝叶斯推断指出，在这次观察之后，我们对陈述 $S$ 的真实性的置信应该是 $P(S)P(O|S)/P(O)$。这里的关键点是，在观察到 $O$ 后，我们对 $S$ 的置信应该根据 $S$ 的真实性使得 $O$ 更有可能发生的程度而改变，这一程度由比率 $P(O|S)/P(O)$ 量化。假设 $S$ 是一对骰子被加载的陈述，而 $O$ 是观察到连续几次掷出双六。这一观察显然增加了 $S$ 为真的概率，根据贝叶斯推断，它应该在加载骰子假设使这些掷骰更有可能发生的程度上增加这一概率。因此，贝叶斯推断提供了一个原则，用于量化证据对置信应产生的影响（具体来说，新证据和先验期望应如何结合），并且它可以用来推导在做出决策时应如何权衡多种形式的证据。在行为层面上，贝叶斯推断已被应用于量化受试者如何根据其可靠性结合多种信息来源（Ernst 和 Banks，2002），并解释了运动控制（Kording 和 Wolpert，2006）和感知（Knill 和 Richards，1996；Stocker 和 Simoncelli，2006）的某些方面。此外，还提出了如何在神经元水平上执行此类计算的建议（Ma 等人，2006）。
A major problem we face in contemplating memory storage or perceptual processing is that we do not understand how neural activity represents information beyond fairly early stages in sensory processing. Generative models provide a guiding principle for thinking about higher-order neural representations (Hinton and Ghahramani, 1997; Rao et al., 2002). In a generative model, patterns of neural activity in a high-level area must not only represent the data, they must also be capable of generating patterns of activity at earlier sensory stages, through back-projections, that resemble the activity evoked by the external world. Representations with such generative capabilities provide a good basis for constructing networks that perform complex tasks (see, for example, Hinton et al., 2006).