Abstract

Continuous attractors offer a unique class of solutions for storing continuousvalued variables in recurrent system states for indefinitely long time intervals.

Unfortunately, continuous attractors suffer from severe structural instability in general—they are destroyed by most infinitesimal changes of the dynamical law that defines them.

This fragility limits their utility especially in biological systems as their recurrent dynamics are subject to constant perturbations.

连续吸引子为在循环系统状态中无限期存储连续值变量提供了一类独特的解决方案。

不幸的是，连续吸引子通常存在严重的结构不稳定性——它们会被定义它们的动力学规律的大多数微小变化所破坏。

这种脆弱性限制了它们的实用性，尤其是在生物系统中，因为它们的循环动力学受到持续扰动。

We observe that the bifurcations from continuous attractors in theoretical neuroscience models display various structurally stable forms. Although their asymptotic behaviors to maintain memory are categorically distinct, their finite-time behaviors are similar.

We build on the persistent manifold theory to explain the commonalities between bifurcations from and approximations of continuous attractors. Fast-slow decomposition analysis uncovers the existence of a persistent slow manifold that survives the seemingly destructive bifurcation, relating the flow within the manifold to the size of the perturbation. Moreover, this allows the bounding of the memory error of these approximations of continuous attractors.

我们观察到理论神经科学模型中连续吸引子的 分岔 显示出各种结构稳定的形式。尽管它们维持记忆的渐近行为在类别上是不同的，但它们的有限时间行为是相似的。

我们基于持久流形理论来解释连续吸引子的分岔和近似之间的共性。快慢分解分析揭示了一个持久慢流形的存在，它在看似破坏性的分岔中幸存下来，将流动与扰动大小联系起来。此外，这允许对这些连续吸引子近似的记忆误差进行界定。

Finally, we train recurrent neural networks on analog memory tasks to support the appearance of these systems as solutions and their generalization capabilities. Therefore, we conclude that continuous attractors are functionally robust and remain useful as a universal analogy for understanding analog memory.

最后，我们在模拟记忆任务上训练循环神经网络，以支持这些系统作为解决方案的出现及其泛化能力。因此，我们得出结论，连续吸引子在功能上是稳健的，并且仍然有用，作为理解模拟记忆的普遍类比。

Introduction

Biological systems exhibit robust behaviors that require neural information processing of analog variables such as intensity, direction, and distance. Virtually all neural models of working memory for continuous-valued information rely on persistent internal representations through recurrent dynamics.

The continuous attractor structure in their recurrent dynamics has been a pivotal theoretical tool due to their ability to maintain activity patterns indefinitely through neural population states 1–4. They are hypothesized to be the neural mechanism for the maintenance of eye positions, heading direction, self-location, target location, sensory evidence, working memory, and decision variables, to name a few 5–7. Observations of persistent neural activity across many brain areas, organisms, and tasks have corroborated the existence of continuous attractors.

生物系统表现出稳健的行为，这些行为需要对 模拟变量（如强度、方向和距离）进行神经信息处理。几乎所有用于连续值信息的工作记忆的神经模型都依赖于通过循环动力学实现的持久内部表示。

由于它们能够通过神经群体状态无限期地维持活动模式，它们在循环动力学中的连续吸引子结构一直是一个关键的理论工具。人们假设它们是维持 眼睛位置、航向方向、自我位置、目标位置、感官证据、工作记忆和决策变量 等的神经机制。在许多大脑区域、生物体和任务中观察到的持续神经活动证实了连续吸引子的存在。

Despite their widespread adoption as models of analog memory, continuous attractors are brittle mathematical objects, casting significant doubts on their ontological value and hence suitability in accurately representing biological functions.

Even the smallest arbitrary change in recurrent dynamics can be problematic, destroying the continuum of fixed points essential for continuous-valued working memory. In neuroscience, this vulnerability is well-known and often referred to as the “fine-tuning problem”.

尽管它们被广泛采用作为模拟记忆的模型，但连续吸引子是 脆弱的数学对象，这对它们的本体价值以及准确表示生物功能的适用性提出了重大质疑。

即使是循环动力学中的最小任意变化也可能是有问题的，破坏了连续值工作记忆所必需的固定点连续体。在神经科学中，这种脆弱性是众所周知的，通常被称为 “微调问题”。

There are two primary sources of perturbations in the recurrent network dynamics:

(1) the stochastic nature of online learning signals that act via synaptic plasticity, and

(2) spontaneous fluctuations in synaptic weights.

Thus, additional mechanisms are necessary to compensate for the degradation in particular implementations, by bringing the short-term behavior closer to that of a continuous attractor.

循环网络动力学中扰动的两个主要来源是：

（1）通过突触可塑性作用的在线学习信号的随机性，以及

（2）突触权重的自发涨落。

因此，需要额外的机制来补偿特定实现中的退化，通过使短期行为更接近连续吸引子的行为。

However, we lack the theoretical basis to understand how much this matters in practice, i.e. what are the effects of different levels of degradation on memory. This is fundamental to justify relying on the brittle concept of continuous attractors for understanding biological analog working memory.

然而，我们缺乏理论基础来理解这实际上有多重要，即不同水平退化对记忆的影响是什么。这对于证明依赖于脆弱的连续吸引子概念来理解生物模拟工作记忆是基本的。

In this study, we explore perturbations and approximations of continuous attractors in the space of dynamical models. We first report on the differences and similarities between the various structurally stable dynamics in the vicinity of continuous attractors in the space of dynamical systems models.

Our analysis reveals the presence of a “ghost” continuous attractor (a.k.a. slow manifold) in all of them (Sec. 2).

By assuming normal hyperbolicity we separate the time scales to obtain a decomposition of the dynamics by separating out the fast flow normal to and the slow flow within the slow manifold. We derive theoretical results that ensure the existence of a slow manifold and determine its closeness to a continuous attractor (Sec. 3).

在本研究中，我们探索了动力学模型空间中连续吸引子的扰动和近似。我们首先报告了动力系统模型空间中连续吸引子附近各种结构稳定动力学之间的差异和相似性。

我们的分析揭示了它们中都存在一个 “幽灵” 连续吸引子（又名慢流形）（第2节）。

通过假设正常超曲率，我们 分离时间尺度，通过分离快流（垂直于慢流形）和慢流（在慢流形内）来获得动力学的分解。我们推导了理论结果，确保慢流形的存在，并确定其与连续吸引子的接近程度（第3节）。

We explore task-trained recurrent neural networks (RNNs) to show that these systems appear naturally as solutions to the task (Sec. 4) and that their generalization capabilities can easily be studied as the distance to the continuous attractor (Sec. 5).

The proposed decomposition applied to theoretical models and task-trained RNNs reveals a “universal motif” of analog memory mechanism with various potential topologies. This leads to the connection of different systems with different topologies as approximate continuous attractors (Sec. 6).

Our theory guarantees that systems close to a continuous attractor (in the space of vector fields) will have similar behavior to it, implying that the concept of continuous attractors remains a crucial framework for understanding the neural computation underlying analog memory (Sec. 3.4).

我们探索了任务训练的循环神经网络（RNNs），以显示这些系统自然地作为任务的解决方案出现（第4节），并且它们的泛化能力可以很容易地作为与连续吸引子的距离来研究（第5节）。

将所提出的分解应用于理论模型和任务训练的 RNNs 揭示了具有各种潜在拓扑结构的模拟记忆机制的 “通用模式”。这导致了不同拓扑结构的不同系统作为近似连续吸引子的连接（第6节）。

我们的理论保证了接近连续吸引子的系统（在向量场空间中）将具有类似的行为，这意味着连续吸引子的概念仍然是理解模拟记忆背后的神经计算的重要框架（第3.4节）。

A critique of pure continuous attractors

We will first lay out a number of observations about the dynamics of bifurcations and approximations of continuous attractors used in theoretical neuroscience. Ordinary differential equations (ODEs) are commonly used to describe the dynamical laws governing the temporal evolution of firing rates or latent population states 1. In this framework, neural systems are viewed as implementing the continuous time evolution of neural states to perform computations. We will consider a continuous attractor as a mechanism that implements analog memory computation: carrying a particular memory representation over time. To define it formally, let $\mathbf{x}(t)\in\mathbb{R}^{d}$ denote the neural state, and $\dot{\mathbf{x}}=\mathbf{f}(\mathbf{x})$ represent its dynamics. Let $\mathcal{M}\subset \mathbb{R}^{d}$ be a manifold. We say $\mathcal{M}$ is a continuous attractor, if (1) every state on the manifold is a fixed point, $\forall\mathbf{x}\in\mathcal{M}$, $\mathbf{f} (\mathbf{x}) = 0$, and (2) the fixed points are marginally stable tangent to the manifold and stable normal to the manifold. In other words, the continuous attractor is a continuum of equilibrium points such that the neural state near the manifold is attracted to it, and on the manifold, the state does not move. Marginal stability implies that continuous systems are structurally unstable, meaning that small perturbations or variations in the system’s parameters lead to significant changes in the system’s behavior or stability. We will now study some examples of continuous attractors and how perturbations change their dynamics.

我们首先将提出一些关于理论神经科学中使用的连续吸引子的分岔和近似动力学的观察。常微分方程（ODEs）通常用于描述控制发射率或潜在群体状态时间演化的动力学规律。在这个框架中，神经系统被视为实现神经状态的连续时间演化以执行计算。我们将考虑连续吸引子作为实现模拟记忆计算的机制：随时间携带特定的记忆表示。为了正式定义它，让 $\mathbf{x}(t)\in\mathbb{R}^{d}$ 表示神经状态，并且 $\dot{\mathbf{x}}=\mathbf{f}(\mathbf{x})$ 表示其动力学。设 $\mathcal{M}\subset \mathbb{R}^{d}$ 为流形。如果满足以下条件，我们称 $\mathcal{M}$ 为连续吸引子：(1) 流形上的每个状态都是一个固定点，$\forall\mathbf{x}\in\mathcal{M}$, $\mathbf{f} (\mathbf{x}) = 0$，以及 (2) 固定点在流形切线方向上是边际稳定的，在流形法线方向上是稳定的。换句话说，连续吸引子是一系列平衡点，使得流形附近的神经状态被吸引到它上面，并且在流形上，状态不会移动。边际稳定性意味着连续系统在结构上是不稳定的，这意味着系统参数的小扰动或变化会导致系统行为或稳定性的显著变化。我们现在将研究一些连续吸引子的例子，以及扰动如何改变它们的动力学。