Abstract

Intrinsic dynamics within the brain can accelerate learning by providing a prior scaffolding for dynamics aligned with task objectives.

Such intrinsic dynamics should self-organize and self-sustain in the face of fluctuating inputs and biological noise, including synaptic turnover and cell death.

大脑内部的内在动力学能够通过为与任务目标相一致的动态过程提供先验框架，从而加速学习。

此类内在动力学应在面对波动的输入和生物噪声（包括突触更新和细胞死亡）时，能够自组织并自我维持.

An example of such dynamics is the formation of sequences, a ubiquitous motif in neural activity. The sequencegenerating circuit in zebra finch HVC provides a reliable timing scaffold for motor output in song and demonstrates a remarkable capacity for unsupervised recovery following perturbation.

序列的形成便是此类动力学的一个例子，它是神经活动中无处不在的模式。

斑胸草雀 HVC 中的序列生成回路为鸣唱时的运动输出提供了可靠的时序支架，并展现出在受到扰动后进行无监督恢复的卓越能力。

Inspired by HVC, we seek a local plasticity rule capable of organizing and maintaining sequence-generating dynamics despite continual network perturbations.

We adopt a meta-learning approach introduced by Confavreux et al, which parameterizes a learning rule using basis functions constructed from pre- and postsynaptic activity and synapse size, with tunable time constants.

Candidate rules are simulated within initially random networks, and their fitness is evaluated according to a loss function that measures the fidelity with which the resulting dynamics encode time.

We use this approach to introduce biological noise, forcing meta-learning to find robust solutions.

受 HVC 启发，我们致力于寻找一种局部可塑性规则，使其能够在持续的网络扰动下组织并维持序列生成动力学。

我们采用 Confavreux 等人提出的元学习方法，该方法利用由突触前和突触后活动以及突触大小构成的基函数对学习规则进行参数化，并配备可调的时间常数。

候选规则在初始随机网络中进行模拟，其适应度根据一个损失函数进行评估，该函数衡量生成的动力学编码时间的精密度。

我们利用该方法引入生物噪声，迫使元学习寻找稳健的解。

We first show that, in the absence of perturbation, meta-learning identifies a temporally asymmetric generalization of Oja’s rule that reliably organizes sparse sequential activity.

When synaptic turnover is introduced, the learned rule incorporates an additional form of homeostasis, better maintaining sequential dynamics relative to other previously proposed rules.

Additionally, inspired by recent findings demonstrating plasticity in synapses from inhibitory interneurons in HVC, we explore the role of inhibitory plasticity in sequence-generating circuits.

首先，我们证明在无扰动条件下，元学习能识别出 Oja 规则的一种时间不对称推广形式，该规则能可靠地组织稀疏的序列活动。

当引入突触更新时，所学习的规则会整合一种额外的稳态机制，相较于先前提出的其他规则，更能有效维持序列动力学。

此外，受近期关于 HVC 中抑制性中间神经元突触可塑性的研究启发，我们探讨了抑制性可塑性在序列生成回路中的作用。

We find that learned plasticity adjusts both excitation and inhibition in response to manipulations, outperforming rules applied only to excitatory connections.

We demonstrate how plasticity acting on both excitatory and inhibitory synapses can better shape excitatory cell dynamics to scaffold timing representations.

我们发现，学习到的可塑性会根据操作调整兴奋性和抑制性，其表现优于仅作用于兴奋性连接的规则。

我们证明了作用于兴奋性和抑制性突触的可塑性如何能更好地塑造兴奋性细胞动力学，从而构建时间表征的框架。

Introduction

How computational structures are organized and maintained within the brain is a central question within neuroscience.

While feedback is clearly essential for learning, self-organization of neural circuits can unfold without feedback, e.g. during development.

Brains have evolved specific cell types with nonrandom spatial organization, plasticity rules, and connectivity that likely introduce a strong set of inductive biases on the information processing they perform.

How might organization of useful circuit dynamics be established and maintained throughout life without the need for feedback?

神经系统中计算结构是如何组织和维持的，是神经科学中的一个核心问题。

虽然反馈对于学习显然是必不可少的，但神经回路的自组织可以在没有反馈的情况下展开，例如在发育过程中。

大脑已经进化出具有非随机空间组织、可塑性规则和连接性的特定细胞类型，这些可能对它们执行的信息处理引入了一套强烈的归纳偏差。

没有反馈的情况下，如何在整个生命过程中建立和维持有用的回路动力学的组织？

Recent work suggests self-organized computations, once established, can accelerate learning and improve performance when experience is limited: Nicola and Clopath [1] demonstrated that a stable high dimensional time signal could improve a network’s performance on sequential motor tasks (Fig. 1a).

Najarro and Risi [2] learned Hebbian plasticity that orchestrated spontaneous walking behavior in quadruped agents; similar work has shown architectural priors increase the sample efficiency and generalization of RL approaches to locomotion [3, 4].

Additionally, in RL settings, supplying agents with a time input permits them to adopt time-dependent policies [5]. The ability of computational primitives, such as timing representations, to self-organize is challenged by the shifting structure of neural circuits. Synaptic loss, synaptogenesis, cell death, and neurogenesis pose challenges for all learning algorithms, but particularly for self-organization which must be based solely on local information rather than global task performance.

最近的工作表明，一旦建立，自组织计算可以加速学习并在经验有限时提高性能：Nicola 和 Clopath 证明了稳定的高维时间信号可以改善网络在序列运动任务上的表现（图 1a）。

Najarro 和 Risi 学习了协调四足动物代理自发行走行为的 Hebbian 可塑性；类似的工作表明，架构先验增加了 RL 方法在运动方面的样本效率和泛化能力。

此外，在 RL 设置中，为代理提供时间输入使它们能够采用时间依赖的策略。计算原语（如时间表征）的自组织能力受到神经回路结构变化的挑战。突触丢失、突触发生、细胞死亡和神经发生对所有学习算法都构成挑战，但特别是对于必须完全基于局部信息而非全局任务性能进行自组织的算法来说更是如此。

Figure 1: Meta-learning approach to discovering plasticity rules that organize sequences

图 1: 发现组织序列的可塑性规则的元学习方法

(a) In zebra finch song learning, a neural representation of time (left) in HVC simplifies the sequential motor learning task of producing the correct spectral output.

(a) 在斑胸草雀的鸣唱学习中，HVC 中的时间神经表征（左）简化了产生正确频谱输出的序列运动学习任务。

(b) Putative network structure of zebra finch HVC: a feed-forward, excitatory network with recurrent inhibition (left). HVC excitatory neurons (red) fire sparsely in time while interneurons (blue) fire tonically (right).

(b) 斑胸草雀 HVC 的假定网络结构：具有递归抑制的前馈兴奋性网络（左）。HVC 兴奋性神经元（红色）在时间上稀疏地放电，而中间神经元（蓝色）持续放电（右）。

(c) Strategy for learning plasticity underlying sequence organization: candidate plasticity rules, parameterized by a set of coefficients and time constants, are simulated. A loss function is evaluated on the resulting dynamics, and new candidate rules are generated.

(c) 学习组织序列的可塑性基础的策略：通过一组系数和时间常数参数化的候选可塑性规则被模拟。对生成的动力学评估一个损失函数，并生成新的候选规则。

(d) Test procedure for representation of time. Networks are activated 400 times (red bars). From the final 50 activations, six are chosen to train a decoder and six to test the representation by decoding time from neural activity.

(d) 时间表征的测试程序。网络被激活 400 次（红色条）。从最后 50 次激活中，选择六次来训练解码器，并选择六次来测试通过神经活动解码时间的表征。

(e) Discovery of robust plasticity rules is encouraged by introducing synaptic turnover, the stochastic addition and removal of synapses, into simulations.

(e) 通过在模拟中引入突触更新（突触的随机添加和移除）来鼓励发现稳健的可塑性规则。

Here, we aim to find plasticity rules that self-organize and maintain one useful computational primitive: sparse, sequential activity. Such activity is widely seen in many areas of the brain including hippocampus [6], cortex [7], and basal ganglia [8].

In the songbird zebra finch, area HVC (used as a proper noun), a cortical-like region, displays sequential activity representing time [9], reducing the problem of motor learning to driving the correct motor neuron at the correct moment [10].

在这里，我们的目标是找到能够自组织和维持一种有用的计算原语的可塑性规则：稀疏的序列活动。这种活动在大脑的许多区域都广泛存在，包括海马、皮层和基底神经节。

在鸣唱鸟斑胸草雀中，HVC 区域（用作专有名词）是一个类似于皮层的区域，显示出代表时间的序列活动，将运动学习的问题简化为在正确的时刻驱动正确的运动神经元.

Extensive literature has explored how such sequence-generating circuits could emerge in the absence of feedback, but has largely focused on either how these structures organize or how they self-maintain, using guessed plasticity rules, and neglecting the effects of ongoing synaptic noise. Further, previous work on sequence organization within HVC has focused on plasticity between excitatory (E) neurons. Recent experimental findings show unsupervised recovery of HVC dynamics is accompanied by changes in both E→E and also inhibitory-to-excitatory (I→E) synaptic strength [18].

大量文献探讨了在没有反馈的情况下如何出现这样的序列生成回路，但主要关注这些结构是如何组织或自我维持的，使用猜测的可塑性规则，并忽略了持续突触噪声的影响。此外，以前关于 HVC 中序列组织的工作主要关注兴奋性（E）神经元之间的可塑性。最近的实验发现表明，HVC 动态的无监督恢复伴随着 E→E 和抑制性到兴奋性（I→E）突触强度的变化。