Introduction

Decades of cognitive research indicate that our brain maintains a model of the world, based on which it can make predictions about upcoming stimuli [35, 7]. Predicting the sensory experience is useful for both perception and learning: Perception becomes more tolerant to uncertainty and noise when sensory information and predictions are integrated [43]. Learning can happen when predictions are compared to sensory information, as the resulting prediction error indicates how to improve the internal model. In both cases, the uncertainties (associated with both the sensory information and the internal model) should determine how much weight we give to the sensory information relative to the predictions, according to theoretical accounts. Behavioural and electrophysiological studies indicate that humans indeed estimate uncertainty and adjust their behaviour accordingly [43, 58, 20, 6, 33]. The neural mechanisms underlying uncertainty and prediction error computation are, however, less well understood. Recently, the activity of individual neurons of layer 2/3 cortical circuits in diverse cortical areas of mouse brains has been linked to prediction errors (visual, [29, 64, 15, 1, 18], auditory [11, 30], somatosensory [2], and posterior parietal [49]). Importantly, prediction errors could be associated with learning [27]. Prediction error neurons are embedded in neural circuits that consist of heterogeneous cell types, most of which are inhibitory. It has been suggested that prediction error activity results from an imbalance of excitatory and inhibitory inputs [23, 22], and that the prediction is subtracted from the sensory input [see e.g. 50, 1], possibly mediated by so-called somatostatin-positive interneurons (SSTs) [1]. How uncertainty is influencing these computations has not yet been investigated. Prediction error neurons receive inputs from a diversity of inhibitory cell types (Fig. 1), the role of which is not completely understood. Here, we hypothesise that one role of inhibition is to modulate the prediction error neuron activity by uncertainty.

Figure 1.

In this study, we use both analytical calculations and numerical simulations of rate-based circuit models with different inhibitory cell types to study circuit mechanisms leading to uncertainty-modulated prediction errors. First, we derive that uncertainty should divisively modulate prediction error activity and introduce uncertainty-modulated prediction errors (UPEs). We hypothesise that layer 2/3 prediction error neurons reflect such UPEs, and that different inhibitory cell types are involved in calculating the difference between predictions and stimuli compared to the uncertainty modulation. Based on experimental findings, we suggest that SSTs and PVs play the respective roles. We then derive biologically plausible plasticity rules that enable those cell types to learn the means and variances from their inputs. Notably, because the information about the stimulus distribution is stored in the connectivity, single inhibitory cells encode the means and variances of their inputs in a context-dependent manner. Layer 2/3 pyramidal cells in this model hence encode uncertainty-modulated prediction errors context-dependently. We show that error neurons can additionally implement out-of-distribution detection by amplifying large errors and reducing small errors with a nonlinear fI-curve (activation function). Finally, we demonstrate that UPEs effectively mediate an adjustable learning rate, which allows fast learning in high-certainty contexts and reduces the learning rate, thus suppressing fluctuations in uncertain contexts.

Results

Normative theories suggests uncertainty-modulated prediction errors (UPEs)

In a complex, uncertain, and hence partly unpredictable world, it is impossible to avoid prediction errors. Some prediction errors will be the result of this variability or noise, other prediction errors will be the result of a change in the environment or new information. Ideally, only the latter should be used for learning, i.e., updating the current model of the world. The challenge our brain faces is to learn from prediction errors that result from new information, and less from prediction errors that result from noise. Hence, intuitively, if we learned that a kind of stimulus or context is very variable (high uncertainty), then a prediction error should have only little influence on our model. Consider a situation in which a person waits for a bus to arrive. If they learned that the bus is not reliable, another late arrival of the bus does not surprise them and does not change their model of the bus (Fig. 1A). If, on the contrary, they learned that the kind of stimulus or context is not very variable (low uncertainty), a prediction error should have a larger impact on their model. For example, if they learned that buses are reliable, they will notice that the bus is late and may use this information to update their model of the bus (Fig. 1A). This intuition of modulating prediction errors by the uncertainty associated with the stimulus or context is supported by both behavioural studies and normative theories of learning. Here we take the view that uncertainty is computed and represented on each level of the cortical hierarchy, from early sensory areas to higher level brain areas, as opposed to a task-specific uncertainty estimate at the level of decision-making in higher level brain areas (Fig. 1B) [see this review for a comparison of these two accounts: 65].

Before we suggest how cortical circuits compute such uncertainty-modulated prediction errors, we consider the normative solution to a simple association that a mouse can learn. The setting we consider is to predict a somatosensory stimulus based on an auditory stimulus (Fig. 1A). The auditory stimulus a is fixed, and the subsequent somatosensory stimulus s is variable and sampled from a Gaussian distribution (s∼ 𝒩 (μ, σ), Fig. 1B). The optimal (maximum-likelihood) prediction is given by the mean of the stimulus distribution. Framed as an optimisation problem, the goal is to adapt the internal model of the mean such that the probability of observing samples s from the true distribution of whisker deflections is maximised given this model.

Hence, stochastic gradient ascent learning on the log likelihood suggests that with each observation s, the prediction, corresponding to the internal model of the mean, should be updated as follows to approach the maximum likelihood solution:

According to this formulation, the update for the internal model should be the prediction error scaled inversely by the variance σ². Therefore, we propose that prediction errors should be modulated by uncertainty.

Computation of UPEs in cortical microcircuits

How can cortical microcircuits achieve uncertainty modulation? Prediction errors can be positive or negative, but neuronal firing rates are always positive. Because baseline firing rates are low in layer 2/3 pyramidal cells [e.g., 42], positive and negative prediction errors were suggested to be represented by distinct neuronal populations [31], which is in line with experimental data [26]. We, therefore, decompose the UPE into a positive UPE⁺ and a negative UPE⁻ component (Fig. 1C,D):

where ⌊… ⌋ ⁺ denotes rectification at 0.

It has been suggested that error neurons compute prediction errors by subtracting the prediction from the stimulus input (or vice versa) [1]. Inhibitory interneurons provide the subtraction, resulting in an excitation-inhibition balance when they match [23]. To represent a UPE, error neurons need additionally be divisively modulated by the uncertainty. Depending on synaptic properties, such as reversal potentials, inhibitory neurons can have subtractive or divisive influences on their postsynaptic targets. Therefore, we propose that an inhibitory cell type that divisively modulates prediction error activity represents the uncertainty. We hypothesise, first, that in positive prediction error circuits, inhibitory interneurons with subtractive inhibitory effects represent the mean μ of the prediction. Second, we hypothesise that inhibitory interneurons with divisive inhibitory effects represent the uncertainty σ² of the prediction (Fig. 1C,D). A layer 2/3 pyramidal cell that receives these sources of inhibition then reflects the uncertainty-modulated prediction error.

More specifically, we propose that the SSTs are involved in the computation of the difference between predictions and stimuli, as suggested before [1], and that the PVs provide the uncertainty modulation. In line with this, prediction error neurons in layer 2/3 receive subtractive inhibition from somatostatin (SST) and divisive inhibition from parvalbumin (PV) interneurons [63]. However, SSTs can also have divisive effects, and PVs can have subtractive effects, dependent on circuit and postsynaptic properties [54, 38, 10].

Local inhibitory cells learn to represent the mean and the variance given an associative cue

As discussed above, how much an individual sensory input contributes to updating the internal model should depend on the uncertainty associated with the sensory stimulus in its current context. Uncertainty estimation requires multiple stimulus samples. Therefore, our brain needs to have a context-dependent mechanism to estimate uncertainty from multiple past instances of the sensory input. Let us consider the simple example from above, in which a sound stimulus represents a context with a particular amount of uncertainty. Here, we investigate whether the presentation of the sound can elicit activity in the PVs that reflects the expected uncertainty of the situation. To investigate whether a sound can cause activity in SSTs and PVs that reflects the mean and the variance of the whisker stimulus distribution, respectively, we simulated a rate-based circuit model consisting of pyramidal cells and the relevant inhibitory cell types. This circuit receives both the sound and the whisker stimuli as inputs.

SSTs learn to estimate the mean

With our circuit model, we first investigate whether SSTs can learn to represent the mean of the stimulus distribution. In this model, SSTs receive whisker stimulus inputs s, drawn from Gaussian distributions (Fig. 2B), and an input from a higher level representation of the sound a (which is either on or off, see Methods). The connection weight from the sound representation to the SSTs is plastic according to a local activity-dependent plasticity rule. The aim of this rule is to minimise the difference between the activation of the SSTs caused by the sound input (which has to be learned) and the activation of the SSTs by the whisker stimulus (which nudges the SST activity in the right direction). The learning rule ensures that the auditory input itself causes SSTs to fire at the desired rate. After learning, the weight and the average SST firing rate reflect the mean of the presented whisker stimulus intensities (Fig. 2C-F).

Figure 2:

PVs learn to estimate the variance context-dependently

We next addressed whether PVs can estimate and learn the variance locally. To estimate the variance of the whisker deflections s, the PVs have to estimate σ²[s] = 𝔼_s[(s − 𝔼 [s])²] = 𝔼_s[(s − μ)²]. To do so, they need to have access to both the whisker stimulus s and the mean μ. PVs in PPC respond to sensory inputs in diverse cortical areas [S1: 53] and are inhibited by SSTs in layer 2/3, which we assumed to represent the mean. Finally, for calculating the variance, these inputs need to be squared. PVs were shown to integrate their inputs supralinearly [8], which could help PVs to approximately estimate the variance.

In our circuit model, we next tested whether the PVs can learn to represent the variance of an upcoming whisker stimulus based on a context provided by an auditory input (Fig. 3A). Two different auditory inputs (Fig. 3B purple, green) are paired with two whisker stimulus distributions that differ in their variances (green: low, purple: high). The synaptic connection from the auditory input to the PVs is plastic according to the same local activity-dependent plasticity rule as the connection to the SSTs. With this learning rule, the weight onto the PV becomes proportional to σ (Fig. 3C), such that the PV firing rate becomes proportional to σ² on average (Fig. 3D). The average PV firing rate is exactly proportional to σ² with a quadratic activation function φ_{P V} (x) (Fig. 3D-F,H) and monotonically increasing with σ² with other choices of activation functions (Suppl. Fig. 9), both when the sound input is presented alone (Fig. 3D,E,H) or when paired with whisker stimulation (Fig. 3F). Notably, a single PV neuron is sufficient for encoding variances of different contexts because the context-dependent σ is stored in the connection weights.

Figure 3:

To estimate the variance, the mean needs to be subtracted from the stimulus samples. A faithful mean subtraction is only ensured if the weights from the SSTs to the PVs (w_PV,SST) match the weights from the stimuli s to the PVs (w_PV,s). The weight w_PV,SST can be learned to match the weight w_PV,s with a local activity-dependent plasticity rule (see Suppl. Fig. 10 and Suppl. Methods).

The PVs can similarly estimate the uncertainty in negative prediction error circuits (Suppl. Fig. 11). In these circuits, SSTs represent the current sensory stimulus, and the mean prediction is an excitatory input to both negative prediction error neurons and PVs.

Calculation of the UPE in Layer 2/3 error neurons

Layer 2/3 pyramidal cell dendrites can generate NMDA and calcium spikes, which cause a nonlinear integration of inputs. Such a nonlinear integration of inputs is convenient when the mean input changes and the current prediction differs strongly from the new mean of the stimulus distribution. In this case, the PV firing rate will increase for larger errors and inhibit error neurons more strongly than indicated by the learned variance estimate. The nonlinearity compensates for this increased inhibition by PVs, such that in the end, layer 2/3 cell activity reflects an uncertainty-modulated prediction error (Fig. 4E) in both negative (Fig. 4A) and positive (Fig. 4B) prediction error circuits. A stronger nonlinearity has an interesting effect: error neurons elicit much larger responses to outliers than to stimuli that match the predicted distribution—a cell-intrinsic form of out-of-distribution detection.

Figure 4:

To ensure a comparison between the stimulus and the prediction, the weights from the SSTs to the UPE neurons need to match the weights from the stimulus s to the UPE neuron and from the mean representation to the UPE neuron, respectively. With inhibitory plasticity (target-based, see Suppl. Methods), the weights from the SSTs can learn to match the incoming excitatory weights (Suppl. Fig. 12).

Interactions between representation neurons and error neurons

The theoretical framework of predictive processing includes both prediction error neurons and representation neurons, the activity of which reflects the internal model and should hence be compared to the sensory information. To make predictions for the activity of representation neurons, we expand our circuit model with this additional cell type. We first show that a representation neuron R can learn a representation of the stimulus mean given inputs from L2/3 error neurons. The representation neuron receives inputs from positive and negative prediction error neurons and from a higher level representation of the sound a (Fig. 5A). It sends its current mean estimate to the error circuits by either targeting the SSTs (in the positive circuit) or the pyramidal cells directly (in the negative circuit). Hence in this recurrent circuit, the SSTs inherit the mean representation instead of learning it. After learning, the weights from the sound to the representation neuron and the average firing rate of this representation neuron reflects the mean of the stimulus distribution (Fig. 5B,C).

Figure 5:

Second, we show that a circuit with prediction error neurons that exhibit NMDA spikes (as in Fig. 4) approximates an idealised circuit, in which the PV rate perfectly represents the variance (Fig. 5D,E, see inset for comparison of the two models). Also in this recurrent circuit, PVs learn to reflect the variance, as the weight from the sound representation a is learned to be proportional to σ (Suppl. Fig. 13).

Predictions for different cell types

Our model makes predictions for the activity of different cell types for positive and negative prediction errors (e.g. when a mouse receives whisker stimuli that are larger (Fig. 6A, black) or smaller (Fig. 6G, grey) than expected) in contexts associated with different amounts of uncertainty (e.g., the high-uncertainty (purple) versus the low-uncertainty (green) context are associated with different sounds). Our model suggests that there are two types of interneurons that provide subtractive inhibition to the prediction error neurons (presumably SST subtypes): in the positive prediction error circuit (SST⁺), they signal the expected value of the whisker stimulus intensity (Fig. 6B,H). in the negative prediction error circuit (SST⁻) they signal the whisker stimulus intensity (Fig. 6C,I). We further predict that interneurons that divisively modulate prediction error neuron activity represent the uncertainty (presumably PVs). Those do not differ in their activity between positive and negative circuits and may even be shared across the two circuits: in both positive and negative prediction error circuits, these cells signal the variance (Fig. 6D,J). L2/3 pyramidal cells that encode prediction errors signal uncertainty-modulated positive prediction errors (Fig. 6E) and uncertainty-modulated negative prediction errors (Fig. 6L), respectively. Finally, the existence of so-called internal representation neurons has been proposed [31]. In our case, those neurons represent the predicted mean of the associated whisker deflections. Our model predicts that upon presentation of an unexpected whisker stimulus, those internal representation neurons adjust their activity to represent the new whisker deflection depending on the variability of the associated whisker deflections: they adjust their activity more (given equal deviations from the mean) if the associated whisker deflections are less variable (see the next section and Fig. 7).

Figure 6:

Figure 7:

The following experimental results are compatible with our predictions: First, putative inhibitory neurons (narrow spiking units) in the macaque anterior cingulate cortex increased their firing rates in periods of high uncertainty [3]. These could correspond to the PVs in our model. Second, prediction error activity seems to be indeed lower for less predictable, and hence more uncertain, contexts: Mice trained in a predictable environment (where locomotion and visual flow match) were compared to mice trained in an unpredictable, uncertain environment [1, they saw a video of visual flow that was independent of their locomotion:]. Layer 2/3 activity towards mismatches in locomotion and visual flow was lower in the mice trained in the unpredictable environment.

The effective learning rate is automatically adjusted with UPEs

To test whether UPEs can automatically adjust the effective learning rate of a downstream neural population, we looked at two contexts that differed in uncertainty and compared how the mean representation evolves with and without UPEs. Indeed, in a low-uncertainty setting, the mean representation can be learned faster with UPEs (in comparison to unmodulated, Fig. 7A,C). In a high-uncertainty setting, the effective learning rate is smaller, and the mean representation is less variable than in the unmodulated case (Fig. 7B,D). The standard deviation of the firing rate increases only sublinearly with the standard deviation of the inputs (Fig. 7E). In summary, uncertainty-modulation of prediction errors enables an adaptive learning rate modulation.

Discussion

Based on normative theories, we propose that the brain uses uncertainty-modulated prediction errors. In particular, we hypothesise that layer 2/3 prediction error neurons represent prediction errors that are inversely modulated by uncertainty. Here we showed that different inhibitory cell types in layer 2/3 cortical circuits can compute means and variances and thereby enable pyramidal cells to represent uncertainty-modulated prediction errors. We further showed that the cells in the circuit are able to learn to predict the means and variances of their inputs with local activity-dependent plasticity rules. Our study makes experimentally testable predictions for the activity of different cell types, PV and SST interneurons, in particular, prediction error neurons and representation neurons. Finally, we showed that circuits with uncertainty-modulated prediction errors enable adaptive learning rates, resulting in fast learning when uncertainty is low and slow learning to avoid detrimental fluctuations when uncertainty is high.

Our theory has the following notable implications: The first implication concerns the hierarchical organisation of the brain. At each level of the hierarchy, we find similar canonical circuit motifs that receive both feedforward (from a lower level) and feedback (from a higher level, predictive) inputs that need to be integrated. We propose that uncertainty is computed on each level of the hierarchy. This enables uncertainty estimates specific to the processing level of a particular area. Experimental evidence is so far insufficient to favour this fully Bayesian account of uncertainty estimation over the idea that uncertainty is only computed on the level of decisions in higher level brain areas such as the parietal cortex [32], orbitofrontal cortex [41], or prefrontal cortex [52]. Our study provides a concrete suggestion for an implementation and, therefore, experimentally testable predictions. The Bayesian account has clear computational advantages for task-flexibility, information integration, active sensing, and learning (see [65] for a recent review of the two accounts). Additionally, adding uncertainty-modulated prediction errors from different hierarchical levels according to the predictive coding model [50, 59] yields Bayes-optimal weighting of feedback and feedforward information, which can be reconciled with human behaviour [43]. Two further important implications result from storing uncertainty in the afferent connections to the PVs. First, this implies that the same PV cell can store different uncertainties depending on the context, which is encoded in the pre-synaptic activation. Second, fewer PVs than pyramidal cells are required for the mechanism, which is compatible with the 80/20 ratio of excitatory to inhibitory cells in the brain.

We claim that the uncertainty represented by PVs in our theoretical framework corresponds to expected uncertainty that results from noise or irreducible uncertainty in the stimuli and should therefore decrease the learning rate. Another common source of uncertainty are changes in the environment, also referred to as the unexpected uncertainty. In volatile environments with high unexpected uncertainty, the learning rate should increase. We suggest that vasointestinalpeptide-positive interneurons (VIPs) could be responsible for signalling the unexpected uncertainty, as they respond to reward, punishment and surprise [47], which can be indicators of high unexpected uncertainty. They provide potent disinhibition of pyramidal cells [45], and also inhibit PVs in layer 2/3 [46]. Hence, they could increase error activity resulting in a larger learning signal. In general, interneurons are innervated by different kinds of neuromodulators [39, 48] and control pyramidal cell’s activity and plasticity [24, 17, 62, 61, 60]. Therefore, neuromodulators could have powerful control over error neuron activity and hence perception and learning.

A diversity of proposals about the neural representation of uncertainty exist. For example, it has been suggested that uncertainty is represented in single neurons by the width [14], or amplitude of their responses [40], or implicitly via sampling [neural sampling hypothesis; 44, 5, 4], or rather than being represented by a single feature, can be decoded from the activity of an entire population [9]. While we suggest that PVs represent uncertainty to modulate prediction error responses, we do not claim that this is the sole representation of uncertainty in neuronal circuits.

Uncertainty estimation is relevant for Bayes-optimal integration of different sources of information, e.g., different modalities [multi-sensory integration; 12, 13] or priors and sensory information. Here, we present a circuit implementation for weighing sensory information according to its uncertainty. It has previously been suggested that Bayes-optimal multi-sensory integration could be achieved in single neurons [13, 25]. Our proposal is complementary to this solution in that uncertainty-modulated errors can be forwarded to other cortical and subcortical circuits at different levels of the hierarchy, where they can be used for inference and learning. It further allows for a context-dependent integration of sensory inputs.

Multiple neurological disorders, such as autism spectrum disorder or schizophrenia, are associated with maladaptive contextual uncertainty-weighting of sensory and prior information [19, 37, 57, 36, 55]. These disorders are also associated with aberrant inhibition, e.g. ASD is associated with an excitation-inhibition imbalance [51] and reduced inhibition [21, 16]. Interestingly, PV cells, in particular chandelier PV cells, were shown to be reduced in number and synaptic strength in ASD [28]. Our theory provides one possible explanation of how deficits in uncertainty-weighting on the behavioural level could be linked to altered PVs on the circuit level.

Finally, uncertainty-modulated errors could advance deep hierarchical neural networks. In addition to propagating gradients, propagating uncertainty may have advantages for learning. The additional information on uncertainty could enable calculating distances between distributions, which can provide an informative and parameter-independent metric for learning [e.g. natural gradient learning, 34].

To provide experimental predictions that are immediately testable, we suggested specific roles for SSTs and PVs, as they can subtractively and divisively modulate pyramidal cell activity, respectively. In principle, our theory more generally posits that any subtractive or divisive inhibition could implement the suggested computations. With the emerging data on inhibitory cell types, subtypes of SSTs and PVs or other cell types may turn out to play the proposed role.

To compare predictions and stimuli in a subtractive manner, the encoded prediction/stimulus needs to be translated into a direct variable code. In this framework, we assume that this can be achieved by the weight matrix defining the synaptic connections from the neural populations representing predictions and stimuli (possibly in a population code).

Conclusion

To conclude, we proposed that prediction error activity in layer 2/3 circuits is modulated by uncertainty and that the diversity of cell types in these circuits achieves the appropriate scaling of the prediction error activity. The proposed model is compatible with Bayes-optimal behaviour and makes predictions for future experiments.

Methods

Derivation of the UPE

The goal is to learn to maximise the log likelihood:

We consider the log likelihood for one sample s of the stimulus distribution:

Stochastic gradient ascent on the log likelihood gives the update for :

Circuit model

Prediction error circuit

We modelled a circuit consisting of excitatory prediction error neurons in layer 2/3, and two inhibitory populations, corresponding to PV and SST interneurons.

Layer 2/3 pyramidal cells receive divisive inhibition from PVs [63]. We, hence, modelled the activity of prediction error neurons as

where φ(x) is the activation function, defined in Eq. 21, I_dend = ⌊w_UPE,s r_s − w_UPE,SST r_SST⌋^k is the dendritic input current to the positive prediction error neuron (see section Neuronal dynamics below for r_x and for the negative prediction error neuron, and Table 1 for w_x). The nonlinearity in the dendrite is determined by the exponent k, which is by default k = 2, unless otherwise specified as in Fig. 4G-J. I₀ > 1 is a constant ensuring that the divisive inhibition does not become excitatory, when σ < 1.0.

Table 1:

The PV firing rate is determined by the input from the sound representation and the whisker stimuli, from which their mean is subtracted (, where the mean is given by ). The mean-subtracted whisker stimuli serve as a target for learning the weight from the sound representation to the PV . The PV firing rate evoles over time according to:

where φ_PV(x) is a rectified quadratic activation function, defined in Eq. 22.

In the positive prediction error circuit, in which the SSTs learn to represent the mean, the SST activity is determined by

Recurrent circuit model

In the recurrent circuit, shown in Fig. 5, we added an internal representation neuron to the circuit with firing rate r_R. In this circuit the SSTs inherit the mean representation from the representation neuron instead of learning it themselves. In this recurrent circuit, the firing rate of each population r_i where i [SST⁺, SST⁻, PV⁺, PV⁻, UPE⁺, UPE⁻, R] evolves over time according to the following neuronal dynamics. φ denotes a rectified linear activation function with saturation, φ_{P V} denotes a rectified quadratic activation function with saturation, defined in the section below.

Activation functions

and

Inputs

The inputs to the circuit were the higher level representation of the sound a, which was either on (1.0) or off (0.0), and N samples from the Gaussian distribution of whisker stimulus intensities. Each whisker stimulus intensity was presented for D timesteps (see Table 2).

Table 2:

Synaptic dynamics / Plasticity rules

Synapses from the higher level representation of the sound a to the SSTs, PVs, and to R were plastic according to the following activity-dependent plasticity rules [56].

where η_PV = 0.01η_R.

Explanation of the synaptic dynamics

The connection weight from the sound representation to the SSTs w_SST,a is plastic according to the following local activity-dependent plasticity rule [56]:

where η is the learning rate, r_a is the pre-synaptic firing rate, r_SST is the post-synaptic firing rate of the SSTs, φ(x) is a rectified linear activation function of the SSTs, and the SST activity is determined by

The SST activity is influenced (nudged with a factor β) by the somatosensory stimuli s, which provide targets for the desired SST activity. The learning rule ensures that the auditory input alone causes SSTs to fire at their target activity. As in the original proposal [56], the terms in the learning rule can be mapped to local neuronal variables, which could be represented by dendritic (w_SST,a r_a) and somatic (r_SST) activity.

The connection weight from the sound representation to the PVs w_PV,a is plastic according to the same local activitydependent plasticity rule as the SSTs [56]:

The weight from the sound representation to the PV approaches σ (instead of μ as the weight to the SSTs), because the PV activity is a function of the mean-subtracted whisker stimuli (instead of the whisker stimuli as the SST activity), and for a Gaussian-distributed stimulus s ∼ 𝒩 (s|μ, σ), it holds that 𝔼 [⌊s − μ⌋⁺] ∝ σ.

Estimating the variance correctly

The PVs estimate the variance of the sensory input from the variance of the teaching input (s− μ), which nudges the membrane potential of the PVs with a nudging factor β. The nudging factor reduces the effective variance of the teaching input, such that in order to correctly estimate the variance, this reduction needs to be compensated by larger weights from the SSTs to the PVs (w_PV,SST) and from the sensory input to the PVs (w_PV,s). To determine how strong the weights w_s = w_PV,SST = w_PV,s need to be to compensate for the downscaling of the input variance by β, we require that 𝔼 [wa]² = σ² when the average weight change 𝔼 [∆w] = 0. The learning rule for w is as follows:

where and .

Using that φ(u) = u², the average weight change becomes:

Given our objective 𝔼 [(wa)²] = σ², we can write:

Then for 𝔼 [∆w] = 0:

Here, we assumed that φ(u) = u² instead of φ(u) = ⌊ u ⌋ ². To test how well this approximation holds, we simulated the circuit for different values of β and hence w_s, and plotted the PV firing rate r_PV(a) given the sound input a and the weight from a to PV, w_PV,a, for different values of β (Fig. 8). This analysis shows that the approximation holds for small β up to a value of β = 0.2.

Figure 8:

Simulation

We initialised the circuit with the initial weight configuration in Tables 1 and 3 and neural firing rates were initialised to be 0 (r_i(0) = 0 with i ∈ [SST⁺, SST⁻, PV⁺, PV⁻, UPE⁺, UPE⁻, R]). We then paired a constant tone input with N samples from the whisker stimulus distribution, the parameters of which we varied and are indicated in each Figure. Each whisker stimulus intensity was presented for D timesteps (see Table 2). All simulations were written in Python. Differential equations were numerically integrated with a time step of dt = 0.1.

Table 3:

Table 4:

Table 5:

Eliciting responses to mismatches (Fig. 4 and Fig. 6)

We first trained the circuit with 10000 stimulus samples to learn the variances in the a-to-PV weights. Then we presented different mismatch stimuli to calculate the error magnitude for each mismatch of magnitude s − μ.

Comparing the UPE circuit with an unmodulated circuit (Fig. 7)

To ensure a fair comparison, the unmodulated control has an effective learning rate that is the mean of the two effective learning rates in the uncertainty-modulated case.

Acknowledgements

We would like to thank Loreen Hertäg and Sadra Sadeh for feedback on the manuscript. This work has received funding from the European Union 7th Framework Programme under grant agreement 604102 (HBP), the Horizon 2020 Framework Programme under grant agreements 720270, 785907 and 945539 (HBP) and the Manfred Stärk Foundation.

Competing Interests Statement

The authors declare that they have no competing interests.

Code availability

All simulation code used for this paper will be made available on GitHub upon publication (https://github.com/k47h4/UPE) and is attached to the submission as supplementary file for the reviewers.

Supplementary Information

Supplementary Methods

Synaptic dynamics/plasticity rules

Different choice of supralinear activation function for PV

Figure 9:

Plastic weights from SST to PV learn to match weights from s to PV

Figure 10:

PVs learn the variance in the negative prediction error circuit

Figure 11:

Learning the weights from the SSTs to the prediction error neurons

Figure 12:

PV activity is proportional to the variance in the recurrent circuit

Figure 13: