Sergey Shuvaev, Ph.D.

My research focuses on biologically-informed models of decision-making. Towards this goal, I combine computational neuroscience, reinforcement learning, Bayesian methods, and game theory. I am interested in merging this work with control theory to facilitate safe and efficient human-AI interaction. I hold a B.Sc., M.Sc., and Ph.D. in Applied Mathematics and Physics from the Moscow Institute of Physics and Technology. My Ph.D. focused on machine learning models of behavior, which I designed in residence at Cold Spring Harbor Laboratory. My B.Sc. comprised developing computer vision pipelines for neuroimaging. During my M.Sc., I complemented my computational background with experience in electrical engineering. My work led to 10+ publications with 100+ citations; it was featured in Nautilus, Spectrum, Coexisting with AI, and Eye on AI. I was honored to be selected as a Swartz Fellow (Ph.D.), an Alexandrov Scholar (M.Sc.), and an Abramov and Frolov Scholar (B.Sc.). My service as a reviewer (ICML, ICLR, NeurIPS) was recognized with two Top Reviewer awards.

Featured projects

Multi-agent: normative theory of conflict

Social conflict is a survival mechanism leading to normal and pathological behaviors. To understand the principles of social conflict, we used behavioral and neural data from mice advancing through its stages. We modeled their interactions as a normal-form game to define the optimal actions and used Bayesian inference to account for partial observability. With our model and behavioral data, we formed and validated hypotheses about the reward schedule, information availability, and evidence accumulation corresponding to conflict behaviors. We found that the animals’ actions were consistent with the first-level Theory of Mind (1-ToM) where agents maintain “primary” beliefs about the strengths and “secondary” beliefs about the beliefs of their opponents. We further found that the 1-ToM beliefs – the latent variables in our model – are significantly correlated with patterns of neural activity, unexplained by other task variables. Overall, our work offers a robust biologically-informed model of social conflict and a framework for future studies of social behaviors in partially observable settings. | paper

Single-agent: algorithmic basis of stay-or-leave decisions

Individuals choose whether to maintain the status quo or to pursue better options. Multiple models have been proposed to explain such choices; however, new evidence has revealed their inconsistency with data. To bridge this gap, we compared behavioral data to predictions of reinforcement learning (RL) models. We show that stay-or-leave decisions are non-Markovian. We find that, provided the state history, these choices are consistent with R-learning but not with V- or Q-learning. We further show that R-learning converges to a leaky version of the marginal value theorem (MVT), a classical result from optimal control theory. The leaky MVT posits that R-learning agents leave depleting resources when reward falls below its leaky average, which, we argue, is Bayes optimal in dynamic natural environments. The leaky average can be computed with a single recurrent neuron, enabling compute-efficient implementation of the decision rule. Overall, our work links RL, optimal control, and Bayesian inference to describe a learning algorithm, a decision rule, and a rationale for stay-or-leave choices. | paper

Experience

Cold Spring Harbor Laboratory

Koulakov Laboratory
Cold Spring Harbor, NY

Postdoctoral Fellow (Nov 2022 - present), Student in Residence (Jul 2016 - Oct 2022)

Used machine-learning approaches to develop normative models of reward-driven behaviors and relate them to neuronal activity observed in the brain.

  • Developed data-driven models of sequential decision-making, motivation, and conflict;
  • Co-developed approaches to compressibility and self-assembly of neural network models;
  • Worked on the function of olfactory receptors and the structure of olfactory connectivity.

Publications: 5 first-author (incl. 2 NeurIPS and 2 PNAS), 2 co-authored (incl. ICML)

Moscow Institute of Physics and Technology

Enikolopov Laboratory
Moscow, Russia

Research Associate (Jul 2016 - Dec 2018), Research Assistant (Jan 2012 - Jun 2016)

Designed computer-vision models to identify brain-wide changes in neuronal populations during brain development and neurodegenerative disorders.

  • Developed pipelines for microscopy, detection, and alignment of brain-wide cell populations;
  • Used these pipelines to study how antidepressants, development, and radiation affect neurogenesis;
  • The pipelines are now used in studies of traumatic brain injury and neurodegenerative disorders.

Publications: 2 first-author and 2 co-authored; led to a body of follow-up research.

Kurchatov Institute

Superconductivity Department
Moscow, Russia

Research Assistant (Aug 2013 - Jul 2015)

Developed numerical models and worked towards measurements of electrical and thermal properties of high-current superconductive cables to pursue requirement-based design.

Publications

Behavior

A normative theory of social conflict

NeurIPS, 2023 | Shuvaev, S., Amelchenko, E., Smagin, D., Kudryavtseva, N., Enikolopov, G., and Koulakov, A. | summary

Neural networks with motivation

Front Sys Neurosci, 2021 | Shuvaev, S., Tran, N., Stephenson-Jones, M., Li, B., and Koulakov, A. | summary

R-learning in actor-critic model offers a biologically relevant mechanism for sequential decision-making

NeurIPS, 2020 | Shuvaev, S.*, Starosta, S.*, Kvitsiani, D., Kepecs, A., and Koulakov, A. | summary

Olfaction

The primacy model and the structure of olfactory space

PLOS Comp Bio, 2024 | Giaffar, H., Shuvaev, S., Rinberg, D., and Koulakov, A. | summary

DeepNose: Using artificial neural networks to represent the space of odorants

ICML, 2019 | Tran, N., Kepple, D., Shuvaev, S., and Koulakov, A. | summary

Neurodevelopment

Encoding innate ability through a genomic bottleneck

PNAS, 2024 | Shuvaev, S., Lachi, D., Koulakov, A., and Zador, A. | summary

Network cloning using DNA barcodes

PNAS, 2019 | Shuvaev, S., Başerdem, B., Zador, A., and Koulakov, A. | summary

Neuroimaging

Spatiotemporal 3D image registration for mesoscale studies of brain development

Scientific Reports, 2022 | Shuvaev, S., Lazutkin, A., Kiryanov, R., Anokhin, K., Enikolopov, G., and Koulakov, A. | summary

DALMATIAN: an algorithm for automatic cell detection and counting in 3D

Front Neuroanat, 2017 | Shuvaev, S., Lazutkin, A., Kedrov, A., Anokhin, K., Enikolopov, G., and Koulakov, A. | summary

Click histochemistry for whole-mount staining of brain structures

MethodsX, 2019 | Lazutkin, A., Shuvaev, S., and Barykina, N. | summary

Neurogenesis

3D topography and dynamics of neurogenic zones in mouse brain

In preparation | summary

Effects of Memantine and Fluoxetine on cell proliferation in adult mouse brain

In preparation | summary

Suppressed neurogenesis without cognitive deficits: effects of fast neutron irradiation in mice

Neuroreport, 2019 | Mineyeva, O., Barykina, N., Bezriadnov, D., ..., Shuvaev, S., Usova, S., and Lazutkin, A. | summary

Outside of work, I am a fan of hiking, cycling, and grilling. I play guitar and look forward to picking up the piano. Like many of us, I'm excited to visit places, enjoy food, and meet people. Check out my Instagram!