Sarah Marzen
  • Main
  • People
  • Contact
  • Google Scholar
  • Random ruminations
  • Research Program
  • Conferences, Workshops, and Working Groups
  • Teaching
  • Main
  • People
  • Contact
  • Google Scholar
  • Random ruminations
  • Research Program
  • Conferences, Workshops, and Working Groups
  • Teaching

Random musings

Stray thoughts on my research, related research, education research, and sweeping commentaries on entire fields
Picture

What does maximizing mutual information between stimulus and neural response do?

3/9/2021

0 Comments

 
Many people have written about this, and I do know this literature as well as I'd like, but I did a random calculation that I thought was semi-useful based on this paper.  I haven't touched the channel coding aspect of this paper, but I did try to understand the mutual information between input and neural response in the small noise limit.  In this case, I'm assuming that the neurons are nearly noiseless, and that their information is coded by a firing rate.  The approximation was heavily inspired by this paper.

The details are not pretty, but the story ends up being pretty simple.

Essentially, we'll assume that the firing rate of the neuron is some nonlinear function of some gain multiplied by the input signal, plus some Gaussian noise that ends up being rather irrelevant.  We'll also assume that the probability density function of the input is highly peaked around some value.  This is essentially ICA, but we learn a few things about the optimal nonlinearity:
  • the neurons want to be maximally silent or maximally firing when the stimulus is most probable;
  • if the neurons are maximally firing when the stimulus is maximally probable, silence is nearby, and soon after an increase in firing rate;
  • and the gain should be large.
It's the second prediction that surprised me, as it would be a very small effect, but maybe noticeable.  Interestingly, all the intuition that comes from doing the multi-dimensional case can be obtained from the single-neuron case.

Please let me know if someone has already done this calculation!  It seems like an obvious move, but I am unfortunately not familiar enough with the literature to know.

Here are some calculations that I will never publish.
0 Comments

    Author

    Write something about yourself. No need to be fancy, just an overview.

    Archives

    February 2025
    January 2025
    December 2024
    August 2024
    July 2024
    May 2024
    December 2023
    October 2023
    November 2022
    July 2022
    December 2021
    November 2021
    March 2021
    February 2021
    May 2020
    July 2019
    May 2017

    Categories

    All

    RSS Feed

Proudly powered by Weebly