What is the definition of information from Shannon?
Information is the reduction of uncertainty
Shannon’s axioms of uncertainty measure
For equiprobable states uncertainty grows with the number that states
Uncertainty should be continuous in probability of states
…Look up
Shannon’s Entropy
Entropy of a system is a measure of uncertainty
From uncertainty to information
Stimulus (S) —> Channel (e.g. neuron) -> Response (R)
Mutual information is the overlap of the Stimulus entropy and response entropy
What is mutual information
A gerneralization of the Pearson corrrelation
Efficient Coding Hypothesis
Information maximisation as a normative objective for neural systems
Why does a moth fly against the wind/a smell and then suddenly casts perpendicular when the wind stops?
Increase probability of encountering odor
summary slide
Last changed13 days ago
