Exploration vs. exploitation

• to make good predictions:

  • consistency (exploiting)

  • flexibility (exploring)

• both at once impossible

• best captured by inverse temperature parameter

• humans: both random and directed exploration

Exploration -> dynamic environments, unexpected uncertainty/surprise

————-> noradrenaline

Expoitation -> stable environments, expected uncertainty

—————-> acetylcholin

-> is explore-exploit trade off a general principle?

• more task relevant behavioral variability —> predicts better final performance

• beginning: more variability and bad performance

• end: less variability and peak performance

• Childhood - more exploration

  • unfocused

  • curiosity&neophilia

  • active play

  • less knowledge

• adulthood - more exploitation

  • attentional focus

  • inhibition

  • executive function

  • goals-directed

  • more knowledge

• exceptionally extended childhood compared to other animals (including primates)

• impressive capacities for

  • physical cognition

  • social cognition

• longer developmental period adaptive to build capacity and motivation for internal and external exploration!

search for new data

• find new hypotheses

• exploitative search

  • narrow

  • only revising current hypotheses when evidence is very strong

  • make small adjustments to current theories to accomodate new evidence

  • “good enough” solutions

• exploratory search

  • broad

  • moving to new hypotheses

  • with only small amount of evidence

  • may waste time imagining unlikely possibilities

  • learner more likely to discover genuinely new ideas

beginning: exploring —> gradually exploiting

1. less knowledge in the beginning

2. with time elapse less opportunities to make use of information that was acquired through exploration

ontogenetic time might be the same with extended childhood

-> in contrast with precocial animals -> elaborate but narrow computational capacities in place at birth

• high dimensionality and complexity in environment

• first explore then exploit necessary

• this is in line with sensitive periods early in development

• baseline

• follow up after 2 years

• 6-7 year old children

• more exploitation in immediate and delayed feedback conditions

• Higher SES

  • via parental education and income

  -> enhanced exploitation and lower exploration

• in stable conditions: