Skip to content

Pruning: sparsifying weight matrice⚓︎

Untitled

Pruning: Motivation⚓︎

Trillion of synapses are generated in the human brain during the first few months of birth. Christopher A. Walsh Peter Huttenlocher (1931–2013)

Untitled

  • 1 year old, peaked at 1000 trillion
  • Pruning begins to occur.
  • 10 years old, a child has nearly 500 trillion synapses
  • This 'pruning' mechanism removes redundant connections in the brain.

Pruning strategies⚓︎

Untitled - 01 - Pruning weights - 02 - Pruning neurons - 03 - Block Prunning - 04 - Pruning heads and layers