The Cartesian Cafe

Richard Borcherds | Monstrous Moonshine: From Group Theory to String Theory

Richard Borcherds is a mathematician and professor at University of California Berkeley known for his work on lattices, group theory, and infinite-dimensional algebras. His numerous accolades include being awarded the Fields Medal in 1998 and being elected a fellow of the American Mathematical Society and the National Academy of Sciences.
Patreon (bonus materials + video chat): https://www.patreon.com/timothynguyen
In this episode, Richard and I give an overview of Richard's most famous result: his proof of the Monstrous Moonshine conjecture relating the monster group on the one hand and modular forms on the other. A remarkable feature of the proof is that it involves vertex algebras inspired from elements of string theory. Some familiarity with group theory and representation theory are assumed in our discussion.
I. Introduction
00:25: Biography
02:51 : Success in mathematics
04:04 : Monstrous Moonshine overview and John Conway
09:44 : Technical overview
II. Group Theory
11:31 : Classification of finite-simple groups + history of the monster group
18:03 : Conway groups + Leech lattice
22:13 : Why was the monster conjectured to exist + more history 28:43 : Centralizers and involutions
32:37: Griess algebra
III. Modular Forms
36:42 : Definitions
40:06 : The elliptic modular function
48:58 : Subgroups of SL_2(Z)
IV. Monstrous Moonshine Conjecture Statement
57:17: Representations of the monster
59:22 : Hauptmoduls
1:03:50 : Statement of the conjecture
1:07:06 : Atkin-Fong-Smith's first proof
1:09:34 : Frenkel-Lepowski-Meurman's work + significance of Borcherd's proof
V. Sketch of Proof
1:14:47: Vertex algebra and monster Lie algebra
1:21:02 : No ghost theorem from string theory
1:25:24 : What's special about dimension 26?
1:28:33 : Monster Lie algebra details
1:32:30 : Dynkin diagrams and Kac-Moody algebras
1:43:21 : Simple roots and an obscure identity
1:45:13: Weyl denominator formula, Vandermonde identity
1:52:14 : Chasing down where modular forms got smuggled in
1:55:03 : Final calculations
VI. Epilogue
1:57:53 : Your most proud result?
2:00:47 : Monstrous moonshine for other sporadic groups?
2:02:28 : Connections to other fields. Witten and black holes and mock modular forms.
 
Further reading: V Tatitschef. A short introduction to Monstrous Moonshine. https://arxiv.org/pdf/1902.03118.pdf
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Announcements for 2024 and a Message to Viewers

Thought I'd share some exciting news about what's happening at The Cartesian Cafe in 2024 and also a personal message to viewers on how they can support the cafe.
Patreon:
https://www.patreon.com/timothynguyen

Tim Maudlin | Bell’s Theorem and Beyond: Nobody Understands Quantum Mechanics

Tim Maudlin is a philosopher of science specializing in the foundations of physics, metaphysics, and logic. He is a professor at New York University, a member of the Foundational Questions Institute, and the founder and director of the John Bell Institute for the Foundations of Physics.
Patreon (bonus materials + video chat):https://www.patreon.com/timothynguyen
In this very in-depth discussion, Tim and I probe the foundations of science through the avenues of locality and determinism as arising from the Einstein-Poldosky-Rosen (EPR) paradox and Bell's Theorem. These issues are so intricate that even the Nobel Prize committee incorrectly described the significance of Bell's work in their press release for the 2022 prize in physics. Viewers motivated enough to think deeply about these ideas will be rewarded with a conceptually proper understanding of the nonlocal nature of physics and its manifestation in quantum theory.
I. Introduction 00:00 :
00:25: Biography
05:26: Interdisciplinary work
11:54 : Physicists working on the wrong things
16:47 : Bell's Theorem soft overview
24:14: Common misunderstanding of "God does not play dice."
25:59: Technical outline
II. EPR Paradox / Argument
29:14 : EPR is not a paradox
34:57 : Criterion of reality
43:57 : Mathematical formulation
46:32 : Locality: No spooky action at a distance
49:54 : Bertlmann's socks
53:17 : EPR syllogism summarized
54:52 : Determinism is inferred not assumed
1:02:18 : Clarifying analogy: Coin flips
1:06:39 : Einstein's objection to determinism revisited
III. Bohm Segue
1:11:05 : Introduction
1:13:38: Bell and von Neumann's error
1:20:14: Bell's motivation: Can I remove Bohm's nonlocality?
IV. Bell's Theorem and Related Examples
1:25:13 : Setup
1:27:59 : Decoding Bell's words: Locality is the key!
1:34:16 : Bell's inequality (overview)
1:36:46 : Bell's inequality (math)
1:39:15 : Concrete example of violation of Bell's inequality
1:49:42: GHZ Example
V. Miscellany
2:06:23 : Statistical independence assumption
2:13:18: The 2022 Nobel Prize
2:17:43: Misconceptions and hidden variables
2:22:28: The assumption of local realism? Repeat: Determinism is a conclusion not an assumption.
VI. Interpretations of Quantum Mechanics
2:28:44: Interpretation is a misnomer
2:29:48: Three requirements. You can only pick two.
2:34:52: Copenhagen interpretation?
 
Further Reading:
J. Bell. Speakable and Unspeakable in Quantum Mechanics
T. Maudlin. Quantum Non-Locality and Relativity
Wikipedia: Mermin's device, GHZ experiment
 
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Antonio Padilla | Fantastic Numbers, Naturalness, and Anthropics in Physics

Antonio (Tony) Padilla is a theoretical physicist and cosmologist at the University of Nottingham. He serves as the Associate Director of the Nottingham Centre of Gravity, and in 2016, Tony shared the Buchalter Cosmology Prize for his work on the cosmological constant. Tony is also a star of the Numberphile YouTube channel, where his videos have received millions of views and he is also the author of the book Fantastic Numbers and Where to Find Them: A Cosmic Quest from Zero to Infinity.
Patreon: https://www.patreon.com/timothynguyen
This episode combines some of the greatest cosmological questions together with mathematical imagination. Tony and I go through the math behind some oft-quoted numbers in cosmology and calculate the age, size, and number of atoms in the universe. We then stretch our brains and consider how likely it would be to find your Doppelganger in a truly large universe, which takes us on a detour through black hole entropy. We end with a discussion of naturalness and the anthropic principle to round out our discussion of fantastic numbers in physics.
Part I. Introduction
00:00 : Introduction
01:06 : Math and or versus physics
12:09 : Backstory behind Tony's book
14:12 : Joke about theoreticians and numbers
16:18 : Technical outline
Part II. Size, Age, and Quantity in the Universe
21:42 : Size of the observable universe
22:32 : Standard candles
27:39 : Hubble rate
29:02 : Measuring distances and time
37:15 : Einstein and Minkowski
40:52 : Definition of Hubble parameter
42:14 : Friedmann equation
47:11 : Calculating the size of the observable universe
51:24 : Age of the universe
56:14 : Number of atoms in the observable universe
1:01:08 : Critical density
1:03:16: 10^80 atoms of hydrogen
1:03:46 : Universe versus observable universe
Part III. Extreme Physics and Doppelgangers
1:07:27 : Long-term fate of the universe
1:08:28 : Black holes and a googol years
1:09:59 : Poincare recurrence
1:13:23 : Doppelgangers in a googolplex meter wide universe
1:16:40 : Finitely many states and black hole entropy
1:25:00 : Black holes have no hair
1:29:30 : Beckenstein, Christodolou, Hawking
1:33:12 : Susskind's thought experiment: Maximum entropy of space
1:42:58 : Estimating the number of doppelgangers
1:54:21 : Poincare recurrence: Tower of four exponents.
Part IV: Naturalness and Anthropics
1:54:34 : What is naturalness? Examples.
2:04:09 : Cosmological constant problem: 10^120 discrepancy
2:07:29 : Interlude: Energy shift clarification. Gravity is key.
2:15:34 : Corrections to the cosmological constant
2:18:47 : String theory landscape: 10^500 possibilities
2:20:41 : Anthropic selection
2:25:59 : Is the anthropic principle unscientific? Weinberg and predictions.
2:29:17 : Vacuum sequestration
Further reading: Antonio Padilla. Fantastic Numbers and Where to Find Them: A Cosmic Quest from Zero to Infinity
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Boaz Barak | Cryptography: The Art of Mathematical Secrecy

Boaz Barak is a professor of computer science at Harvard University, having previously been a principal researcher at Microsoft Research and a professor at Princeton University. His research interests span many areas of theoretical computer science including cryptography, computational complexity, and the foundations of machine learning. Boaz serves on the scientific advisory boards for Quanta Magazine and the Simons Institute for the Theory of Computing and he was selected for Foreign Policy magazine’s list of 100 leading global thinkers for 2014.
www.patreon.com/timothynguyen
Cryptography is about maintaining the privacy and security of communication. In this episode, Boaz and I go through the fundamentals of cryptography from a foundational mathematical perspective. We start with some historical examples of attempts at encrypting messages and how they failed. After some guesses as to how one might mathematically define security, we arrive at the one due to Shannon. The resulting definition of perfect secrecy turns out to be too rigid, which leads us to the notion of computational secrecy that forms the foundation of modern cryptographic systems. We then show how the existence of pseudorandom generators (which remains a conjecture) ensures that such computational secrecy is achievable, assuming P does not equal NP. Having covered private key cryptography in detail, we then give a brief overview of public key cryptography. We end with a brief discussion of Bitcoin, machine learning, deepfakes, and potential doomsday scenarios.
I. Introduction
00:17 : Biography: Academia vs Industry
10:07 : Military service
12:53 : Technical overview
17:01 : Whiteboard outline
II. Warmup
24:42 : Substitution ciphers
27:33 : Viginere cipher
29:35 : Babbage and Kasiski
31:25 : Enigma and WW2
33:10 : Alan Turing
III. Private Key Cryptography: Perfect Secrecy
34:32 : Valid encryption scheme
40:14 : Kerckhoffs's Principle
42:41 : Cryptography = steelman your adversary
44:40 : Attempt #1 at perfect secrecy
49:58 : Attempt #2 at perfect secrecy
56:02 : Definition of perfect secrecy (Shannon)
1:05:56 : Enigma was not perfectly secure
1:08:51 : Analogy with differential privacy
1:11:10 : Example: One-time pad (OTP)
1:20:07 : Drawbacks of OTP and Soviet KGB misuse
1:21:43 : Important: Keys cannot be reused!
1:27:48 : Shannon's Impossibility Theorem
IV. Computational Secrecy
1:32:52 : Relax perfect secrecy to computational secrecy
1:41:04 : What computational secrecy buys (if P is not NP)
1:44:35 : Pseudorandom generators (PRGs)
1:47:03 : PRG definition
1:52:30 : PRGs and P vs NP
1:55:47: PRGs enable modifying OTP for computational secrecy
V. Public Key Cryptography
2:00:32 : Limitations of private key cryptography
2:09:25 : Overview of public key methods
2:13:28 : Post quantum cryptography
VI. Applications
2:14:39 : Bitcoin
2:18:21 : Digital signatures (authentication)
2:23:56 : Machine learning and deepfakes
2:30:31 : A conceivable doomsday scenario: P = NP
Further reading: Boaz Barak. An Intensive Introduction to Cryptography
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Sean Carroll | The Many Worlds Interpretation & Emergent Spacetime

Sean Carroll is a theoretical physicist and philosopher who specializes in quantum mechanics, cosmology, and the philosophy of science. He is the Homewood Professor of Natural Philosophy at Johns Hopkins University and an external professor at the Sante Fe Institute. Sean has contributed prolifically to the public understanding of science through a variety of mediums: as an author of several physics books including Something Deeply Hidden and The Biggest Ideas in the Universe, as a public speaker and debater on a wide variety of scientific and philosophical subjects, and also as a host of his podcast Mindscape which covers topics spanning science, society, philosophy, culture, and the arts.
www.patreon.com/timothynguyen
In this episode, we take a deep dive into The Many Worlds (Everettian) Interpretation of quantum mechanics. While there are many philosophical discussions of the Many Worlds Interpretation available, ours marries philosophy with the technical, mathematical details. As a bonus, the whole gamut of topics from philosophy and physics arise, including the nature of reality, emergence, Bohmian mechanics, Bell's Theorem, and more. We conclude with some analysis of Sean's speculative work on the concept of emergent spacetime, a viewpoint which naturally arises from Many Worlds. This video is most suitable for those with a basic technical understanding of quantum mechanics.
Part I: Introduction
00:00:00 : Introduction
00:05:42 : Philosophy and science: more interdisciplinary work?
00:09:14 : How Sean got interested in Many Worlds (MW)
00:13:04 : Technical outline
Part II: Quantum Mechanics in a Nutshell
00:14:58 : Textbook QM review
00:24:25 : The measurement problem
00:25:28 : Einstein: "God does not play dice"
00:27:49 : The reality problem
Part III: Many Worlds
00:31:53 : How MW comes in
00:34:28 : EPR paradox (original formulation)
00:40:58 : Simpler to work with spin
00:42:03 : Spin entanglement
00:44:46 : Decoherence
00:49:16 : System, observer, environment clarification for decoherence
00:53:54 : Density matrix perspective (sketch)
00:56:21 : Deriving the Born rule
00:59:09 : Everett: right answer, wrong reason. The easy and hard part of Born's rule.
01:03:33 : Self-locating uncertainty: which world am I in?
01:04:59 : Two arguments for Born rule credences
01:11:28 : Observer-system split: pointer-state problem
01:13:11 : Schrodinger's cat and decoherence
01:18:21 : Consciousness and perception
01:21:12 : Emergence and MW
01:28:06 : Sorites Paradox and are there infinitely many worlds
01:32:50 : Bad objection to MW: "It's not falsifiable."
Part IV: Additional Topics
01:35:13 : Bohmian mechanics
01:40:29 : Bell's Theorem. What the Nobel Prize committee got wrong
01:41:56 : David Deutsch on Bohmian mechanics
01:46:39 : Quantum mereology
01:49:09 : Path integral and double slit: virtual and distinct worlds
Part V. Emergent Spacetime
01:55:05 : Setup
02:02:42 : Algebraic geometry / functional analysis perspective
02:04:54 : Relation to MW
Part VI. Conclusion
02:07:16 : Distribution of QM beliefs
02:08:38 : Locality
 
Further reading:
Hugh Everett. The Theory of the Universal Wave Function, 1956.
Sean Carroll. Something Deeply Hidden, 2019.
More Sean Carroll & Timothy Nguyen:
Fragments of the IDW: Joe Rogan, Sam Harris, Eric Weinstein: https://youtu.be/jM2FQrRYyas
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org
 

Daniel Schroeder | Introduction to Thermal Physics

Daniel Schroeder is a particle and accelerator physicist and an editor for The American Journal of Physics. Dan received his PhD from Stanford University, where he spent most of his time at the Stanford Linear Accelerator, and he is currently a professor in the department of physics and astronomy at Weber State University. Dan is also the author of two revered physics textbooks, the first with Michael Peskin called An Introduction to Quantum Field Theory (or simply Peskin & Schroeder within the physics community) and the second An Introduction to Thermal Physics. Dan enjoys teaching physics courses at all levels, from Elementary Astronomy through Quantum Mechanics.
In this episode, I get to connect with one of my teachers, having taken both thermodynamics and quantum field theory courses when I was a university student based on Dan's textbooks. We take a deep dive towards answering two fundamental questions in the subject of thermodynamics: what is temperature and what is entropy? We provide both a qualitative and quantitative analysis, discussing good and bad definitions of temperature, microstates and macrostates, the second law of thermodynamics, and the relationship between temperature and entropy. Our discussion was also a great chance to shed light on some of the philosophical assumptions and conundrums in thermodynamics that do not typically come up in a physics course: the fundamental assumption of statistical mechanics, Laplace's demon, and the arrow of time problem (Loschmidt's paradox) arising from the second law of thermodynamics (i.e. why is entropy increasing in the future when mechanics has time-reversal symmetry).
Patreon: https://www.patreon.com/timothynguyen
Outline:
00:00:00 : Introduction
00:01:54 : Writing Books
00:06:51 : Academic Track: Research vs Teaching
00:11:01 : Charming Book Snippets
00:14:54 : Discussion Plan: Two Basic Questions
00:17:19 : Temperature is What You Measure with a Thermometer
00:22:50 : Bad definition of Temperature: Measure of Average Kinetic Energy
00:25:17 : Equipartition Theorem
00:26:10 : Relaxation Time
00:27:55 : Entropy from Statistical Mechanics
00:30:12 : Einstein solid
00:32:43 : Microstates + Example Computation
00:38:33: Fundamental Assumption of Statistical Mechanics (FASM)
00:46:29 : Multiplicity is highly concentrated about its peak
00:49:50 : Entropy is Log(Multiplicity)
00:52:02 : The Second Law of Thermodynamics
00:56:13 : FASM based on our ignorance?
00:57:37 : Quantum Mechanics and Discretization
00:58:30 : More general mathematical notions of entropy
01:02:52 : Unscrambling an Egg and The Second Law of Thermodynamics
01:06:49 : Principle of Detailed Balance
01:09:52 : How important is FASM?
01:12:03 : Laplace's Demon
01:13:35 : The Arrow of Time (Loschmidt's Paradox)
01:15:20 : Comments on Resolution of Arrow of Time Problem
01:16:07 : Temperature revisited: The actual definition in terms of entropy
01:25:24 : Historical comments: Clausius, Boltzmann, Carnot
01:29:07 : Final Thoughts: Learning Thermodynamics
 
Further Reading:
Daniel Schroeder. An Introduction to Thermal Physics
L. Landau & E. Lifschitz. Statistical Physics.
 
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Ethan Siegel | Demystifying Dark Matter

Ethan Siegel is a theoretical astrophysicist and science communicator. He received his PhD from the University of Florida and held academic positions at the University of Arizona, University of Oregon, and Lewis & Clark College before moving on to become a full-time science writer. Ethan is the author of the book Beyond The Galaxy, which is the story of “How Humanity Looked Beyond Our Milky Way And Discovered The Entire Universe” and he has contributed numerous articles to ScienceBlogs, Forbes, and BigThink. Today, Ethan is the face and personality behind Starts With A Bang, both a website and podcast by the same name that is dedicated to explaining and exploring the deepest mysteries of the cosmos.
In this episode, Ethan and I discuss the mysterious nature of dark matter: the evidence for it and the proposals for what it might be.
Patreon: https://www.patreon.com/timothynguyen
Part I. Introduction
00:00:00 : Biography and path to science writing
00:07:26 : Keeping up with the field outside academia
00:11:42 : If you have a bone to pick with Ethan...
00:12:50 : On looking like a scientist and words of wisdom
00:18:24 : Understanding dark matter = one of the most important open problems
00:21:07 : Technical outline
Part II. Ordinary Matter
23:28 : Matter and radiation scaling relations
29:36 : Hubble constant
31:00 : Components of rho in Friedmann's equations
34:14 : Constituents of the universe
41:21 : Big Bang nucleosynthesis (BBN)
45:32 : eta: baryon to photon ratio and deuterium formation
53:15 : Mass ratios vs eta
Part III. Dark Matter
1:01:02 : rho = radiation + ordinary matter + dark matter + dark energy
1:05:25 : nature of peaks and valleys in cosmic microwave background (CMB): need dark matter
1:07:39: Fritz Zwicky and mass mismatch among galaxies of a cluster
1:10:40 : Kent Ford and Vera Rubin and and mass mismatch within a galaxy
1:11:56 : Recap: BBN tells us that only about 5% of matter is ordinary
1:15:55 : Concordance model (Lambda-CDM)
1:21:04 : Summary of how dark matter provides a common solution to many problems
1:23:29 : Brief remarks on modified gravity
1:24:39 : Bullet cluster as evidence for dark matter
1:31:40 : Candidates for dark matter (neutrinos, WIMPs, axions)
1:38:37 : Experiment vs theory. Giving up vs forging on
1:48:34 : Conclusion
Image Credits: http://timothynguyen.org/image-credits/
Further learning:
E. Siegel. Beyond the Galaxy
Ethan Siegel's webpage: www.startswithabang.com
 
More Ethan Siegel & Timothy Nguyen videos:
Brian Keating’s Losing the Nobel Prize Makes a Good Point but …https://youtu.be/iJ-vraVtCzw
Testing Eric Weinstein's and Stephen Wolfram's Theories of Everythinghttps://youtu.be/DPvD4VnD5Z4
 
Twitter: @iamtimnguyenWebpage: http://www.timothynguyen.org

Alex Kontorovich | Circle Packings and Their Hidden Treasures

Alex Kontorovich is a Professor of Mathematics at Rutgers University and served as the Distinguished Professor for the Public Dissemination of Mathematics at the National Museum of Mathematics in 2020–2021. Alex has received numerous awards for his illustrious mathematical career, including the Levi L. Conant Prize in 2013 for mathematical exposition, a Simons Foundation Fellowship, an NSF career award, and being elected Fellow of the American Mathematical Society in 2017. He currently serves on the Scientific Advisory Board of Quanta Magazine and as Editor-in-Chief of the Journal of Experimental Mathematics.
In this episode, Alex takes us from the ancient beginnings to the present day on the subject of circle packings. We start with the Problem of Apollonius on finding tangent circles using straight-edge and compass and continue forward in basic Euclidean geometry up until the time of Leibniz whereupon we encounter the first complete notion of a circle packing. From here, the plot thickens with observations on surprising number theoretic coincidences, which only received full appreciation through the craftsmanship of chemistry Nobel laureate Frederick Soddy. We continue on with more advanced mathematics arising from the confluence of geometry, group theory, and number theory, including fractals and their dimension, hyperbolic dynamics, Coxeter groups, and the local to global principle of advanced number theory. We conclude with a brief discussion on extensions to sphere packings.
Patreon: http://www.patreon.com/timothynguyen
I. Introduction
00:00: Biography
11:08: Lean and Formal Theorem Proving
13:05: Competitiveness and academia
15:02: Erdos and The Book
19:36: I am richer than Elon Musk
21:43: Overview
II. Setup
24:23: Triangles and tangent circles
27:10: The Problem of Apollonius
28:27: Circle inversion (Viette’s solution)
36:06: Hartshorne’s Euclidean geometry book: Minimal straight-edge & compass constructions
III. Circle Packings
41:49: Iterating tangent circles: Apollonian circle packing
43:22: History: Notebooks of Leibniz
45:05: Orientations (inside and outside of packing)
45:47: Asymptotics of circle packings
48:50: Fractals
50:54: Metacomment: Mathematical intuition
51:42: Naive dimension (of Cantor set and Sierpinski Triangle)
1:00:59: Rigorous definition of Hausdorff measure & dimension
IV. Simple Geometry and Number Theory
1:04:51: Descartes’s Theorem
1:05:58: Definition: bend = 1/radius
1:11:31: Computing the two bends in the Apollonian problem
1:15:00: Why integral bends?
1:15:40: Frederick Soddy: Nobel laureate in chemistry
1:17:12: Soddy’s observation: integral packings
V. Group Theory, Hyperbolic Dynamics, and Advanced Number Theory
1:22:02: Generating circle packings through repeated inversions (through dual circles)
1:29:09: Coxeter groups: Example
1:30:45: Coxeter groups: Definition
1:37:20: Poincare: Dynamics on hyperbolic space
1:39:18: Video demo: flows in hyperbolic space and circle packings
1:42:30: Integral representation of the Coxeter group
1:46:22: Indefinite quadratic forms and integer points of orthogonal groups
1:50:55: Admissible residue classes of bends
1:56:11: Why these residues? Answer: Strong approximation + Hasse principle
2:04:02: Major conjecture
2:06:02: The conjecture restores the "Local to Global" principle (for thin groups instead of orthogonal groups)
2:09:19: Confession: What a rich subject
2:10:00: Conjecture is asymptotically true
2:12:02: M. C. Escher
VI. Dimension Three: Sphere Packings
2:13:03: Setup + what Soddy built
2:15:57: Local to Global theorem holds
VII. Conclusion
2:18:20: Wrap up
2:19:02: Russian school vs Bourbaki
Image Credits: http://timothynguyen.org/image-credits/

Greg Yang | Large N Limits: Random Matrices & Neural Networks

Greg Yang is a mathematician and AI researcher at Microsoft Research who for the past several years has done incredibly original theoretical work in the understanding of large artificial neural networks. Greg received his bachelors in mathematics from Harvard University in 2018 and while there won the Hoopes prize for best undergraduate thesis. He also received an Honorable Mention for the Morgan Prize for Outstanding Research in Mathematics by an Undergraduate Student in 2018 and was an invited speaker at the International Congress of Chinese Mathematicians in 2019.
In this episode, we get a sample of Greg's work, which goes under the name "Tensor Programs" and currently spans five highly technical papers. The route chosen to compress Tensor Programs into the scope of a conversational video is to place its main concepts under the umbrella of one larger, central, and time-tested idea: that of taking a large N limit. This occurs most famously in the Law of Large Numbers and the Central Limit Theorem, which then play a fundamental role in the branch of mathematics known as Random Matrix Theory (RMT). We review this foundational material and then show how Tensor Programs (TP) generalizes this classical work, offering new proofs of RMT. We conclude with the applications of Tensor Programs to a (rare!) rigorous theory of neural networks.
Patreon: https://www.patreon.com/timothynguyen
Part I. Introduction
00:00:00 : Biography
00:02:45 : Harvard hiatus 1: Becoming a DJ
00:07:40 : I really want to make AGI happen (back in 2012)
00:09:09 : Impressions of Harvard math
00:17:33 : Harvard hiatus 2: Math autodidact
00:22:05 : Friendship with Shing-Tung Yau
00:24:06 : Landing a job at Microsoft Research: Two Fields Medalists are all you need
00:26:13 : Technical intro: The Big Picture
00:28:12 : Whiteboard outline
Part II. Classical Probability Theory
00:37:03 : Law of Large Numbers
00:45:23 : Tensor Programs Preview
00:47:26 : Central Limit Theorem
00:56:55 : Proof of CLT: Moment method
1:00:20 : Moment method explicit computations
Part III. Random Matrix Theory
1:12:46 : Setup
1:16:55 : Moment method for RMT
1:21:21 : Wigner semicircle law
Part IV. Tensor Programs
1:31:03 : Segue using RMT
1:44:22 : TP punchline for RMT
1:46:22 : The Master Theorem (the key result of TP)
1:55:04 : Corollary: Reproof of RMT results
1:56:52 : General definition of a tensor program
Part V. Neural Networks and Machine Learning
2:09:05 : Feed forward neural network (3 layers) example
2:19:16 : Neural network Gaussian Process
2:23:59 : Many distinct large N limits for neural networks
2:27:24 : abc parametrizations (Note: "a" is absorbed into "c" here): variance and learning rate scalings
2:36:54 : Geometry of space of abc parametrizations
2:39:41: Kernel regime
2:41:32 : Neural tangent kernel
2:43:35: (No) feature learning
2:48:42 : Maximal feature learning
2:52:33 : Current problems with deep learning
2:55:02 : Hyperparameter transfer (muP)
3:00:31 : Wrap up
Further Reading:
Tensor Programs I, II, III, IV, V by Greg Yang and coauthors.
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org