lecture September 13, 2024 24:07 YouTube

AI can't cross this line and we don't know why.

Welch Labs

AIScaling

Train a model and its error falls fast, then flattens. Train a bigger one and it reaches lower, for more compute. Plot the family of curves on log axes and a boundary appears that no model crosses — the compute-optimal frontier. It is one of three neural scaling laws, and error tracks compute, model size, and dataset size in much the same way almost regardless of architecture. The question the video ends on is whether this is an ideal gas law for intelligence, or a coincidence we have mistaken for one.

Theme
Language
Support
© funclosure 2025