Mike Gioia (@intelligentjello)

Sutton's The Bitter Lesson is actually the impetus for this article. My previous post talks about that essay at length. I corresponded with Rich while writing that post asking him several questions. His response to the final draft was simply "Not too bad", which is about as good as I can expect to get from him on an essay that eschewed the bitter lesson's lesson. Predicting the next frame is definitely a simplification, but I think it's not a bad metaphor. In reality I think the models like compress data and do the prediction in latent space, but they're still predicting how those pixels plausibly change over the course of the video. I could be off, but it feels like a helpful example for readers.