Are we hurtling toward a future where AI can do everything humans can? Edwin Chen (@echen) believes we might...

Are we hurtling toward a future where AI can do everything humans can?

Edwin Chen (@echen) believes we might be. He’s the CEO of Surge AI, one of the largest providers of expert data for frontier labs. Surge passed over $1 billion in revenue without raising any outside capital, and that gives Edwin a unique perspective on how quickly AI progress is accelerating.

I’m on the record arguing that AI automation actually creates more human work. I also believe that even though AI progress is accelerating exponentially, we’re much farther away from AI replacing humans than it might seem.

That’s why I had Edwin on @every’s AI & I. We batted around different visions of the future, and discussed whether humanity will retain its unique place in the universe, and what that might be.

We get into:
• If Chen’s version of the future materializes, he’s worried it’ll make people stop trying. One answer comes from a short story by science fiction writer Ted Chiang: Behave as if your decisions matter, even when you know they don’t.
• AI may soon be able to take a nebulous goal like “win a Fields Medal” and execute. What it can’t do, I argue, is set its own goals—LLMs have no intrinsic motivation, no drive to explore, no ability to just change their mind.
• A model optimized for engagement doesn’t provide the most valuable user experience. Edwin spent 20 rounds polishing a pointless email with one model before Claude told him to just send it.
• Why AI is still bad at writing: models learn to hack the metrics they're trained on. Edwin's Hemingway Bench found models outputting a metaphor in every single sentence, an overindexxing that makes for a terrible reading experience.

This is a must-watch for anyone interested in where we fit as models get more capable.

Watch below!

Timestamps
1. Introduction: 00:00:54
2. Surge as a "school for AGI": 00:01:49
3. What AI's capacity for novel mathematics says about human achievement: 00:04:46
4. Motivation in an era when AI can do everything: 00:07:29
5. The trap of optimizing AI models for engagement: 00:14:34
6. Training using datasets versus training using environments: 00:29:34
7. The value of personal data: 00:35:09
8. Why models are bad at writing: 00:39:40
9. Chen's AGI timeline: 00:42:00

YouTube: https://www.youtube.com/watch?v=omX6wrLuX08&feature=youtu.be

Spotify: https://open.spotify.com/episode/3u4hnxGB6tefawfPwdmZuI?si=uPUjqlEKRrG8GcyFFk-Hlw