LessWrong (Curated & Popular)

This is a link post. METR recently estimated that Anthropic is now making AI progress about twice as fast as its historical rate (they would not be surprised by anything from 1.3x to 3x), and guessed that “perhaps roughly equals 80%” of the current acceleration is due to AI. If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario.

This level of acceleration due to AI is very unexpected. Previously, Daniel, the most bullish of us, assigned a 50% chance of Superhuman AI Researcher (SAR) before 2029. SAR is roughly the point at which a company would rather fire all their human researchers and rely solely on automated AI researchers than have a team of the best human researchers in the world with no AIs.

Now, all of us forecast that Superhuman AI Researcher (SAR) will likely arrive at some point in 2027 (assuming no substantial deliberate slowdown). We also predict a 35-65% chance that Top-Expert-Dominating AI (TED-AI), AIs that can automate all white-collar work, will arrive before 2028. While there were many factors and sources of evidence that influenced us, the single biggest source was the METR report.

If SAR is [...]

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/f9PajMLmBYzmWzyDv/aifp-looks-like-the-intelligence-explosion-will-happen-in


Linkpost URL:
https://blog.aifutures.org/p/q35-timelines-update-looks-like-the

---



Narrated by TYPE III AUDIO.

Note: As with any research done in a couple of days, this is all very tentative.

I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days.

% rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs.


This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution).

Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...]

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

November 11th, 2026


Source:

https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought:

“Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114).

It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below.

After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...]

---

Outline:

(02:54) Is this somehow about positive vs negative rewards?

[... 6 more sections]

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question:

How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph.

You get a bunch of fairly similar answers. I’ll bold the common themes below.

Gemini 4 Carbon says:

There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:

There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:

There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...]

---

Outline:

(02:25) Yudkowsky on AI Consciousness and Welfare

(06:20) Yudkowsky on The Moral Duties We Have to AI

(08:56) Soares on AI Consciousness and Welfare

(12:30) On Claude, Control, and Building ASI

(16:59) Misc Longform Writing

(19:41) Conclusion

The original text contained 1 footnote which was omitted from this narration.

---

First published:

November 11th, 2026


Source:

https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of

---



Narrated by TYPE III AUDIO.

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular.

In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill.

People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement.

People are starting to feel like it's the endgame, “the board is set”.

I do agree with everyone else that it's starting to look like we’re in the endgame.

The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent.

But the fight is not yet over:

“We have very little time to stop full recursive self improvement

But ASI hasn't taken over yet

If we all work really hard we might still have a chance

But this means that every single person who is aware of what is going on must be *actually trying*

Like "we're [...]

---

Outline:

(01:41) Actually Trying

[... 4 more sections]

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.