LessWrong (Curated & Popular)

Note: As with any research done in a couple of days, this is all very tentative.

I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days.

% rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs.


This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution).

Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...]

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

November 11th, 2026


Source:

https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought:

“Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114).

It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below.

After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...]

---

Outline:

(02:54) Is this somehow about positive vs negative rewards?

[... 6 more sections]

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question:

How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph.

You get a bunch of fairly similar answers. I’ll bold the common themes below.

Gemini 4 Carbon says:

There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:

There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:

There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...]

---

Outline:

(02:25) Yudkowsky on AI Consciousness and Welfare

(06:20) Yudkowsky on The Moral Duties We Have to AI

(08:56) Soares on AI Consciousness and Welfare

(12:30) On Claude, Control, and Building ASI

(16:59) Misc Longform Writing

(19:41) Conclusion

The original text contained 1 footnote which was omitted from this narration.

---

First published:

November 11th, 2026


Source:

https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of

---



Narrated by TYPE III AUDIO.

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular.

In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill.

People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement.

People are starting to feel like it's the endgame, “the board is set”.

I do agree with everyone else that it's starting to look like we’re in the endgame.

The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent.

But the fight is not yet over:

“We have very little time to stop full recursive self improvement

But ASI hasn't taken over yet

If we all work really hard we might still have a chance

But this means that every single person who is aware of what is going on must be *actually trying*

Like "we're [...]

---

Outline:

(01:41) Actually Trying

[... 4 more sections]

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Tl;dr:

Lightcone Commons’ first funding round is complete! 17 evaluators made funding recommendations, including Zvi Mowshowitz, Scott Siskind and Eliezer Yudkowsky. 23 funders coordinated to grant 45.1 million dollars, and 147 projects received funding.

Round two is open. Applications for round two close on November 15th and funders confirm their grants by December 14th. Funders can join anytime before December 7th.

In 2027, we will be hosting funding rounds monthly. Yes. Every month.

You can join as a funder, express interest in joining as an evaluator, and apply for funding at lightconecommons.com.

Lightcone Commons staff do not decide what grants get made. Pls stop saying that "Lightcone Commons funded us", that's a bit like saying "PayPal funded me".

Lightcone Commons is an end-to-end philanthropic coordination platform.

We host an open funding application, and invite evaluators to make ordered funding recommendations. Participating funders then decide how much they want to give, and to whom – either by following the recommendations of evaluators or choosing specific projects to fund. When multiple funders want to fund the same project they ‘split the bill’.

Lightcone Commons is not a foundation. We don’t decide to make specific grants. Funders decide [...]

---

Outline:

(02:16) Round One Overview

(02:22) Funders

(03:08) Evaluators

(03:50) Next Round

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

November 10th, 2026


Source:

https://www.lesswrong.com/posts/3ASLq8ePNqkbeGqdj/lightcone-commons-round-1-complete-round-2-underway

---



Narrated by TYPE III AUDIO.