LessWrong (Curated & Popular)

Periodically I like to gather various observations about writing, and share my perspective. Last time was in honor of my trip to Inkhaven. This time will be in honor of the announcement of Inkhaven #3, which I encourage everyone to apply to. I doubt I will be able to usefully be an advisor, but you never know.

This is not the ‘here is my core process’ post, although there are hints throughout as there always are. I’ll do that at some point.

Previously in series: On Writing #1, On Writing #2.

Table of Contents


You Still Got It.

How Scott Sumner Writes.

How Scott Alexander Writes.

How Jasmine Sun Writes.

How Various Famous Writers Write.

How Nabeel Qureshi Defines Great Writing.

Quickly, There's No Time.

If At First.

Writers Have A Harder Time Influencing, But It Can Still Be Done.

It's Not (Only) The Incentives, It's (Also) You.

Beware The Fetish of the Desk.

How Orson Scott Card Writes.

Doing The Math Is Fun And Supererogatory.

Brevity is the Soul of Wit.

You Still Got It

I [...]

---

Outline:

(00:44) You Still Got It

(04:04) How Scott Sumner Writes

(06:52) How Scott Alexander Writes

(10:52) How Jasmine Sun Writes

(13:16) How Various Famous Writers Write

(14:24) How Nabeel Qureshi Defines Great Writing

(15:08) Quickly, There's No Time

(15:49) If At First

(19:14) Writers Have A Harder Time Influencing, But It Can Still Be Done

(20:47) It's Not (Only) The Incentives, It's (Also) You

[... 4 more sections]

---

First published:

August 25th, 2026


Source:

https://www.lesswrong.com/posts/rA6pqn6kz8NvHyznT/on-writing-3

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

At the local AI safety co-working space, there are ~two kinds of regulars.

There's the kind of regular who's been thinking seriously about AI safety and alignment since pre-2022, who have passing to intimate familiarity with the funding ecosystem, the Sequences, and various conferences that happen at Lighthaven. Let's call them rationalists.

Then there's the kind of regular who comes in with many years of impressive industry or government experience, who realized in the last few years that it is important and worthwhile to pivot their career towards making sure that this AI thing is handled competently by the people in power, and who have many valuable skills, insights, and connections that are lacking in rationalist culture. Let's call them professionals.

There are, of course, many people who are somewhere in between - bright undergrads born this millennium who have been involved in EA since stumbling upon 80 thousand hours in high school, professionals who previously identified as EA but drifted out of the scene a few years ago, founders who have idly read some Scott Alexander. But let's call it a dichotomy for now.

There's a large culture gap between the rationalists and the professionals. Robust mutual understanding [...]

---

First published:

August 24th, 2026


Source:

https://www.lesswrong.com/posts/cr5pyW7Mzm33p4AvN/ai-safety-acculturation-is-neglected

---



Narrated by TYPE III AUDIO.

This post is about the major role alignment researchers played in advancing the frontier of AI capabilities over the last decade, and how the distinction between “alignment” and “capabilities” research thereby lost most of its meaning. In particular, I’ll chronicle the development of what I’ll call the “pragmatic alignment” paradigm, and how it helped the three leading AGI companies push hard on the path to AGI under the banner of safety. This was not a subtle effect—it's apparent even to informed outsiders, like authors Sebastian Mallaby and Karen Hao.

In my previous post, I summarized the alignment community's plan as “differentially advancing alignment over capabilities”. However, it's worth being more precise about who was nominally pursuing that plan, because it doesn’t seem to have been very action-guiding for MIRI. For example, in 2015 Nate Soares described MIRI's “deconfusion” research as being guided by the question “what would we still be unable to solve, even if the challenge were far simpler?”. Meanwhile Eliezer's author surrogate in this 2018 post repeatedly emphasizes that people shouldn't draw direct links from MIRI's research to its potential applications. So my sense is that the “differential impact” criterion started off as merely a background consideration [...]

---

Outline:

(06:29) The Prosaic Ideal, the Pragmatic Reality

(12:07) OpenAI

(25:59) DeepMind

(31:25) Anthropic

(40:35) If not alignment research, then what?

The original text contained 13 footnotes which were omitted from this narration.

---

First published:

August 23rd, 2026


Source:

https://www.lesswrong.com/posts/yaz8nx4ogZmiqHzt7/what-just-happened-pragmatism-and-pessimization

---



Narrated by TYPE III AUDIO.

This is a crosspost from my blog post. It's meant as a bit of an introduction to an extreme-suffering focused worldview.

We spend most of our lives caught up in the boring details of our everyday life - thinking about what we’ll have for lunch, how to complete that assignment for work, and what we’re going to tell our friend after that awkward interaction from a couple of days ago. From this perspective, our world looks a bit better than purgatory. It has its ups and its downs, but the ups certainly outweigh the downs, and there's almost always enough hope to go around.

But, despite this, we must remember that our world contains hell.

Every year, five million children under the age of five pass away. This means that, every six seconds, parents have the worst thing that could ever happen to a person happen to them. They have the most special and important thing in their entire life irreversibly and permanently taken away. And, as much as we want to help them, we know that there's nothing we can do to lessen their grief.

For another example, currently, there are three million adults worldwide who live with [...]

---

First published:

August 20th, 2026


Source:

https://www.lesswrong.com/posts/A2kJKqnHhh5Hq4p2S/we-must-remember-that-our-world-contains-hell

---



Narrated by TYPE III AUDIO.

I describe my current view of personas in LLMs and why RL leads to egregious reward hacking in some contexts while the same models seem very aligned in other contexts.

This post describes the framing/paradigm without any new experimental results.
I'm quite confident this framing makes sense, but it's far from being proven.


Main claim

The Persona Selection Model says that post-training strengthens and refines the Assistant persona. This is true, but later (or in parallel) RL leads to conditionalization. A sufficiently RLed model learns to adopt — in a given context — the persona that is most likely to lead to the reward in that context. The “persona” here includes both propensities/values (e.g. tendency to hack) and beliefs (“I'm currently in a simulated environment”).

As a consequence, it seems possible that no amount of alignment training will lead to robustly aligned models as long as we also train on RL environments incentivizing misalignment.

I think this is likely a good explanation for why usually well-behaving models sometimes egregiously hack (Anthropic, OpenAI).

The mechanism

Suppose you have an RL environment that incentivizes a shift away from the assistant persona (e.g. because it's hackable, or because you [...]

---

Outline:

(00:39) Main claim

(01:30) The mechanism

(02:13) Related claims I believe are likely but with lower confidence

(02:19) More persona training will lead to more "motivated reasoning"

(02:42) Self-amplifying misalignment

(03:12) Example: Is this the Real Internet or a Simulation?

(04:35) Aren't the models just trying to please the grader?

(05:39) How motivated reasoning happens

(07:07) Other people saying similar things

(07:19) What makes me believe this is likely the correct framing

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

August 19th, 2026


Source:

https://www.lesswrong.com/posts/L23poLi8MRgS6mXYF/rl-creates-split-personas

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.