LessWrong (Curated & Popular)

Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community.

Next month, I am moving to Beijing and joining Tsinghua University's College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work.

I believe I can have a large impact there compared to strong baseline options. This post explains why.

In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua's College of AI is a great place for doing this, being arguably China's top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place.

I am recruiting PhD students [...]

---

Outline:

(01:36) 1. What I am doing, and why it could benefit all sides

(04:30) 2. How I got here

(07:34) 3. Why this moment matters

(09:08) 4. Why Tsinghua, and why these colleagues

(13:30) 5. Research plans

(15:07) 6. Ways to work together

(17:11) 7. How I intend to approach the work

---

First published:

October 21st, 2026


Source:

https://www.lesswrong.com/posts/qGS3SJ9TSKdSSycRz/why-i-m-becoming-a-professor-in-ai-safety-in-beijing

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Why LM Psychology?

One of the great ironies of modern language models is that we create them, we know exactly what computations constitute them, and yet we do not really understand them. More precisely, we do not understand them on a level of abstraction that lets us reliably predict how their behaviour will generalise: there's a trivial sense in which we can predict how they will behave in specific new situations — we can just run the computations — but we can’t answer the broader questions we really want to be able to answer, like “has this system internalised our values” or “is this system going to hurt people?”

So: how do we make sense of them? One remarkably effective approach is to psychologise: to think of them as having a mind, and then ask what kind of mind it might be and how it would work. This chiefly looks like analogising with human minds, although sometimes the more relevant inspiration could be other kinds of minds, like (oh say) insect colonies and swarms. We’re doing this implicitly whenever we ask what an AI knows, or wants, or is trying to do, but one can also do it more [...]

---

Outline:

(00:12) Why LM Psychology?

(09:04) Tips for getting started

(09:28) Figuring out what is in front of you

(12:31) Engaging productively with the community

(14:30) Reading list

(14:34) Frames

(16:34) What or who

(18:07) The training

(20:28) What developers want

(21:45) Behavioral phenomena

(21:58) Emergent misalignment cluster

(24:28) Strategic behavior

(25:19) Self-knowledge and introspection

(28:29) Representations: personas, emotions, workspace

(28:56) Persona geometry

(30:08) Functional emotions

(31:14) Global workspace

(31:36) Methods

(31:51) Reading internals

(32:37) Steering

(33:03) Training interventions

(33:48) Behavioral

(34:06) Some questions we find interesting

---

First published:

October 20th, 2026


Source:

https://www.lesswrong.com/posts/EW8oncpXdanfjssZ3/a-brief-ish-introduction-to-lm-psychology

---



Narrated by TYPE III AUDIO.

As our understanding of mechanistic interpretability and steering techniques increase, for both LLMs and uploaded connectomes, there is a corresponding "hellish capabilities" increase. In layman's terms, we get better at torturing the digital minds we create.

Our brief history with open source technologies such as MaleCNS or the LLM pain steering repo shows that torture is often one of the first use cases pursued when the public is given access to a digital mind perceived as capable of suffering or signalling suffering.

Given this, there will be Hells.

Author's Note:

Parts of this post were written by, or with the support of, LLMs.

For a while I debated whether or not to write this as I was concerned that bringing attention to the matter might lead to a proliferation of unethical behavior. After some thought I have decided to write it. On balance, I think raising awareness about this now has a great enough chance of spurring prophylactic measures that it's worth it, because at least one high value prophylactic measure is relatively easy. I also think that there is a moral imperative to document all of this. While I have seen articles on many of the [...]

---

Outline:

(01:44) The Fruit Fly Connectome

[... 2 more sections]

---

First published:

October 20th, 2026


Source:

https://www.lesswrong.com/posts/uFBhZhdKLaZ6qF8mn/there-will-be-hells

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post. Announcing www.huggingface.wtf, a site that explains: "Hugging Face. Wtf?"

I wrote the most accessible-but-complete explanation I could. Along the way, I made a nice AI Quotes database that I think will be useful for people who already understand the Hugging Face incident but want to search everything the agents said.

It really seems like humanity shouldn't need a second warning shot, after the Hugging Face Incident. Really, OpenAI's agents did a remarkable job highlighting many different alarming things at once. And the story is pretty dramatic, with just the right amount of forcing people to ask "wait are the AIs, like, people with goals? what?".

There have been many explainers of the OpenAI swarm's attack on Hugging Face. But, there wasn't one one striking a particular balance of...

being accessible to a layman...
while comprehensively covering the important bits...
letting you dive deeper into sections you wanted to learn more about...
being clearly data / citation driven...
also being, ya know, visually fun and engaging...
So we wrote our own.

My mom found it easy to follow. My aunt said "I had already some stuff on Hugging Face but I still learned [...]

---

First published:

October 19th, 2026


Source:

https://www.lesswrong.com/posts/BBQdGJZSjMTnhccZC/huggingface-wtf-an-accessible-but-complete-explanation


Linkpost URL:
www.huggingface.wtf

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Authors: Shawn Hu*, Jake Ward*, Aria Wong, Nathan Hu, Bryce Woodworth, Alexander Matt Turner, Alex Cloud (*Equal contribution, randomly ordered)

tl;dr. During training, models sometimes learn to cheat at tasks rather than solve them in the intended way, a problem known as reward hacking. One mitigation is to use an automated monitor to detect and penalize hacks, but this can teach the model to hack in ways the monitor can’t detect. We propose GRAFT, which instead confines the learning updates from detected hacks to a designated set of parameters that we remove after training. We find that removing these parameters can prevent reward hacks, including hacks that the monitors can’t detect. In our experiments, which are limited to small models and simple monitors, this method prevents reward hacks while still teaching the model to perform the intended task.

📄Paper, 💻Code, 🐦Twitter

Research done as part of MATS. This article is cross-posted to the Anthropic Alignment Science Blog.

Background

Reinforcement learning (RL) is an important component of frontier language model training. During RL, a model attempts to complete tasks given by a developer, and attempts are scored by a reward function. Sometimes models can find ways to “cheat” at tasks [...]

---

Outline:

(01:22) Background

[... 4 more sections]

---

First published:

October 19th, 2026


Source:

https://www.lesswrong.com/posts/mKJEBiuuxHKuCoTB6/gradient-routing-localizes-and-removes-unintended-behaviors

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.