LessWrong (Curated & Popular)

For those who are trying to bring about a glorious transhuman utopia with the help of hopefully-aligned ASI, I think it's worth thinking explicitly about what utopia might actually look like and where it's likely to fall short.

To that end, some have helpfully written depictions of utopian (or utopia-adjacent) worlds: The Adventure, Just another day in utopia, The Culture, The Gentle Seduction, The Gentle Romance, Machines of Loving Grace, Friendship is Optimal, Dath Ilan, The Maker of MIND, Failed Utopia #4-2.

Unfortunately, the best utopian story I've ever read is also a massive spoiler, since it appears at the very end of a much longer story (see below for the title and author):

Worth the Candle by Alexander Wales

Inspired by this tweet[1] and with the original author's permission, I adapted the epilogue of that story so it can be enjoyed without 1.5 million words of context!

What I love most about this depiction is its exploration of the inherent imperfection of utopia: even when you have literally unlimited power, flaws will remain, and some (many?) people will even prefer the pre-utopia world.

The primary purpose of this adaptation is to recontextualize the epilogue so it's accessible and [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

June 3rd, 2026


Source:

https://www.lesswrong.com/posts/to9cSGgD6nALByKjg/my-favorite-depiction-of-utopia

---



Narrated by TYPE III AUDIO.

ARC has teamed up with AIcrowd to launch the ARC White-Box Estimation Challenge, a contest to improve upon our estimation algorithms for random MLPs. The warm-up round begins this week, and later rounds will have a total prize pool of at least $100,000.

We are very grateful to Sharada Mohanty, Sneha Nanavati, Dipam Chakraborty and everyone else at AIcrowd for working with us to host this contest, as well as to Paul Rosu for testing the contest and to Harshita Khera for operational support.

Introduction to the Challenge

Our challenge follows the same setup as our recent paper on wide random MLPs: we consider MLPs with weights , defined by



where the activation function is , applied coordinatewise.




To begin with, we are fixing the width and the number of hidden layers , but we expect to change this setup in future rounds.[1]

Contestants must design an algorithm that takes in a set of weights and produces an estimate for the expected output


Algorithms will be evaluated on MLPs with randomly-sampled Gaussian weights. The goal is to achieve as low mean squared error as possible, subject to certain computational [...]

---

Outline:

(00:41) Introduction to the Challenge

(01:58) Why run this contest?

(03:39) Use of LLMs

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

June 2nd, 2026


Source:

https://www.lesswrong.com/posts/Kben8CzS4awCwNw5c/announcing-the-arc-white-box-estimation-challenge

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As a bureaucrat, my role is to annoy my friends. Someone voices an idea, “Wouldn’t it be nice if…” or “I wonder if we could…” I make a note. I do some estimates. If it pencils out, I’ll bring it back up, week after week. The discussions are fun, but also practical. We’ll test the waters, what would be a minimum viable scheme? What's easy, what's hard? Who could do the hard parts? Over time the idea gets more detailed, specific, feasible. I’ll pull out a calendar. Soon our scheme has co-conspirators, action items, even a budget. It's just good staff work.

I’ve been hearing whispers in the wind for a year now.

“Imagine if we had something like this in DC.” 
“Where can I host an event that might get a dozen or a hundred people?” 
“It's such a pain in the ass to book event space in the Capitol.” 
“I think this person has started to see what's coming, where can they go to get caught up?”
“The community seems to be growing but it's all fragmented in group chats.” 
“How is no one planning an afterparty, that's clearly the highest leverage intervention!?”
“Why can’t [...]
---

Outline:

(02:11) How Lighthaven Works

(05:45) What Does DC Need?

(06:52) A Day in the Life

(10:19) Minimum Viable Lighthaven

(12:04) ...so you mean a Group House?

(14:27) ...so you mean a Co-Working Space?

(16:27) Feasibility Study

(17:35) Property

(22:19) Funding

(24:55) What is the Minimally Viable Funding?

(28:03) Leadership

(31:06) Cultural Fit

(33:21) Name and Brand Positioning

(35:20) Ability to Scale

(37:48) Risks

(41:09) First Steps

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

May 31st, 2026


Source:

https://www.lesswrong.com/posts/95NgkvZKJx8tJbtn5/lighthaven-east-a-feasibility-study

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

1.1 Tl;dr

Alignment is often conceptualized as AIs helping humans achieve their goals: AIs that increase people's agency and empowerment; AIs that are helpful, corrigible, and/or obedient; AIs that avoid manipulating people. But that last one—manipulation—points to a challenge for all these desiderata: a human's goals are themselves under-determined and manipulable, and it's awfully hard to pin down a principled distinction between changing people's goals in a good way (“providing counsel”, “providing information”, “sharing ideas”) versus a bad way (“manipulating”, “brainwashing”).

The manipulability of human desires is hardly a new observation in the alignment literature, but it remains unsolved (see lit review in §3 below).

In this post I will propose an explanation of how we humans intuitively conceptualize the distinction between guidance (good) vs manipulation (bad), in case it helps us brainstorm how we might put that distinction into AI.

…But (spoiler alert) it turns out not to really help, because I’ll argue that we humans think about it in a deeply incoherent way, intimately tied to our scientifically-inaccurate intuitions around free will.

I jump from there into a broader review of every approach that I can think of for writing a “True Name” for manipulation or [...]

---

Outline:

(00:13) 1.1. Tl;dr

(02:04) 1.2. Bigger-picture context: why is this issue so important to me?

(04:48) 2. How do humans intuitively define empowerment, agency, manipulation, etc.?

(04:56) 2.1. Background: human free will intuitions

(09:20) 2.2. Our free-will-infused intuitive notions of empowerment, agency, manipulation, corrigibility, responsibility, etc.

(12:00) 2.3. Another dimension: counsel vs manipulation as an emotive conjugation

(13:07) 3. If the intuitive definitions of manipulation etc. reside in a messed-up ontology, has the alignment literature found any alternative, better way to define these concepts?

[... 12 more sections]

---

First published:

May 11th, 2026


Source:

https://www.lesswrong.com/posts/vzHtHHBJoKATi5SeK/empowerment-corrigibility-etc-are-simple-abstractions-of-a

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

At the risk of embarrassing myself, I’ll share a confession.

For context, I took five years of Latin: four in high school and one in college. In addition to learning the language, all my Latin classes taught a lot about Roman history. Emperors, internal politics, Caesar, etc. I was always learning some random bag of facts about Roman history. In high school, I won the award for top Latin student in my graduating class. So I wasn’t a bad Latin student.

Here's the confession: I somehow don’t even vaguely remember the rough timespan the Roman Empire existed. Maybe Jesus time? I know he was killed by the Romans (is that right?). Were they around for a long time after? A long time before that? When was Romulus and Remus allegedly fighting? Virgil wrote the Aeneid when? I don’t have a clue. Despite being a kind of “Latin expert” I am missing a much more important foundational fact: when all of this was happening.

When I say trees are made out of air I’m not talking about the fact that there is a lot of empty space inside a tree (or actually anything made out of atoms). I mean something [...]

---

First published:

May 28th, 2026


Source:

https://www.lesswrong.com/posts/xiTBpBDwubnr4MLRe/trees-are-mostly-made-of-air-and-a-generalizable-lesson-for

---



Narrated by TYPE III AUDIO.