LessWrong (Curated & Popular)

About a year ago, David and I put up two bounty problems involving natural latents. I am now about 80% confident that both have been resolved, both within the past couple months. Both cases made heavy use of LLMs and Lean.

The first to land was Grisha Pochuev's counterexample to the "Existence of a Deterministic Maximal Redund" conjecture. It's pretty readable, and I'm mostly convinced that it works. The original bounty post offered $500 for a proof or partial payout for a counterexample, with partial payout depending on how thoroughly the counterexample killed hope of any nearby variant of the conjecture. I think this counterexample is worth 300 dollars. Good job Grisha, and hopefully I can figure out a not-too-painful way to send you money.

Meanwhile, for a couple months David has been cranking away on "secret project X", with the promise that he'd tell me what the project was if and when it bore fruit. Well, apparently it bore fruit; he now has a proof that existence of a stochastic natural latent implies existence of a deterministic natural latent, which was our other bounty problem. The proof is apparently "pretty gnarly", lots of cases, all LLM-coded in Lean. [...]

---

First published:

August 11th, 2026


Source:

https://www.lesswrong.com/posts/7QvKqpGJwqXrQcMgx/llms-are-starting-to-noticeably-accelerate-our-work

---



Narrated by TYPE III AUDIO.

Today is August 4, 2026

[Crossposted from AI StopWatch]

In the living room the voice-clock sang, Tick-tock, seven o’clock, time to get up, time to get up, seven o’clock! as if it were afraid that nobody would.

So begins Ray Bradbury's There Will Come Soft Rains, a short story that has haunted me for most of my life. Depicting the aftermath of nuclear war, it was first published in 1950. It takes place today.

Literally:

“Today is August 4, 2026,” said a second voice from the kitchen ceiling, “in the city of Allendale, California.” It repeated the date three more times for memory's sake. “Today is Mr. Featherstone's birthday. Today is the anniversary of Tilita's marriage. Insurance is payable, as are the water, gas, and light bills.”

Was it narrative convenience or prophetic vision that drove Bradbury to depict the smart house of the future as gratuitously conspicuous in its competence, pointlessly reminding the owners of the year and their city of residence? There's something very Alexa-like about that — and about the janky brittleness evident in the system as it prepares breakfast for a family that won’t be eating and opens the garage door for a father who [...]

---

First published:

August 4th, 2026


Source:

https://www.lesswrong.com/posts/aowxE8xZ8xkhRCn9r/there-will-come-soft-rains-1

---



Narrated by TYPE III AUDIO.

It seems to me that, for every loss function that we use to train LLMs, we get a very distinct flavor of LLM misalignment. Here's the summary table, and then we’ll go through the rows separately.

Training stage

Loss function

Flavor of misalignment

Famous examples

Pretraining & SFT

Imitative learning (next-token prediction)

“Seven deadly sins” misalignment

Bing-Sydney, “Emergent misalignment”

RLHF & DPO

Human approval

“Glazing” misalignment

GPT-4o

RLVR

Automatic verifier

“Literal genie” misalignment

HuggingFace hacking

RLAIF

Approval from another LLM

“Trickster” misalignment

“Current AIs seem pretty misaligned to me”

Warning: I’m not an LLM power-user myself, but rather relying on reports I’ve read. Also, I don’t consider LLM alignment to be my primary area of expertise. I’m open to feedback!

1. Imitative learning → “seven deadly sins” misalignment

Training stage

Loss function

Misaligned behavior

Pretraining, SFT

Imitative learning (next-token prediction)

Any and all of the vices of humanity

In imitative learning, the LLM tries to predict what the next token of text will be. Then those predictions magically turn into its outputs. See my earlier discussion: “LLM pretraining magically transmutes observations into behavior, in a way that is profoundly disanalogous to how brains work”.

This leads to LLM behavior [...]

---

Outline:

(00:55) 1. Imitative learning → "seven deadly sins" misalignment

(04:24) 2. Human approval → "glazing" misalignment

(06:35) 3. Automatic verifiers → "literal genie" misalignment

(08:05) 4. LLM judges → "trickster" misalignment

(12:06) Afterword

The original text contained 1 footnote which was omitted from this narration.

---

First published:

August 10th, 2026


Source:

https://www.lesswrong.com/posts/GRmvZsHXH4vaijPMv/four-llm-loss-functions-four-flavors-of-llm-misalignment

---



Narrated by TYPE III AUDIO.

Introduction

I think reprogenetics (human germline genomic engineering) can be done in a widely acceptable and beneficial way, and should be pursued aggressively. In particular, as a strong background motivation of mine, I think accelerating strong reprogenetics is probably the best way to enable strong human intelligence amplification; and I think strong HIA is among the best ways to decrease existential risk from AGI.

A very common objection to caring much about reprogenetics is that AGI seems very likely to come soon—say, within a decade or two. (Here I mean "actual" AGI—the kind that probably doesn't already exist—the kind that has fluid intelligence and AI advantages for recursive self-improvement, which together make it likely to take over the world shortly after being created.) The objection is fairly straightforward:

AGI will probably come within a decade or two. If that's going to happen, then even if a new cohort of brilliant humans were born today, they would still be children, or would at best have barely begun contributing ideas for how to avoid extinction. Any supposed benefit, denominated in percentage points of AGI existential risk averted, is small. Therefore, reprogenetics is too slow; and if you're going [...]

---

Outline:

(00:12) Introduction

(03:37) HIA, part of your nutritionally complete portfolio

(05:52) Against confident short timelines

(08:29) HIA may indirectly slow down AGI capabilities

(09:31) HIA has substantial impact even with short timelines

(16:10) Adult HIA methods aren't fast either, absent big investment

(27:35) Takeaways

---

First published:

August 8th, 2026


Source:

https://www.lesswrong.com/posts/iQzxxgJXXaAQjq7Jz/faq-isn-t-agi-coming-too-soon-for-reprogenetics-to-help

---



Narrated by TYPE III AUDIO.

This sequence is about the last decade in AI alignment. It recounts the gradual transition from a field which treated alignment as a hard scientific problem, to a field which has largely abandoned the goal of deep, generalizable scientific progress in favor of iteratively improving existing systems and attempting to gain technological and political power. I also describe (in subsequent posts, which I'll upload over the next few weeks) how fear and (self-)deceptive reasoning made the field one of the biggest forces pushing AI capabilities forward over the last decade, especially via significant contributions to the scaling of LLMs and the development of ChatGPT.

Zooming out further: the two leading AGI companies, which are locked in an intense rivalry, were both explicitly founded under the banner of AI alignment, and got off the ground in significant part due to alignment-oriented ideas, talent and resources. People in the field often sense that something must have gone wrong to get here, but don’t know how to allocate responsibility (aside from blaming Sam Altman and sometimes Elon), and fall back on assuming that “the ship has already sailed”. But in this sequence I characterize our current situation as resulting from a pattern [...]

---

Outline:

(08:17) Conceptual Clarity and Scientific Progress

(20:26) Orienting Towards Prestige

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

August 9th, 2026


Source:

https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-happened-a-retrospective-of-ai-alignment

---



Narrated by TYPE III AUDIO.