ForeCast

Lukas Finnveden is a Research Analyst at Redwood Research. We talk about making deals with early scheming AIs: paying them to cooperate rather than take over.

You can read a full transcript here.

To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.

Rose Hadshar (Forethought) and Nora Ammann (ARIA) talk about power concentration, checks and balances, coups, and post-AGI political economy.

You can read a full transcript here.

This is the audio version of a video podcast. Watch the video on YouTube: youtu.be/PB855Xpx1kk?si=v05LVM-2VQ3XiT7N

Prompted in part by Rose's recent article for 80,000 Hours on ‘Extreme power concentration’ — 80000hours.org/problem-profiles/extreme-power-concentration

Timstamps:

00:00:00 Intro

00:01:26 What is ‘power concentration’? Is power concentration the right framing?

00:05:25 When can it be good for power to be centralised?

00:09:40 Is ‘checks and balances’ a better framing?

00:11:16 How AI undermines existing checks and balances

00:15:25 Economic power, meme complexes, and cultural influence

00:28:50 AI companies vs governments as centres of power

00:31:25 The difficulty of knowing where power actually lies

00:40:05 Do humans and AIs concentrate power differently?

00:48:38 Should we be trying to imagine a better post-AGI political economy?

00:56:28 Concrete actions: transparency & whistleblower protections

00:59:07 Human-AI teaming and collaborative intelligence

01:01:16 AI agent economies and proving trustworthiness

01:13:16 Can we train AI to follow the law?

01:16:33 Building resilient coalitions of trustworthy agents

01:18:42 Closing reflections

Joe Carlsmith is a writer, researcher, and philosopher. He works at Anthropic on the character/constitution/spec for Claude. Before that, he was a senior advisor at Open Philanthropy.

You can read a full transcript here.

To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.

This is a bonus episode to say that Forethought is hiring researchers. After an overview of the roles, we hear from Research Fellow Mia Taylor about working at Forethought.

You can read a full transcript here.

The application deadline has been extended to November 1st 2025.

Apply here: forethought.org/careers/researcher

Chapters

(00:00:00) Forethought hiring overview and roles

(00:03:21) Interview with Mia begins

(00:03:34) Why Mia joined Forethought

(00:05:59) Daily work, how work at Forethought was different from expected

(00:08:38) Research examples

(00:10:58) Who should and shouldn't apply

(00:14:11) Disagreements and closing thoughts

Links

Apply for researcher positions

Referral form
(if you know someone who might be a good fit)

This is a narration of ‘Introducing Better Futures
’ by William MacAskill; published 3rd August 2025.

Narration by Perrin Walker (@perrinjwalker).