Jenneral HQ 🌠

On Friends at Frontier Labs

I am deeply uncertain about whether or not it is a good idea to get my friends to quit working at frontier AI labs. I am against people working at frontier AI labs in general, but perhaps downstream of this, all my friends in the labs have sophisticated justifications for their being there that seem plausibly true to me. More broadly, if this was not a painfully difficult question, the community wouldn't have split over it, and there are enough rationalists and EAs in the labs that it seems obvious to me that the community has split.

So I don't really get my friends to try to quit their jobs, as much as try to help them regularly orient to whether or not they should — and they generally humour me by treating this as a live question. I let them know frequently that I have reservations about their work, but also that I generally trust their judgement and care about them as people.

Here are my intuitions for what my friends are experiencing, while they are working at a frontier lab:

They value their job. Their job is giving them silly amounts of money and validation and prestige. Their job lets them work on a really important project with really smart people in comfortable settings. Their job gives them purpose. They believe that they are doing the right thing, and that they can more or less trust the leadership, or that even if they can't trust the leadership fully, every other player is less trustworthy.

They likely know that some people disagree with their choice to work at a frontier AI company and want them to feel shame about it. They actually know much more about the really cursed tradeoffs and clusterfucks that are happening inside their company than I do. But they will take pains to hide this ammunition from me, an outsider with reservations about their work — even if they haven't signed NDAs or have permissive ones, they will feel protective of the company, all the brilliant wonderful people that they know as colleagues, and their hopes for the work itself and the future that it might bring into being.

My friends are often scared, stressed, sleep-deprived, and in what I perceive to be an actively adversarial information environment. But I am also scared, stressed, and in an actively adversarial information environment (according to them). For my part I do try to prioritize good sleep hygiene.

There are people, including some that they look up to intellectually, who are in the process of repeatedly and loudly expressing disapproval at their choice to work at a frontier AI lab. Oliver Habryka and Eliezer Yudkowsky yells at respond disagreeably to their compatriots on X every time they choose to voice a pro-lab take. Ben Pace (disclosure: long Pace1) might be messaging them deranged stuff on Signal like "Good seeing you at the house party. By the way I think if we are living in a sane world you would be sentenced to ten thousand years in prison." High decouplers, man.

I think it's good that the deeprats (shhhh just go with it) do this. There should be some voices of general disapproval, and there is much value in saying socially aggressive and unpleasant true things. And they sure do have the right temperament for doing that job, unlike me.

The failure mode I see in this strategy is that they might make the lab employees tense and guarded and defensive around them, and nudge them that much further into an echo chamber comprised only of their sympathetic colleagues inside the frontier labs, all with their own well reasoned and sophisticated justifications for staying. Would it be too galaxy brained to claim that this might make them less likely to quit a harmful job?

For my part, as I know about the existence of the deeprats and their general MO, I think it is reasonable for me to try a different tack. The risk of this tack is that not only will I feel like an enabler, I might actually be an enabler. In this way, I might be making them less likely to quit a plausibly harmful job.

I mean, the deal is, I can't make people quit their job. And maybe it doesn't make sense for them to quit their job — like I said, I am deeply unsure. But I think my friends appreciate the ear of an unsettled third party, who has reservations about their work but is open to hearing them out about their beliefs. I think it's bad for people who are conflicted, as my friends tend to be, to only be able to talk to people who have already chosen sides. I think it makes it harder for them to figure out what they believe in, and whether or not their beliefs are true.

I have never gotten a friend to quit working at a frontier AI lab, though one of them quit of their own volition after one year, to resume their work on technical alignment.

It is fortunate that I am actually uncertain about whether or not it is reasonable for people who care about x-risk to work inside frontier labs, because I think it would be somewhat dishonest to pretend that you are uncertain for the purposes of "helping" your friend "orient". How convenient for me.






  1. ...also, I've smooched him approximately fifty times today.↩

#ai #blog #longform