Alignment Research Center

The Alignment Research Center (ARC) is a nonprofit that does theoretical research on aligning machine-learning systems with human interests. It was founded by Paul Christiano, who had led alignment research at OpenAI from 2017 to 2021.¹ Open Philanthropy described it in 2022 as focused on "developing strategies for AI alignment that can be adopted by industry today and scaled to future machine learning systems."² Open Philanthropy's 2017 grant page for its 30-million-dollar grant to OpenAI had disclosed that Christiano and Dario Amodei were "technical advisors to Open Philanthropy" and lived "in the same house as Holden," meaning the grantmaker's chief executive, Holden Karnofsky.³

Filings

The organization is registered with the Internal Revenue Service as Alignment Research Center, EIN 86-3605182, a 501(c)(3) whose exemption was ruled on March 1, 2022. It files from 440 North Barranca Avenue, box 9807, in Covina, California.⁴ METR, the evaluations group ARC incubated, files from box 3345 at the same Covina street address.⁵

ARC reported revenue of 475,650 dollars and expenses of 153,352 dollars for 2021, ending the year with assets of 323,598 dollars. For 2022 it reported revenue of 4,466,022 dollars, expenses of 1,425,170 dollars and assets of 4,716,204 dollars, and paid its officers nothing. For 2023 revenue rose to 10,043,597 dollars and expenses to 6,272,135 dollars, with year-end assets of 7,411,763 dollars and compensation of current officers of 376,083 dollars.⁴

Funding

Open Philanthropy recommended 265,000 dollars to ARC for general support in March 2022, filed under its focus area "Navigating Transformative AI," and 1,250,000 dollars over two years in November 2022, which the grant page said "follows our March 2022 support."²⁶ No later Open Philanthropy grant page for ARC has been found.

Earlier in 2022 ARC also received 1.25 million dollars from the FTX Foundation. On November 15, 2022, amid the collapse of Sam Bankman-Fried's exchange FTX, ARC wrote: "We now believe that this money morally (if not legally) belongs to FTX customers or creditors, so we intend to return $1.25M to them." It added that the decision was "particularly straightforward" for ARC "because we haven't spent most of the money and have other supporters happy to fill our funding gap." A 2024 update to the same page stated that "ARC has now returned this grant (less legal and administrative expenses) to the FTX bankruptcy estate."⁷

Governance

ARC's team page, published April 30, 2024, lists Christiano, Jacob Hilton, Victor Lecomte, Wilson Wu and Mike Winer as full-time researchers and Harshita Khera in operations. Christiano is executive director and president; the board consists of Christiano, Buck Shlegeris and Ben Hoskin; Kyle Scott is treasurer and Khera secretary.⁸ Shlegeris is chief executive of Redwood Research, whose board Christiano sat on from its founding in 2021 until June 2023, alongside Karnofsky.⁹ From September 2023 to April 2024 Christiano was also one of the five original trustees of Anthropic's Long-Term Benefit Trust, leaving it to become head of AI safety at the U.S. AI Safety Institute.¹⁰

ARC Evals and METR

In 2022 ARC hired Beth Barnes to build an evaluations team, ARC Evals. Barnes wrote in September 2023 that Christiano "remained focused on theory, but spent a significant minority of his time overseeing and advising on evaluations work," and that the team had completed evaluations of OpenAI's GPT-4 and Anthropic's Claude in partnership with the two companies.¹¹ On December 4, 2023, the team announced that it was "wrapping up our incubation period at ARC" and becoming a standalone 501(c)(3) named METR, with Barnes continuing to lead it and ARC Evals staff transferring to the new entity. The announcement added: "While we had initially planned for Paul Christiano to be a board member and advisor, he declined involvement in light of his new role as Head of Safety at the US AI Safety Institute."¹²