Part of the EA Melbourne Hackathon, March 2023 # To do - [ ] Continue to write up Custodian Initiative - [ ] Think about base rates - how likely this is to matter - [ ] Think about the SPC (MacAskill et al) framework - how valuable is this - [ ] Continue to trawl the EA Forum for any related content # Research EA Forum search terms: - Custodian - Revisit - To try: - Changing minds - Someone / something was wrong Key orgs / initiatives: - 80k doesn't have anything much more explicitly retrospective or ruling things 'out', but might be doing research behind the scenes that doesn't get published as it is indeed a 'no action' result - Rethink Priorities obviously has a backlog of research, but nothing quite like this proposal - Open Philanthropy - Some, but incidental (e.g. https://www.openphilanthropy.org/research/tobacco-control/) - There's some things that look backwards (e.g. the History of Philanthropy work) but nothing that quite seems the same - Centre for Effective Altruism has gotten out of this space - Copenhagen Consensus is another one 80k links to, but again no dice - Global Priorities Institute - no luck - Future of Humanity Institute - no luck - Forethought Foundation - no luck - EA Forum - By its nature more ad hoc - Red teaming / Change Our Mind contests also don't seem to foster this systematically. - ❓ Non-EA - e.g. Gates or Bloomberg Foundations? Could these be relied on? - If so, theres still probably room for more of an aggregator? - How do governments generally do this - feels like they don't do it very well maybe? - Also reliant on that push factor - having advocates/lobbyists who are aware of an update come and push that update Tangentially related orgs: - Founders Pledge ❌ - Happier Lives Institute ❌ - Global Catastrophic Risk Institute ❌ - Center on Long-Term Risk ❌ - ❓ The Center for High Impact Philanthropy - Campbell Collaboration ❌ - Behavioural Insights, JPAL, Pew... ❌ Looks like most places that either produce "first-order research" or are specifically aligned to cause areas will be producing new best-practice content regularly, such that they may not specifically need to revisit old research or will do so naturally. Any *particular* literature reviews or meta-analyses are probably also not that relevant to this question; high-quality ones will no doubt catch the sorts of gaps we're concerned about here. But the question is - how do we make sure we're *doing* these surveys at the right cadence? # Examples - https://forum.effectivealtruism.org/posts/RGwJviLRnrSojwYwu/opportunities-that-surprised-us-during-our-clearer-thinking#5__We_hadn_t_considered_that_there_may_be_significant_new_ideas_about_how_to_reduce_the_chance_of_nuclear_war__given_the_age_of_the_field__ - _Almost_ related: https://forum.effectivealtruism.org/posts/RGwJviLRnrSojwYwu/opportunities-that-surprised-us-during-our-clearer-thinking#7__We_hadn_t_realized_that_1Day_Sooner_conducts_activities_outside_of_their_human_challenge_trial_work__and_that_these_activities_have_significant_room_for_funding__ - adjacent due to the fact that it's an "explored" thing with a surprising additional thing # UVP - The Custodian Initiative addresses a particular failure mode that I haven't seen addressed elswhere. It's one thing to make a moral mistake - and people are conducting plenty of research on this - but another to get something morally right *at the time*, and fail to notice a discovery that changes that calculus. - It takes its name from the 4X strategy game *Stellaris*, which implemented a Custodian Initiative to steward old content - not just fixing bugs, but actively supporting and enriching existing content, while the rest of the team pushes ahead with new content, expansions and so on. - This connects with the concept of "looking for your keys under the streetlight". - Another analogy is with investing / financial advice: confirmation that you're on the right track is also important (what are the trades one *didn't* make?) - It's probably really a sub-field of global priorities research - but looking backwards, rather than forwards. - Another link is to the replication crisis # Similar ideas - Priorities research - Various red team initiatives - Changes in cause areas: - EA Survey - 2020 survey results @ https://forum.effectivealtruism.org/posts/83tEL2sHDTiWR6nwo/ea-survey-2020-cause-prioritization - David Moss estimated 2022 results circa late March 2023 (comment on the announcement post dated 6 March) - Changes in interventions: - https://forum.effectivealtruism.org/posts/RcSf7eGKF9E8XNAra/a-major-update-in-our-assessment-of-water-quality-3 - Similar attempts: - https://forum.effectivealtruism.org/posts/BFvtrhaea85CDYhBQ/ea-has-gotten-it-very-wrong-on-climate-change-a-canadian - Individual people changing their mind on particular things - but maybe more "under the streetlight": - https://forum.effectivealtruism.org/posts/wc2LwxagsgwmoXnrr/revisiting-why-global-poverty - Doing EA Better - Notably the reforms: - EA should set up a counter foundation that has as its main goal critical reporting, investigative journalism and “counter research” about EA and other philanthropic institutions - Overall related via 'better epistemics' but there's not much focus on any *time* component in the post. # Random thoughts - Is this specifically a long-term issue more than anything? It feels like maybe near-term work isn't as vulnerable to this for some reason? More of a coherent field with better-aligned academic disciplines...? # ITN ## Importance - Might also play into the recent discussion of epistemics; this proposal effectively calls for systematic or even institutional red teaming of priorities (though in the 'backward' direction). ❓ What's the impact of a missed update? ## Neglectedness - Particular individuals are well suited to updating the community on issues in particular fields. For instance, I would expect that new discoveries in AI safety or moral uncertainty that impact their ITN assessment will be disseminated pretty quickly, as there are people actively looking ("under the streetlight"). - Similarly, funding bodies have incentives / do / should keep an eye on the effectiveness (i.e. ITN) of a particular cause area or specific intervention.[^1] - However, these are cases where people have 'dropped their keys', and are looking for them 'under the streetlight'. There may be particular issues or fields that no one is keeping an eye on, or not enough people are keeping an eye on, or no one with enough social capital is keeping an eye on. - I think a good candidate for this is climate change. I found Will MacAskill's analysis in *What We Owe the Future* compelling: it's an area that is important and seems to be tractable in some ways and less tractable in others, but not really neglected, and not as neglected as other cause areas. Whenever I discuss this with friends or colleagues who are less inclined to this way of thinking, the stance I generally end up taking is that 'yes it's important, but the next dollar is probably better spent elsewhere; *I'm not explicitly advocating for decreasing funding*'. - Now, it may very well be that actually the current amount of spending does not represent an optimal marginal rate of substitution between issues - maybe governments *ought* to spend less on climate change, and more on something else. - However, my claim is about the minimum, and about my placatory corollary. I don't think EA writ large should spend much time or effort on climate change, *given how things stand today*. I could well imagine that this could change on any of the ITN dimensions, such that it does become sensible for EA to engage. It could be that we discover some fact that makes it look more important (e.g. some cascade-flavoured theory is shown to be much more likely); much more neglected (e.g. coordination actively worsens and governments wind back effort); or much more tractable, in some way that EA is particularly well-positioned to leverage. - The risk I propose is that because EA generally prioritises, and suggests that interested individuals prioritise, other areas. This means there are areas where we are not cultivating expertise, and are therefore reliant on other sources for updates. - For some issues, this may be sufficient - meaningful shifts in climate change, e.g. in technological solutions, seem likely to be heavily publicised. - The key questions I see are: - Which areas present this risk? - What is the risk actually presented? - What is the right base case, and what is its base rate? - Neglectedness may also go down if we organise around a different principle, such as bringing in or focusing more on existing academic fields etc. - more in line with the Doing EA Better paradigm. ## Tractability - 'Pull' methods - ❓ Build this into GPR: a "good" paper should define minimal conditions that call for revisiting - Taking this to zany town: smart contracts might be useful? - 'Push' methods - ❓ Fund custodians (e.g. some number of GPI researchers) to rotate through key areas on a regular basis - could be a lower barrier to entry, as it likely looks like literature reviews, rather than cutting-edge research # Candidate areas - Climate change - Nanotechnology? That one's fallen out of favour - More classical areas like homelessness and criminal justice - these have come up in my EA social circles recently - # Footnotes [^1]: https://forum.effectivealtruism.org/posts/RD9ztxR4y6jYaPbT7/room-for-more-funding-why-doesn-t-the-gates-foundation-just - details of the Gates Foundation and how they move their funding around # "Rediscovered Research" Prompting ChatGPT (Mar 14) on a variety of prompts, such as "What are some examples where people have abandoned a particular line of research, only to come back to it later and make important discoveries?" These are of varying relevance and probably need to be vetted. There's a bit of 1. Gregor Mendel and the discovery of genetics: In the 1860s, Gregor Mendel conducted experiments on pea plants to study heredity. Despite his groundbreaking findings, his work was largely ignored by the scientific community until the early 20th century, when it was rediscovered and recognized as the foundation of modern genetics. 2. Barry Marshall and the discovery of H. pylori: In the 1980s, Barry Marshall became interested in the role of bacteria in causing stomach ulcers. Despite facing skepticism from the medical community, Marshall conducted experiments that showed the bacterium Helicobacter pylori was responsible for many cases of ulcers. Marshall was awarded the Nobel Prize in Physiology or Medicine in 2005 for his work. 3. Lisa Meitner and the discovery of nuclear fission: In the 1930s, Lisa Meitner and her colleague Otto Hahn conducted experiments on nuclear reactions. Although they discovered nuclear fission, Hahn was initially credited with the discovery while Meitner's contributions were largely overlooked. It wasn't until later that Meitner's work was recognized and she was awarded numerous honors for her contributions to the field. 4. Jonas Salk and the development of the polio vaccine: In the 1940s, Jonas Salk became interested in developing a vaccine for polio, a disease that had devastated the world for centuries. Despite facing numerous setbacks and failures, Salk persisted in his research and eventually developed the first successful polio vaccine in 1955. 5. Jean Piaget and the development of cognitive psychology: In the early 20th century, Jean Piaget became interested in the way children learn and develop. He conducted numerous experiments and observations to develop his theory of cognitive development. However, his work was initially met with skepticism from other psychologists, and he turned his attention to other topics. It wasn't until the 1960s that his work was widely recognized and became a foundational theory in cognitive psychology. 6. John Bowlby and attachment theory: In the 1950s, John Bowlby began studying the bond between infants and their primary caregivers. He developed his theory of attachment, which posits that infants develop an emotional bond with their caregiver that is crucial for their social and emotional development. Although his work was initially dismissed by many psychologists, it has since become a widely accepted theory in developmental psychology. 7. Carol Dweck and growth mindset: In the 1980s, Carol Dweck became interested in the way people approach learning and achievement. She developed the concept of growth mindset, which posits that individuals who believe that their abilities can be developed through effort and persistence are more likely to achieve success than those who believe their abilities are fixed. Although her work was initially met with skepticism, it has since become an important concept in educational psychology and has been widely applied in educational settings. 8. Stanley Milgram and obedience to authority: In the 1960s, Stanley Milgram conducted a series of controversial experiments on obedience to authority. Although his work was initially criticized for its ethics, it has since become a widely studied and debated topic in social psychology, with implications for understanding phenomena such as conformity, group behavior, and social influence. 9. Harry Harlow and maternal deprivation: In the 1950s and 1960s, Harry Harlow conducted a series of experiments on infant rhesus monkeys to study the effects of maternal deprivation. His work showed that social and emotional development were as important as physical needs, and that lack of social and emotional support can have long-lasting negative effects on development. Although his work was initially controversial and faced criticism, it has since become an important area of study in developmental psychology. 10. Aaron Beck and cognitive therapy: In the 1960s, Aaron Beck developed cognitive therapy as a new approach to psychotherapy. His work emphasized the role of thoughts and beliefs in the development and maintenance of psychological problems. Although his ideas were initially met with skepticism from other psychologists, cognitive therapy has since become one of the most widely used approaches to psychotherapy. 11. Jane Goodall and chimpanzee behavior: In the 1960s, Jane Goodall began studying chimpanzee behavior in the wild. Her observations challenged the prevailing view at the time that humans were unique in their capacity for tool use, communication, and social behavior. Her work has since become an important area of study in primatology and has led to a greater understanding of the evolutionary roots of human behavior. 12. Robert Sternberg and intelligence: In the 1980s, Robert Sternberg proposed a new theory of intelligence, which he called "triarchic theory." His work emphasized the importance of practical intelligence, creative intelligence, and analytical intelligence, and challenged the traditional view of intelligence as a single, unitary construct. Although his theory was initially met with skepticism, it has since become an important area of study in cognitive psychology and has led to a greater understanding of the different facets of human intelligence. 13. Ignaz Semmelweis and hand washing: In the mid-19th century, Ignaz Semmelweis, a Hungarian physician, discovered that hand washing could significantly reduce the incidence of infections in hospitals. Despite his findings, his ideas were initially met with skepticism and resistance from the medical establishment, and it took several decades for his ideas to be widely accepted and implemented. This delay likely contributed to the continued high incidence of hospital-acquired infections during this time period. 14. Rosalind Franklin and DNA structure: In the early 1950s, Rosalind Franklin conducted x-ray crystallography experiments that provided crucial insights into the structure of DNA. However, her contributions were not fully recognized at the time, and it was only after the discovery of the double helix structure by James Watson and Francis Crick that her work was fully appreciated. This delay in recognition likely delayed progress in understanding the structure and function of DNA. 15. Gregor Mendel and genetics: In the mid-19th century, Gregor Mendel, an Austrian monk, conducted experiments on pea plants that laid the foundation for the field of genetics. However, his work was largely ignored by the scientific community at the time, and it was only several decades later that his contributions were recognized and widely adopted. This delay likely slowed progress in understanding the mechanisms of inheritance and the role of genetics in biology. 16. The discovery of deep-sea hydrothermal vents: In the 1970s, scientists exploring the deep sea discovered hydrothermal vents, which were home to previously unknown communities of organisms that could thrive in extreme conditions. This discovery had implications for the study of evolution, ecology, and astrobiology. For example, the study of these organisms and their adaptations to extreme environments has provided insights into the origins of life and the potential for life beyond Earth. 17. The influence of physics on biology: Advances in physics have had a major influence on the development of biology in recent decades. For example, the use of techniques such as X-ray crystallography and nuclear magnetic resonance imaging has allowed biologists to study the structure and function of proteins and other biomolecules at a level of detail previously impossible. Similarly, the development of mathematical models and simulations has allowed biologists to explore complex biological systems and make predictions about their behavior. 18. The impact of computer science on neuroscience: Advances in computer science have revolutionized the field of neuroscience in recent decades. For example, the development of high-performance computing and machine learning techniques has allowed neuroscientists to analyze vast amounts of data and make predictions about brain function. Similarly, the development of brain-computer interfaces has allowed researchers to study and manipulate neural activity in real-time. 19. The discovery of the ozone hole: In the 1970s and 1980s, scientists studying atmospheric chemistry discovered that certain chemicals, particularly chlorofluorocarbons (CFCs), were damaging the Earth's ozone layer. However, this discovery was initially met with skepticism and resistance from some scientists and industries, who argued that the evidence was not sufficient to justify regulatory action. It was only after further research and political pressure that the Montreal Protocol was signed in 1987, banning the production of CFCs. This discovery had important implications for the field of climate science, as it highlighted the potential for human activities to have a significant impact on the Earth's atmosphere. 20. The role of gut microbiota in health and disease: In recent years, researchers have made significant progress in understanding the role of the gut microbiota in human health and disease. However, this field was largely ignored by medical researchers for many years, who focused primarily on the role of genetics and environmental factors in disease. It was only in the past few decades that researchers began to appreciate the importance of the microbiota, and its potential role in conditions such as obesity, diabetes, and inflammatory bowel disease. 21. The impact of biodiversity on ecosystem function: Ecologists have long recognized the importance of biodiversity in maintaining healthy ecosystems. However, this field was largely ignored by policymakers and other stakeholders for many years, who focused primarily on economic development and resource extraction. It was only in the past few decades that the importance of biodiversity was fully appreciated, and efforts were made to protect and conserve ecosystems and the services they provide.