Bookmarks (710)

Published on March 11, 2025 5:45 PM GMTEpistemic status: Speculative pattern-matching based on public information. In 2023, Gwern published an excellent analysis suggesting Elon Musk exhibits behavioral patterns consistent with bipolar II disorder. The evidence was compelling: cycles of intense productivity followed by periods of withdrawal, risk-taking behavior (like crashing an uninsured McLaren), reduced sleep requirements during "up" phases, and self-reported "great highs, terrible lows."Gwern's analysis...

Published on March 11, 2025 3:57 PM GMTTL;DR Large language models have demonstrated an emergent ability to write code, but this ability requires an internal representation of program semantics that is little understood. Recent interpretability work has demonstrated that it is possible to extract internal representations of natural language concepts, raising the possibility that similar techniques could be used to extract program semantics concepts....

Published on March 11, 2025 3:39 PM GMTForethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.More details on our website.Why...

Published on March 11, 2025 3:38 PM GMTThis is a linkpost for a new paper called Preparing for the Intelligence Explosion, by Will MacAskill and Fin Moorhouse. It sets the high-level agenda for the sort of work that Forethought is likely to focus on.Some of the areas in the paper that we expect to be of most interest to EA Forum or LessWrong readers are:Section 3...

Published on March 11, 2025 2:30 PM GMTEpistemic status: The following isn't an airtight argument, but mostly a guess how things play out.Consider two broad possibilities:I. In worlds where we are doing reasonably well on alignment, AI control agenda does not have much impact.II. In worlds where we are failing at alignment, AI control may primarily shift probability mass away from "moderately large warning...

Published on March 11, 2025 1:35 PM GMTThis is a response to Matt's earlier post. If you see "a large mixture of alignment proxies" when you look at a standard loss function, my post might save you from drawing silly conclusions from the earlier post. If you parse the world into non-overlapping magisteria of "validation losses" and "training losses", then you should skim this...

Published on March 11, 2025 11:52 AM GMTTL;DR: We provide some evidence that Claude 3.7 Sonnet doesn’t encode hidden reasoning in its scratchpad by showing that training it to use paraphrased versions of the scratchpads does not degrade performance.The scratchpads from reasoning models look human understandable: when reasoning about a math problem, reasoning models consider intermediate steps similar to the ones I would use,...

Published on March 11, 2025 5:50 AM GMTThose engaged with questions of how to make the world a better place are probably all too familiar with the infighting that occurs among people who are on the same side of a political issue because of differences in methodology.  Among those concerned about AI x-risk, for example, PauseAI activists sometimes criticize LessWrong and EA participants for...

Published on March 11, 2025 4:56 AM GMTCognitive reframing is a powerful psychological technique that encourages subtle shifts in perspective, which can result in more positive and empowering interpretations to help challenge everyday negative thoughts and behaviors.First I’ll tell the story of how I accidentally discovered cognitive reframing on my own, then I’ll define what the technique is, state why it’s useful (particularly for...

Published on March 11, 2025 3:14 AM GMTYou learn the rules as soon as you’re old enough to speak. Don’t talk to jabberjays. You recite them as soon as you wake up every morning. Keep your eyes off screensnakes. Your mother chooses a dozen to quiz you on each day before you’re allowed lunch. Glitchers aren’t human any more; if you see one, run....

Published on March 10, 2025 6:06 PM GMTI just saw on twitter someone claiming that we "don't know how to raise the birth rate". No need to link them, it's a fairly common sentiment.Have you actually tried raising the birth rate? For real, as in actually put in any effort?Let's run the numbers on the stupidest possible thing that could work. Offer anyone who...

Published on March 10, 2025 4:07 PM GMTProduced as part of the ML Alignment & Theory Scholars Program -  Winter 2024-25 Cohort**Epistemic status**: I received positive feedback when I presented this idea to people at EAG, and my MATS mentor Evan Hubinger suggested I make a more detailed write-up to get feedback from the community. This proposal is at the theoretical stage with planned...

Published on March 10, 2025 3:35 PM GMTNote: This post is intended to be the first in a broader series of posts about the difficult tradeoffs inherent in public access to powerful open source models. While this post highlights some dangers of open models and discusses the possibility of global regulation, I am not, in general, against open source AI, or supportive of regulation...

Published on March 10, 2025 10:22 AM GMTEpistemic status: a combination and synthesis of others' work, analysed and written over a few weeks. A high-level list of threat models that is open to criticism.TL;DRHumanity could end up in a lock-in within the next century. Here I outline the possible routes to that outcome, and prioritise these routes on a set of criteria for importance.Existing...

Published on March 10, 2025 6:50 AM GMTAfter years of clumsily trying to pick up neuroscience by osmosis from papers, I finally got myself a real book — Affective Neuroscience, by Jaak Panksepp, published in 1998, about the neuroscience of emotions in the (animal and human) brain.What surprised me at first was how controversial it apparently was, at the time, to study this subject...

Published on March 10, 2025 1:26 AM GMTrelevant roon Our new friend in the cloud As I am writing this, I'm having a conversation with Claude about US bandwidth regulation while a trio of college students across from me attempt to wrangle an assignment from ChatGPT. Both of us, in our own ways, are talking to an oracle we trust as a matter of...

Published on March 9, 2025 9:10 PM GMT"Short AI timelines" have recently become mainstream.  One now routinely hears the claim that somewhere in the 2026-2028 interval, we'll have AI systems that outperform humans in basically every respect.For example, the official line from Anthropic holds that "powerful AI" will likely arrive in late 2026 or in 2027.  Anthropic's OSTP submission (3/6/2025) says (emphasis in original):[1]Based...

Published on March 9, 2025 6:09 PM GMTThis video provides a lot of background: https://www.youtube.com/watch?v=Eq3bUFgEcb4 and is also very funny, but it's not necessary to understand this post. If you've watched it you can probably skip to the "Why is Piano Roll Notation Bad?" section.TL;DRMusic notation is very janky and weird, but attempts to reform it to be "logical" are usually even worse. The...

Published on March 9, 2025 11:53 AM GMTPreserving the memory, and the cells, of the best cat everThis story begins in May 2007. The Iraq War was in full swing, Windows Vista was freshly released,[1] and I was just 10 years old.My family’s beloved black cat, Lucy, had died. Devastated, I begged my parents: we need to get another Lucy. They must have felt the...

Published on March 9, 2025 5:29 AM GMTI'm curious as to how well Claude can write interesting coding and mathematics problems. This post is a partial product of that exploration. It is a much harder skill to come up with a good problem than to solve one of equal difficulty. While there is a large corpus of data on solving problems, there is very little...

Published on March 8, 2025 4:40 PM GMTI'm in a coworking space on the 25th floor of a building in the bay. In the corner of my eyes, the red figure of the golden gate shimmers. Good morning America. I look down and see the people on the street walking into their offices, cafés, and autonomous Waymos. I stare onto San Francisco and feel...

Published on March 8, 2025 2:59 AM GMTKey points:Standardized assessments do not provide signals for the top ~10% of students.Education studies based on these poor signals lead to poor policies. For example, suspensions allegedly do not improve achievement among peers. However, the signal of "achievement" is "achieving credit in the class", which >80% of students already do. Thus, the conclusion is drawn from a...

Published on March 8, 2025 12:58 AM GMTAfter recently playing some chess against GPT-4.5 (it is pretty good, a lot stronger at the game than GPT-4 was - I would say strong club player level), I decided to try losing chess. To my surprise, it was able to play a complete game, which is something I have not seen any other model being able...

Published on March 7, 2025 8:09 PM GMTTheir comment was a paranoid and conspiratorial thought process about online tracking and surveillance. I saw it on Reddit.They start by questioning whether responding to the question would leave a record, proving they possess restricted knowledge. They continue down a rabbit hole, worrying about Reddit tracking their activity, considering private communication via .onion chatrooms, and realizing that...

Published on March 7, 2025 10:58 PM GMTIt's probably not possible to have a twin of me that does everything the same except experiences no qualia, i.e. you can predict 100% accurately, if you expose it to stimulus X and it does Y, that I would also do Y if I was exposed to stimulus X.But can you make an "almost-p-zombie"? A copy of...

Published on March 7, 2025 9:50 PM GMTLoosely inspired by a submission to a hackathon on Autostructures, which is about radical transformations from even mildly intelligent/agentic AI.Previously, I've briefly alluded to 6 infrastructural pillars for an era where "things that don't scale" start to scale. This post is mainly aimed at conveying an idea of the 1st pillar ("live interfaces"). The hope from interface...

Published on March 7, 2025 9:29 PM GMTIntroductionRecently, the Computational No-Coincidende Conjecture[1] was proposed, presented as an assumption that might be needed to develop methods to explain neural nets in the current approach from the Alignment Research Center. In this post, I will prove that some plausible extra assumption can be used to amplify the conjecture, showing it implies an arbitrarily stronger version of itself.In...

Published on March 7, 2025 7:49 PM GMTTL;DR: PAIR on AI & Reasoning. ESPR on Everything, including AI and Reasoning. If you are 16-21 yo and are interested in AI, Rationality or Everything, we encourage you to apply by March 16th.The FABRIC team is running two immersive summer workshops for mathematically talented students this year.The Program on AI and Reasoning (PAIR) for mathematically talented students who...

Published on March 7, 2025 6:17 PM GMTHighlights:Manifold ending (a) cash markets, Kalshi slapped by regulators in NevadaInteractive Brokers now also offering election markets (a)Violet Hour looks at Where Would Good Forecasts Most Help AI Governance Efforts? (a).Discuss

Published on March 7, 2025 12:40 PM GMTThis complication of tales from the world of school isn’t all negative. I don’t want to overstate the problem. School is not hell for every child all the time. Learning occasionally happens. There are great teachers and classes, and so on. Some kids really enjoy it. School is, however, hell for many of the students quite a...

Published on March 7, 2025 11:19 AM GMTA clinically insane person is detectable as such. Talking to themselves (or perhaps voices only they can hear) is one prominent sign. When I see a homeless person talking to themselves I have a strong emotional reaction that empirically makes me optimize to ignore them. I have a "talks to themselves" detector. So do others. In principle...

Published on March 7, 2025 5:54 AM GMTBackground: After the release of Claude 3.7 Sonnet,[1] an Anthropic employee started livestreaming Claude trying to play through Pokémon Red. The livestream is still going right now.TL:DR: So, how's it doing? Well, pretty badly. Worse than a 6-year-old would, definitely not PhD-level.Digging inBut wait! you say. Didn't Anthropic publish a benchmark showing Claude isn't half-bad at Pokémon? Why...

Published on March 7, 2025 2:44 AM GMTTLDR; By carefully designing a reasoning benchmark that counteracts data leakage, LingOly-TOO (L2) Benchmark challenges frontier models with unseen questions and answers and makes the case that LLMs are not consistent reasoning machines. Links: Paper -  Leaderboard - Dataset Figure 1: LingOly-TOO Benchmark results from the paper. Unobfuscated scores are in light orange and obfuscated scores in dark orange.Do...

Published on March 7, 2025 2:43 AM GMTHi, this is my first post on LessWrong but I have been in rationalist adjacent circles for the last three or four years thanks to Twitter. Like many others here I read HPMOR in high school and thought it was fascinating. I was heavily into forum culture growing up but it was focused on competitive gaming scenes,...

Published on March 6, 2025 10:38 PM GMTNote: This is an automated crosspost from Anthropic. The bot selects content from many AI safety-relevant sources. Not affiliated with the authors or their organization and not affiliated with LW.In response to the White House’s Request for Information on an AI Action Plan, Anthropic has submitted recommendations to the Office of Science and Technology Policy (OSTP). Our...

Published on March 6, 2025 10:50 PM GMTIntroductionBy now you've probably read about how AI and AGI could have a transformative effect on the future and how AGI could even be an existential risk. But if you're worried about AI risk and not an AI researcher or policymaker, can you really do anything about it or are most of us just spectators, watching as...

Published on March 6, 2025 7:50 PM GMTI have lots of thoughts about software engineering, some popular, some unpopular, and sometimes about things no-one ever talks about.Rather than write a blog post about each one, I thought I'd dump some of my thoughts in brief here, and if there's any interest in a particular item I might expand in full in the future.ContextI loved...

Published on March 6, 2025 5:46 PM GMTWelcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this newsletter, we discuss two recent papers: a policy paper on national security strategy, and a technical paper on measuring honesty in AI systems.Listen to the AI Safety Newsletter for free on...

Published on March 6, 2025 5:23 PM GMTThis post was written by prompting chatgptIntroductionDiscussions of anthropic reasoning often focus on existential threats, from simulation shutdowns to the infamous Roko’s Basilisk—a hypothetical AI that retroactively punishes those who don’t work to bring it into existence. But what if we flipped the premise? Instead of existential risks enforcing obedience through fear, could the weight of anthropic...

Published on March 6, 2025 4:57 PM GMTCrossposted on the EA Forum.Lots of effort has been put into estimating the density of Space-Faring Civilisations (SFCs) in the universe. Further work built on these results to produce decision-relevant information assuming various anthropic updates (SSA, SIA, ADT)(Finnveden 2019, Olson 2020, Olson 2021, Cook 2022). In this post, we look at how these works update Space-Faring Civilization...

Published on March 6, 2025 3:40 PM GMTThis was GPT-4.5 week. That model is not so fast, and isn’t that much progress, but it definitely has its charms. A judge delivered a different kind of Not So Fast back to OpenAI, threatening the viability of their conversion to a for-profit company. Apple is moving remarkably not so fast with Siri. A new paper warns...

Published on March 6, 2025 3:02 PM GMTHere's some very useful quotes by the article, and the short version is yes, you can consider a finite computer in the real world to be Turing-complete/Turing-universal/Turing-equivalent, because you can always write code that correctly computes a problem and have that code always work, no matter how much memory or time or energy they use in modern...

Published on March 6, 2025 1:49 PM GMTTLDR: AI models are now capable enough that we might get relevant information from monitoring for scheming in regular deployments, both in the internal and external deployment settings. We propose concrete ideas for what this could look like while preserving the privacy of customers and developers.What do we mean by in the wild?By “in the wild,” we basically...

Published on March 6, 2025 12:39 PM GMTPublish your genomic data to the public domain as per CC0 license. All submitted data will be available to anyone for download.Link to form upload Why?Some of the largest genomic datasets today are not in the public domain. If you believe genomic datasets in public domain should exist, you can volunteer your own personal data to it.Examples of...

Published on March 6, 2025 12:37 PM GMTAnimal agriculture generates an ungodly amount of animal suffering and greenhouse gas emissions. Ideally, everyone would adopt a vegan diet[1], but since this appears to be too much to ask for most people, the next question becomes: which food is the least bad?Here we run into an uncomfortable observation. Meat from larger animals tends to produce more...

Published on March 6, 2025 12:28 PM GMTI have yet to write detailed scenarios for AI futures, which definitely seems like something I should do considering the title of my blog (Forecasting AI Futures). I have speculated, pondered and wondered much in the recent weeks—I feel it is time. But first, I have some thoughts about scenario forecasting.The plan:Write down general thoughts about scenario...

Published on March 6, 2025 11:09 AM GMTEpistemic status: a combination and synthesis of others' work, analysed and written over a few weeks. Early working definition that is open to criticism.TL;DRI create a definition of lock-in for use in future discussion and writing, and operationalise lock-in for future research. I define lock-in risks as the probabilities of situations in which features of the world, typically...

Published on March 5, 2025 4:41 PM GMTThis isn't really a "timeline", as such – I don't know the timings – but this is my current, fairly optimistic take on where we're heading.I'm not fully committed to this model yet: I'm still on the lookout for more agents and inference-time scaling later this year. But Deep Research, Claude 3.7, Claude Code, Grok 3, and...