~www_lesswrong_com | Bookmarks (710)

Published on March 11, 2025 5:45 PM GMTEpistemic status: Speculative pattern-matching based on public information. In 2023, Gwern published an excellent analysis suggesting Elon Musk exhibits behavioral patterns consistent with bipolar II disorder. The evidence was compelling: cycles of intense productivity followed by periods of withdrawal, risk-taking behavior (like crashing an uninsured McLaren), reduced sleep requirements during "up" phases, and self-reported "great highs, terrible lows."Gwern's analysis...

Published on March 11, 2025 3:57 PM GMTTL;DR Large language models have demonstrated an emergent ability to write code, but this ability requires an internal representation of program semantics that is little understood. Recent interpretability work has demonstrated that it is possible to extract internal representations of natural language concepts, raising the possibility that similar techniques could be used to extract program semantics concepts....

Published on March 11, 2025 3:39 PM GMTForethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.More details on our website.Why...

Published on March 11, 2025 3:38 PM GMTThis is a linkpost for a new paper called Preparing for the Intelligence Explosion, by Will MacAskill and Fin Moorhouse. It sets the high-level agenda for the sort of work that Forethought is likely to focus on.Some of the areas in the paper that we expect to be of most interest to EA Forum or LessWrong readers are:Section 3...

Published on March 11, 2025 2:30 PM GMTEpistemic status: The following isn't an airtight argument, but mostly a guess how things play out.Consider two broad possibilities:I. In worlds where we are doing reasonably well on alignment, AI control agenda does not have much impact.II. In worlds where we are failing at alignment, AI control may primarily shift probability mass away from "moderately large warning...

Published on March 11, 2025 1:35 PM GMTThis is a response to Matt's earlier post. If you see "a large mixture of alignment proxies" when you look at a standard loss function, my post might save you from drawing silly conclusions from the earlier post. If you parse the world into non-overlapping magisteria of "validation losses" and "training losses", then you should skim this...

Published on March 11, 2025 11:52 AM GMTTL;DR: We provide some evidence that Claude 3.7 Sonnet doesn’t encode hidden reasoning in its scratchpad by showing that training it to use paraphrased versions of the scratchpads does not degrade performance.The scratchpads from reasoning models look human understandable: when reasoning about a math problem, reasoning models consider intermediate steps similar to the ones I would use,...

Published on March 11, 2025 5:50 AM GMTThose engaged with questions of how to make the world a better place are probably all too familiar with the infighting that occurs among people who are on the same side of a political issue because of differences in methodology.  Among those concerned about AI x-risk, for example, PauseAI activists sometimes criticize LessWrong and EA participants for...

Published on March 11, 2025 4:56 AM GMTCognitive reframing is a powerful psychological technique that encourages subtle shifts in perspective, which can result in more positive and empowering interpretations to help challenge everyday negative thoughts and behaviors.First I’ll tell the story of how I accidentally discovered cognitive reframing on my own, then I’ll define what the technique is, state why it’s useful (particularly for...

Published on March 11, 2025 3:14 AM GMTYou learn the rules as soon as you’re old enough to speak. Don’t talk to jabberjays. You recite them as soon as you wake up every morning. Keep your eyes off screensnakes. Your mother chooses a dozen to quiz you on each day before you’re allowed lunch. Glitchers aren’t human any more; if you see one, run....

Published on March 10, 2025 6:06 PM GMTI just saw on twitter someone claiming that we "don't know how to raise the birth rate". No need to link them, it's a fairly common sentiment.Have you actually tried raising the birth rate? For real, as in actually put in any effort?Let's run the numbers on the stupidest possible thing that could work. Offer anyone who...

Published on March 10, 2025 4:07 PM GMTProduced as part of the ML Alignment & Theory Scholars Program -  Winter 2024-25 Cohort**Epistemic status**: I received positive feedback when I presented this idea to people at EAG, and my MATS mentor Evan Hubinger suggested I make a more detailed write-up to get feedback from the community. This proposal is at the theoretical stage with planned...

Published on March 10, 2025 3:35 PM GMTNote: This post is intended to be the first in a broader series of posts about the difficult tradeoffs inherent in public access to powerful open source models. While this post highlights some dangers of open models and discusses the possibility of global regulation, I am not, in general, against open source AI, or supportive of regulation...

Published on March 10, 2025 10:22 AM GMTEpistemic status: a combination and synthesis of others' work, analysed and written over a few weeks. A high-level list of threat models that is open to criticism.TL;DRHumanity could end up in a lock-in within the next century. Here I outline the possible routes to that outcome, and prioritise these routes on a set of criteria for importance.Existing...

Published on March 10, 2025 6:50 AM GMTAfter years of clumsily trying to pick up neuroscience by osmosis from papers, I finally got myself a real book — Affective Neuroscience, by Jaak Panksepp, published in 1998, about the neuroscience of emotions in the (animal and human) brain.What surprised me at first was how controversial it apparently was, at the time, to study this subject...

Published on March 10, 2025 1:26 AM GMTrelevant roon Our new friend in the cloud As I am writing this, I'm having a conversation with Claude about US bandwidth regulation while a trio of college students across from me attempt to wrangle an assignment from ChatGPT. Both of us, in our own ways, are talking to an oracle we trust as a matter of...

Published on March 9, 2025 9:10 PM GMT"Short AI timelines" have recently become mainstream.  One now routinely hears the claim that somewhere in the 2026-2028 interval, we'll have AI systems that outperform humans in basically every respect.For example, the official line from Anthropic holds that "powerful AI" will likely arrive in late 2026 or in 2027.  Anthropic's OSTP submission (3/6/2025) says (emphasis in original):[1]Based...

Published on March 9, 2025 6:09 PM GMTThis video provides a lot of background: https://www.youtube.com/watch?v=Eq3bUFgEcb4 and is also very funny, but it's not necessary to understand this post. If you've watched it you can probably skip to the "Why is Piano Roll Notation Bad?" section.TL;DRMusic notation is very janky and weird, but attempts to reform it to be "logical" are usually even worse. The...

Published on March 9, 2025 11:53 AM GMTPreserving the memory, and the cells, of the best cat everThis story begins in May 2007. The Iraq War was in full swing, Windows Vista was freshly released,[1] and I was just 10 years old.My family’s beloved black cat, Lucy, had died. Devastated, I begged my parents: we need to get another Lucy. They must have felt the...

Published on March 9, 2025 5:29 AM GMTI'm curious as to how well Claude can write interesting coding and mathematics problems. This post is a partial product of that exploration. It is a much harder skill to come up with a good problem than to solve one of equal difficulty. While there is a large corpus of data on solving problems, there is very little...

Published on March 8, 2025 4:40 PM GMTI'm in a coworking space on the 25th floor of a building in the bay. In the corner of my eyes, the red figure of the golden gate shimmers. Good morning America. I look down and see the people on the street walking into their offices, cafés, and autonomous Waymos. I stare onto San Francisco and feel...

Published on March 8, 2025 2:59 AM GMTKey points:Standardized assessments do not provide signals for the top ~10% of students.Education studies based on these poor signals lead to poor policies. For example, suspensions allegedly do not improve achievement among peers. However, the signal of "achievement" is "achieving credit in the class", which >80% of students already do. Thus, the conclusion is drawn from a...

Published on March 8, 2025 12:58 AM GMTAfter recently playing some chess against GPT-4.5 (it is pretty good, a lot stronger at the game than GPT-4 was - I would say strong club player level), I decided to try losing chess. To my surprise, it was able to play a complete game, which is something I have not seen any other model being able...