~www_lesswrong_com | Bookmarks (664)

Published on October 10, 2024 12:50 PM GMTJoshua Achiam is the OpenAI Head of Mission Alignment I start off this post with an apology for two related mistakes from last week. The first is the easy correction: I incorrectly thought he was the head of ‘alignment’ at OpenAI rather than his actual title ‘mission alignment.’ Both are important, and make one’s views important, but...

Published on October 10, 2024 9:32 AM GMTI am testing a new debate format and am looking for people to debate any of the following (or make a case for other things):Policy decisionsEA Cause prioritisationWhat should/will happen in Taiwan/Ukraine/GazaUS electionsEconomicsAnimal consciousnessThe debate would happen via a video call and then be put on my youtube (and I guess other places if you want). Why am...

Published on October 10, 2024 9:06 AM GMTGnosticism was an old Christian sect which solved the problem of evil by asserting that the material world was created by an evil entity, and that only inner enlightment can reveal the path to the true, good and loving god. I think many rationalists have a worldview similar to this, perhaps most strongly illustrated by The Goddess...

Published on October 10, 2024 3:27 AM GMTI’ve held the prediction for some time that we are likely to see a fall from grace in Sam Altman, akin to Sam Bankman-Fried’s.This is a difficult intuition to pin down, so I thought I should write it out. I believe it’s also pertinent to the Rationalist community: it is a warning to head a lesson from...

Published on October 9, 2024 11:42 PM GMTImagine a TV showing a video of a bizarre, unfamiliar object - let’s call it a squirgle. The video was computer generated by a one-time piece of code, so there's no "real squirgle" somewhere else in the world which the video is showing. Nonetheless, there's still some substantive sense in which the squirgle on screen is "a...

Published on October 9, 2024 8:19 PM GMTI think that most people underestimate how many scientific mysteries remain, even on questions that sound basic.My favourite candidate for "the most basic thing that is still unknown" is the momentum carried by light, when it is in a medium (for example, a flash of light in glass or water). If a block of glass has a refractive...

Published on October 9, 2024 5:54 PM GMTI'm extremely Noticing-pilled. I think the ability to notice subtle things in your mind, or in the world around you, is one of the foundational rationality skills. It's necessary for building habits[1], for seeing clues that help you solve confusing problems, for diagnosing how you could have thought that faster, and for generally building a mechanistic model of...

Published on October 9, 2024 4:38 PM GMTThanks to Evan Hubinger for funding this project and for introducing me to predictive models, Johannes Treutlein for many fruitful discussions on related topics, and Dan Valentine for providing valuable feedback on my code implementation. In September 2023, I received four months of funding through Manifund to extend my initial results on avoiding self-fulfilling prophecies in predictive models. Eleven months...

Published on October 9, 2024 4:01 PM GMTHave you ever felt deeply connected to the world around you? Like your effortless existence makes a positive difference?I’ve been pondering this state lately, especially since re-reading The Courage to be Disliked. The book introduces a concept I like called “Community Feeling”.Community Feeling is fully believing “I am beneficial to the world” and “my effortless existence contributes...

Published on October 9, 2024 12:56 PM GMTGeoffrey Hinton received the Nobel Prize in Physics for his role in creating the modern field of deep learning. This will strengthen his reputation as the "Godfather of AI" which was already used to amplify his public statements about AI risk.[1]The Nobel Prize in Physics 2024 was awarded to John J. Hopfield and Geoffrey E. Hinton “for...

Published on October 9, 2024 5:51 AM GMTAt least in the early days of LessWrong this community was mostly focused on all the ways Man's reasoning failed him, the myriad ways we are irrational. What foolish humans, making mistakes even trivial mathematics can show you are wrong! Evolution aims to maximise number of offspring. But people don't have a direct drive for that because...

Published on October 9, 2024 2:17 AM GMTIf you are interested in supporting the development of research projects and researchers in AI safety, the MATS team is currently hiring for Community Manager, Research Manager, and Operations Generalist roles and would love to hear from you. Please apply by Nov 3 to join our team for the Winter 2024-25 Program over the period of Dec...

Published on October 9, 2024 12:04 AM GMTPalisade is hiring for the following roles:Executive Assistant to the Executive Director (Jeffrey Ladish)Content LeadOperations LeadPolicy Lead.Applications are rolling, and you can fill out our short (~10 minutes) application form here. Feel free to indicate interest in multiple positions, or make a general expression of interest. We’re flexible in structuring roles and open to a single candidate...

Published on October 8, 2024 10:30 PM GMT(Cross-posted from my website.)This is the video and transcript for a lecture I gave about my essay series “Otherness and control in the age of AGI” (slides available here). The lecture attempts to distill down what I see as the core thread explored in the series, which I summarize in the following chart:The “Otherness and control in...

Published on October 8, 2024 8:25 PM GMTSome friends and I were discussing "what makes one a rationalist", which is a less strict criteria than "what makes one a member of the rationalist community", which is a less strict criteria still than "what makes one a high status, well liked, or enthusiastic member of the rationalist community", and I am most interested in the...

Published on October 8, 2024 1:30 PM GMT4.1 Post summary / Table of contentsThis is the fourth of a series of eight blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)“Trance” is an umbrella term for various states of consciousness in which “you lose yourself”, somehow. The first kind that I learned about was hypnotic...

Published on October 7, 2024 6:08 PM GMTIf I say "the store is 500 meters away," is this strictly true? Strictly false? Either strictly true or strictly false, with probabilies of true or false summing to one? Fuzzily true, because the store is 500.1 meters away? My thesis is that it's strictly true or strictly false, with associated probabilities. Bear with me.The 500 meter...

Published on October 7, 2024 4:46 PM GMTI interact with journalists quite a lot and I have specific preferences. Not just for articles, but for behaviour. And journalists do behave pretty strangely at times. This account comes from talking to journalists on ~10 occasions. Including being quoted in ~5 articles. PrivacyI do not trust journalists to abide by norms of privacy. If I talk to a...

Published on October 7, 2024 3:15 PM GMTPreviously: https://www.lesswrong.com/posts/RydETq379eoWqBFvj/updates-and-reflections-on-optimal-exercise-after-nearly-ahttps://www.lesswrong.com/posts/bZ2w99pEAeAbKnKqo/optimal-exerciseI figured it would be useful to summarize how I train when I go for time efficiency these days, based on my understanding of the relevant meta-analyses on exercise selection and parameters. The primary time savers are supersets and lowered rest times. A routine like this is not totally optimized for max hypertrophy or anything, but...

Published on October 7, 2024 1:02 PM GMTWe have published A Narrow Path: our best attempt to draw out a comprehensive plan to deal with AI extinction risk. We propose concrete conditions that must be satisfied for addressing AI extinction risk, and offer policies that enforce these conditions.A Narrow Path answers the following: assuming extinction risk from AI, what would be a response that...

Published on October 7, 2024 9:56 AM GMTTLDR;In previous work, we found a problematic form of feature splitting called "feature absorption" when analyzing Gemma Scope SAEs. We hypothesized that this was due to SAEs struggling to separate co-occurrence between features, but we did not prove this. In this post, we set up toy models where we can explicitly control feature representations and co-occurrence rates...

Published on October 7, 2024 9:45 AM GMTI think consequentialism describes only a subset of my wishes. For example, maximizing money is well modeled by it. But when I'm playing with something, it's mostly about the process, not the end result. Or when I want to respect the wishes of other people, I don't really know what end result I'm aiming for, but I...

Published on October 7, 2024 8:53 AM GMTThis is an archival link-post for our preprint, which can be found here.Figure 1: SAE-Rad identifies clinically relevant and interpretable features within radiological images. We illustrate a number of pathological and instrumentation features relevant for producing radiology reports. We add annotations (green arrows) to emphasize the presence of each feature.Executive SummaryThis post is a heavily slimmed down...

Published on October 6, 2024 7:53 PM GMTEpistemic status: This essay attempts to communicate intuitions strong shaped by @Joe Carlsmith's writing, particularly "On Green."Warning: very minor spoilers, mostly for "The Dark Knight".Some people have a sense of weight about them. They know exactly what they want and can explain why it is Right. Presented with a moral dilemma, they resolve conflicts between local values...