~www_lesswrong_com | Bookmarks (690)

Published on October 10, 2024 1:40 PM GMTBoth Geoffrey Hinton and Demis Hassabis were given the Nobel Prize this week, in Physics and Chemistry respectively. Congratulations to both of them along with all the other winners. AI will be central to more and more of scientific progress over time. This felt early, but not as early as you would think. The two big capability...

Published on October 10, 2024 1:36 PM GMTOne strategy for mitigating risk from schemers (that is, egregiously misaligned models that intentionally try to subvert your safety measures) is behavioral red-teaming (BRT). The basic version of this strategy is something like: Before you deploy your model, but after you train it, you search really hard for inputs on which the model takes actions that are very...

Published on October 10, 2024 12:50 PM GMTJoshua Achiam is the OpenAI Head of Mission Alignment I start off this post with an apology for two related mistakes from last week. The first is the easy correction: I incorrectly thought he was the head of ‘alignment’ at OpenAI rather than his actual title ‘mission alignment.’ Both are important, and make one’s views important, but...

Published on October 10, 2024 9:32 AM GMTI am testing a new debate format and am looking for people to debate any of the following (or make a case for other things):Policy decisionsEA Cause prioritisationWhat should/will happen in Taiwan/Ukraine/GazaUS electionsEconomicsAnimal consciousnessThe debate would happen via a video call and then be put on my youtube (and I guess other places if you want). Why am...

Published on October 10, 2024 9:06 AM GMTGnosticism was an old Christian sect which solved the problem of evil by asserting that the material world was created by an evil entity, and that only inner enlightment can reveal the path to the true, good and loving god. I think many rationalists have a worldview similar to this, perhaps most strongly illustrated by The Goddess...

Published on October 10, 2024 3:27 AM GMTI’ve held the prediction for some time that we are likely to see a fall from grace in Sam Altman, akin to Sam Bankman-Fried’s.This is a difficult intuition to pin down, so I thought I should write it out. I believe it’s also pertinent to the Rationalist community: it is a warning to head a lesson from...

Published on October 9, 2024 11:42 PM GMTImagine a TV showing a video of a bizarre, unfamiliar object - let’s call it a squirgle. The video was computer generated by a one-time piece of code, so there's no "real squirgle" somewhere else in the world which the video is showing. Nonetheless, there's still some substantive sense in which the squirgle on screen is "a...

Published on October 9, 2024 8:19 PM GMTI think that most people underestimate how many scientific mysteries remain, even on questions that sound basic.My favourite candidate for "the most basic thing that is still unknown" is the momentum carried by light, when it is in a medium (for example, a flash of light in glass or water). If a block of glass has a refractive...

Published on October 9, 2024 5:54 PM GMTI'm extremely Noticing-pilled. I think the ability to notice subtle things in your mind, or in the world around you, is one of the foundational rationality skills. It's necessary for building habits[1], for seeing clues that help you solve confusing problems, for diagnosing how you could have thought that faster, and for generally building a mechanistic model of...

Published on October 9, 2024 4:38 PM GMTThanks to Evan Hubinger for funding this project and for introducing me to predictive models, Johannes Treutlein for many fruitful discussions on related topics, and Dan Valentine for providing valuable feedback on my code implementation. In September 2023, I received four months of funding through Manifund to extend my initial results on avoiding self-fulfilling prophecies in predictive models. Eleven months...

Published on October 9, 2024 4:01 PM GMTHave you ever felt deeply connected to the world around you? Like your effortless existence makes a positive difference?I’ve been pondering this state lately, especially since re-reading The Courage to be Disliked. The book introduces a concept I like called “Community Feeling”.Community Feeling is fully believing “I am beneficial to the world” and “my effortless existence contributes...

Published on October 9, 2024 12:56 PM GMTGeoffrey Hinton received the Nobel Prize in Physics for his role in creating the modern field of deep learning. This will strengthen his reputation as the "Godfather of AI" which was already used to amplify his public statements about AI risk.[1]The Nobel Prize in Physics 2024 was awarded to John J. Hopfield and Geoffrey E. Hinton “for...

Published on October 9, 2024 5:51 AM GMTAt least in the early days of LessWrong this community was mostly focused on all the ways Man's reasoning failed him, the myriad ways we are irrational. What foolish humans, making mistakes even trivial mathematics can show you are wrong! Evolution aims to maximise number of offspring. But people don't have a direct drive for that because...

Published on October 9, 2024 2:17 AM GMTIf you are interested in supporting the development of research projects and researchers in AI safety, the MATS team is currently hiring for Community Manager, Research Manager, and Operations Generalist roles and would love to hear from you. Please apply by Nov 3 to join our team for the Winter 2024-25 Program over the period of Dec...

Published on October 9, 2024 12:04 AM GMTPalisade is hiring for the following roles:Executive Assistant to the Executive Director (Jeffrey Ladish)Content LeadOperations LeadPolicy Lead.Applications are rolling, and you can fill out our short (~10 minutes) application form here. Feel free to indicate interest in multiple positions, or make a general expression of interest. We’re flexible in structuring roles and open to a single candidate...

Published on October 8, 2024 10:30 PM GMT(Cross-posted from my website.)This is the video and transcript for a lecture I gave about my essay series “Otherness and control in the age of AGI” (slides available here). The lecture attempts to distill down what I see as the core thread explored in the series, which I summarize in the following chart:The “Otherness and control in...

Published on October 8, 2024 8:25 PM GMTSome friends and I were discussing "what makes one a rationalist", which is a less strict criteria than "what makes one a member of the rationalist community", which is a less strict criteria still than "what makes one a high status, well liked, or enthusiastic member of the rationalist community", and I am most interested in the...

Published on October 8, 2024 1:30 PM GMT4.1 Post summary / Table of contentsThis is the fourth of a series of eight blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)“Trance” is an umbrella term for various states of consciousness in which “you lose yourself”, somehow. The first kind that I learned about was hypnotic...