Bookmarks (667)

Published on November 23, 2024 6:34 PM GMTThe following is a short supplement to an upcoming post on the implications for conscious significance (a framework for understanding the free will and determinism debate) but is a more general observation.The world has undergone many paradigm shifts; where a profound truth has been revealed about the nature of the universe and our place within it. Individuals...

Published on November 23, 2024 5:30 PM GMTWe thank Madeline Brumley, Joe Kwon, David Chanin and Itamar Pres for their helpful feedback.IntroductionControlling LLM behavior through directly intervening on internal activations is an appealing idea. Various methods for controlling LLM behavior through activation steering have been proposed. Most steering methods add a 'steering vector' (SV) to the model’s activations at a given layer and token position...

Published on November 23, 2024 12:50 PM GMT Another round of liberating kid posts from Facebook. For reference, in 2018 Lily turned 4 and Anna turned 3. (Some of these were from me; some were from Julia. Ones saying "me" could mean either of us. Ones from others are labeled.) 2018-01-12 Since Lily started listening to "Heidi" a lot, her language has started sounding...

Published on November 23, 2024 12:45 PM GMTAuthors: Beren Millidge ,Yuhang Song, Armin Lak, Mark E. Walton, Rafal Bogacz.Abstract:Animals can adapt their preferences for different types of reward according to physiological state, such as hunger or thirst. To explain this ability, we employ a simple multi-objective reinforcement learning model that learns multiple values according to different reward dimensions such as food or water. We...

Published on November 23, 2024 9:08 AM GMTIntroductionFirst, a bit of blatant advertising: The East Coast Rationalist Megameetup is the evening of December 13th through the morning of December 16th, in Brooklyn, New York City. It's a weekend long unconference timed to coincide with NYC Secular Solstice. Despite the name, people come from all over North America, some for their first rationalist meetup and...

Published on November 23, 2024 2:36 AM GMTHaving recognized that I have asked these same questions repeatedly across a wide range of channels and have never gotten satisfying answers for them, I'm compiling them here so that they can be discussed by a wide range of people in an ongoing way.Why has EV made many moves in the direction of decentralizing EA, rather than...

Published on November 22, 2024 8:53 PM GMTOn the heels of Donald Trump’s election and his promises to end the Department of Education, you may have seen claims like these spreading around X.SourceThis claim is based on two datapoints. First, is the literacy rate of around 99% in 1979 which was measured by the US Census. After the Department of Education was created in...

Published on November 22, 2024 8:53 PM GMTI've been thinking about something I will clumsily call paraddictions: desires that don't quite rise to the level of an addiction, but that have a disproportionate and hard-to-moderate influence over your behavior. Can they be used as a tool for motivation and behavior change?epistemic status: Most of the ideas here are generally solid behavior change principles that...

Published on November 22, 2024 8:11 PM GMTIntroductionIn software design, several key principles share a common foundation: the concept of factoring, or breaking complex problems into smaller, manageable parts. Three prominent examples are:Unix philosophy: Design programs to do one thing well.Black box abstraction: Isolate functionality to focus on core problems.Interface-based programming: Build systems from well-defined, interconnected modules.These principles, though known by different names, all...

Published on November 22, 2024 7:01 PM GMTThis is a crosspost from https://chillphysicsenjoyer.substack.com/p/pursuing-physics-research-part-time. IntroDisclaimer - I’m a part-time research associate doing biophysics with a uni research group in the UK. But I have a day job in an unrelated field that pays the bills.Whilst I’ve read many personal accounts of research from full-time research students in the academic system, I haven’t heard as much from...

Published on November 22, 2024 6:00 PM GMTSummary Over the past year I’ve investigated potential interventions against respiratory illnesses. Previous results include “Enovid nasal spray is promising but understudied”, “Povidone iodine is promising but understudied” and “Humming will solve all your problems no wait it’s useless”. Two of the iodine papers showed salt water doing as well or almost as well as iodine. I...

Published on November 22, 2024 4:41 PM GMTTLDR. We can create a relatively good model of a person by prompting a long-context LLM with a list of facts about this person. We can get much better results by iteratively improving the prompt based on the person's feedback. Sideloading is the only technology for immortality that actually works as of 2024.Fig. 1. Here are my...

Published on November 22, 2024 4:16 PM GMT(If you’re in a hurry, you can just read the “Background and summary” section, and skip the other 85%.)0. Background and summary0.1 Background: What’s the problem and why should we care?My primary neuroscience research goal for the past couple years has been to solve a certain problem, a problem which has had me stumped since the very...

Published on November 22, 2024 4:12 PM GMTCrosspost of this on my blog. I recently wrote an article making the case for giving to shrimp welfare. Lots of people were convinced to give (enough to help around half a billion shrimp avoid a painful death), but because the idea sounds silly, many people on Twitter, hacker news, and various other places made fun of it,...

Published on November 22, 2024 2:01 PM GMTWhich of the following do you think is bigger? A.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display:...

Published on November 22, 2024 12:56 PM GMTA short science fiction story about our ancestors and the ethical responsibility we have towards them. Old Ana's legs fought against her now, but still she led her granddaughter up the mountain path. The girl's torch made shadows dance on the rocks. The river sang in the distance – the same river that had shown Ana her great...

Published on November 22, 2024 3:46 AM GMTI love this question, but I've enjoyed noticing answers to its opposite more. What are some things you thought weren't metaphors but were surprised to learn actually are metaphors.A classic example is that, at least in English, time is often described using distance metaphors. For example, we talk about things taking a "long" or a "short" amount...

Published on November 22, 2024 3:26 AM GMTPreceded by: "Consciousness as a conflationary alliance term for intrinsically valued internal experiences"tl;dr: Chatbots are "conscious" in a variety of important ways. We humans should probably be nice to each other about the moral disagreements and confusions we're about to uncover in our concept of "consciousness".Executive Summary: Turing Prize laureate Geoffrey Hinton is most likely correct that LLM...

Published on November 22, 2024 12:59 AM GMTSome people in my orbit suggested reading Robert F. Kennedy Jr’s book The Real Anthony Fauci.Here’s my story of wading through a few pages of the book and trying to understand the basis of the claims. My takeaway: there's a lot of sloppiness here, and several times the book’s claim isn’t supported by the citation. Often, though, the citation...

Published on November 21, 2024 10:43 PM GMTIn everyday life, if something looks good to a human, then it is probably actually good (i.e. that human would still think it’s good if they had more complete information and understanding). Obviously there are plenty of exceptions to this, but it works most of the time in day-to-day dealings. But if we start optimizing really hard...

Published on November 20, 2024 6:19 PM GMTI am grateful to Noosphere89 for prompting me to start this discussion. In human evolution, the fundamental goal was to survive and reproduce, passing on our genes to the next generation. But somewhere along the way, humans developed complex societies and technologies that go beyond just survival and reproduction. Now, we often find ourselves pursuing goals that...

Published on November 20, 2024 4:48 PM GMTCrosspost of this on my blog. 1 My basic view 'Cause everybody criesEverybody hurts sometimesSometimes everything is wrong—Rem “Everybody hurts”You didn’t think the era of absurdly long posts was over, did you?Prior to the 2024 election, there were a series of swing states that pundits forecasted could have gone either way. All of them ended up going to Trump...

Published on November 20, 2024 1:40 PM GMTPreviously: Long-Term Charities: Apply For SFF Funding, Zvi’s Thoughts on SFF There are lots of great charitable giving opportunities out there right now. I recently had the opportunity to be a recommender in the Survival and Flourishing Fund for the second time. As a recommender, you evaluate the charities that apply and decide how worthwhile you think...

Published on November 20, 2024 11:48 AM GMTAuthors: Anonymous (I'm not one of them).Abstract:Most analysis of transformer expressivity treats the depth (number of layers) of a model as a fixed constant, and analyzes the kinds of problems such models can solve across inputs of unbounded length. In practice, however, the context length of a trained transformer model is bounded. Thus, a more pragmatic question...

Published on November 20, 2024 11:04 AM GMTCurrently, it looks like Robert F. Kennedy Jr. will become HHS secretary and change a lot of things at the HHS. There will likely be both good and bad changes.It's a time when the ideas that lay around matter. What changes do you think should be made? What clever policy ideas should be adopted?Discuss

Published on November 20, 2024 10:13 AM GMTSummary thread: https://x.com/GarrisonLovely/status/1859022323799699474. An influential congressional commission is calling for a militarized race to build superintelligent AI based on threadbare evidenceDiscuss

Published on November 20, 2024 6:04 AM GMTI run a weekly sequences-reading meetup with some friends, and I want to add a film-component, where we watch films that have some tie-in to what we've read.I got to talking with friends about what good rationality films there are. We had some ideas but I wanted to turn it to LessWrong to find out.So please, submit...

Published on November 20, 2024 1:37 AM GMTAs I relayed in my last post, in "Theory of Games and Economic Behavior", von Neumann and Morgenstern defended the validity of using utility to predict economic outcomes despite the fact that cross-individual utility cannot apparently be measured or apprehended directly: "The [ . . . ] difficulties of the notion of utility, and particularly of the...

Published on November 20, 2024 12:57 AM GMTI work in a niche company in a niche industry mostly with a nearly-obsolete programming language. I have worked for the same company for an extremely long time by programmer standards, basically right out of high-school. I have not looked for work in a very long time.Reading HackerNews lately, the market looks pretty grim. I am reading...

Published on November 18, 2024 6:25 PM GMTWe wanted to run an event where controversial opinions could be shared without restraint: “a safe space for what you can't say in ‘safe spaces’”So a mechanism was devised:https://x.com/tobi/status/1795619548579918222Examples of what may or may have been on the slips:“Only white men can save us.”“Intelligent people are worth more.”“Bestiality is good, actually.” We also advertised the event so only...

Published on November 18, 2024 4:00 PM GMTWhat kinds of ethical implications should we expect from the Many-Worlds Interpretation of Quantum Mechanics (MWI)? I’ll argue that we shouldn’t expect decision-making to change. The implications are more about how we should think or feel about events in our lives, and the virtues of taking a cosmic perspective.According the many-worlds interpretation (MWI) of quantum mechanics, the universe...

Published on November 18, 2024 2:25 PM GMTGreat. Another crucial consideration I missed. I was convinced that working on reducing the existential risk for humanity should be a global priority.Upholding our potential and ensuring that we can create a truly just future seems so wonderful.Well, recently I was introduced to the idea that this might actually not be the case. The argument is rooted in...

Published on November 18, 2024 4:58 PM GMTPeople often ask me this. It’s a good question. It seems potentially important for anyone who doesn’t want to die, or doesn’t want other people to have to die. I don’t feel like I have a good answer, which is one reason I want to do more research. But recently I think my answer has gotten a...

Published on November 18, 2024 4:05 PM GMT(thanks to Alex Mallen, Cody Rushing, Zach Stein-Perlman, Hoagy Cunningham, Vlad Mikulik, and Fabien Roger for comments)Sometimes I hear people argue against AI control as follows: if your control measures rely on getting good judgments from "trusted" AI models, you're doomed because AI models aren't adversarially robust. I agree that better adversarial robustness would definitely help with...

Published on November 18, 2024 1:20 PM GMTThis is your monthly roundup. Let’s get right to it. Young People are Young and Stupid As a reminder that yes college students are often young and stupid and wrong about everything, remember the time they were behind a ban on paid public toilets? This is a central case of the kind of logic that often gets...

Published on November 18, 2024 12:45 PM GMTThis post was written during the agent foundations fellowship with Alex Altair funded by the LTFF. Thanks to Alex, Jose, Daniel, Cole, and Einar for reading and commenting on a draft.The Good Regulator Theorem, as published by Conant and Ashby in their 1970 paper (cited over 1700 times!) claims to show that 'every good regulator of a...

Published on November 18, 2024 12:30 PM GMTOn the emergence of history's reinsOne general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die.― Charles DarwinWe are a way for the universe to know itself.— Carl SaganIf you don’t know where you are going, you’ll end up someplace else.— Yogi BerraImagine you were trying...

Published on November 18, 2024 4:21 AM GMTFertility rates in the developed world are too low and that's bad, blah blah.Here's my proposal to raise fertility rates (coming from a current uni student):Promote parenthood in universities via student run clubs. Reading groups about different parenting styles, or more subtle pro-fertility clubs like knitting groups where students make clothes for sick kids (very un-EA, I...

Published on November 18, 2024 2:13 AM GMTNote: I would have done this as a Quick Take, but it doesn't allow images.In the spirit of How I got 4.2M YouTube views without making a single video, I improved a diagram on the wikipedia page for Pareto Efficiency.ORIGINAL VERSIONNEW VERSION Based on the text, I don't think the author understood that Cooperate/Cooperate isn't the only Pareto...

Published on November 16, 2024 2:31 PM GMTAt every turn there was a Gwern, Within this dream I couldn't discern If to Heaven I did rise Or in Satan's pit - my soul's demise. Creatures short and stout and keen of mind Wrote and wrote, words flowed like wine. "Good sir," I said, "how did you come to be In this place with him....

Published on November 16, 2024 2:24 PM GMTI want to make a serious effort to create a bigger evals field. I’m very interested in which resources you think would be most helpful. I’m also looking for others to contribute and potentially some funding. Which resources would be most helpful? Suggestions include:Open Problems in Evals list: A long list of open relevant problems & projects in...

Published on November 16, 2024 6:38 AM GMTAs part of the court case between Elon Musk and Sam Altman, a substantial number of emails between Elon, Sam Altman, Ilya Sutskever, and Greg Brockman have been released. I have found reading through these really valuable, and I haven't found an online source that compiles all of them in an easy to read format. So I made...

Published on November 16, 2024 4:29 AM GMTRob Miles has released a new video, this time on control!Youtube Transcript (Very rough)0:00hi so this channel is about AI safety a lot of people when they hear AI safety0:05the first thing that comes to mind is they think it's about how to keep potentially dangerous AI systems under control that we're trying to answer0:11questions like what...

Published on November 16, 2024 2:59 AM GMTEpistemic status: Toy model. Oversimplified, but has been anecdotally useful to at least a couple people, and I like it as a metaphor.IntroductionI’d like to share a toy model of willpower: your psyche’s conscious verbal planner “earns” willpower (earns trust with the rest of your psyche) by choosing actions that nourish your fundamental, bottom-up processes in the...

Published on November 16, 2024 12:57 AM GMTI wrote a whole book! What's next?I'm currently doing an edit pass on the entire book. I need to rewrite some of the early sections, fix some consistency issues, and generally look with fresh eyes on words I wrote months or years ago. Many of you provided helpful comments, and I'm using those to make the second...

Published on November 15, 2024 6:55 PM GMTTrump and the Republican party will yield broad governmental control during what will almost certainly be a critical period for AGI development. In this post, we want to briefly share various frames and ideas we’ve been thinking through and actively pitching to Republican lawmakers over the past months in preparation for the possibility of a Trump win.Why...

Published on November 15, 2024 3:47 PM GMTIn this post, I’ll make a technical point that comes up when thinking about risks from scheming AIs from a control perspective.In brief: Consider a deployment of an AI in a setting where it’s going to be given a sequence of tasks, and you’re worried about a safety failure that can happen suddenly. Every time the AI...

Published on November 15, 2024 1:59 PM GMTTechnological progress can excite us, politics can infuriate us, and wars can mobilize us. But faced with the risk of human extinction that the rise of artificial intelligence is causing, we have remained surprisingly passive. In part, perhaps this was because there did not seem to be a solution. This is an idea I would like to...