~www_lesswrong_com | Bookmarks (662)

Published on December 25, 2024 10:44 AM GMTWhen I was in college, I rented a group house with 5 other nerds. There were 5 bedrooms to divide among the 6 of us, so I negotiated an agreement where I paid less rent in exchange for sleeping in the hallway. This wasn't as bad as it sounds. My bedroom wasn't hallway-sized. It was bedroom-sized and...

Published on December 25, 2024 5:49 AM GMTI've heard repeatedly from many people that the highest-value part of conferences is not the talks or structured events, but rather the casual spontaneous conversations. Yet my own experience does not match this at all; the casual spontaneous conversations are consistently low-value.My current best model is that the casual spontaneous conversations mostly don't have much instrumental value,...

Published on December 24, 2024 9:57 AM GMTBy Sophie Bridgers, Rishub Jain, Rory Greig, and Rohin ShahBased on work by the Google DeepMind Rater Assist Team: Vladimir Mikulik, Sophie Bridgers, Tian Huey Teh, Rishub Jain, Rory Greig, Lili Janzer (randomized order, equal contributions) Human oversight is critical for ensuring that Artificial Intelligence (AI) models remain safe and aligned to human values. But AI systems are...

Published on December 24, 2024 10:45 PM GMTThis is a link to the latest Bayesian Conspiracy episode. Oliver tells us how Less Wrong instantiated itself into physical reality via LightHaven, along with a bit of deep lore of foundational Rationalist/EA orgs. He also gives a surprisingly nuanced (IMO) view on Leverage! Do you like transcripts? We got one of those at the link as well....

Published on December 24, 2024 6:50 PM GMTEpistemic Status: This was composed in late 2017, sat editing limbo, was still worth publishing in 2021 and got some edits, and by 2023 was overdue for being published. Now, in 2024, it is almost just embarrassing that I haven't published it yet.The 2006 book "Data Analysis: A Bayesian Tutorial" by Devinderjit Sivia & John Skilling exposed...

Published on December 24, 2024 1:37 PM GMTI find that most communities I am part of, where AI discussions occur, fall into two categories: The community is too broad, and the discussion is fragmented. ACX would fall into this category. The AI discussion focuses on AGI/ASI and related topics (alignment, safety, how it will affect humanity, etc.). Lesswrong would fall into this category. I...

Published on December 24, 2024 1:00 PM GMTThis post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models. This is by far the best demonstration so far of the principle that AIs Will Increasingly Attempt Shenanigans. This was their announcement thread. New Anthropic research: Alignment faking in large...

Published on December 24, 2024 10:25 AM GMTTLDR: We're hosting a 3-month, fully-funded fellowship to do AI safety research drawing on inspiration from fields like evolutionary biology, neuroscience, dynamical systems theory, and more. Past fellows have been mentored by John Wentworth, Davidad, Abram Demski, Jan Kulveit and others, and gone on to work at places like Anthropic, AISI UK, Apart research, or as full-time PIBBSS...

Published on December 23, 2024 5:37 PM GMTThis essay was part of my application to UKAISI. Unclear how much is novel, but I like the decomposition.Some AI-enabled catastrophic risks involve an actor purposefully bringing about the negative outcome, as opposed to multi-agent tragedies where no single actor necessarily desired it. Such purposeful catastrophes can be broken down into a conjunction: they materialize if and only...

Published on December 23, 2024 3:44 PM GMTAs a holiday gift to myself, I put together a printable book of some creative writing by Scott Alexander & Eliezer. I'm sharing it here in case anyone else is interested :).It contains most of Scott's published fiction since he switched to Substack, plus some select creative non-fiction pieces (including a few from the old SSC days)....

Published on December 23, 2024 2:20 PM GMTI took a trip to San Francisco early in December. Ever since then, things in the world of AI have been utterly insane. Google and OpenAI released endless new products, including Google Flash 2.0 and o1. Redwood Research and Anthropic put out the most important alignment paper of the year, on the heels of Apollo’s report on...

Published on December 23, 2024 1:17 PM GMTtl;dr: The glitch tokens ' petertodd' and ' Leilan' were studied extensively in the context of GPT-3 before its decommissioning at the end of 2023 [1] [2]. Here, the conception of these two tokens and their relationship is studied for GPT-2, GPT-2-xl and GPT-J (which share the same token vocabulary as GPT-3). An appendix explores possible connections...

Published on December 23, 2024 9:38 AM GMTI'm seeing a lot of people on LW saying that they have very short timelines (say, five years or less) until AGI. However, the arguments that I've seen often seem to be just one of the following:"I'm not going to explain but I've thought about this a lot""People at companies like OpenAI, Anthropic etc. seem to believe...

Published on December 23, 2024 9:32 AM GMTThe AI should talk like a team of many AI. Each AI only uses the word "I" when referring to itself, and calls other AI in the team by their name. I argue that this may massively reduce Self-Allegiance by making it far more coherent for one AI to whistleblow or fight another AI which is unethical...

Published on December 23, 2024 8:51 AM GMTProject summaryAI Safety Camp has a seven-year track record of enabling participants to try their fit, find careers and start new orgs in AI Safety. We host up-and-coming researchers outside the Bay Area and London hubs.If this fundraiser passes…$15k, we won’t run a full program, but can facilitate 10 projects.$40k, we can organise the 11th edition, for...

Published on December 23, 2024 6:32 AM GMTThis is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback. The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.Compute governance is a type of AI governance that focuses on controlling access to the computing hardware needed...

Published on December 23, 2024 5:16 AM GMTEpistemic Status: Seven years and one day ago, I wrote this. In 2017 we faced years of Trump and couldn't imagine covid. Here in 2024 we face years of Trump and I'm trying to "imagine what we can't imagine". I dislike the title (except for the part where it might introduce people to music from a previous...

Published on December 23, 2024 3:58 AM GMTOf the posts I've delayed writing for years, I maybe regret this one the most.I think more people (x-risk focused people in particular) should consider becoming (and hiring) metacognitive assistants. This is the single largest performance boost I know of – hire someone to sit with you and help you think. It doesn't help me (much) when...

Published on December 21, 2024 5:17 PM GMTI used to think that all generic strategy advice was pointless. After all, the point of a strategy is to achieve a thing, and to achieve a thing you just think hard about how to best do it and then work hard to do it. I said this to my friend Dewi, who said that this is...

Published on December 21, 2024 8:28 AM GMTEpistemic status: We are moderately confident in the feasibility of serially filtered shelters for 14-log protection as well as cost estimates, but more uncertain about how mirror biology might develop in the real world.Originally published on the EA Forum. Minor edits for LW adaptation.Please sign up here for ~bi-monthly updates on shelter/refuges work.Summary and main take-awaysI want to be...

Published on December 21, 2024 2:55 AM GMT2024 is the year it became clear that we're actually going to do battery energy storage. That it was time to stop soberly reminding anyone too excited about solar that storage is an unsolved problem. In fact I personally became very excited about solar, and briefly thought that this year's wave of battery installations was just the...