~www_lesswrong_com | Bookmarks (657)

Published on September 26, 2024 9:20 PM GMTSeptember 26 is the anniversary of the 1983 Soviet nuclear false alarm incident—item #24 on Wikipedia’s upsettingly long “List of Nuclear Close Calls”—in which Lieutenant Colonel Stanislav Petrov did an excellent job at not starting World War III.Quarterly Performance Review, Autumn 1983Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe...

Published on September 26, 2024 1:44 PM GMTThis research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated...

Published on September 26, 2024 1:40 PM GMT One thing I like about renting cars when I travel is that it's an opportunity to get a sense for a car that's a lot more detailed than what you'd get with a test drive. Traveling to DC for work a few days ago, I took the opportunity to rent a 2023 Chevy Bolt. This is...

Published on September 26, 2024 12:00 PM GMTWe interrupt Nate Silver week here at Don’t Worry About the Vase to bring you some rather big AI news: OpenAI and Sam Altman are planning on fully taking their masks off, discarding the nonprofit board’s nominal control and transitioning to a for-profit B-corporation, in which Sam Altman will have equity. We now know who they are...

Published on September 26, 2024 10:56 AM GMTI was reading about honeypots today. Honeypots (in AI evaluations, at least) are described as testing scenarios that could trick an AI into revealing deceptive or misaligned intent. My first thought was that, especially for current generation large language models, a honeypot scenario and a genuine user scenario would probably be indistinguishable. After all, if we say...

Published on September 26, 2024 9:17 AM GMTRecently, there was a post on SB-1047 and how it's quite mild regulation. I'm not expert on it and don't know how it works. In the comment section I was asking:Why wouldn't deep fake porn or voice cloning technology to engage in fraud be powerful enough to materially contribute to critical harm?There are cases of fraud that could...

Published on September 26, 2024 8:08 AM GMTToday we honor the actions of Stanislav Petrov (1939 – 2017) once again.Half an hour past midnight on September 26, 1983, he saw the first apparent launch on his computer monitor in a glass-walled room on the top floor of the Ballistic Missile Early Warning System (BMEWS) command and control post. The warning system was by now...

Published on September 26, 2024 1:50 AM GMT When I'm doing exploratory work I want to run many analyses. I'm usually optimizing for getting something quick, but I want to document what I'm doing enough that if there are questions about my analysis or I later want to draw on it I can reconstruct what I did. I've taken a few approaches to this...

Published on September 25, 2024 10:35 PM GMTWe (Goodfire) just put our research preview live - you can play with Llama 3 and use sparse autoencoders to read & write from its internal activations. This is a linkpost for:The research preview.Our blog post about building it.Taking research and turning it into something you can actually use and play with has been great. It's surprising...

Published on September 25, 2024 9:15 PM GMTOpenAI to remove non-profit controlReuters reports: Exclusive: OpenAI to remove non-profit control and give Sam Altman equityChatGPT-maker OpenAI is working on a plan to restructure its core business into a for-profit benefit corporation that will no longer be controlled by its non-profit board, people familiar with the matter told Reuters, in a move that will make the...

Published on September 24, 2024 1:58 PM GMTA lot is happening. How do you keep on top of things?For example:What's your system for finding out about important developments?What information sources do you find most useful?What are some problems with your current system?I'll share my answers in a comment below.Motivation:  I've noticed many cases where AI safety professionals—including leaders in the field—find out about important...

Published on September 24, 2024 1:49 PM GMTAuthors: John Hewitt, Nelson F. Liu, Percy Liang, Christopher D. Manning.Abstract:Instruction tuning commonly means finetuning a language model on instruction-response pairs. We discover two forms of adaptation (tuning) that are deficient compared to instruction tuning, yet still yield instruction following; we call this implicit instruction tuning. We first find that instruction-response pairs are not necessary: training solely...

Published on September 24, 2024 11:50 AM GMTPreviously: Book Review: On the Edge: The Fundamentals As I said in the Introduction, I loved this part of the book. Let’s get to it. Poker and Game Theory When people talk about game theory, they mostly talk solving for the equilibrium, and how to play your best game or strategy (there need not be a formal...

Published on September 24, 2024 11:30 AM GMT Lily recently wrote a song, and I've been helping them record it. I got us set up to record four tracks (vocals, drums, keyboard x2) and we did a bunch of takes: Most of the takes are full-length, but after we got a really nice version of the end we did some rounds with just the...

Published on September 24, 2024 11:00 AM GMTCurrent LLM based AI systems are getting pretty good at maths by writing formal proofs in Lean or similar. https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/So, how can we use these models to help align AI? The Simple Alignment Solution Assumption states that many problems in alignment, for example corrigibility or value learning, have simple solutions. I mean "simple" in the sense that logical...

Published on September 24, 2024 7:54 AM GMTThis is a response to Scott Alexanders Give Up Seventy Percent Of The Way Through The Hyperstitious Slur Cascade. Maybe read that post first.I think it becomes more important to struggle against a cascade the more annoying the new word is. There are a variety of factors that contribute how annoying it is, here are five of...

Published on September 24, 2024 5:08 AM GMTI’m not Christian now, but I used to be. I parted ways with Christianity because I came to identify as more of an Empiricist - believing that knowledge should come from experiment and observation, not writings in old sacred texts.But some of the writing in those old sacred texts is actually really good.The King James Bible contains...

Published on September 24, 2024 2:00 AM GMTAwhile ago, Nate Soares wrote the posts Decision theory does not imply that we get to have nice things and Cosmopolitan values don't come free and But why would the AI kill us? Paul Christiano put forth some arguments that "it seems pretty plausible that AI will be at least somewhat 'nice'", similar to how humans are somewhat...

Published on September 24, 2024 12:47 AM GMTThis post is probably hazardous for one type of person in one particular growth stage, and necessary for people in a different growth stage, and I don't really know how to tell the difference in advance.If you read it and feel like it kinda wrecked you, send me a DM. I'll try to help bandage it. One of...

Published on September 22, 2024 5:44 PM GMTThis essay was written by @Metacelsus.In March 2023, a research team at Osaka University announced an extraordinary achievement: the production of viable eggs using cells taken from male mice.Building on years of research into stem cells and reproductive development,1 the researchers, led by Professor Katsuhiko Hayashi, developed a method to induce male stem cells to lose their...

Published on September 22, 2024 5:10 PM GMT One of the things I like most about contra dance is how it works with a very wide range of skill levels, including having one of the best learning curves of any dance form I know. As a community we are pretty committed to the idea that people should just be able to show up and...

Published on September 22, 2024 4:50 PM GMTI’ve been fascinated recently by historical linguistics, and in particular reconstructed proto-languages such as Proto-Indo-European.But what even is a “proto-language”?Well, if we go for the literal operationalization, Wikipedia defines a proto-language as:a postulated ancestral language from which a number of attested languages are believed to have descended by evolution, forming a language family.[…]It is by definition a...