~www_lesswrong_com | Bookmarks (657)

Published on September 17, 2024 5:31 PM GMTAs part of the AI Safety, Ethics and Society Course, I’m conducting a survey to better understand the psychological and emotional effects of long-term engagement with AI technologies, particularly within the AI safety community. This is an invitation for you to take part in this anonymous questionnaire, which explores how engagement with AI could influence emotions, stress levels, and mental...

Published on September 17, 2024 2:53 PM GMTFor a while, I thought music was harmful, due largely to pervasive and arbitrary earworms. More recently, I started to find that earworms are ephemeral and lawful. A contrarian belief held like the former for years gets stuck as part of my identity, but maybe I should find the truth."Music is harmful" is hard to measure and...

Published on September 17, 2024 12:20 PM GMTIt’s that time again for all the sufficiently interesting news that isn’t otherwise fit to print, also known as the Monthly Roundup. Bad News Beware the failure mode in strategy and decisions that implicitly assumes competence, or wishes away difficulties, and remember to reverse all advice you hear. Stefan Schubert (quoting Tyler Cowen on raising people’s ambitions...

Published on September 16, 2024 6:15 PM GMTMIRI updatesAaron Scher and Joe Collman have joined the Technical Governance Team at MIRI as researchers. Aaron previously did independent research related to sycophancy in language models and mechanistic interpretability, while Joe previously did independent research related to AI safety via debate and contributed to field-building work at MATS and BlueDot Impact.In an interview with PBS News...

Published on September 16, 2024 4:31 PM GMTIntroductionEvery single time I design a protein — using ML or otherwise — I am confident that it is capable of being manufactured. I simply reach out to Twist Biosciences, have them create a plasmid that encodes for the amino acids that make up my proteins, push that plasmid into a cell, and the cell will pump...

Published on September 16, 2024 4:07 PM GMTTL;DR: We introduce the first comprehensive theoretical framework for understanding and mitigating secret collusion among advanced AI agents, along with CASE, a novel model evaluation framework. CASE assesses the cryptographic and steganographic capabilities of agents, while exploring the emergence of secret collusion in real-world-like multi-agent settings. Whereas current AI models aren't yet proficient in advanced steganography, our...

Published on September 16, 2024 1:40 PM GMTTerrible name (with a terrible reason, that this ‘resets the counter’ on AI capability to 1, and ‘o’ as in OpenAI when they previously used o for Omni, very confusing). Impressive new capabilities in many ways. Less impressive in many others, at least relative to its hype. Clearly this is an important capabilities improvement. However, it is...

Published on September 16, 2024 12:39 PM GMTSuppose that an embedded agent models its environment using an approximate simplicity prior, would it acquire a physicalist agent ontology or an algorithmic/logical agent ontology?One argument for the logical agent ontology is that it allows the agent to compress different parts of its observations that are subjunctively dependent: If two physical systems are computing the same function,...

Published on September 16, 2024 9:43 AM GMTA framing I quite like is that of germs vs soma, body vs eggs and cum, consciousness vs replicators.My first foray into age reversal was a (successful) attempt to increase fluid IQ, the loss of which is among less than a handful of ubiquitous symptoms of aging.At the time, I found it odd that most people working...

Published on September 16, 2024 8:11 AM GMTI notice that reasoning about logical uncertainty does not appear more confusing to me than reasoning about empirical one. Am I missing something?Consider the classical example from the description of the tag:Is the googolth digit of pi odd? The probability that it is odd is, intuitively, 0.5. Yet we know that this is definitely true or false...

Published on September 16, 2024 1:04 AM GMTMarkets for information are inefficient, in large part due to the Buyer’s Inspection Paradox: you can’t “inspect” information like you would any other good before buying — the moment you inspect the information, you have obtained it and cannot return it. More generally, the problem is an inability to reliably commit to forgetting. A comment by John...

Published on September 15, 2024 9:37 PM GMTInterpolation, Extrapolation, Hyperpolation: Generalising into new dimensionsby Toby Ord Abstract:This paper introduces the concept of hyperpolation: a way of generalising from a limited set of data points that is a peer to the more familiar concepts of interpolation and extrapolation. Hyperpolation is the task of estimating the value of a function at new locations that lie outside the...

Published on September 15, 2024 9:24 PM GMTSupposedly intelligence is some kind of superpower.  And they're now selling intelligence for pennies/million tokens.  Logically, it seems like I should be spending way more of my income than I currently am on intelligence.  But what should I spend it on?For context, I currently spend ~$50/month on AI:ChatGPT $20/monthGithub Copilot $10/monthVarious AI art apps ~$20/monthSuppose I wanted...

Published on September 14, 2024 5:51 PM GMTImagine that you want to predict the behavior of some system -- the weather, a computer program, a real-world language... However you go about it, I bet you will do some compression: you'll find simple and minimal and concrete ways to think about the vast complexity of your system that helps you predict what you want to...

Published on September 14, 2024 3:39 AM GMTThanks to Kaj Sotala, Stag Lynn, and Ulisse Mini for reviewing. Thanks to Kaj Sotala, Brian Toomey, Alex Zhu, Damon Sasi, Anna Salamon, and CFAR for mentorship and financial supportA few months ago I made the claim “Radically effective and rapid growth [motivationally / emotionally / socially] is possible with the right combination of facilitator and method”. Eg: for anxiety,...

Published on September 13, 2024 9:24 PM GMTImagine if you will, a map of a landscape. On this map, I will draw some vague regions. Their boundaries are uncertain, for it is a new and under-explored land. This map is drawn as a graph, but I want to emphasize that the regions are vague guesses, and the true borders could be very convoluted.  So here's...

Published on September 13, 2024 8:30 PM GMTI mean big in terms of number of tokens, and I am thinking about this question specifically in the context of training windows vs context windows. This question is inspired by an Andrew Mayne tweet covered in AI #80: Never Have I Ever:Most AI systems are trained on less than 2,000 words per sample. They can generalize...

Published on September 13, 2024 8:00 PM GMTMachine learning systems are typically trained to maximize average-case performance. However, this method of training can fail to meaningfully control the probability of tail events that might cause significant harm. For instance, while an artificial intelligence (AI) assistant may be generally safe, it would be catastrophic if it ever suggested an action that resulted in unnecessary large-scale...

Published on September 13, 2024 7:38 PM GMTHolden just published this paper on the Carnegie Endowment website. I thought it was a decent reference, so I figured I would crosspost it (included in full for convenience, but if either Carnegie Endowment or Holden has a preference for just having an excerpt or a pure link post, happy to change that) If-then commitments are an...

Published on September 13, 2024 7:00 PM GMTWith Lakera's strides in securing LLM APIs, Goodfire AI's path to scaling interpretability, and 20+ model evaluations startups among much else, there's a rising number of technical startups attempting to secure the model ecosystem.Of course, they have varying levels of impact on superintelligence containment and security and even with these companies, there's a lot of potential for aligned, ambitious and high-impact startups within the...

Published on September 13, 2024 5:29 PM GMTEpistemic status and trigger warnings: Not rigorous in either math or physics. Not proof-read by any third party yet. May contain original research. Lengthy. Some sections are tongue-in-cheek. Anthrophics. Cosmology. Theism.IntroductionCatching up The Bayesian Conspiracy, I have recently listened to the episode from a month ago where they talked with Bentham's Bulldog about his argument for God,...

Published on September 13, 2024 5:09 PM GMTOnce upon a time, the sun let out a powerful beam of light which shattered the world. The air and the liquid was split, turning into body and breath. Body and breath became fire, trees and animals. In the presence of the lightray, any attempt to reunite simply created more shards, of mushrooms, carnivores, herbivores and humans....

Published on September 13, 2024 3:52 PM GMTEpistemic Status: I wrote this back in January, and have been uncertain about whether to publish it. I expect that most people who read this here will be unconvinced. But I still want to express my intuition.In the last month, these ideas have come up in conversation again. Then, Toby Ord published a paper on arxiv a...

Published on September 13, 2024 3:47 PM GMTThis article revisits and expands upon the AI alignment difficulty scale, a framework for understanding the increasing challenges of aligning artificial intelligence systems with human values. We explore how alignment difficulties evolve from simple goal misalignment to complex scenarios involving deceptive alignment and gradient hacking as we progress up the scale. We examine the changing dynamics across different...