~www_lesswrong_com | Bookmarks (664)

Published on October 2, 2024 9:49 AM GMTThis is a selection of AI safety paper highlights in September 2024, from my blog "AI Safety at the Frontier". The selection primarily covers ML-oriented research. It's only concerned with papers (arXiv, conferences etc.), not LessWrong or Alignment Forum posts. As such, it should be a nice addition for people primarily following the forum, who might otherwise...

Published on October 2, 2024 8:33 AM GMTThe more I work on myself, the more I realize that the first step for me is almost always to just notice I’m stuck in a loop.As a teenager, I didn’t make progress towards getting good at anything because I was stuck in the following loop:Decide X is my passionGo all in for X, think about it...

Published on October 2, 2024 6:48 AM GMTSuppose you can tell your AI to meet a certain spec (e.g. cure cancer), but most plans that meet the spec are unsafe (e.g. involve killing everyone, or so Rob Bensinger thinks). In these cases, a quantilizer is insufficient for safety due to instrumental convergence.[1]But suppose we can also give the agent a dispreference for unsafe actions...

Published on October 2, 2024 2:20 AM GMT The keyboard is a bit of an awkward instrument to travel with. It's quite large, to the point that you have to give up at least one seat in a typical car. What makes this especially frustrating is that I don't actually use the whole 88 keys: The very lowest notes tend to be boomy, while...

Published on October 1, 2024 9:59 PM GMTVision Weekend US, Foresight’s annual festival, is approaching, offering an opportunity to engage with the world’s most forward-thinking ideas and individuals through the conference, unconference, office hours, mentorship sessions, tech demos, art exhibits, cyberfuture dinners, tea ceremonies, and aftershow parties. Dates: December 6–8, 2024Location: Across iconic Bay Area locations, such as The Institute, Internet Archive, Future House and...

Published on September 30, 2024 4:00 PM GMTThis series of posts presents the idea of building a knowledge database that allows for the collection and exchange of reliable information between people, computers, and between people and computers. It also discusses how this database could increase the intelligence of both people and computers, and increase the safety of AGI.Usefulness of credible information for people and...

Published on September 30, 2024 3:57 PM GMTChatGPT 4o’s interpretation of semiconductor grade quartzWe rarely think about where our stuff comes from or how it’s made. We go through our lives expecting that the things we consume are easily acquired. That is the beauty of modern society: supply chain logistics work so well that we seldom think about the consequences if these systems are...

Published on September 30, 2024 3:56 PM GMTYou may have heard that tooth decay is caused by bacteria producing lactic acid. Let's consider that a little more deeply. criteria for decay To effectively cause cavities, bacteria must meet 4 criteria: Anchoring (with special proteins) to either the tooth surface or something connected to it. Biofilm production to trap nutrients and protect the bacteria. Metabolism...

Published on September 30, 2024 3:49 PM GMTJust what it says on the tin. Covered most everywhere, but I found the quote in this Reuters article stuck out to me the most:Newsom said the bill "does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data" and would apply "stringent standards to...

Published on September 30, 2024 2:54 PM GMTNPR writes:Gov. Gavin Newsom of California on Sunday vetoed a bill that would have enacted the nation’s most far-reaching regulations on the booming artificial intelligence industry. California legislators overwhelmingly passed the bill, called SB 1047, which was seen as a potential blueprint for national AI legislation.The measure would have made tech companies legally liable for harms caused by...

Published on September 30, 2024 10:52 AM GMTThe HierarchyThere is a hierarchy in life from simple cells to complex cells to multi-cellular creatures to creatures that often live in a groups like birds and bees. As we go up the hierarchy the lower levels have less individual say. For example the mitochondria have no individual life or fitness outside of the cell, and the...

Published on September 30, 2024 8:57 AM GMTRecently my team and I have been working on FindingConsensus.AI. It’s a site to show how public figures would answer specific questions about Artificial Intelligence (but it could work for anything). Often when thinking, I defer to figures I trust, so seeing a spread of public figures gives me a head start. I hope that if we...

Published on September 30, 2024 2:35 AM GMTSummaryThis winter, MATS will be running our seventh program. In early-mid 2024, 46% of alumni from our first four programs (Winter 2021-22 to Summer 2023) completed a survey about their career progress since participating in MATS. This report presents key findings from the responses of these 72 alumni.49% of respondents described their current work as “Working/interning on AI...

Published on September 29, 2024 10:07 PM GMT[Meta note: quickly written, unpolished. Also, it's possible that there's some more convincing work on this topic that I'm unaware of – if so, let me know. Also also, it's possible I'm arguing with an imaginary position here and everyone already agrees with everything below.]In research discussions about LLMs, I often pick up a vibe of casual,...

Published on September 27, 2024 5:49 PM GMTThis post has some ablation results around the thesis of the ICML 2024 Mech. Interp. workshop 1st prize winning paper: The Geometry of Categorical and Hierarchical Concepts in Large Language Models The main takeaway is that the orthogonality they observe in categorical and hierarchical concepts occurs practically everywhere, even at places where it really should not.Overview of...

Published on September 27, 2024 5:49 PM GMTepistemic status: my intuition after reading and watching a bunch of stuff; no new information You take a die in your hand. If you throw it, the result will be what people usually call a random number. Let's say you get 2. What do we mean when we say that this number is random? To answer these questions,...

Published on September 27, 2024 5:27 PM GMTThe o1 system card reports:0.8% of o1-preview’s responses got flagged as being ‘deceptive’ [...] roughly two thirds of which appear to be intentional (0.38%), meaning that there was some evidence in the chain of thought that o1-preview was aware that the answer was incorrect [...] Intentional hallucinations primarily happen when o1-preview is asked to provide references to...

Published on September 27, 2024 4:47 PM GMTI recently wrote a twitter thread for Works In Progress summarizing their article on gene drives as a tool to end malaria. There’s a section in that piece that I didn’t get to highlight in the thread which I want to talk about here: the offense-defense balance of gene drives.What Are Gene Drives and How do They...

Published on September 27, 2024 2:00 PM GMTPreviously: The Fundamentals, The Gamblers, The Business We have now arrived at the topics most central to this book, aka ‘The Future.’ Rationalism and Effective Altruism (EA) The Manifest conference was also one of the last reporting trips that I made for this book. And it confirmed for me that the River is real—not just some literary...

Published on September 27, 2024 8:44 AM GMTMany sources report that cybercrime costs the global economy trillions of dollars per year. It is the top Google search result and it is quoted on Wikipedia. But I am not able to track down how the number was computed, or find criticism of these numbers.This would be insanely high if true: the world GDP is only...

Published on September 27, 2024 12:40 AM GMTWe're excited to announce the inaugural Australian AI Safety Forum, taking place on November 7-8, 2024, in Sydney, Australia. This event aims to foster the growth of the AI safety community within Australia.Apply now! Key DetailsDates: November 7-8, 2024Location: Sydney Knowledge Hub, The University of SydneyWebsite: https://aisafetyforum.auDescriptionThe Australian AI Safety Forum is a two-day interdisciplinary event scheduled for...

Published on September 26, 2024 10:45 PM GMTtl;dr: "Gell-Mann amnesia" is a cognitive bias—an observation of human failure. The mechanism behind it can be instrumentalized. Call it "Gell-Mann checks". Importantly, Gell-Mann checks are only valid insofar as the truth-finding mechanism being judged is good at generalizing.That is, you can only judge a whole mind by its part if it treats every part the same way. I...

Published on September 26, 2024 10:05 PM GMTOne of the questions I've heard asked is "how do you design a utility function that would make the AI do nothing?"  That is, how could we put a pause button on an AI so that we could pause it if we wanted to?  I had an idea about how one could do this, and am sure...