# The Book in a Few Sentences In the book [[The Coming Wave|The Coming Wave]] Mustafa Suleyman argues that we are about to face a new technological wave due mainly to the development of [[Inteligência artificial]] and Biotechnology. To this end, the author defines [[O que é uma onda tecnologica]], what were the [[3 ondas tecnológicas anteriores]] and what is [[A próxima onda]]. The main point of the book is that [[Tecnologias fundamentais se expandem rapidamente]] and that we must deal with [[O problema de contenção tecnológico]]. Therefore, we need to face the [[Armadilha do pessimismo-aversão]] to try to prevent the [[Apocalipse informacional]] and the deterioration of the relationship between [[Tecnologias e o estado]]. To do this, we need to [[Não foque nas minuciais mas no grande esquema|look not just at the detail, but at the big picture]] of what is happening in the world of technology, since [[Tecnologias são ideias]] and there are several [[Forças tecnológicas de proliferação]]. It is necessary to understand that [[We depend on technologies]] but that [[Technological solutions are not a panacea]]. Therefore, when developing technologies, we must understand the [[Drivers of the search for better technologies]] and use the [[Estratégia para identificar novas ondas tecnológicas]] in order to try to contain the development of this technology so that it is safe. # How This Book Changed Me One of the interesting parts of the book is the beginning, which was written by an [[Inteligência artificial]] and which presents a writing style that is practically indistinguishable from the author. This alone is a demonstration of the impact that this technology has on our lives. In addition, the author also brings worrying reports of the advance of synthetic biology both in terms of technical capacity and cost reduction. Taken together, these technologies clearly have the potential to have a huge impact on our future, leaving us with no idea what might emerge and what path societies and states will take. The book left me with a sense of urgency about the need to contain the development of these technologies. Not in the sense of stopping them, but in the sense of controlling their development and ensuring that they are aligned with best ethical and social practices. This includes appropriate regulation, risk assessment, and measures to mitigate misuse. However, Suleyman also acknowledges the challenges of containing these technologies due to their enormous economic and control potential. The power and profits that can be gained from their development and use will make it difficult to implement effective containment measures. Despite the challenges, I believe it is essential that we have an open and informed discussion about the future of [[Inteligência artificial|AI]] and synthetic biology # Notes - [[Tecnologias fundamentais se expandem rapidamente]] - [[Inteligência Artificial Geral]] - [[Sociedades tecnologicamente estagnadas tendem a ruir]] - [[Technological development dilemma]] - [[Biotecnologia tem ficado mais acessível]] - [[Armadilha do pessimismo-aversão]] - [[O problema de contenção tecnológico]] - [[02 - Notas de literatura/4 características fundamentais das tecnologias da próxima onda]] - [[O que é uma onda tecnologica]] - [[Tecnologias gerais são rapidamente assimiladas e esquecidas]] - [[3 ondas tecnológicas anteriores]] - [[Não foque nas minuciais mas no grande esquema]] - [[Forças tecnológicas de proliferação]] - [[Principal desafio para entender tecnologias]] - [[Tecnologias são ideias]] - [[Energia nuclear foi contida]] - [[A próxima onda]] - [[Hipótese de escala]] - [[Planejamento hierárquico]] - [[Biology and engineering will be unique to new technologies]] - [[CRISPR]] - [[Materiais biológicos podem replicar a tecnologia]] - [[Níveis de autonomia]] - [[Drivers of the search for better technologies]] - [[Abertura é fundamental para tecnologia e ciência]] - [[Soluções tecnológicas não são uma panacea]] - [[Tecnologias e o estado]] - [[Apocalipse informacional]] - [[Ganho de função na Biologia]] - [[We depend on technologies]] - [[Estratégia para identificar novas ondas tecnológicas]] # My 3 Favorite Highlights - Almost every foundational technology ever invented, from pickaxes to plows, pottery to photography, phones to planes, and everything in between, follows a single, seemingly immutable law: it gets cheaper and easier to use, and ultimately it proliferates, far and wide. - A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it's not. How it is created, used, owned, and managed all makes a difference. - Put simply, technology and political order are intimately connected. The introduction of new technologies has major political consequences. . # Destaques (432) And now we stand at the brink of another such moment as we face the rise of a coming wave of technology that includes both advanced AI and biotechnology. — Página: [202](kindle://book?action=open&asin=B0BSKW45KB&location=202) ^ref-35487 Nota: Those are two of the most transformagive technologies --- With AI, we could unlock the secrets of the universe, cure diseases that have long eluded us, and create new forms of art and culture that stretch the bounds of imagination. With biotechnology, we could engineer life to tackle diseases and transform agriculture, creating a world that is healthier and more sustainable. — Página: [205](kindle://book?action=open&asin=B0BSKW45KB&location=205) ^ref-48007 --- But on the other hand, the potential dangers of these technologies are equally vast and profound. With AI, we could create systems that are beyond our control and find ourselves at the mercy of algorithms that we don’t understand. With biotechnology, we could manipulate the very building blocks of life, potentially creating unintended consequences for both individuals and entire ecosystems. — Página: [207](kindle://book?action=open&asin=B0BSKW45KB&location=207) ^ref-52860 --- The above was written by an AI. The rest is not, although it soon could be. This is what’s coming. — Página: [215](kindle://book?action=open&asin=B0BSKW45KB&location=215) ^ref-56969 Nota: An ai can write like a human!? --- Permeating humanity’s oral traditions and ancient writings is the idea of a giant wave sweeping everything in its path, leaving the world remade and reborn. — Página: [225](kindle://book?action=open&asin=B0BSKW45KB&location=225) ^ref-21944 --- Look again at history and you can see it marked by a series of metaphorical waves: the rise and fall of empires and religions, and bursts of commerce. — Página: [231](kindle://book?action=open&asin=B0BSKW45KB&location=231) ^ref-15446 Nota: There arw other forms of waves that are not literal. --- Almost every foundational technology ever invented, from pickaxes to plows, pottery to photography, phones to planes, and everything in between, follows a single, seemingly immutable law: it gets cheaper and easier to use, and ultimately it proliferates, far and wide. — Página: [236](kindle://book?action=open&asin=B0BSKW45KB&location=236) ^ref-1169 --- Humanity’s quest to improve—ourselves, our lot, our abilities, and our influence over our environment—has powered a relentless evolution of ideas and creation. — Página: [238](kindle://book?action=open&asin=B0BSKW45KB&location=238) ^ref-35377 --- Almost every object in your line of sight has, in all likelihood, been created or altered by human intelligence. — Página: [244](kindle://book?action=open&asin=B0BSKW45KB&location=244) ^ref-36077 Nota: Vivemos num mundo nao natural --- Every principle and abstract concept, every small creative endeavor or project, every encounter in your life, has been mediated by our species’ unique and endlessly complex capacity for imagination, creativity, and reason. Human ingenuity is an astonishing thing. — Página: [247](kindle://book?action=open&asin=B0BSKW45KB&location=247) ^ref-4843 Nota: This is amazing! --- It’s no exaggeration to say the entirety of the human world depends on either living systems or our intelligence. — Página: [252](kindle://book?action=open&asin=B0BSKW45KB&location=252) ^ref-44296 --- The coming wave is defined by two core technologies: artificial intelligence (AI) and synthetic biology. — Página: [255](kindle://book?action=open&asin=B0BSKW45KB&location=255) ^ref-64885 --- From where we stand today, it appears that containing this wave—that is, controlling, curbing, or even stopping it—is not possible. This book asks why that might be true and what it means if it is. — Página: [259](kindle://book?action=open&asin=B0BSKW45KB&location=259) ^ref-21542 Nota: This is the main ideia in the book. --- What if we could distill the essence of what makes us humans so productive and capable into software, into an algorithm? — Página: [266](kindle://book?action=open&asin=B0BSKW45KB&location=266) ^ref-8782 --- To achieve this objective, we would need to create a system that could imitate and then eventually outperform all human cognitive abilities, from vision and speech to planning and imagination, and ultimately empathy and creativity. — Página: [273](kindle://book?action=open&asin=B0BSKW45KB&location=273) ^ref-44630 Nota: This is the foal of an AGI. --- AI has been climbing the ladder of cognitive abilities for decades, and it now looks set to reach human-level performance across a very wide range of tasks within the next three years. — Página: [287](kindle://book?action=open&asin=B0BSKW45KB&location=287) ^ref-62802 --- Beyond AI, a wider revolution was underway, with AI feeding a powerful, emerging generation of genetic technologies and robotics. Further progress in one area accelerates the others in a chaotic and cross-catalyzing process beyond anyone’s direct control. — Página: [291](kindle://book?action=open&asin=B0BSKW45KB&location=291) ^ref-1999 Nota: There is a positiive feedback loop --- Once matured, these emerging technologies will spread rapidly, becoming cheaper, more accessible, and widely diffused throughout society. They will offer extraordinary new medical advances and clean energy breakthroughs, creating not just new businesses but new industries and quality of life improvements in almost every imaginable area. — Página: [300](kindle://book?action=open&asin=B0BSKW45KB&location=300) ^ref-23116 Nota: Arent those difffente type of AI with different tasks?! --- The likelihood of each may be small, but the possible consequences are huge. Even a slim chance of outcomes like these requires urgent attention. — Página: [307](kindle://book?action=open&asin=B0BSKW45KB&location=307) ^ref-7532 Nota: Pequenos eventos podem ter graves consequencias --- Some countries will react to the possibility of such catastrophic risks with a form of technologically charged authoritarianism to slow the spread of these new powers. — Página: [308](kindle://book?action=open&asin=B0BSKW45KB&location=308) ^ref-58165 Nota: This is a possibility --- Equally plausible is a Luddite reaction. Bans, boycotts, and moratoriums will ensue. — Página: [311](kindle://book?action=open&asin=B0BSKW45KB&location=311) ^ref-45654 Nota: This is also possible --- Moreover, attempting to ban development of new technologies is itself a risk: technologically stagnant societies are historically unstable and prone to collapse. Eventually, they lose the capacity to solve problems, to progress. — Página: [314](kindle://book?action=open&asin=B0BSKW45KB&location=314) ^ref-3910 Nota: Is it mandatory to keep going foward?! --- This is the core dilemma: that, sooner or later, a powerful generation of technology leads humanity toward either catastrophic or dystopian outcomes. — Página: [320](kindle://book?action=open&asin=B0BSKW45KB&location=320) ^ref-55290 --- I see this as an interlocking set of technical, social, and legal mechanisms constraining and controlling technology working at every possible level: a means, in theory, of evading the dilemma. Yet even technology’s harshest critics tend to dodge this language of hard containment. That needs to change; I hope this book shows why, and hints at how. — Página: [325](kindle://book?action=open&asin=B0BSKW45KB&location=325) ^ref-23691 --- In the past, new jobs were created at the same time as old ones were made obsolete, but what if AI could simply do most of those as well? There was, I suggested, little precedent for the new forms of concentrated power that were coming. — Página: [337](kindle://book?action=open&asin=B0BSKW45KB&location=337) ^ref-6711 Nota: When technology starts to act more like humans we might have a problem. --- The message was clear, but I spelled it out anyway. “The pitchforks are coming,” I said. Coming for us, the makers of technology. It was up to us to ensure the future was better than this. — Página: [341](kindle://book?action=open&asin=B0BSKW45KB&location=341) ^ref-25821 --- One stood out. The presenter showed how the price of DNA synthesizers, which can print bespoke strands of DNA, was falling rapidly. Costing a few tens of thousands of dollars, they are small enough to sit on a bench in your garage and let people synthesize—that is, manufacture—DNA. — Página: [348](kindle://book?action=open&asin=B0BSKW45KB&location=348) ^ref-19391 Nota: This is a poblem with uninmagined consequences. --- Given the increasing availability of the tools, the presenter painted a harrowing vision: Someone could soon create novel pathogens far more transmissible and lethal than anything found in nature. — Página: [351](kindle://book?action=open&asin=B0BSKW45KB&location=351) ^ref-53689 Nota: Isso e bastante perigoso --- No one wanted to confront the implications of the hard facts and cold probabilities they’d heard. — Página: [361](kindle://book?action=open&asin=B0BSKW45KB&location=361) ^ref-51087 --- This widespread emotional reaction I was observing is something I have come to call the pessimism-aversion trap: the misguided analysis that arises when you are overwhelmed by a fear of confronting potentially dark realities, and the resulting tendency to look the other way. — Página: [367](kindle://book?action=open&asin=B0BSKW45KB&location=367) ^ref-42977 --- Spend time in tech or policy circles, and it quickly becomes obvious that head-in-the-sand is the default ideology. — Página: [396](kindle://book?action=open&asin=B0BSKW45KB&location=396) ^ref-9499 Nota: Not diving deeper in the problem is a problem. --- Waves are everywhere in human life. This one is just the latest. — Página: [406](kindle://book?action=open&asin=B0BSKW45KB&location=406) ^ref-60273 Nota: This is similar to other ones --- I am biased when it comes to the question of whether this matters. Nonetheless, having been up close to this unfurling revolution over the last decade and a half, I am convinced we’re on the cusp of the most important transformation of our lifetimes. — Página: [410](kindle://book?action=open&asin=B0BSKW45KB&location=410) ^ref-51620 Nota: This is someone from within the developmsnt of technology --- But without containment, every other aspect of technology, every discussion of its ethical shortcomings, or the benefits it could bring, is inconsequential. — Página: [416](kindle://book?action=open&asin=B0BSKW45KB&location=416) ^ref-21961 Nota: Withiou thinking thare is no understanting of its responsability --- Instead of turning away from technologies, the past is marked by a pronounced pattern of proliferation, resulting in sprawling chains of both intended and unintended consequences. — Página: [421](kindle://book?action=open&asin=B0BSKW45KB&location=421) ^ref-53349 --- I call this “the containment problem.” How do we keep a grip on the most valuable technologies ever invented as they get cheaper and spread faster than any in history? — Página: [423](kindle://book?action=open&asin=B0BSKW45KB&location=423) ^ref-30343 Nota: How do we control it?! --- details of the coming wave itself. At its heart lie two general-purpose technologies of immense promise, power, and peril: artificial intelligence and synthetic biology. — Página: [425](kindle://book?action=open&asin=B0BSKW45KB&location=425) ^ref-54005 --- The various technologies I’m speaking of share four key features that explain why this isn’t business as usual: they are inherently general and therefore omni-use, they hyper-evolve, they have asymmetric impacts, and, in some respects, they are increasingly autonomous. — Página: [429](kindle://book?action=open&asin=B0BSKW45KB&location=429) ^ref-51931 Nota: These as the characteristics of the comming wave --- In part 4 the discussion moves to what we can do about it. — Página: [440](kindle://book?action=open&asin=B0BSKW45KB&location=440) ^ref-52387 --- Our great challenge is that we still haven’t reckoned with the latter mode of failure. — Página: [450](kindle://book?action=open&asin=B0BSKW45KB&location=450) ^ref-39267 Nota: The ones were technology does more harm --- Guarding against failure means understanding and ultimately confronting what can go wrong. — Página: [455](kindle://book?action=open&asin=B0BSKW45KB&location=455) ^ref-32359 --- Today some 2 billion combustion engines are in everything from lawnmowers to container ships. Around 1.4 billion of them are in cars. They have grown steadily more accessible, efficient, powerful, and adaptable. A whole way of life, arguably a whole civilization, developed around them, from sprawling suburbs to industrial farms, drive-thru restaurants to car mod culture. Vast highways were built, sometimes right through cities, severing neighborhoods but connecting far-flung regions. — Página: [495](kindle://book?action=open&asin=B0BSKW45KB&location=495) ^ref-24857 --- Once there was momentum, the spread of the internal combustion engine became unstoppable. From a few oil-soaked German workshops grew a technology that has affected every human being on earth. — Página: [503](kindle://book?action=open&asin=B0BSKW45KB&location=503) ^ref-40243 Nota: Some technologies arw similar to this onw --- Technology has a clear, inevitable trajectory: mass diffusion in great roiling waves. — Página: [507](kindle://book?action=open&asin=B0BSKW45KB&location=507) ^ref-59356 --- Over time demand for the best new products and services grows, driving competition to produce cheaper versions bursting with yet more features. This in turn drives yet more demand for the technologies that create them, and they also become easier and cheaper to — Página: [509](kindle://book?action=open&asin=B0BSKW45KB&location=509) ^ref-24106 --- So, what is a wave? Put simply, a wave is a set of technologies coming together around the same time, powered by one or several new general-purpose technologies with profound societal implications. — Página: [515](kindle://book?action=open&asin=B0BSKW45KB&location=515) ^ref-40672 Nota: This concept is interesting --- By “general-purpose technologies,” I mean those that enable seismic advances in what human beings can do. — Página: [516](kindle://book?action=open&asin=B0BSKW45KB&location=516) ^ref-13040 --- From the very beginning, we are never separate from the waves of technology we create. We evolve together, in symbiosis. — Página: [520](kindle://book?action=open&asin=B0BSKW45KB&location=520) ^ref-12857 Nota: Do we use technology or does it uses us? --- Stonework and fire were proto-general-purpose technologies, meaning they were pervasive, in turn enabling new inventions, goods, and organizational behaviors. — Página: [531](kindle://book?action=open&asin=B0BSKW45KB&location=531) ^ref-35291 --- The irony of general-purpose technologies is that, before long, they become invisible and we take them for granted. — Página: [535](kindle://book?action=open&asin=B0BSKW45KB&location=535) ^ref-18012 --- Language, agriculture, writing—each was a general-purpose technology at the center of an early wave. These three waves formed the foundation of civilization as we know it. Now we take them for granted. — Página: [536](kindle://book?action=open&asin=B0BSKW45KB&location=536) ^ref-5056 --- number of general-purpose technologies — Página: [538](kindle://book?action=open&asin=B0BSKW45KB&location=538) ^ref-52457 Nota: What are they? --- The Agricultural Revolution (9000 –7500 BCE), one of history’s most significant waves, marked the arrival of two massive general-purpose technologies that gradually replaced the nomadic, hunter-gatherer way of life: the domestication of plants and animals. — Página: [550](kindle://book?action=open&asin=B0BSKW45KB&location=550) ^ref-36062 --- Eventually, this coupled with a new revolution in tools—hoes and plows. These simple innovations marked the beginning of modern civilizations. — Página: [553](kindle://book?action=open&asin=B0BSKW45KB&location=553) ^ref-386 Nota: Civikization started with agriculture --- Over time, this dynamic accelerated. Beginning around the 1770s in Europe, the first wave of the Industrial Revolution combined steam power, mechanized looms, the factory system, and canals. — Página: [558](kindle://book?action=open&asin=B0BSKW45KB&location=558) ^ref-49547 --- In the 1840s came the age of railways, telegraphs, and steamships, and a bit later steel and machine tools; together they formed the First Industrial Revolution. — Página: [560](kindle://book?action=open&asin=B0BSKW45KB&location=560) ^ref-19808 --- Then, just a few decades later, came the Second Industrial Revolution. You’ll be familiar with its greatest hits: the internal combustion engine, chemical engineering, powered flight, and electricity. — Página: [561](kindle://book?action=open&asin=B0BSKW45KB&location=561) ^ref-50622 --- This isn’t, however, an orderly process. Technological waves don’t arrive with the neat predictability of the tides. Over the long term, waves erratically intersect and intensify. — Página: [564](kindle://book?action=open&asin=B0BSKW45KB&location=564) ^ref-27424 Nota: Technology is based on caos --- They are part of us. There is no such thing as a non-technological human being. — Página: [570](kindle://book?action=open&asin=B0BSKW45KB&location=570) ^ref-18275 Nota: We are one with technology --- Most people in technology are stuck in the minutiae of today and dreaming of tomorrow. It is tempting to think of inventions in discrete and lucky moments. But do so and you’ll miss the stark patterns of history, the sheer, almost innate tendency for technology’s waves to come again and again. — Página: [578](kindle://book?action=open&asin=B0BSKW45KB&location=578) ^ref-14795 --- General-purpose technologies become waves when they diffuse widely. Without an epic and near-uncontrolled global diffusion, it’s not a wave; it’s a historical curiosity. — Página: [584](kindle://book?action=open&asin=B0BSKW45KB&location=584) ^ref-43943 --- Proliferation is catalyzed by two forces: demand and the resulting cost decreases, each of which drives technology to become even better and cheaper. — Página: [606](kindle://book?action=open&asin=B0BSKW45KB&location=606) ^ref-63054 --- The long and intricate dialogue of science and technology produces a chain of insights, breakthroughs, and tools that build and reinforce over time, productive recombinations that drive the future. — Página: [607](kindle://book?action=open&asin=B0BSKW45KB&location=607) ^ref-49043 Nota: This is a good relationship between the two. --- Civilization’s appetite for useful and cheaper technologies is boundless. This will not change. — Página: [614](kindle://book?action=open&asin=B0BSKW45KB&location=614) ^ref-1692 --- Transistors are now produced in the tens of trillions per second, at billionths of a dollar per transistor: the fastest, most extensive proliferation in history. — Página: [638](kindle://book?action=open&asin=B0BSKW45KB&location=638) ^ref-8597 --- It took smartphones a few years to go from niche product to utterly essential item for two-thirds of the planet. — Página: [642](kindle://book?action=open&asin=B0BSKW45KB&location=642) ^ref-18090 --- Now humans produce hundreds of billions of emails, messages, images, and videos daily and store them in the cloud. Eighteen million gigabytes of data are added to the global sum every single minute of every day. — Página: [646](kindle://book?action=open&asin=B0BSKW45KB&location=646) ^ref-56596 --- Our phones are the first thing we see in the morning and the last at night. Every aspect of human life is affected: they help us find love and new friends while turbocharging supply chains. They influence who gets elected and how, where our money is invested, our children’s self-esteem, our music tastes, our fashion, our food, and everything in between. — Página: [650](kindle://book?action=open&asin=B0BSKW45KB&location=650) ^ref-24628 --- No previous wave has mushroomed as quickly, but the historical pattern nonetheless repeats. At first it seems impossible and unimaginable. Then it appears inevitable. And each wave grows bigger and stronger still. — Página: [655](kindle://book?action=open&asin=B0BSKW45KB&location=655) ^ref-61597 --- History tells us that technology diffuses, inevitably, eventually to almost everywhere, from the first campfires to the fires of the Saturn V rocket, from the first scrawled letters to the endless text of the internet. — Página: [661](kindle://book?action=open&asin=B0BSKW45KB&location=661) ^ref-38623 --- Technology’s unavoidable challenge is that its makers quickly lose control over the path their inventions take once introduced to the world. — Página: [672](kindle://book?action=open&asin=B0BSKW45KB&location=672) ^ref-17579 --- What people actually do with your invention, however well intentioned, can never be guaranteed. — Página: [675](kindle://book?action=open&asin=B0BSKW45KB&location=675) ^ref-44455 --- Understanding technology is, in part, about trying to understand its unintended consequences, to predict not just positive spillovers but “revenge effects.” — Página: [683](kindle://book?action=open&asin=B0BSKW45KB&location=683) ^ref-41195 --- As technology proliferates, more people can use it, adapt it, shape it however they like, in chains of causality beyond any individual’s comprehension. — Página: [687](kindle://book?action=open&asin=B0BSKW45KB&location=687) ^ref-14986 --- Technology’s problem here is a containment problem. If this aspect cannot be eliminated, it might be curtailed. Containment is the overarching ability to control, limit, and, if need be, close down technologies at any stage of their development or deployment. It means, in some circumstances, the ability to stop a technology from proliferating in the first place, checking the ripple of unintended consequences (both good and bad). — Página: [692](kindle://book?action=open&asin=B0BSKW45KB&location=692) ^ref-42557 --- In most cases, containment is about meaningful control, the capability to stop a use case, change a research direction, or deny access to harmful actors. — Página: [700](kindle://book?action=open&asin=B0BSKW45KB&location=700) ^ref-25097 --- Containment encompasses regulation, better technical safety, new governance and ownership models, and new modes of accountability and transparency, all as necessary (but not sufficient) precursors to safer technology. — Página: [712](kindle://book?action=open&asin=B0BSKW45KB&location=712) ^ref-30432 --- Think of containment, then, as a set of interlinked and mutually reinforcing technical, cultural, legal, and political mechanisms for maintaining societal control of technology during a time of exponential change; an architecture up to the task of containing what would have once been centuries or millennia of technological change happening now in a matter of years or even months, where consequences ricochet around the world in seconds. — Página: [715](kindle://book?action=open&asin=B0BSKW45KB&location=715) ^ref-57392 --- In hindsight, waves might appear smooth and inevitable. But there is an almost infinite array of small, local, and often arbitrary factors that affect a technology’s trajectory. — Página: [737](kindle://book?action=open&asin=B0BSKW45KB&location=737) ^ref-60973 --- Where there is demand, technology always breaks out, finds traction, builds users. — Página: [757](kindle://book?action=open&asin=B0BSKW45KB&location=757) ^ref-28530 --- As technologies let you do more, for less, their appeal only grows, along with their adoption. — Página: [766](kindle://book?action=open&asin=B0BSKW45KB&location=766) ^ref-46204 --- Technologies are ideas, and ideas cannot be eliminated. — Página: [774](kindle://book?action=open&asin=B0BSKW45KB&location=774) ^ref-57903 --- Technology is an eternally dangling carrot, constantly promising more, better, easier, cheaper. — Página: [775](kindle://book?action=open&asin=B0BSKW45KB&location=775) ^ref-9112 --- That nuclear technology remained contained was no accident; it was a conscious nonproliferation policy of the nuclear powers, helped by the fact that nuclear weapons are incredibly complex and expensive to produce. — Página: [798](kindle://book?action=open&asin=B0BSKW45KB&location=798) ^ref-32585 --- Nuclear history is still a chilling succession of accidents, near misses, and misunderstandings. — Página: [817](kindle://book?action=open&asin=B0BSKW45KB&location=817) ^ref-20022 --- Although its history is one of enabling people to do more, increasing capabilities, driving improvements in well-being, it’s not a one-sided story: Technology creates more lethal and destructive weapons as well as better tools. It produces losers, eliminates some jobs and ways of life, and creates harm up to the planetary, existential scale of climate change. — Página: [868](kindle://book?action=open&asin=B0BSKW45KB&location=868) ^ref-1278 --- As long as a technology is useful, desirable, affordable, accessible, and unsurpassed, it survives and spreads and those features compound. — Página: [873](kindle://book?action=open&asin=B0BSKW45KB&location=873) ^ref-4692 --- While technology doesn’t tell us when, or how, or whether to walk through the doors it opens, sooner or later we do seem to walk through them. — Página: [874](kindle://book?action=open&asin=B0BSKW45KB&location=874) ^ref-64380 --- By and large we’ve let the waves wash over us, managing on an uncoordinated, ad hoc basis, accepting that capabilities spreading inevitably and uncontrollably is, whether welcomed or reviled, a fact of life. — Página: [881](kindle://book?action=open&asin=B0BSKW45KB&location=881) ^ref-36702 --- In the coming decades, a new wave of technology will force us to confront the most foundational questions our species has ever faced. — Página: [884](kindle://book?action=open&asin=B0BSKW45KB&location=884) ^ref-17449 --- For most of history, the challenge of technology lay in creating and unleashing its power. That has now flipped: the challenge of technology today is about containing its unleashed power, ensuring it continues to serve us and our planet. — Página: [889](kindle://book?action=open&asin=B0BSKW45KB&location=889) ^ref-1762 --- The “general” in AGI refers to the technology’s intended broad scope; we wanted to build truly general learning agents that could exceed human performance at most cognitive tasks. — Página: [901](kindle://book?action=open&asin=B0BSKW45KB&location=901) ^ref-10143 --- For the first time I’d witnessed a very simple, very elegant system that could learn valuable knowledge, arguably a strategy that wasn’t obvious to many humans. — Página: [918](kindle://book?action=open&asin=B0BSKW45KB&location=918) ^ref-59378 --- When IBM’s Deep Blue beat Garry Kasparov at chess in 1997, it used the so-called brute-force technique, where an algorithm aims to systematically crunch through as many possible moves as it can. — Página: [935](kindle://book?action=open&asin=B0BSKW45KB&location=935) ^ref-36136 --- AlphaGo initially learned by watching 150,000 games played by human experts. Once we were satisfied with its initial performance, the key next step was creating lots of copies of AlphaGo and getting it to play against itself over and over. — Página: [938](kindle://book?action=open&asin=B0BSKW45KB&location=938) ^ref-59877 --- Until recently, the history of technology could be encapsulated in a single phrase: humanity’s quest to manipulate atoms. — Página: [960](kindle://book?action=open&asin=B0BSKW45KB&location=960) ^ref-46131 --- First bits and then increasingly genes supplanted atoms as the building blocks of invention. — Página: [969](kindle://book?action=open&asin=B0BSKW45KB&location=969) ^ref-42170 --- They began parallel revolutions—those of bits and genes—that dealt in the currency of information, working at new levels of abstraction and complexity. Eventually, the technologies matured and gave us everything from smartphones to genetically modified rice. But there were limits to what we could do. — Página: [971](kindle://book?action=open&asin=B0BSKW45KB&location=971) ^ref-8731 --- artificial intelligence and synthetic biology. For the first time core components of our technological ecosystem directly address two foundational properties of our world: intelligence and life. In other words, technology is undergoing a phase transition. No longer simply a tool, it’s going to engineer life and rival—and surpass—our own intelligence. — Página: [976](kindle://book?action=open&asin=B0BSKW45KB&location=976) ^ref-37209 --- While AI and synthetic biology are the coming wave’s central general-purpose technologies, a bundle of technologies with unusually powerful ramifications surrounds them, encompassing quantum computing, robotics, nanotechnology, and the potential for abundant energy, among others. — Página: [984](kindle://book?action=open&asin=B0BSKW45KB&location=984) ^ref-14597 --- Technology is a set of evolving ideas. New technologies evolve by colliding and combining with other technologies. — Página: [989](kindle://book?action=open&asin=B0BSKW45KB&location=989) ^ref-8394 --- Each technology described here intersects with, buttresses, and boosts the others in ways that make it difficult to predict their impact in advance. They are all deeply entangled and will grow more so. — Página: [998](kindle://book?action=open&asin=B0BSKW45KB&location=998) ^ref-15676 --- We see this now on a large scale, with AI helping design better chips and production techniques that enable more sophisticated forms of AI and so on. — Página: [1002](kindle://book?action=open&asin=B0BSKW45KB&location=1002) ^ref-47732 --- In the era of abundant venture capital, distinguishing shiny objects from genuine breakthroughs is not so straightforward. — Página: [1011](kindle://book?action=open&asin=B0BSKW45KB&location=1011) ^ref-54369 --- Nonetheless, understanding the coming wave is not about making a snap judgment about where things will be this or that year; it is about closely tracking the development of multiple exponential curves over decades, projecting them into the future, and asking what that means. — Página: [1017](kindle://book?action=open&asin=B0BSKW45KB&location=1017) ^ref-59996 --- Deep learning uses neural networks loosely modeled on those of the human brain. In simple terms, these systems “learn” when their networks are “trained” on large amounts of data. — Página: [1030](kindle://book?action=open&asin=B0BSKW45KB&location=1030) ^ref-24751 --- Everywhere you look, software has eaten the world, opening the path for collecting and analyzing vast amounts of data. — Página: [1066](kindle://book?action=open&asin=B0BSKW45KB&location=1066) ^ref-42979 --- Within a few years AIs will be able to talk about, reason over, and even act in the same world that we do. — Página: [1071](kindle://book?action=open&asin=B0BSKW45KB&location=1071) ^ref-52066 Nota: It this real? --- A big part of what makes humans intelligent is that we look at the past to predict what might happen in the future. — Página: [1095](kindle://book?action=open&asin=B0BSKW45KB&location=1095) ^ref-6620 --- LLMs take advantage of the fact that language data comes in a sequential order. Each unit of information is in some way related to data earlier in a series. — Página: [1100](kindle://book?action=open&asin=B0BSKW45KB&location=1100) ^ref-4750 --- The challenge lies in designing an algorithm that “knows where to look” for signals in a given sentence. What are the key words, the most salient elements of a sentence, and how do they relate to one another? In AI this notion is commonly referred to as “attention.” — Página: [1103](kindle://book?action=open&asin=B0BSKW45KB&location=1103) ^ref-24470 --- In the attention map, every token bears some relationship to every token before it, and for a given input sentence the strength of this relationship describes something about the importance of that token in the sentence. — Página: [1109](kindle://book?action=open&asin=B0BSKW45KB&location=1109) ^ref-53485 --- Based on this, it then suggests which tokens should come next in the sequence, what output logically follows the input. In other words, it autocompletes what might come next. — Página: [1114](kindle://book?action=open&asin=B0BSKW45KB&location=1114) ^ref-38293 Nota: It is just autocompletting stuff --- All of this is just the start. We are only beginning to scratch at the profound impact [[Grandes modelos de linguagem|large language models]] are about to have. If DQN and AlphaGo were the early signs of something lapping at the shore, [[ChatGPT|ChatGPT]] and LLMs are the first signs of the wave beginning to crash around us. — Página: [1132](kindle://book?action=open&asin=B0BSKW45KB&location=1132) ^ref-41480 Nota: As coisas mudarao rapidamente, mesmo! --- This kind of vast, almost instantaneous consumption of information is not just difficult to comprehend; it’s truly alien. — Página: [1145](kindle://book?action=open&asin=B0BSKW45KB&location=1145) ^ref-6004 Nota: Muitos dados nao necessariamente corretos pode ser um problema --- This is the reality of the coming wave. It advances at an unprecedented rate, taking even its proponents by surprise. — Página: [1158](kindle://book?action=open&asin=B0BSKW45KB&location=1158) ^ref-53548 --- To get a sense of one petaFLOP, imagine a billion people each holding a million calculators, doing a complex multiplication, and hitting “equals” at the same time. I find this extraordinary. — Página: [1164](kindle://book?action=open&asin=B0BSKW45KB&location=1164) ^ref-63696 --- While this is true, it misses the fact that in AI training we can just keep connecting larger and larger arrays of chips, daisy-chaining them into massively parallel supercomputers. — Página: [1170](kindle://book?action=open&asin=B0BSKW45KB&location=1170) ^ref-28715 --- Researchers meanwhile see more and more evidence for “the scaling hypothesis,” which predicts that the main driver of performance is, quite simply, to go big and keep going bigger. — Página: [1172](kindle://book?action=open&asin=B0BSKW45KB&location=1172) ^ref-7035 --- It’s true that we are, more widely, complex emotional and social beings. But humans’ ability to complete given tasks—human intelligence itself—is very much a fixed target, as large and multifaceted as it is. Unlike the scale of available compute, our brains do not radically change year by year. In time this gap will be closed. — Página: [1182](kindle://book?action=open&asin=B0BSKW45KB&location=1182) ^ref-62776 --- At the other end of the spectrum, you can now create a nanoLLM based on just three hundred lines of code capable of generating fairly plausible imitations of Shakespeare. In short, AI increasingly does more with — Página: [1195](kindle://book?action=open&asin=B0BSKW45KB&location=1195) ^ref-46241 --- Over the next decade, there will almost certainly be dramatic capability increases, even as costs further decline by multiple orders of magnitude. Progress is accelerating so much that benchmarks get eclipsed before new ones are even made. — Página: [1198](kindle://book?action=open&asin=B0BSKW45KB&location=1198) ^ref-2984 --- EleutherAI, — Página: [1201](kindle://book?action=open&asin=B0BSKW45KB&location=1201) ^ref-23896 Nota: Procurar por essa empresa --- burgeoning field of generative AI. — Página: [1209](kindle://book?action=open&asin=B0BSKW45KB&location=1209) ^ref-43191 --- The potential for harm, abuse, and misinformation is real. But the positive news is that many of these issues are being improved with larger and more powerful models. — Página: [1225](kindle://book?action=open&asin=B0BSKW45KB&location=1225) ^ref-63813 Nota: Is this true?! --- but rather that AI had reached a point where it could convince otherwise intelligent people—indeed, someone with a real understanding of how it actually worked—that it was conscious. — Página: [1266](kindle://book?action=open&asin=B0BSKW45KB&location=1266) ^ref-43343 Nota: As cezes saber cono algo funciona nao significa que voce compreende aquilo e menos ainda que suas crencas nao sao cpazes de afetar seu julgamento. --- There’s a recurrent problem with making sense of progress in AI. We quickly adapt, even to breakthroughs that astound us initially, and within no time they seem routine, even mundane. — Página: [1271](kindle://book?action=open&asin=B0BSKW45KB&location=1271) ^ref-2909 --- In the words of John McCarthy, who coined the term “artificial intelligence”: “As soon as it works, no one calls it AI anymore.” AI is—as those of us building it like to joke—“what computers can’t do.” Once they can, it’s just software. — Página: [1273](kindle://book?action=open&asin=B0BSKW45KB&location=1273) ^ref-38577 --- The eminent professor of complexity Melanie Mitchell rightly points out that present-day AI systems have many limitations: they can’t transfer knowledge from one domain to another, provide quality explanations of their decision-making process, and so on. — Página: [1281](kindle://book?action=open&asin=B0BSKW45KB&location=1281) ^ref-63120 --- People interpret unsolved problems as evidence of lasting limitations; I see an unfolding research process. — Página: [1285](kindle://book?action=open&asin=B0BSKW45KB&location=1285) ^ref-32409 --- Over the last decade, intellectual and political elites in tech circles became absorbed by the idea that a recursively self-improving AI would lead to an “intelligence explosion” known as the Singularity. — Página: [1301](kindle://book?action=open&asin=B0BSKW45KB&location=1301) ^ref-9514 --- Debating timelines to AGI is an exercise in reading crystal balls. — Página: [1305](kindle://book?action=open&asin=B0BSKW45KB&location=1305) ^ref-34406 --- I’ve gone to countless meetings trying to raise questions about synthetic media and misinformation, or privacy, or lethal autonomous weapons, and instead spent the time answering esoteric questions from otherwise intelligent people about consciousness, the Singularity, and other matters irrelevant to our world right now. — Página: [1307](kindle://book?action=open&asin=B0BSKW45KB&location=1307) ^ref-48189 Nota: Algumas questoes deveriam ser prioritarias. --- AGI is binary—you either have it or you don’t, a single, identifiable threshold that would be crossed by a given system. I’ve always thought that this characterization is wrong. Rather, it’s a gradual transition, where AI systems become increasingly capable, consistently nudging toward AGI. It’s not a vertical takeoff so much as a smooth evolution already underway. — Página: [1310](kindle://book?action=open&asin=B0BSKW45KB&location=1310) ^ref-20550 --- All that matters is what the system can do. Focus on that, and the real challenge comes into view: systems can do more, much more, with every passing day. — Página: [1315](kindle://book?action=open&asin=B0BSKW45KB&location=1315) ^ref-62837 --- In a paper published in 1950, the computer scientist Alan Turing suggested a legendary test for whether an AI exhibited human-level intelligence. When AI could display humanlike conversational abilities for a lengthy period of time, such that a human interlocutor couldn’t tell they were speaking to a machine, the test would be passed: the AI, conversationally akin to a human, deemed intelligent. — Página: [1318](kindle://book?action=open&asin=B0BSKW45KB&location=1318) ^ref-62205 --- We don’t just care about what a machine can say; we also care about what it can do. — Página: [1324](kindle://book?action=open&asin=B0BSKW45KB&location=1324) ^ref-12271 Nota: At the moment there is not a lot of use cases outside the narrow vison of it. --- Put simply, passing a Modern Turing Test would involve something like the following: an AI being able to successfully act on the instruction “Go make $1 million on Amazon in a few months with just a $100,000 investment.” — Página: [1327](kindle://book?action=open&asin=B0BSKW45KB&location=1327) ^ref-32903 --- Should my Modern Turing Test for the twenty-first century be met, the implications for the global economy are profound. — Página: [1334](kindle://book?action=open&asin=B0BSKW45KB&location=1334) ^ref-28317 --- We’ll come to robots later, but the truth is that for a vast range of tasks in the world economy today all you need is access to a computer; — Página: [1338](kindle://book?action=open&asin=B0BSKW45KB&location=1338) ^ref-49742 --- The challenge is in advancing what AI developers call hierarchical planning, stitching multiple goals and subgoals and capabilities into a seamless process toward a singular end. — Página: [1340](kindle://book?action=open&asin=B0BSKW45KB&location=1340) ^ref-11147 Nota: When this is possible, we might have a greate problem --- I think of this as “artificial capable intelligence” (ACI), the point at which AI can achieve complex goals and tasks with minimal oversight. — Página: [1348](kindle://book?action=open&asin=B0BSKW45KB&location=1348) ^ref-20367 --- ACI represents the next stage of AI’s evolution. A system that not only could recognize and generate novel images, audio, and language appropriate to a given context, but also would be interactive—operating in real time, with real users. — Página: [1354](kindle://book?action=open&asin=B0BSKW45KB&location=1354) ^ref-58842 --- All of this, then, enables tool use and the emergence of real capability to perform a wide range of complex, useful actions. It adds up to a genuinely capable AI, an ACI. — Página: [1359](kindle://book?action=open&asin=B0BSKW45KB&location=1359) ^ref-52611 --- The future of AI is, at least in one sense, fairly easy to predict. Over the next five years, vast resources will continue to be invested. Some of the smartest people on the planet are working on these problems. Orders of magnitude more computation will train the top models. All of this will lead to more dramatic leaps forward, including breakthroughs toward AI that can imagine, reason, plan, and exhibit common sense. — Página: [1365](kindle://book?action=open&asin=B0BSKW45KB&location=1365) ^ref-14865 --- It won’t be long before AI can transfer what it “knows” from one domain to another, seamlessly, as humans do. — Página: [1368](kindle://book?action=open&asin=B0BSKW45KB&location=1368) ^ref-9076 --- It’s not just a tool or platform but a transformative meta-technology, the technology behind technology and everything else, itself a maker of tools and platforms, not just a system but a generator of systems of any and all kinds. — Página: [1372](kindle://book?action=open&asin=B0BSKW45KB&location=1372) ^ref-29356 --- Just as everything from the steam engine to the microprocessor was driven by an intense dialogue between physics and engineering, so the coming decades will be defined by a convergence of biology and engineering. — Página: [1385](kindle://book?action=open&asin=B0BSKW45KB&location=1385) ^ref-22972 --- Much of civilization would have been impossible without selective breeding—the insistent process of refining crops and animals to select for more desirable traits. — Página: [1393](kindle://book?action=open&asin=B0BSKW45KB&location=1393) ^ref-28056 --- The Economist calls the Carlson curve: the epic collapse in costs for sequencing DNA. — Página: [1414](kindle://book?action=open&asin=B0BSKW45KB&location=1414) ^ref-59780 --- [[CRISPR]] gene editing (the acronym stands for clustered regularly interspaced short palindromic repeats) — Página: [1420](kindle://book?action=open&asin=B0BSKW45KB&location=1420) ^ref-46086 --- [[CRISPR]] edits DNA sequences with the help of Cas9, an enzyme acting as a pair of finely tuned DNA scissors, cutting parts of a DNA strand for precise genetic editing and modification of anything ranging from a minute bacterium to large mammals like human beings, with edits anywhere from tiny changes to significant interventions in the genome. — Página: [1423](kindle://book?action=open&asin=B0BSKW45KB&location=1423) ^ref-5785 --- After the initial [[CRISPR]] paper was published, progress applying it was rapid; the first gene-edited plants were created within a year, the first animals—mice—even before that. — Página: [1427](kindle://book?action=open&asin=B0BSKW45KB&location=1427) ^ref-34873 --- [[CRISPR]] use cases are multiplying, from tomatoes ultrarich in vitamin D to treatments for conditions including sickle-cell disease and beta-thalassemia (a blood disorder producing abnormal hemoglobin). — Página: [1433](kindle://book?action=open&asin=B0BSKW45KB&location=1433) ^ref-15418 --- Companies like the Odin will sell you a genetic engineering kit including live frogs and crickets for $1,999, while another kit includes a mini-centrifuge, a polymerase chain reaction machine, and all the reagents and materials you need to get going. — Página: [1439](kindle://book?action=open&asin=B0BSKW45KB&location=1439) ^ref-32169 Nota: This is both incredible and dangerous --- If sequencing is reading, synthesizing is writing. And writing doesn’t just involve reproducing known strands of DNA; it also enables scientists to write new strands, to engineer life itself. — Página: [1447](kindle://book?action=open&asin=B0BSKW45KB&location=1447) ^ref-52281 --- Where nature takes a long and winding path to reach extraordinarily effective results, this bio-revolution puts the power of concentrated design at the heart of these self-replicating, self-healing, and evolving processes. — Página: [1458](kindle://book?action=open&asin=B0BSKW45KB&location=1458) ^ref-25103 --- Biotech is following a similar trajectory, only at a much earlier phase; organisms will soon be designed and produced with the precision and scale of today’s computer chips and software. — Página: [1467](kindle://book?action=open&asin=B0BSKW45KB&location=1467) ^ref-32930 Nota: Isso e interessante e assustador ao mesmo tempo --- Countless experiments are underway in the strange and emerging landscape of synthetic biology: viruses that produce batteries, proteins that purify dirty water, organs grown in vats, algae that draw down carbon from the atmosphere, plants that consume toxic waste. — Página: [1476](kindle://book?action=open&asin=B0BSKW45KB&location=1476) ^ref-12388 --- No one can fully say what the consequences might be. — Página: [1480](kindle://book?action=open&asin=B0BSKW45KB&location=1480) ^ref-43020 Nota: E as vezes isso poder ir no caminho errado --- Eventually, it might be possible to reconfigure ourselves to enhance our immune responses. That, in turn, might open the door to even more ambitious experimentation like longevity and regenerative technologies, already a burgeoning area of research. — Página: [1489](kindle://book?action=open&asin=B0BSKW45KB&location=1489) ^ref-5750 --- A world where life spans are set to average a hundred years or more is achievable in the next decades. Nor is this just about longer life; it’s about healthier lives as we get older. — Página: [1497](kindle://book?action=open&asin=B0BSKW45KB&location=1497) ^ref-4984 --- It won’t be long before “gene doping” becomes a live issue in sports, education, and professional life. — Página: [1501](kindle://book?action=open&asin=B0BSKW45KB&location=1501) ^ref-59541 --- Next-generation DNA printers will produce DNA with an increasing degree of precision. If improvements can be made in not only expressing that DNA but then using it to genetically engineer a diverse array of new organisms, automating and scaling the processes, a device or set of devices could, theoretically, produce an enormous range of biological materials and constructions using only a few basic inputs. — Página: [1523](kindle://book?action=open&asin=B0BSKW45KB&location=1523) ^ref-53563 --- There is still a long way to go before this technology can be harnessed. But all the functional parts of a computer—data storage, information transmission, and a basic system of logic—can in principle be replicated using biological materials. — Página: [1532](kindle://book?action=open&asin=B0BSKW45KB&location=1532) ^ref-27876 --- Forty-five percent of the global disease burden could be met with “science that is conceivable today.” — Página: [1536](kindle://book?action=open&asin=B0BSKW45KB&location=1536) ^ref-13047 --- Understand proteins, and you’ve taken a giant leap forward in understanding—and mastering—biology. But there’s a problem. Simply knowing the DNA sequence isn’t enough to know how a protein works. Instead, you need to understand how it folds. Its shape, formed by this knotted folding, is core to its function: collagen in our tendons has a rope-like structure, while enzymes have pockets to hold the molecules they act on. — Página: [1541](kindle://book?action=open&asin=B0BSKW45KB&location=1541) ^ref-11261 --- Whereas once it might have taken researchers weeks or months to determine a protein’s shape and function, that process can now begin in a matter of seconds. This is what we mean by exponential change. This is what the coming wave makes possible. — Página: [1571](kindle://book?action=open&asin=B0BSKW45KB&location=1571) ^ref-53624 --- All intelligence to date has come from life. Call them synthetic intelligence and artificial life and they still mean the same thing. Both fields are about re-creating, engineering these utterly foundational and interrelated concepts, two core attributes of humanity; change the view and they become one single project. — Página: [1576](kindle://book?action=open&asin=B0BSKW45KB&location=1576) ^ref-34607 --- It likely won’t be too long before neural “laces” made from carbon nanotubes plug us directly into the digital world. — Página: [1590](kindle://book?action=open&asin=B0BSKW45KB&location=1590) ^ref-59200 --- Technological waves are bigger than just one or two general-purpose technologies. They are clusters of technologies arriving at around the same time, anchored by one or more general-purpose technologies but extending far beyond them. — Página: [1599](kindle://book?action=open&asin=B0BSKW45KB&location=1599) ^ref-11605 --- Rather, they develop in rippling amplificatory loops. Where you find a general-purpose technology, you also find other technologies developing in constant dialogue, spurred on by it. — Página: [1606](kindle://book?action=open&asin=B0BSKW45KB&location=1606) ^ref-1092 --- The John Deere company still makes agricultural technology today. You might be thinking tractors, sprinklers, and combines, and it’s true that John Deere does make all these things. Increasingly, though, the company builds robots. — Página: [1624](kindle://book?action=open&asin=B0BSKW45KB&location=1624) ^ref-15358 --- As with AI, robotics proved much more difficult in practice than early engineers assumed. The real world is a strange, uneven, unexpected, and unstructured environment, exquisitely sensitive to things like pressure: — Página: [1639](kindle://book?action=open&asin=B0BSKW45KB&location=1639) ^ref-56235 --- But as we’ve seen in so many other applications of machine learning, what starts with close human supervision ends up with the AI learning to do the task better by itself, eventually generalizing to new settings. — Página: [1653](kindle://book?action=open&asin=B0BSKW45KB&location=1653) ^ref-30312 --- Robots can operate with precision in a far greater range of environments for far longer periods than humans. — Página: [1669](kindle://book?action=open&asin=B0BSKW45KB&location=1669) ^ref-50514 --- I think we’re now getting to the point where AI is pushing robots toward their original promise: machines that can replicate all the physical actions of a human and more. — Página: [1671](kindle://book?action=open&asin=B0BSKW45KB&location=1671) ^ref-35659 Nota: Maybe they are capable, but this does bot mean that they are easily deployable --- AIs are products of bits and code, existing within simulations and servers. Robots are their bridge, their interface with the real world. — Página: [1690](kindle://book?action=open&asin=B0BSKW45KB&location=1690) ^ref-40615 --- While very much a nascent technology, there are huge implications when quantum computing does materialize. Its key attraction is that each additional qubit doubles a machine’s total computing power. — Página: [1702](kindle://book?action=open&asin=B0BSKW45KB&location=1702) ^ref-37930 --- In theory, solving any optimization problem could be greatly sped up—almost anything that involves minimizing costs in complex circumstances, whether that’s efficiently loading a truck or running a national economy. — Página: [1714](kindle://book?action=open&asin=B0BSKW45KB&location=1714) ^ref-34066 --- Energy rivals intelligence and life in its fundamental importance. Modern civilization relies on vast amounts of it. — Página: [1727](kindle://book?action=open&asin=B0BSKW45KB&location=1727) ^ref-11970 --- In 2000, solar energy cost $4.88 per watt, but by 2019 it had fallen to just 38 cents. — Página: [1737](kindle://book?action=open&asin=B0BSKW45KB&location=1737) ^ref-30554 --- As the elements of AI, advanced biotechnology, quantum computing, and robotics combine in new ways, prepare for breakthroughs like advanced nanotechnology, a concept that takes the ever-growing precision of technology to its logical conclusion. — Página: [1755](kindle://book?action=open&asin=B0BSKW45KB&location=1755) ^ref-14410 --- The dream of the physical universe rendered a completely malleable platform, the plaything of tiny, dexterous nanobots or effortless replicators, is still the province, — Página: [1765](kindle://book?action=open&asin=B0BSKW45KB&location=1765) ^ref-54603 --- At its core, the coming wave is a story of the proliferation of power. If the last wave reduced the costs of broadcasting information, this one reduces the costs of acting on it, giving rise to technologies that go from sequencing to synthesis, reading to writing, editing to creating, imitating conversations to leading them. — Página: [1769](kindle://book?action=open&asin=B0BSKW45KB&location=1769) ^ref-42808 --- We are reaching the decisive point of what, in geological or human evolutionary timescales, is a technological explosion unfolding in successive waves, a compounding, accelerating cycle of innovation steadily getting faster and more impactful, breaking first over a period of thousands of years, then hundreds of years, and now single years or even months. — Página: [1778](kindle://book?action=open&asin=B0BSKW45KB&location=1778) ^ref-50292 Nota: Things are moving fast, too fast i dare to say --- Drones and AI played a small but important part in the early days of the conflict in Ukraine, new technologies with a pronounced asymmetric potential that closed some — Página: [1811](kindle://book?action=open&asin=B0BSKW45KB&location=1811) ^ref-38099 --- The real question is what this means for conflict when production costs fall by another order of magnitude and capabilities multiply. — Página: [1818](kindle://book?action=open&asin=B0BSKW45KB&location=1818) ^ref-30478 Nota: This might be a problem --- The coming wave is, however, characterized by a set of four intrinsic features compounding the problem of containment. — Página: [1822](kindle://book?action=open&asin=B0BSKW45KB&location=1822) ^ref-48141 --- First among them is the primary lesson of this section: hugely asymmetric impact. — Página: [1823](kindle://book?action=open&asin=B0BSKW45KB&location=1823) ^ref-57967 --- Second, they are developing fast, a kind of hyper-evolution, iterating, improving, and branching into new areas at incredible speed. — Página: [1825](kindle://book?action=open&asin=B0BSKW45KB&location=1825) ^ref-28001 --- Third, they are often omni-use; that is, they can be used for many different purposes. — Página: [1826](kindle://book?action=open&asin=B0BSKW45KB&location=1826) ^ref-7451 --- And fourth, they increasingly have a degree of autonomy beyond any previous technology. — Página: [1826](kindle://book?action=open&asin=B0BSKW45KB&location=1826) ^ref-23499 --- Emerging technologies have always created new threats, redistributed power, and removed barriers to entry. — Página: [1830](kindle://book?action=open&asin=B0BSKW45KB&location=1830) ^ref-10029 Nota: Asymetrical technilogy and power --- In the words of the security expert Audrey Kurth Cronin, “Never before have so many had access to such advanced technologies capable of inflicting death and mayhem.” — Página: [1838](kindle://book?action=open&asin=B0BSKW45KB&location=1838) ^ref-31808 --- A single AI program can write as much text as all of humanity. A single two-gigabyte image-generation model running on your laptop can compress all the pictures on the open web into a tool that generates images with extraordinary creativity and precision. — Página: [1846](kindle://book?action=open&asin=B0BSKW45KB&location=1846) ^ref-24241 Nota: These are examples of ghe great vimpsct that technology might have. --- However many safeguards and security protocols are in place, the scale of impact is far wider than we’ve seen before. — Página: [1857](kindle://book?action=open&asin=B0BSKW45KB&location=1857) ^ref-20651 --- AI creates asymmetric risks beyond those of a bad batch of food, a plane accident, or a faulty product. Its risks extend to entire societies, making it not so much a blunt tool as a lever with global consequences. — Página: [1858](kindle://book?action=open&asin=B0BSKW45KB&location=1858) ^ref-8262 Nota: It is a toolt that can act globaly. It already does. --- And we already live in an age of interlinked global systems. In the coming wave a single point—a given program, a genetic change—can alter everything. — Página: [1861](kindle://book?action=open&asin=B0BSKW45KB&location=1861) ^ref-64533 --- If you want to contain technology, you might hope it develops at a manageable pace, giving society time and space to understand and adapt to it. — Página: [1863](kindle://book?action=open&asin=B0BSKW45KB&location=1863) ^ref-3458 --- Put simply, innovation in the “real world” could start moving at a digital pace, in near-real time, with reduced friction and fewer dependencies. — Página: [1878](kindle://book?action=open&asin=B0BSKW45KB&location=1878) ^ref-62575 --- You will be able to experiment in small, speedy, malleable domains, creating near-perfect simulations, and then translate them into concrete products. And then do it again, and again, learning, evolving, and improving at rates previously impossible in the expensive, static world of atoms. — Página: [1879](kindle://book?action=open&asin=B0BSKW45KB&location=1879) ^ref-636 Nota: os this the concepg or busebof digital twin? --- The physicist César Hidalgo argues that configurations of matter are significant because of the information they contain. — Página: [1881](kindle://book?action=open&asin=B0BSKW45KB&location=1881) ^ref-30448 --- Just as today’s models produce detailed images based on a few words, so in decades to come similar models will produce a novel compound or indeed an entire organism with just a few natural language prompts. — Página: [1897](kindle://book?action=open&asin=B0BSKW45KB&location=1897) ^ref-14221 --- This is what we mean by hyper-evolution—a fast, iterative platform for creation. — Página: [1901](kindle://book?action=open&asin=B0BSKW45KB&location=1901) ^ref-11142 Nota: Each technology fuel the next one --- Dual-use technologies are those with both civilian and military applications. — Página: [1917](kindle://book?action=open&asin=B0BSKW45KB&location=1917) ^ref-23111 --- Dual-use technologies are both helpful and potentially destructive, tools and weapons. — Página: [1922](kindle://book?action=open&asin=B0BSKW45KB&location=1922) ^ref-12206 --- And the more powerful the technology, the more concern there should be about how many uses it might have. — Página: [1926](kindle://book?action=open&asin=B0BSKW45KB&location=1926) ^ref-40602 --- Omni-use technologies like steam or electricity have wider societal effects and spillovers than narrower technologies. — Página: [1930](kindle://book?action=open&asin=B0BSKW45KB&location=1930) ^ref-32293 --- It’s still very early days, and truly general systems are still some way off, but at some point these capabilities will expand to many thousands of activities. — Página: [1937](kindle://book?action=open&asin=B0BSKW45KB&location=1937) ^ref-54982 Nota: We might see what these general porpuse systems looks like in the near future. --- Over time, technology tends toward generality. What this means is that weaponizable or harmful uses of the coming wave will be possible regardless of whether this was intended. — Página: [1944](kindle://book?action=open&asin=B0BSKW45KB&location=1944) ^ref-1469 --- What’s different about the coming wave is how quickly it is being embedded, how globally it spreads, how easily it can be componentized into swappable parts, and just how powerful and above all broad its applications could be. — Página: [1948](kindle://book?action=open&asin=B0BSKW45KB&location=1948) ^ref-32720 --- Researchers in the field categorize autonomy from level 0, no autonomy whatsoever, to level 5, where a vehicle can drive itself under all conditions and the driver simply inputs a destination and then can fall happily asleep. — Página: [1963](kindle://book?action=open&asin=B0BSKW45KB&location=1963) ^ref-27093 --- Keeping humans “in the loop,” as the saying goes, is desirable, but optional. — Página: [1968](kindle://book?action=open&asin=B0BSKW45KB&location=1968) ^ref-9237 Nota: This might ve the case for a lot of tasks --- A paradox of the coming wave is that its technologies are largely beyond our ability to comprehend at a granular level yet still within our ability to create and use. — Página: [1985](kindle://book?action=open&asin=B0BSKW45KB&location=1985) ^ref-15945 --- Autonomous systems can and may be explainable, but the fact that so much of the coming wave operates at the edge of what we can understand should give us pause. We won’t always be able to predict what these autonomous systems will do next; that’s the nature of autonomy. — Página: [1989](kindle://book?action=open&asin=B0BSKW45KB&location=1989) ^ref-15234 --- Humans dominate our environment because of our intelligence. A more intelligent entity could, it follows, dominate us. The AI researcher Stuart Russell calls it the “gorilla problem”: gorillas are physically stronger and tougher than any human being, but it is they who are endangered or living in zoos; they who are contained. — Página: [1999](kindle://book?action=open&asin=B0BSKW45KB&location=1999) ^ref-31616 --- An “intelligence explosion” is the point at which an AI can improve itself again and again, recursively making itself better in ever faster and more effective ways. — Página: [2004](kindle://book?action=open&asin=B0BSKW45KB&location=2004) ^ref-42969 --- Nobody really knows how we can contain the very features being researched so intently in the coming wave. — Página: [2008](kindle://book?action=open&asin=B0BSKW45KB&location=2008) ^ref-65061 --- the real question is why it’s so hard to see it as anything other than inevitable. — Página: [2013](kindle://book?action=open&asin=B0BSKW45KB&location=2013) ^ref-55213 --- This implication of national rivalry was an aspect of the contest I soon came to regret. — Página: [2027](kindle://book?action=open&asin=B0BSKW45KB&location=2027) ^ref-39821 Nota: Things ae not as simple as we thibk --- it became clear AlphaGo was part of a much bigger story than one trophy, system, or company; it was that of great powers engaging in a new and dangerous game of technological competition—and a series of overwhelmingly powerful and interlocking incentives that ensure the coming wave really is coming. — Página: [2046](kindle://book?action=open&asin=B0BSKW45KB&location=2046) ^ref-24556 --- From curiosity to crisis, fortune to fear, at its heart technology emerges to fill human needs. — Página: [2050](kindle://book?action=open&asin=B0BSKW45KB&location=2050) ^ref-34091 --- The first driver has to do with what I experienced with AlphaGo: great power competition. — Página: [2055](kindle://book?action=open&asin=B0BSKW45KB&location=2055) ^ref-29464 --- Second comes a global research ecosystem with its ingrained rituals rewarding open publication, curiosity, and the pursuit of new ideas at all costs. — Página: [2057](kindle://book?action=open&asin=B0BSKW45KB&location=2057) ^ref-7861 --- Then come the immense financial gains from technology and the urgent need to tackle our global social challenges. — Página: [2058](kindle://book?action=open&asin=B0BSKW45KB&location=2058) ^ref-65012 --- And the final driver is perhaps the most human of all: ego. — Página: [2059](kindle://book?action=open&asin=B0BSKW45KB&location=2059) ^ref-540 --- China is already ahead of the United States in green energy, 5G, and AI and is on a trajectory to overtake it in quantum and biotech in the next few years. — Página: [2125](kindle://book?action=open&asin=B0BSKW45KB&location=2125) ^ref-20564 Nota: China is rising sharply --- Xi was stating a bald truth, the self-declared mantra of not just China but virtually every state, from superpower leaders at the frontier to isolated pariahs: who builds, owns, and deploys technology matters. — Página: [2134](kindle://book?action=open&asin=B0BSKW45KB&location=2134) ^ref-8333 --- Technology has become the world’s most important strategic asset, not so much the instrument of foreign policy as the driver of it. — Página: [2137](kindle://book?action=open&asin=B0BSKW45KB&location=2137) ^ref-7817 --- But the reality is that the logic of nation-states is at times painfully simple and yet utterly inevitable. — Página: [2146](kindle://book?action=open&asin=B0BSKW45KB&location=2146) ^ref-63464 --- The debate now isn’t whether we are in a technological and AI arms race; it’s where it will lead. — Página: [2153](kindle://book?action=open&asin=B0BSKW45KB&location=2153) ^ref-46076 Nota: This debate seams to be treated more serious as time goes on. --- [[Grandes modelos de linguagem|Grandes modelos de linguagem]] are still seen as cutting-edge, yet there is no great magic or hidden state secret to them. Access to computation is likely the biggest bottleneck, but plenty of services exist to make it happen. The same goes for [[CRISPR]] or DNA synthesis. — Página: [2180](kindle://book?action=open&asin=B0BSKW45KB&location=2180) ^ref-11842 --- happening. It is a point so obvious it doesn’t often get mentioned: there is no central authority controlling what technologies get developed, who does it, and for what purpose; technology is an orchestra with no conductor. — Página: [2187](kindle://book?action=open&asin=B0BSKW45KB&location=2187) ^ref-22900 --- Raw curiosity, the quest for truth, the importance of openness, evidence-based peer review—these are core values for scientific and technological research. — Página: [2193](kindle://book?action=open&asin=B0BSKW45KB&location=2193) ^ref-54455 --- Openness is science and technology’s cardinal ideology. What is known must be shared; what is discovered must be published. — Página: [2197](kindle://book?action=open&asin=B0BSKW45KB&location=2197) ^ref-22122 --- Moreover, these days papers are announced on Twitter and often written with social media influence in mind. They are designed to be eye-catching and attract attention. — Página: [2209](kindle://book?action=open&asin=B0BSKW45KB&location=2209) ^ref-14051 Nota: This might be a huge problem --- Academics fervently argue for open access to their research. In tech, strong norms around sharing and contributing support a flourishing space of open-source software. — Página: [2211](kindle://book?action=open&asin=B0BSKW45KB&location=2211) ^ref-24601 Nota: The world is starting to be more open? --- All in all, to a degree that is perhaps underappreciated, publication and sharing aren’t just about the process of falsification in science. They’re also for prestige, for peers, for the pursuit of a mission, for the sake of a job, for likes. — Página: [2218](kindle://book?action=open&asin=B0BSKW45KB&location=2218) ^ref-18155 --- All these companies, those most keenly investing in the coming wave, those with the most lavish budgets, have a track record of openly publishing their research. — Página: [2230](kindle://book?action=open&asin=B0BSKW45KB&location=2230) ^ref-54438 --- That’s no discredit to him; it just shows that no one can ever be sure of anything at the research frontier. — Página: [2247](kindle://book?action=open&asin=B0BSKW45KB&location=2247) ^ref-51197 --- Science and technology research is inherently unpredictable, exceptionally open, and growing fast. Governing or controlling it is therefore immensely difficult. — Página: [2254](kindle://book?action=open&asin=B0BSKW45KB&location=2254) ^ref-37084 --- The railway boom of the 1840s was “arguably the greatest bubble in history.” But in the annals of technology, it is more norm than exception. There was nothing inevitable about the coming of the railways, but there was something inevitable about the chance to make money. — Página: [2280](kindle://book?action=open&asin=B0BSKW45KB&location=2280) ^ref-23338 --- Science has to be converted into useful and desirable products for it to truly spread far and wide. Put simply: most technology is made to earn money. — Página: [2285](kindle://book?action=open&asin=B0BSKW45KB&location=2285) ^ref-46892 --- they see them as ways to make a profit. And this, the potential for profit, is built on something even more long-lasting and robust: raw demand. People both want and need the fruits of technology. — Página: [2291](kindle://book?action=open&asin=B0BSKW45KB&location=2291) ^ref-54067 --- The sheer breadth of human wants and needs, and the countless opportunities to profit from them, are integral to the story of technology and will remain so in the future. — Página: [2295](kindle://book?action=open&asin=B0BSKW45KB&location=2295) ^ref-27333 --- In developed economies, people work far less than they used to for far more reward. In Germany, for example, annual working hours have decreased by nearly 60 percent since 1870. — Página: [2304](kindle://book?action=open&asin=B0BSKW45KB&location=2304) ^ref-45005 Nota: What are othe data about work? --- Here’s a slight correction: much of what we see around us is powered by human intelligence in direct pursuit of monetary gain. — Página: [2308](kindle://book?action=open&asin=B0BSKW45KB&location=2308) ^ref-42296 --- When a corporation automates insurance claims or adopts a new manufacturing technique, it creates efficiency savings or improves the product, boosting profits and attracting new customers. Once an innovation delivers a competitive advantage like this, everyone must either adopt it, leapfrog it, switch focus, or lose market share and eventually go bust. The attitude around this dynamic in technology businesses in particular is simple and ruthless: build the next generation of technology or be destroyed. — Página: [2314](kindle://book?action=open&asin=B0BSKW45KB&location=2314) ^ref-6487 --- All of the world’s knowledge, best practices, precedent, and computational power will be available, tailored to you, to your specific needs and circumstances, instantaneously and effortlessly. — Página: [2343](kindle://book?action=open&asin=B0BSKW45KB&location=2343) ^ref-41586 Nota: Does it really? --- Little is ultimately more valuable than intelligence. Intelligence is the wellspring and the director, architect, and facilitator of the world economy. The more we expand the range and nature of intelligences on offer, the more growth should be possible. — Página: [2347](kindle://book?action=open&asin=B0BSKW45KB&location=2347) ^ref-50472 --- In 1945, around 50 percent of the world’s population was seriously undernourished. Today, despite a population well over three times bigger, that’s down to 10 percent. — Página: [2370](kindle://book?action=open&asin=B0BSKW45KB&location=2370) ^ref-3017 --- food sufficient, if not adequately distributed, for the planet’s eight billion and rising human inhabitants. — Página: [2376](kindle://book?action=open&asin=B0BSKW45KB&location=2376) ^ref-34587 Nota: The problem is not on the vproduction side, but rather on the distribuition side. --- We pursue new technologies, including those in the coming wave, not just because we want them, but because, at a fundamental level, we need them. — Página: [2378](kindle://book?action=open&asin=B0BSKW45KB&location=2378) ^ref-47757 --- The energy scholar Vaclav Smil calls ammonia, cement, plastics, and steel the four pillars of modern civilization: the material base underwriting modern society, each hugely carbon-intensive to produce, with no obvious successors. — Página: [2390](kindle://book?action=open&asin=B0BSKW45KB&location=2390) ^ref-28944 --- If we are to stand any chance of keeping global warming under two degrees Celsius, then the world’s scientists working under the UN’s Intergovernmental Panel on Climate Change have been clear: carbon capture and storage is an essential technology. — Página: [2398](kindle://book?action=open&asin=B0BSKW45KB&location=2398) ^ref-28181 --- Technology can and will improve lives and solve problems. — Página: [2407](kindle://book?action=open&asin=B0BSKW45KB&location=2407) ^ref-32897 --- A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it’s not. How it is created, used, owned, and managed all make a difference. — Página: [2416](kindle://book?action=open&asin=B0BSKW45KB&location=2416) ^ref-20955 --- The coming wave is coming partly because there is no way through without it. — Página: [2422](kindle://book?action=open&asin=B0BSKW45KB&location=2422) ^ref-13372 --- Scientists and technologists are all too human. They crave status, success, and a legacy. They want to be the first and best and recognized as such. — Página: [2425](kindle://book?action=open&asin=B0BSKW45KB&location=2425) ^ref-2770 --- polymathic Hungarian American John von Neumann. “What we are creating now,” he said, “is a monster whose influence is going to change history, provided there is any history left, yet it would be impossible not to see it through, not only for military reasons, but it would also be unethical from the point of view of the scientists not to do what they know is feasible, no matter what terrible consequences it may have.” — Página: [2435](kindle://book?action=open&asin=B0BSKW45KB&location=2435) ^ref-34722 Nota: Isso e pesrigoso. Bastante perugoso --- Making history, doing something that matters, helping others, beating others, impressing a prospective partner, impressing a boss, peers, rivals: it’s all in there, all part of the ever-present drive to take risks, explore the edges, go further into the unknown. Build something new. Change the game. Climb the mountain. — Página: [2441](kindle://book?action=open&asin=B0BSKW45KB&location=2441) ^ref-50923 Nota: This vision might br too dangsrous --- Nationalism, capitalism, and science—these are, by now, embedded features of the world. Simply removing them from the scene is not possible in any meaningful time frame. — Página: [2449](kindle://book?action=open&asin=B0BSKW45KB&location=2449) ^ref-22974 --- Technology “emerges” through countless independent contributions all layering on top of one another, a metastasizing, entangled morass of ideas unraveling themselves, driven on by deep-rooted and dispersed incentives. — Página: [2455](kindle://book?action=open&asin=B0BSKW45KB&location=2455) ^ref-27293 Nota: Most of the time we have no ideia about where the ideias come from. --- Technology is now an indispensable mega-system infusing every aspect of daily life, society, and the economy. — Página: [2464](kindle://book?action=open&asin=B0BSKW45KB&location=2464) ^ref-58010 --- This is the ultimate collective action problem. The idea that CRISPR or AI can be put back in the box is not credible. Until someone can create a plausible path to dismantling these interlocking incentives, the option of not building, saying no, perhaps even just slowing down or taking a different path isn’t there. — Página: [2471](kindle://book?action=open&asin=B0BSKW45KB&location=2471) ^ref-48799 --- At its heart, the nation-state, the central unit of the world’s political order today, offers its citizens a simple and highly persuasive bargain: that not only is centralization of power in the sovereign, territorial state possible but its benefits far outweigh the risks. — Página: [2485](kindle://book?action=open&asin=B0BSKW45KB&location=2485) ^ref-53059 --- On the one hand the most dystopian excesses of centralized power must be avoided, and on the other we must accept regular intervention to maintain order. — Página: [2493](kindle://book?action=open&asin=B0BSKW45KB&location=2493) ^ref-38113 --- In this third section of the book, we grapple with the profound consequences of these technologies for the nation-state and for the liberal democratic nation-state above all. — Página: [2501](kindle://book?action=open&asin=B0BSKW45KB&location=2501) ^ref-51862 --- We ended up with maximum divergence on all possible things. It was, in other words, politics as usual. — Página: [2519](kindle://book?action=open&asin=B0BSKW45KB&location=2519) ^ref-56137 --- The idea that technology alone can solve social and political problems is a dangerous delusion. But the idea that they can be solved without technology is also wrongheaded. — Página: [2542](kindle://book?action=open&asin=B0BSKW45KB&location=2542) ^ref-39380 --- While technology is still the single most powerful avenue for addressing the challenges of the twenty-first century, we cannot ignore downsides. — Página: [2554](kindle://book?action=open&asin=B0BSKW45KB&location=2554) ^ref-49532 --- Technology is not just a tool to support the bargain we’ve made in the nation-state; it is also a genuine threat to it. — Página: [2557](kindle://book?action=open&asin=B0BSKW45KB&location=2557) ^ref-59116 --- And anyone who thinks what happened in Syria could never happen “here” is kidding themselves; people are people wherever they are. — Página: [2563](kindle://book?action=open&asin=B0BSKW45KB&location=2563) ^ref-45462 --- Going into the coming wave, many nations are beset by a slew of major challenges battering their effectiveness, making them weaker, more divided, and more prone to slow and faulty decision-making. — Página: [2577](kindle://book?action=open&asin=B0BSKW45KB&location=2577) ^ref-61437 --- Democracies are built on trust. People need to trust that government officials, militaries, and other elites will not abuse their dominant positions. Everyone relies on the trust that taxes will be paid, rules honored, the interests of the whole put ahead of individuals. Without trust, from the ballot box to the tax return, from the local council to the judiciary, societies are in trouble. — Página: [2581](kindle://book?action=open&asin=B0BSKW45KB&location=2581) ^ref-9218 Nota: Rhis might not be true for every state. It looks to me that is more about power than trust --- That so many people profoundly feel society is failing is itself a problem: Distrust breeds negativity and apathy. People decline to vote. — Página: [2593](kindle://book?action=open&asin=B0BSKW45KB&location=2593) ^ref-27006 --- Behind the new authoritarian impulse and political instability lies a growing pool of social resentment. — Página: [2597](kindle://book?action=open&asin=B0BSKW45KB&location=2597) ^ref-26494 --- Forty million people in the United States live in poverty, and more than five million live in “Third World conditions”—all within the world’s richest economy. — Página: [2603](kindle://book?action=open&asin=B0BSKW45KB&location=2603) ^ref-48850 Nota: Even the weath counties has its problems --- Onshoring, national security, resilient supply chains, self-sufficiency—today’s language of trade is once again the language of borders, barriers, and tariffs. — Página: [2611](kindle://book?action=open&asin=B0BSKW45KB&location=2611) ^ref-39076 --- Every previous wave of technology has had profound political implications. We should expect the same in the future. — Página: [2621](kindle://book?action=open&asin=B0BSKW45KB&location=2621) ^ref-20292 --- Social media thrives on heightened emotions and, quite often, outrage. — Página: [2627](kindle://book?action=open&asin=B0BSKW45KB&location=2627) ^ref-5902 --- Too fast in its development, too global, too protean and enticing for any simple model of containment, strategically critical, relied upon by billions, modern technology itself is a prime actor, a monumental force nation-states struggle to manage. — Página: [2634](kindle://book?action=open&asin=B0BSKW45KB&location=2634) ^ref-7339 Nota: Technology is a problem in this case --- Technology didn’t straightforwardly “cause” or create the modern state (or indeed any political structure). But the potential it unleashes is not neutral in that story. — Página: [2640](kindle://book?action=open&asin=B0BSKW45KB&location=2640) ^ref-41210 --- As the historian of technology Langdon Winner puts it, “Technology in its various manifestations is a significant part of the human world. Its structures, processes, and alterations enter into and become part of the structures, processes, and alterations of human consciousness, society, and politics.” In other words, technology is political. — Página: [2642](kindle://book?action=open&asin=B0BSKW45KB&location=2642) ^ref-63646 --- In this, technology is one of the key determinants of history, but never alone and never in a mechanistic, inherently predictable way. — Página: [2649](kindle://book?action=open&asin=B0BSKW45KB&location=2649) ^ref-26463 Nota: the impact might not be predicted at first --- Technologies are ideas, manifested in products and services that have profound and lasting consequences for people, social structures, the environment, and everything in between. — Página: [2652](kindle://book?action=open&asin=B0BSKW45KB&location=2652) ^ref-64530 --- Put simply, technology and political order are intimately connected. The introduction of new technologies has major political consequences. — Página: [2668](kindle://book?action=open&asin=B0BSKW45KB&location=2668) ^ref-3058 --- What emerges will, I think, tend in two directions with a spectrum of outcomes in between. On one trajectory, some liberal democratic states will continue to be eroded from within, becoming a kind of zombie government. — Página: [2684](kindle://book?action=open&asin=B0BSKW45KB&location=2684) ^ref-36172 --- On another, unthinking adoption of some aspects of the coming wave opens pathways to domineering state control, creating supercharged Leviathans whose power goes beyond even history’s most extreme totalitarian governments. — Página: [2687](kindle://book?action=open&asin=B0BSKW45KB&location=2687) ^ref-31103 --- WannaCry tricked some users into opening an email, which released a “worm” replicating and transporting itself to infect a quarter of a million computers across 150 countries in just one day. — Página: [2716](kindle://book?action=open&asin=B0BSKW45KB&location=2716) ^ref-27628 Nota: That is scary --- In other words, thanks to a basic failure of containment, a global superpower became a victim of its own powerful and supposedly secure technology. This is uncontained asymmetry in action. — Página: [2741](kindle://book?action=open&asin=B0BSKW45KB&location=2741) ^ref-57869 --- With the coming wave that assumption is a serious mistake. Such attacks demonstrate that there are those who would use cutting-edge technologies to degrade and disable key state functions. — Página: [2747](kindle://book?action=open&asin=B0BSKW45KB&location=2747) ^ref-12915 Nota: As technology progresses attacks can make more damage --- This is, in part, a story of who can do what, a story of power and where it lies. — Página: [2761](kindle://book?action=open&asin=B0BSKW45KB&location=2761) ^ref-40746 --- Power is “the ability or capacity to do something or act in a particular way;…to direct or influence the behavior of others or the course of events.” — Página: [2763](kindle://book?action=open&asin=B0BSKW45KB&location=2763) ^ref-53949 --- Technology is ultimately political because technology is a form of power. — Página: [2766](kindle://book?action=open&asin=B0BSKW45KB&location=2766) ^ref-5555 --- As we saw in part 2, it will enable people to do things in the real world. I think of it like this: just as the costs of processing and broadcasting information plummeted in the consumer internet era, the cost of actually doing something, taking action, projecting power, will plummet with the next wave. Knowing is great, but doing is much more impactful. — Página: [2767](kindle://book?action=open&asin=B0BSKW45KB&location=2767) ^ref-51130 Nota: Been able to do things and not just know things is a dramatcally diffent thing --- Wherever power is today, it will be amplified. Anyone with goals—that is, everyone—will have huge help in realizing them. — Página: [2774](kindle://book?action=open&asin=B0BSKW45KB&location=2774) ^ref-25078 --- Everyone will have a world-class team on their side and in their corner. — Página: [2781](kindle://book?action=open&asin=B0BSKW45KB&location=2781) ^ref-35921 Nota: Does it really? Been an expert is different than having only knowledge. --- Democratizing access necessarily means democratizing risk. — Página: [2784](kindle://book?action=open&asin=B0BSKW45KB&location=2784) ^ref-12709 --- More sophisticated armed robots will further reduce barriers to violence. — Página: [2803](kindle://book?action=open&asin=B0BSKW45KB&location=2803) ^ref-33268 --- The cost of military-grade drones has fallen by three orders of magnitude over the last decade. By 2028, $26 billion a year will be spent on military drones, and at that point many are likely to be fully autonomous. — Página: [2811](kindle://book?action=open&asin=B0BSKW45KB&location=2811) ^ref-16819 Nota: That is scary --- Imagine a huge cache of documents from a company leaked. A legal AI might be able to parse this against multiple legal systems, figure out every possible infraction, and then hit that company with multiple crippling lawsuits around the world at the same time. — Página: [2828](kindle://book?action=open&asin=B0BSKW45KB&location=2828) ^ref-3673 --- This new dynamic—where bad actors are emboldened to go on the offensive—opens up new vectors of attack thanks to the interlinked, vulnerable nature of modern systems: not just a single hospital but an entire health system can be hit; not just a warehouse but an entire supply chain. — Página: [2844](kindle://book?action=open&asin=B0BSKW45KB&location=2844) ^ref-17072 --- If the state began in new forms of war, perhaps it will end in the same way. — Página: [2856](kindle://book?action=open&asin=B0BSKW45KB&location=2856) ^ref-7127 --- Now powerful, asymmetric, omni-use technologies are certain to reach the hands of those who want to damage the state. — Página: [2860](kindle://book?action=open&asin=B0BSKW45KB&location=2860) ^ref-47647 --- Eventually, the balance might be restored, but not before a wave of immensely destabilizing force is unleashed. And as we’ve seen, the nature of the threat is far more widespread than blunt forms of physical assault. — Página: [2866](kindle://book?action=open&asin=B0BSKW45KB&location=2866) ^ref-744 --- Ask yourself, what happens when anyone has the power to create and broadcast material with incredible levels of realism? — Página: [2878](kindle://book?action=open&asin=B0BSKW45KB&location=2878) ^ref-37310 --- These fakes will be so good our rational minds will find it hard to accept they aren’t real. — Página: [2881](kindle://book?action=open&asin=B0BSKW45KB&location=2881) ^ref-18180 Nota: What is real is almost impossivle to know --- Anyone motivated to sow instability now has an easier time of it. — Página: [2887](kindle://book?action=open&asin=B0BSKW45KB&location=2887) ^ref-46622 --- The threat here lies not so much with extreme cases as in subtle, nuanced, and highly plausible scenarios being exaggerated and distorted. — Página: [2891](kindle://book?action=open&asin=B0BSKW45KB&location=2891) ^ref-58362 Nota: Somethong plausible is more dangerous than something outrageous --- Not everyone would buy it, but those who wanted to believe would find it utterly compelling. — Página: [2899](kindle://book?action=open&asin=B0BSKW45KB&location=2899) ^ref-24924 Nota: Deepfakes needs to be just real enought --- AI-enhanced digital tools will exacerbate information operations like these, meddling in elections, exploiting social divisions, and creating elaborate astroturfing campaigns to sow chaos. — Página: [2912](kindle://book?action=open&asin=B0BSKW45KB&location=2912) ^ref-26164 --- Eighty-two percent of influential users advocating for “reopening America” were bots. — Página: [2917](kindle://book?action=open&asin=B0BSKW45KB&location=2917) ^ref-2717 --- Cue an “Infocalypse,” the point at which society can no longer manage a torrent of sketchy material, where the information ecosystem grounding knowledge, trust, and social cohesion, the glue holding society together, falls apart. — Página: [2926](kindle://book?action=open&asin=B0BSKW45KB&location=2926) ^ref-20806 --- Even as the number of BSL-4 labs booms, only a quarter of them score highly on safety, according to the Global Health Security Index. — Página: [2963](kindle://book?action=open&asin=B0BSKW45KB&location=2963) ^ref-65300 --- Nothing should get out. Yet pathogens do, time and again. Despite being some of the toughest around, the protocols, technologies, and regulations for containment fail. — Página: [2970](kindle://book?action=open&asin=B0BSKW45KB&location=2970) ^ref-24747 --- Few areas of biology are as controversial as gain-of-function (GOF) research. Put simply, gain-of-function experiments deliberately engineer pathogens to be more lethal or infectious, or both. — Página: [2974](kindle://book?action=open&asin=B0BSKW45KB&location=2974) ^ref-11286 --- One way of understanding how it might happen—that is, how viruses might become more lethal and transmissible at the same time—and how we might combat that is to, well, make it happen. That’s where gain-of-function research comes in. — Página: [2977](kindle://book?action=open&asin=B0BSKW45KB&location=2977) ^ref-40142 --- In late 2022, an NIH study at Boston University combined the original, more deadly strain of COVID with the spike protein of the more transmissible omicron variant. Many felt the research shouldn’t have gone ahead, but there it was, funded by public money. — Página: [2993](kindle://book?action=open&asin=B0BSKW45KB&location=2993) ^ref-49124 --- It’s about what goes wrong when powerful tools proliferate, what mistakes get made, what “revenge effects” unfurl, what random, unforeseen mess results from technology’s collision with reality. — Página: [2996](kindle://book?action=open&asin=B0BSKW45KB&location=2996) ^ref-34932 --- If every half-competent lab or even random biohacker can embark on this research, tragedy cannot be indefinitely postponed. — Página: [3003](kindle://book?action=open&asin=B0BSKW45KB&location=3003) ^ref-4543 Nota: This bmight be a huge problem of safety --- As the power and spread of any technology grows, so its failure modes escalate. — Página: [3005](kindle://book?action=open&asin=B0BSKW45KB&location=3005) ^ref-59658 --- Yet stressors might also be less discrete events, less a robot attack, lab leak, or deepfake video, and more a slow and diffuse process undermining foundations. — Página: [3009](kindle://book?action=open&asin=B0BSKW45KB&location=3009) ^ref-6861 --- Maynard Keynes called “technological unemployment.” In Keynes’s view, this was a good thing, with increasing productivity freeing up time for further innovation and leisure. — Página: [3017](kindle://book?action=open&asin=B0BSKW45KB&location=3017) ^ref-26679 --- So far at least, in economic terms, new technologies have not ultimately replaced labor; they have in the aggregate complemented it. — Página: [3022](kindle://book?action=open&asin=B0BSKW45KB&location=3022) ^ref-21405 --- These tools will only temporarily augment human intelligence. They will make us smarter and more efficient for a time, and will unlock enormous amounts of economic growth, but they are fundamentally labor replacing. — Página: [3027](kindle://book?action=open&asin=B0BSKW45KB&location=3027) ^ref-28294 --- With each additional robot per thousand workers there is a decline in the employment-to-population ratio, and consequently a fall in wages. Today algorithms perform the vast bulk of equity trades and increasingly act across financial institutions, and yet, even as Wall Street booms, it sheds jobs as technology encroaches on more and more tasks. — Página: [3036](kindle://book?action=open&asin=B0BSKW45KB&location=3036) ^ref-38050 --- believe this rosy vision is implausible over the next couple of decades; automation is unequivocally another fragility amplifier. — Página: [3045](kindle://book?action=open&asin=B0BSKW45KB&location=3045) ^ref-48939 --- The number of people who can get a PhD in machine learning will remain tiny in comparison to the scale of layoffs. And, sure, new demand will create new work, but that doesn’t mean it all gets done by human beings. — Página: [3053](kindle://book?action=open&asin=B0BSKW45KB&location=3053) ^ref-53640 --- If your sense of self is wedded to a particular kind of work, it’s little consolation if you feel your new job demeans your dignity. — Página: [3059](kindle://book?action=open&asin=B0BSKW45KB&location=3059) ^ref-6411 --- New jobs might be created in the long term, but for millions they won’t come quick enough or in the right places. — Página: [3064](kindle://book?action=open&asin=B0BSKW45KB&location=3064) ^ref-9632 --- Whichever side of the jobs debate you fall on, it’s hard to deny that the ramifications will be hugely destabilizing for hundreds of millions who will, at the very least, need to re-skill and transition to new types of work. — Página: [3072](kindle://book?action=open&asin=B0BSKW45KB&location=3072) ^ref-11156 --- The full amplification of fragility is missed because it often appears as if these impacts were happening incrementally and in convenient silos. They are not. — Página: [3083](kindle://book?action=open&asin=B0BSKW45KB&location=3083) ^ref-39913 --- This is not, then, confined to amplifying specific points of fragility; it is, in the slightly longer term, about a transformation of the very ground on which society is built. And in this great redistribution of power, the state, already fragile and growing more so, is shaken to its core, its grand bargain left tattered and precarious. — Página: [3093](kindle://book?action=open&asin=B0BSKW45KB&location=3093) ^ref-16567 --- new technologies help create new centers of power with new social infrastructures both enabling them and supporting them. — Página: [3119](kindle://book?action=open&asin=B0BSKW45KB&location=3119) ^ref-28658 Nota: Most of the imes we do not know what are those technologies. --- While small changes in technology can fundamentally alter the balance of power, trying to predict exactly how, decades into the future, is incredibly difficult. — Página: [3122](kindle://book?action=open&asin=B0BSKW45KB&location=3122) ^ref-39295 --- Recall that growing access to power means everyone’s power will be amplified. — Página: [3126](kindle://book?action=open&asin=B0BSKW45KB&location=3126) ^ref-5525 --- The nation-state will be subject to massive centrifugal and centripetal forces, centralization and fragmentation. — Página: [3135](kindle://book?action=open&asin=B0BSKW45KB&location=3135) ^ref-4595 Nota: Tuebulence is on the way --- But what this misses is that the most powerful forces in the world are actually groups of individuals coordinating to achieve shared goals. — Página: [3158](kindle://book?action=open&asin=B0BSKW45KB&location=3158) ^ref-45858 Nota: Sometimes intelligence is spread between individuals. --- Big tech companies provide tools for everything from organizing a birthday to running multimillion-dollar businesses. — Página: [3173](kindle://book?action=open&asin=B0BSKW45KB&location=3173) ^ref-63227 Nota: And this is probably a problem --- The world’s top fifty cities have the lion’s share of wealth and corporate power (45 percent of big company HQs; 21 percent of world GDP) despite having only 8 percent of the world’s population. The top 10 percent of global firms take 80 percent of the total profits. — Página: [3182](kindle://book?action=open&asin=B0BSKW45KB&location=3182) ^ref-59043 --- For Koreans today, Samsung is almost like a parallel government, a constant presence throughout people’s lives. — Página: [3192](kindle://book?action=open&asin=B0BSKW45KB&location=3192) ^ref-41853 --- All the big tech platforms either are mainly service businesses or have very large service businesses. — Página: [3207](kindle://book?action=open&asin=B0BSKW45KB&location=3207) ^ref-16991 Nota: Services are better than standalone products. --- In this scenario there will be just a few mega-players whose scale and power will begin to rival traditional states. — Página: [3225](kindle://book?action=open&asin=B0BSKW45KB&location=3225) ^ref-26333 Nota: Companiss can challange governmens --- Whatever the end point, we are heading to a place where unprecedented powers and abilities are out there, in the hands of already powerful actors who’ll no doubt use them to amplify their reach and further their own agenda. — Página: [3234](kindle://book?action=open&asin=B0BSKW45KB&location=3234) ^ref-17003 Nota: Power is tge key for most players --- Almost every detail of life is logged, somewhere, by those with the sophistication to process and act on the data they collect. — Página: [3275](kindle://book?action=open&asin=B0BSKW45KB&location=3275) ^ref-25253 Nota: This is done today. I think maknly as a way to sell the data to advsertisers. --- Societies of overweening surveillance and control are already here, and now all of this is set to escalate enormously into a next-level concentration of power at the center. — Página: [3309](kindle://book?action=open&asin=B0BSKW45KB&location=3309) ^ref-33381 --- It’s no secret that governments monitor and control their own populations, but these tendencies extend deep into Western firms, too. — Página: [3317](kindle://book?action=open&asin=B0BSKW45KB&location=3317) ^ref-12155 --- Consider that a combination of AI, cheap robotics, and advanced biotech coupled with clean energy sources might, for the first time in modernity, make living “off-grid” nearly equivalent to being plugged-in. — Página: [3353](kindle://book?action=open&asin=B0BSKW45KB&location=3353) ^ref-54119 --- The biggest AI models will cost hundreds of millions of dollars to train, and consequently few will have ownership. But paradoxically a countertrend will play out in parallel. AI breakthroughs already make their way into open-source code repositories within days of being published in open-access journals, making topflight models easy for anyone to access, experiment with, build, and modify in turn. Models down to the weights are published, leaked, and stolen. — Página: [3370](kindle://book?action=open&asin=B0BSKW45KB&location=3370) ^ref-32948 --- It’s already a remarkable fact that corporate titans spend most of their lives working on software, like Gmail or Excel, accessible to most people on the planet. Extend that, radically, with the democratization of empowerment, when everyone on the planet has unfettered access to the most powerful technologies ever built. — Página: [3392](kindle://book?action=open&asin=B0BSKW45KB&location=3392) ^ref-18034 Nota: At times it seams like everyone has access to fhs same tools. Hoever they do not have the same power. --- Understanding the future means handling multiple conflicting trajectories at once. — Página: [3426](kindle://book?action=open&asin=B0BSKW45KB&location=3426) ^ref-19357 --- AI systems already make critical decisions with overt political implications: who receives a loan, a job, a place at college, parole; who gets seen by a senior physician. — Página: [3432](kindle://book?action=open&asin=B0BSKW45KB&location=3432) ^ref-62480 --- In all cases, the additional stress and volatility, the unpredictable amplification of power, the wrenching disruption of radical new centers of capability, will further stress the foundation of the liberal democratic nation-state system. — Página: [3443](kindle://book?action=open&asin=B0BSKW45KB&location=3443) ^ref-29034 Nota: Stress and problemss might be greater --- Catastrophic events that once took place over years and decades could happen in minutes, at the push of a button. With the coming wave, we are poised to take yet another such leap, expanding both the upper bound of risk and the number of avenues available to those seeking to unleash catastrophic force. — Página: [3471](kindle://book?action=open&asin=B0BSKW45KB&location=3471) ^ref-27156 --- And yet here’s the dilemma: the most secure solutions for containment are equally unacceptable, leading humanity down an authoritarian and dystopian pathway. — Página: [3482](kindle://book?action=open&asin=B0BSKW45KB&location=3482) ^ref-30448 Nota: The technology bused for containment is a problem to ? --- Once militaries are fully automated, the barriers to entry for conflict will be far lower. — Página: [3519](kindle://book?action=open&asin=B0BSKW45KB&location=3519) ^ref-57189 Nota: This might be true qnd a huge problem for the world since there is majod conflictsbhsppeninf today. --- What if multiple such pathogens were released at once? This goes far beyond fragility amplification; it would be an unfathomable calamity. — Página: [3533](kindle://book?action=open&asin=B0BSKW45KB&location=3533) ^ref-20331 Nota: The next pandemic is on the way. --- As AI gets more and more powerful, the most extreme scenarios will require serious consideration and mitigation. However, well before we get there, much could go wrong. — Página: [3547](kindle://book?action=open&asin=B0BSKW45KB&location=3547) ^ref-14396 Nota: Agibis a distant problem. We have a lot more to Worry now. --- AI is both valuable and dangerous precisely because it’s an extension of our best and worst selves. And as a technology premised on learning, it can keep adapting, probing, producing novel strategies and ideas potentially far removed from anything before considered, even by other AIs. — Página: [3553](kindle://book?action=open&asin=B0BSKW45KB&location=3553) ^ref-18167 --- We don’t yet know the implications of AI for fields as diverse as agriculture, chemistry, surgery, and finance. That’s part of the problem; we don’t know what failure modes are being introduced and how deep they could extend. — Página: [3563](kindle://book?action=open&asin=B0BSKW45KB&location=3563) ^ref-38988 Nota: Should we not find what are the problems and risks? --- Solving the question of AI alignment doesn’t mean doing so once; it means doing it every time a sufficiently powerful AI is built, wherever and whenever that happens. — Página: [3571](kindle://book?action=open&asin=B0BSKW45KB&location=3571) ^ref-27670 --- As the coming wave matures, however, the tools of destruction will, as we’ve seen, be democratized and commoditized. — Página: [3610](kindle://book?action=open&asin=B0BSKW45KB&location=3610) ^ref-40529 Nota: When technology spead sp does the dangers --- Regardless of where we are with BSL-4 protocols or regulatory proposals or technical publications on the AI alignment problem, those incentives grind away, the technologies keep developing and diffusing. — Página: [3620](kindle://book?action=open&asin=B0BSKW45KB&location=3620) ^ref-55858 Nota: Safedy comss after the technology development. --- Technology has penetrated our civilization so deeply that watching technology means watching everything. — Página: [3634](kindle://book?action=open&asin=B0BSKW45KB&location=3634) ^ref-42298 --- The door to dystopia is cracked open. Indeed, in the face of catastrophe, for some dystopia may feel like a relief. — Página: [3641](kindle://book?action=open&asin=B0BSKW45KB&location=3641) ^ref-20040 --- What level of societal control is appropriate to stopping an engineered pandemic? What level of interference in other countries is appropriate toward the same end? The consequences for liberty, sovereignty, and privacy have never been so potentially painful. — Página: [3653](kindle://book?action=open&asin=B0BSKW45KB&location=3653) ^ref-53247 Nota: Fhere is a lsvel where this thibgs are good? --- The philosopher of technology Lewis Mumford talked about the “megamachine,” where social systems combine with technologies to form “a uniform, all-enveloping structure” that is “controlled for the benefit of depersonalized collective organizations.” — Página: [3669](kindle://book?action=open&asin=B0BSKW45KB&location=3669) ^ref-32123 --- Despite the weightlessness of the digital world, there’s a solidness and a profusion to the material world around us. It shapes our everyday expectations. — Página: [3682](kindle://book?action=open&asin=B0BSKW45KB&location=3682) ^ref-19549 --- Twenty-first-century civilization is a long way from the Maya, naturally, but the pressures of a huge and hungry superstructure, a large population, the hard limits of energy and civilizational capacity have not magically gone away; they’ve just been kept at bay. — Página: [3693](kindle://book?action=open&asin=B0BSKW45KB&location=3693) ^ref-57193 Nota: Limits exist. That can be good or bad. --- Our entire edifice is premised on the idea of long-term economic growth. And long-term economic growth is ultimately premised on the introduction and diffusion of new technologies. — Página: [3697](kindle://book?action=open&asin=B0BSKW45KB&location=3697) ^ref-32329 --- it’s worth repeating: solving problems like climate change, or maintaining rising living and health-care standards, or improving education and opportunity is not going to happen without delivering — Página: [3705](kindle://book?action=open&asin=B0BSKW45KB&location=3705) ^ref-46282 Nota: Not sure that I agree with this. If looks to me that thdre is enought technolofy, awhat si lacking is distribuition. Both in technological and finantial terms. --- This is not only about numbers but about expertise, tax base, and investment levels; retirees will be pulling money out of the system, not investing it for the long term. — Página: [3718](kindle://book?action=open&asin=B0BSKW45KB&location=3718) ^ref-55752 --- Even if it were possible, the idea of stopping the coming wave isn’t a comforting thought. Maintaining, let alone improving, standards of living needs technology. Forestalling a collapse needs technology. The costs of saying no are existential. And yet every path from here brings grave risks and downsides. This is the great dilemma. — Página: [3738](kindle://book?action=open&asin=B0BSKW45KB&location=3738) ^ref-12260 --- I’m still convinced that technology remains a primary driver for making improvements to our world and our lives. — Página: [3749](kindle://book?action=open&asin=B0BSKW45KB&location=3749) ^ref-42988 Nota: That is the main point of view from the author . --- My profound worry is that technology is demonstrating the real possibility to sharply move net negative, that we don’t have answers to arrest this shift, and that we’re locked in with no way out. — Página: [3754](kindle://book?action=open&asin=B0BSKW45KB&location=3754) ^ref-11981 Nota: Another pomit of view from the author. --- Exponential change is coming. It is inevitable. That fact needs to be addressed. — Página: [3775](kindle://book?action=open&asin=B0BSKW45KB&location=3775) ^ref-44064 --- On paper regulation looks enticing, even obvious and straightforward; suggesting it lets people sound smart, concerned, and even relieved. — Página: [3788](kindle://book?action=open&asin=B0BSKW45KB&location=3788) ^ref-45527 Nota: It does sound easy. Buf necessary. --- Talking about the ethics of machine learning systems is a world away from, say, the technical safety of synthetic bio. These discussions happen in isolated, echoey silos. They rarely break out. — Página: [3810](kindle://book?action=open&asin=B0BSKW45KB&location=3810) ^ref-32556 --- The central problem for humanity in the twenty-first century is how we can nurture sufficient legitimate political power and wisdom, adequate technical mastery, and robust norms to constrain technologies to ensure they continue to do far more good than harm. How, in other words, we can contain the seemingly uncontainable. — Página: [3821](kindle://book?action=open&asin=B0BSKW45KB&location=3821) ^ref-62387 --- Regulators regulate for things they can anticipate. This, meanwhile, is an age of surprises. — Página: [3835](kindle://book?action=open&asin=B0BSKW45KB&location=3835) ^ref-60168 --- Regulation doesn’t just rely on the passing of a new law. It is also about norms, structures of ownership, unwritten codes of compliance and honesty, arbitration procedures, contract enforcement, oversight mechanisms. All of this needs to be integrated and the public needs to buy in. — Página: [3859](kindle://book?action=open&asin=B0BSKW45KB&location=3859) ^ref-18404 Nota: Regulation needs people that comply to them. --- Regulation is not enough, but at least it’s a start. Bold steps. A real understanding of the stakes involved in the coming wave. In a world where containment seems like it’s not possible, all of this gestures toward a future where it might be. — Página: [3894](kindle://book?action=open&asin=B0BSKW45KB&location=3894) ^ref-17117 --- Contained technology is technology whose modes of failure are known, managed, and mitigated, a situation where the means to shape and govern technology escalate in parallel with its capabilities. — Página: [3906](kindle://book?action=open&asin=B0BSKW45KB&location=3906) ^ref-48284 --- Before outlining a strategy it’s worth asking the following kinds of questions to prompt promising avenues: — Página: [3915](kindle://book?action=open&asin=B0BSKW45KB&location=3915) ^ref-41631 --- Is the technology omni-use and general-purpose or specific? — Página: [3916](kindle://book?action=open&asin=B0BSKW45KB&location=3916) ^ref-53550 --- Is the tech moving away from atoms toward bits? — Página: [3919](kindle://book?action=open&asin=B0BSKW45KB&location=3919) ^ref-61550 --- Are price and complexity coming down, and if so how fast? — Página: [3922](kindle://book?action=open&asin=B0BSKW45KB&location=3922) ^ref-57504 --- Are there viable alternatives ready to go? — Página: [3924](kindle://book?action=open&asin=B0BSKW45KB&location=3924) ^ref-60081 --- Does the technology enable asymmetric impact? — Página: [3926](kindle://book?action=open&asin=B0BSKW45KB&location=3926) ^ref-26939 --- Does it have autonomous characteristics? — Página: [3928](kindle://book?action=open&asin=B0BSKW45KB&location=3928) ^ref-42384 --- Does it confer outsized geopolitical strategic advantage? — Página: [3930](kindle://book?action=open&asin=B0BSKW45KB&location=3930) ^ref-36605 --- Does it favor offense or defense? — Página: [3932](kindle://book?action=open&asin=B0BSKW45KB&location=3932) ^ref-31538 --- Are there resource or engineering constraints on its invention, development, and deployment? — Página: [3934](kindle://book?action=open&asin=B0BSKW45KB&location=3934) ^ref-7956 --- The reality is, we have often not controlled or contained technologies in the past. And if we want to do so now, it would take something dramatically new, an all-encompassing program of safety, ethics, regulation, and control that doesn’t even really have a name and doesn’t seem possible in the first place. — Página: [3946](kindle://book?action=open&asin=B0BSKW45KB&location=3946) ^ref-53951 --- One challenge in even beginning to have this conversation is that technology, in the popular imagination, has become associated with a narrow band of often superfluous applications. “Technology” now mostly means social media platforms and wearable gadgets to measure our steps and heart rate. — Página: [3952](kindle://book?action=open&asin=B0BSKW45KB&location=3952) ^ref-16002 Nota: That is the problem of saying for example that someone works at a technology company. That are a lot of technology outside the so called technology compnay --- Technology is not a niche; it is a hyper-object dominating human existence. — Página: [3956](kindle://book?action=open&asin=B0BSKW45KB&location=3956) ^ref-64422 --- “Technology” is not, on the face of it, a problem in the same sense as a heating planet. And yet it might be. — Página: [3973](kindle://book?action=open&asin=B0BSKW45KB&location=3973) ^ref-44651 --- Technical safety, up close, in the code, in the lab, is the first item on any containment agenda. — Página: [4024](kindle://book?action=open&asin=B0BSKW45KB&location=4024) ^ref-46949 --- Frontier AI safety research is still an undeveloped, nascent field focusing on keeping ever more autonomous systems from superseding our ability to understand or control them. — Página: [4041](kindle://book?action=open&asin=B0BSKW45KB&location=4041) ^ref-41063 --- And yet safety decisions made today will alter the future course of technology and humanity. — Página: [4051](kindle://book?action=open&asin=B0BSKW45KB&location=4051) ^ref-56970 --- Concretely, a good proposal for legislation would be to require that a fixed portion—say, a minimum of 20 percent—of frontier corporate research and development budgets should be directed toward safety efforts, with an obligation to publish material findings to a government working group so that progress can be tracked and shared. — Página: [4053](kindle://book?action=open&asin=B0BSKW45KB&location=4053) ^ref-20989 --- Rather than give an AI a set of fixed external objectives contained in what’s known as a written constitution, he recommends that systems gingerly infer our preferences and ends. They should carefully watch and learn. In theory, this should leave more room for doubt within systems and avoid perverse outcomes. — Página: [4085](kindle://book?action=open&asin=B0BSKW45KB&location=4085) ^ref-17455 --- The highest-level challenge, whether in synthetic biology, robotics, or AI, is building a bulletproof off switch, a means of closing down any technology threatening to run out of control. — Página: [4096](kindle://book?action=open&asin=B0BSKW45KB&location=4096) ^ref-780 --- Actually having meaningful oversight and enforceable rules and reviewing technical implementations are vital. Technical safety advances and regulation will struggle to be effective if you can’t verify that they are working as intended. — Página: [4109](kindle://book?action=open&asin=B0BSKW45KB&location=4109) ^ref-20564 --- There has to be a well-defined, legal route to checking any new technology under the hood, in the code, in the lab, in the factory, or out in the wild. — Página: [4156](kindle://book?action=open&asin=B0BSKW45KB&location=4156) ^ref-32291 --- The wave can be slowed, at least for some period of time and in some areas. — Página: [4197](kindle://book?action=open&asin=B0BSKW45KB&location=4197) ^ref-19081 --- Buying time in an era of hyper-evolution is invaluable. Time to develop further containment strategies. Time to build in additional safety measures. Time to test that off switch. Time to build improved defensive technologies. Time to shore up the nation-state, regulate better, or even just get that bill passed. Time to knit together international alliances. — Página: [4198](kindle://book?action=open&asin=B0BSKW45KB&location=4198) ^ref-34882 --- So, as negative impacts become clear, we must use these choke points to create sensible rate-limiting factors, checks on the speed of development, to better ensure that good sense is implemented as fast as the science evolves. — Página: [4219](kindle://book?action=open&asin=B0BSKW45KB&location=4219) ^ref-5907 --- More than anyone else, those working on technology need to be actively working to solve the problems described in this book. The burden of proof and the burden of solutions rest on them, on us. — Página: [4230](kindle://book?action=open&asin=B0BSKW45KB&location=4230) ^ref-10435 --- The closer you are to a technology’s beating heart, the more you can affect outcomes, steer it in more positive directions, and block harmful applications. But this means also being part of what makes it a reality—for all the good and for all the harm it may do. — Página: [4252](kindle://book?action=open&asin=B0BSKW45KB&location=4252) ^ref-14435 Nota: This is ghe core os responsability. --- In a world of entrenched incentives and failing regulation, technology needs critics not just on the outside but at its beating heart. — Página: [4268](kindle://book?action=open&asin=B0BSKW45KB&location=4268) ^ref-9675 --- I believe that figuring out ways to reconcile profit and social purpose in hybrid organizational structures is the best way to navigate the challenges that lie ahead, but making it work in practice is incredibly hard. — Página: [4276](kindle://book?action=open&asin=B0BSKW45KB&location=4276) ^ref-43172 --- shareholder capitalism works because it is simple and clear, and governance models too have a tendency to default to the simple and clear. In the shareholder model, lines of accountability and performance tracking are quantified and very transparent. It may be possible to design more modern structures in theory, but operating them in practice is another story. — Página: [4302](kindle://book?action=open&asin=B0BSKW45KB&location=4302) ^ref-53098 --- Technology companies that have strong containment mechanisms and goals written in as a fiduciary duty are a next step. — Página: [4332](kindle://book?action=open&asin=B0BSKW45KB&location=4332) ^ref-45643 --- Containment needs a new generation of corporations. It needs founders and those working in tech to contribute positively to society. It also needs something altogether more difficult. It needs politics. — Página: [4336](kindle://book?action=open&asin=B0BSKW45KB&location=4336) ^ref-8442 --- Accountability is enabled by deep understanding. Ownership gives control. Both require governments to get their hands dirty. — Página: [4349](kindle://book?action=open&asin=B0BSKW45KB&location=4349) ^ref-17667 --- in the twenty-first century it doesn’t make sense to have cabinet positions addressing matters like the economy, education, security, and defense without a similarly empowered and democratically accountable position in technology. — Página: [4365](kindle://book?action=open&asin=B0BSKW45KB&location=4365) ^ref-6323 --- The most sophisticated AI systems or synthesizers or quantum computers should be produced only by responsible certified developers. — Página: [4383](kindle://book?action=open&asin=B0BSKW45KB&location=4383) ^ref-64923 Nota: This is a different for of regulation ghat is needed. --- Taxation also needs to be completely overhauled to fund security and welfare as we undergo the largest transition of value creation—from labor to capital—in history. — Página: [4390](kindle://book?action=open&asin=B0BSKW45KB&location=4390) ^ref-11920 --- Countries no more like giving up power than companies like missing out on profit, and yet these are precedents to learn from, shards of hope in a landscape riven with resurgent techno-competition. — Página: [4431](kindle://book?action=open&asin=B0BSKW45KB&location=4431) ^ref-60116 --- Best practices are hence not corporate secrets, an edge over rival airlines: they’re enthusiastically implemented by competitors in the common interests of collective industry trust and safety. — Página: [4502](kindle://book?action=open&asin=B0BSKW45KB&location=4502) ^ref-32753 --- While the tech industry talks a big game when it comes to “embracing failure,” it rarely does so when it comes to privacy or safety or technical breaches. — Página: [4506](kindle://book?action=open&asin=B0BSKW45KB&location=4506) ^ref-48013 Nota: Embracing does not mean been open about its failures. --- ask not just what doing no harm means in an age of globe-spanning algorithms and edited genomes but how that can be enacted daily in what are often morally ambiguous circumstances. — Página: [4543](kindle://book?action=open&asin=B0BSKW45KB&location=4543) ^ref-3379 --- So, if the invocation of the grand “we” is at present meaningless, it prompts an obvious follow-up: let’s build one. Throughout history change came about because people self-consciously worked for it. — Página: [4566](kindle://book?action=open&asin=B0BSKW45KB&location=4566) ^ref-49406 Nota: We need to byild the we that we want. --- The “we” that builds technology is scattered, subject to a mass of competing and different national, commercial, and research incentives. The more the “we” that is subject to it speaks clearly in one voice, a critical public mass agitating for change, demanding an alignment of approaches, the better chance of good outcomes. — Página: [4581](kindle://book?action=open&asin=B0BSKW45KB&location=4581) ^ref-43636 --- Kevin is determined to use every tool to stop them from happening. His program is one of the most holistic containment strategies around. It’s built around three pillars: delay, detect, and defend. — Página: [4597](kindle://book?action=open&asin=B0BSKW45KB&location=4597) ^ref-37208 --- Technical safety: Concrete technical measures to alleviate possible harms and maintain control. 2. Audits: A means of ensuring the transparency and accountability of technology. 3. Choke points: Levers to slow development and buy time for regulators and defensive technologies. 4. Makers: Ensuring responsible developers build appropriate controls into technology from the start. 5. Businesses: Aligning the incentives of the organizations behind technology with its containment. 6. Government: Supporting governments, allowing them to build technology, regulate technology, and implement mitigation measures. 7. Alliances: Creating a system of international cooperation to harmonize laws and programs. 8. Culture: A culture of sharing learning and failures to quickly disseminate means of addressing them. 9. Movements: All of this needs public input at every level, including to put pressure on each component and make it accountable. — Página: [4614](kindle://book?action=open&asin=B0BSKW45KB&location=4614) ^ref-52102 --- Safe, contained technology is, like liberal democracy, not a final end state; rather, it is an ongoing process, a delicate equilibrium that must be actively maintained, constantly fought for and protected. There’s no moment when we say, aha, we’ve solved the problem of proliferating technology! Instead, it’s about finding a way through, ensuring sufficient numbers of people are committed to keeping the unending balance between openness and closure. — Página: [4648](kindle://book?action=open&asin=B0BSKW45KB&location=4648) ^ref-5305 --- We should all get comfortable with living with contradictions in this era of exponential change and unfurling powers. Assume the worst, plan for it, give it everything. Stick doggedly to the narrow path. Get a world beyond the elites engaged and pushing. If enough people start building that elusive “we,” those glimmers of hope will become raging fires of change. — Página: [4689](kindle://book?action=open&asin=B0BSKW45KB&location=4689) ^ref-61790 --- Too many visions of the future start with what technology can or might do and work from there. That’s completely the wrong foundation. Technologists should focus not just on the engineering minutiae but on helping to imagine and realize a richer, social, human future in the broadest sense, a complex tapestry of which technology is just one strand. — Página: [4764](kindle://book?action=open&asin=B0BSKW45KB&location=4764) ^ref-44365 ---