Dr Rajiv Desai

Is artificial intelligence (AI) an existential threat?  

October 13, 2023 by Dr Rajiv Desai

Is artificial intelligence (AI) an existential threat?   

_

_

Section-1

Prologue:  

On September 26, 2022, without most of us noticing, humanity’s long-term odds of survival became ever so slightly better. Some 11,000,000 km away, a NASA spacecraft was deliberately smashed into the minor planet-moon, Dimorphos, and successfully changed its direction of travel. It was an important proof of concept that showed that if we’re ever in danger of being wiped out by an asteroid, we might be able to stop it from hitting the Earth. But what if the existential threat we need to worry about isn’t Deep Impact but Terminator? Despite years of efforts from professionals and researchers to quash any and all comparisons with apocalyptic science fiction and real-world artificial intelligence (AI), the threat of this technology going rogue and posing a serious threat to survival isn’t just for Hollywood movies. As crazy as it sounds, this is increasingly a threat that serious thinkers worry about. 

_

Asteroid impacts, climate change, and nuclear conflagration are all potential existential risks, but that is just the beginning. So are solar flares, super-volcanic eruptions, high-mortality pandemics, and even stellar explosions. All of these deserve more attention in the public debate. But one fear trumps the worries of existential risk researchers: Artificial Intelligence (AI). AI may be the ultimate existential risk. Warnings by prominent scientists like Stephan Hawking, twittering billionaire Elon Musk and an open letter signed in 2015 by more than 11,000 persons have raised public awareness of this still under-appreciated threat. Toby Ord estimates that the likelihood of AI causing human extinction is one in ten for the next hundred years.

_

‘Summoning the demon.’ ‘The new tools of our oppression.’ ‘Children playing with a bomb.’ These are just a few ways the world’s top researchers and industry leaders have described the threat that artificial intelligence poses to mankind. Will AI enhance our lives or completely upend them? There’s no way around it — artificial intelligence is changing human civilization, from how we work to how we travel to how we enforce laws. As AI technology advances and seeps deeper into our daily lives, its potential to create dangerous situations is becoming more apparent. A Tesla Model 3 owner in California died while using the car’s Autopilot feature. In Arizona, a self-driving Uber vehicle hit and killed a pedestrian (though there was a driver behind the wheel). Other instances have been more insidious. For example, when IBM’s Watson was tasked with helping physicians diagnose cancer patients, it gave numerous “unsafe and incorrect treatment recommendations.” Some of the world’s top researchers and industry leaders believe these issues are just the tip of the iceberg. What if AI advances to the point where its creators can no longer control it? How might that redefine humanity’s place in the world?

_

On May 30, 2023, distinguished AI scientists, including Turing Award winners Geoffrey Hinton and Yoshua Bengio, and leaders of the major AI labs, including Sam Altman of OpenAI and Demis Hassabis of Google DeepMind, have signed a single-sentence statement from the Center for AI Safety that reads:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

The idea that AI might become difficult to control, and either accidentally or deliberately destroy humanity, has long been debated by philosophers. But in the past six months, following some surprising and unnerving leaps in the performance of AI algorithms, the issue has become a lot more widely and seriously discussed. The statement comes at a time of growing concern about the potential harms of artificial intelligence. Recent advancements in so-called large language models — the type of AI system used by ChatGPT and other chatbots — have raised fears that AI could soon be used at scale to spread misinformation and propaganda, or that it could eliminate millions of white-collar jobs. President Biden warns artificial intelligence could ‘overtake human thinking’.    

The Centre for AI Safety website suggests a number of possible disaster scenarios:

• AIs could be weaponised – for example, drug-discovery tools could be used to build chemical weapons
• AI-generated misinformation could destabilise society and “undermine collective decision-making”
• The power of AI could become increasingly concentrated in fewer and fewer hands, enabling “regimes to enforce narrow values through pervasive surveillance and oppressive censorship”
• Enfeeblement, where humans become dependent on AI “similar to the scenario portrayed in the film Wall-E”

Not everyone is on board with the AI doomsday scenario, though. Yann LeCun, who won the Turing Award with Hinton and Bengio for the development of deep learning, has been critical of apocalyptic claims about advances in AI and has not signed the letter. And some AI researchers who have been studying more immediate issues, including bias and disinformation, believe that the sudden alarm over theoretical long-term risk distracts from the problems at hand. But others have argued that AI is improving so rapidly that it has already surpassed human-level performance in some areas, and that it will soon surpass it in others. They say the technology has shown signs of advanced abilities and understanding, giving rise to fears that “artificial general intelligence,” or AGI, a type of artificial intelligence that can match or exceed human-level performance at a wide variety of tasks, may not be far off.

_

A March 22, 2023 letter calling for a six-month “pause” on large-scale AI development beyond OpenAI’s GPT-4 highlights the complex discourse and fast-growing, fierce debate around AI’s various stomach-churning risks, both short-term and long-term.  The letter was published by the nonprofit Future of Life Institute, which was founded to “reduce global catastrophic and existential risk from powerful technologies” (founders include by MIT cosmologist Max Tegmark, Skype co-founder Jaan Tallinn, and DeepMind research scientist Viktoriya Krakovna). The letter says that “With more data and compute, the capabilities of AI systems are scaling rapidly. The largest models are increasingly capable of surpassing human performance across many domains. No single company can forecast what this means for our societies.”  The letter points out that superintelligence is far from the only harm to be concerned about when it comes to large AI models — the potential for impersonation and disinformation are others. The letter warned of an “out-of-control race” to develop minds that no one could “understand, predict, or reliably control”.

Critics of the letter — which was signed by Elon Musk, Steve Wozniak, Yoshua Bengio, Gary Marcus and several thousand other AI experts, researchers and industry leaders — say it fosters unhelpful alarm around hypothetical dangers, leading to misinformation and disinformation about actual, real-world concerns. A group of well-known AI ethicists have written a counterpoint to this letter criticizing it for a focus on hypothetical future threats when real harms are attributable to misuse of the tech today; for example inexpensive fake pictures and videos are widely available, and indistinguishable from the real thing, which is completely reshaping the way in which humans associate truth with evidence. Even those who doubt whether Artificial General Intelligence (AGI) capable of accomplishing any goal, will be created in the future, still agree that AI will have profound implications for all domains, including: healthcare, law, and national security. 

_

In 2018 at the World Economic Forum in Davos, Google CEO Sundar Pichai had something to say: “AI is probably the most important thing humanity has ever worked on. I think of it as something more profound than electricity or fire.” Pichai’s comment was met with a healthy dose of skepticism. But nearly five years later, it’s looking more and more prescient. AI translation is now so advanced that it’s on the brink of obviating language barriers on the internet among the most widely spoken languages. College professors are tearing their hair out because AI text generators can now write essays as well as your typical undergraduate — making it easy to cheat in a way no plagiarism detector can catch. AI-generated artwork is even winning state fairs. A new tool called Copilot uses machine learning to predict and complete lines of computer code, bringing the possibility of an AI system that could write itself one step closer. DeepMind’s AlphaFold system, which uses AI to predict the 3D structure of just about every protein in existence, was so impressive that the journal Science named it 2021’s Breakthrough of the Year. While innovation in other technological fields can feel sluggish — as anyone waiting for the metaverse would know — AI is full steam ahead. The rapid pace of progress is feeding on itself, with more companies pouring more resources into AI development and computing power.

_

Many prominent AI experts have recognized the possibility that AI presents an existential risk. Contrary to misrepresentations in the media, this risk need not arise from spontaneous malevolent consciousness. Rather, the risk arises from the unpredictability and potential irreversibility of deploying an optimization process more intelligent than the humans who specified its objectives. For now, we are nowhere near artificial general intelligence i.e., machine thinking. No current technology is even on the pathway to it. Today AI is used as a grandiose label for machine learning. Machine learning systems find patterns in immense datasets, producing models that statistically recognizes same patterns in similar data, or generate new data statistically “like”, resembling intricate tessellations of data they started with. Such programs are valuable and do interesting work, but they are not thinking. We have major advances in the accomplishments of deep neural networks— artificial neural networks with multiple layers between the input and output layers—across a wide range of areas, including game-playing, speech and facial recognition, and image generation. Even with these breakthroughs though, the cognitive capabilities of current AI systems remain limited to domain-specific applications. Nevertheless, many researchers are alarmed by the speed of progress in AI and worry that future systems, if not managed correctly, could present an existential threat. By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it. Although AI is hard, it is very easy for people to think they know far more about Artificial Intelligence than they actually do. Artificial Intelligence is not settled science; it belongs to the frontier, not to the textbook. I have already written on artificial intelligence on March 23, 2017 in this website and concluded that AI can improve human performance and decision-making, and augment human creativity and intelligence but not replicate it. However, AI has advanced so much and so fast in last 6 years that I am compelled to review my thoughts on AI. Today my endeavour is to study whether AI pose existential threat to humanity or such threat is overblown.     

_____

_____

Abbreviations and synonyms:

ML = machine learning

DL = deep learning

GPT = generative pre-trained transformer

AGI = artificial general intelligence

ANI = artificial narrow intelligence

HLAI = human-level artificial intelligence

ASI = artificial superintelligence

GPU = graphics processing unit

TPU = tensor processing unit

NLP = natural language processing

GAI = generative artificial intelligence

LLM = large language model

LaMDA = language model for dialogue applications

RLHF = reinforcement learning from human feedback

DNN = deep neural network

ANN = artificial neural network

CNN = convolutional neural network

RNN = recurrent neural network

NCC = neural correlates of consciousness

ACT = AI consciousness test

_____  

_____   

Section-2

Existential threats to humanity:

A global catastrophic risk or a doomsday scenario is a hypothetical future event that could damage human well-being on a global scale, even endangering or destroying modern civilization. An event that could cause human extinction or permanently and drastically curtail humanity’s potential is known as an “existential risk.” Humanity has suffered large catastrophes before. Some of these have caused serious damage but were only local in scope—e.g., the Black Death may have resulted in the deaths of a third of Europe’s population, 10% of the global population at the time. Some were global, but were not as severe—e.g., the 1918 influenza pandemic killed an estimated 3–6% of the world’s population. Most global catastrophic risks would not be so intense as to kill the majority of life on earth, but even if one did, the ecosystem and humanity would eventually recover (in contrast to existential risks).

_

Existential risks are defined as “risks that threaten the destruction of humanity’s long-term potential.” The instantiation of an existential risk (an existential catastrophe) would either cause outright human extinction or irreversibly lock in a drastically inferior state of affairs. Existential risks are a sub-class of global catastrophic risks, where the damage is not only global but also terminal and permanent, preventing recovery and thereby affecting both current and all future generations. Human extinction is the hypothetical end of the human species due to either natural causes such as population decline from sub-replacement fertility, an asteroid impact, large-scale volcanism, or via anthropogenic destruction (self-extinction). For the latter, some of the many possible contributors include climate change, global nuclear annihilation, biological warfare, and ecological collapse. Other scenarios center on emerging technologies, such as advanced artificial intelligence, biotechnology, or self-replicating nanobots. The scientific consensus is that there is a relatively low risk of near-term human extinction due to natural causes. The likelihood of human extinction through humankind’s own activities, however, is a current area of research and debate.

_

An existential risk is any risk that has the potential to eliminate all of humanity or, at the very least, kill large swaths of the global population, leaving the survivors without sufficient means to rebuild society to current standards of living.

Until relatively recently, most existential risks (and the less extreme version, known as global catastrophic risks) were natural, such as the super volcanoes and asteroid impacts that led to mass extinctions millions of years ago. The technological advances of the last century, while responsible for great progress and achievements, have also opened us up to new existential risks.

Nuclear war was the first man-made global catastrophic risk, as a global war could kill a large percentage of the human population. As more research into nuclear threats was conducted, scientists realized that the resulting nuclear winter could be even deadlier than the war itself, potentially killing most people on earth.

Biotechnology and genetics often inspire as much fear as excitement, as people worry about the possibly negative effects of cloning, gene splicing, gene drives, and a host of other genetics-related advancements. While biotechnology provides incredible opportunity to save and improve lives, it also increases existential risks associated with manufactured pandemics and loss of genetic diversity.

Artificial intelligence (AI) has long been associated with science fiction, but it’s a field that’s made significant strides in recent years. As with biotechnology, there is great opportunity to improve lives with AI, but if the technology is not developed safely, there is also the chance that someone could accidentally or intentionally unleash an AI system that ultimately causes the elimination of humanity.

Climate change is a growing concern that people and governments around the world are trying to address. As the global average temperature rises, droughts, floods, extreme storms, and more could become the norm. The resulting food, water and housing shortages could trigger economic instabilities and war. While climate change itself is unlikely to be an existential risk, the havoc it wreaks could increase the likelihood of nuclear war, pandemics or other catastrophes.

_

Probability of existential risk: Natural vs. Anthropogenic:

Experts generally agree that anthropogenic existential risks are (much) more likely than natural risks. A key difference between these risk types is that empirical evidence can place an upper bound on the level of natural risk. Humanity has existed for at least 200,000 years, over which it has been subject to a roughly constant level of natural risk. If the natural risk were sufficiently high, then it would be highly unlikely that humanity would have survived as long as it has. Based on a formalization of this argument, researchers have concluded that we can be confident that natural risk is lower than 1 in 14,000 per year.

Another empirical method to study the likelihood of certain natural risks is to investigate the geological record. For example, a comet or asteroid impact event sufficient in scale to cause an impact winter that would cause human extinction before the year 2100 has been estimated at one-in-a-million. Moreover, large super volcano eruptions may cause a volcanic winter that could endanger the survival of humanity. The geological record suggests that super volcanic eruptions are estimated to occur on average about once every 50,000 years, though most such eruptions would not reach the scale required to cause human extinction. Famously, the super volcano Mt. Toba may have almost wiped out humanity at the time of its last eruption (though this is contentious).

Since anthropogenic risk is a relatively recent phenomenon, humanity’s track record of survival cannot provide similar assurances. Humanity has only survived 78 years since the creation of nuclear weapons, and for future technologies, there is no track record at all. This has led thinkers like Carl Sagan to conclude that humanity is currently in a “time of perils” – a uniquely dangerous period in human history, where it is subject to unprecedented levels of risk, beginning from when humans first started posing risk to themselves through their actions.

A 2016 survey of AI experts found a median estimate of 5% that human-level AI would cause an outcome that was “extremely bad (e.g., human extinction)”.

_

Existential risks are typically distinguished from the broader category of global catastrophic risks. Bostrom (2013), for example, uses two dimensions—scope and severity—to make this distinction. Scope refers to the number of people at risk, while severity refers to how badly the population in question would be affected. Existential risks are at the most extreme end of both of these spectrums: they are pan-generational in scope (i.e., “affecting humanity over all, or almost all, future generations”), and they are the severest kinds of threats, causing either “death or a permanent and drastic reduction of quality of life”. Perhaps the clearest example of an existential risk is an asteroid impact on the scale of that which hit the Earth 66 million years ago, wiping out the dinosaurs (Schulte et al., 2010; ÓhÉigeartaigh, 2017). Global catastrophic risks, by way of contrast, could be either just as severe but narrower in scope, or just as broad but less severe. Some examples include the destruction of cultural heritage, thinning of the ozone layer, or even a large-scale pandemic outbreak (Bostrom, 2013).

_

Today, the worldwide arsenal of nuclear weapons could lead to unprecedented death tolls and habitat destruction and, hence, it poses a clear global catastrophic risk. Still, experts assign a relatively low probability to human extinction from nuclear warfare (Martin, 1982; Sandberg & Bostrom, 2008; Shulman, 2012). This is in part because it seems more likely that extinction, if it follows at all, would occur indirectly from the effects of the war, rather than directly. This distinction has appeared in several discussions on existential risks (e.g., Matheny, 2007, Liu et al., 2018; Zwetsloot & Dafoe, 2019), but it is made most explicitly in Cotton-Barratt et al. (2020), who explain that a global catastrophe that causes human extinction can do so either directly by “killing everyone”, or indirectly, by “removing our ability to continue flourishing over a longer period.” A nuclear explosion itself is unlikely to kill everyone directly, but the resulting effects it has on the Earth could lead to lands becoming uninhabitable, in turn leading to a scarcity of essential resources, which could (over a number of years) lead to human extinction. Some of the simplest examples of direct risks of human extinction, by way of contrast, are “if the entire planet is struck by a deadly gamma ray burst, or enough of a deadly toxin is dispersed through the atmosphere”. What’s critical here is that for an existential risk to be direct it has to be able to reach everyone.

_

Much like nuclear fallout, the arguments for why and how AI poses an existential risk are not straightforward. This is partly because AI is a general-purpose technology. It has a wide range of potential uses, for a wide range of actors, across a wide range of sectors. Here we are interested in the extent to which the use or misuse of AI can play a sine qua non role in existential risk scenarios, across any of these domains. We are interested not only in current AI capabilities, but also in future (potential) capabilities. Depending on how the technology develops, AI could pose either a direct or indirect risk, although we make the case that direct existential risks from AI are even more improbable than indirect ones. Another helpful way of thinking about AI risks is to divide them into accidental risks, structural risks, or misuse risks (Zwetsloot & Dafoe, 2019). Accidental risks occur due to some glitch, fault, or oversight that causes an AI system to exhibit unexpected harmful behaviour. Structural risks of AI are those caused by how a system shapes the broader environment in ways that could be disruptive or harmful, especially in the political and military realms. And finally misuse risks are those caused by the deliberate use of AI in a harmful manner.

_

In the scientific field of existential risk, which studies the most likely causes of human extinction, AI is consistently ranked at the top of the list. In The Precipice, a book by Oxford existential risk researcher Toby Ord that aims to quantify human extinction risks, the likeliness of AI leading to human extinction exceeds that of climate change, pandemics, asteroid strikes, super volcanoes, and nuclear war combined. One would expect that even for severe global problems, the risk that they lead to full human extinction is relatively small, and this is indeed true for most of the above risks. AI, however, may cause human extinction if only a few conditions are met. Among them is human-level AI, defined as an AI that can perform a broad range of cognitive tasks at least as well as we can. Studies outlining these ideas were previously known, but new AI breakthroughs have underlined their urgency: AI may be getting close to human level already. Recursive self-improvement is one of the reasons why existential-risk academics think human-level AI is so dangerous. Because human-level AI could do almost all tasks at our level, and since doing AI research is one of those tasks, advanced AI should therefore be able to improve the state of AI. Constantly improving AI would create a positive feedback loop with no scientifically established limits: an intelligence explosion. The endpoint of this intelligence explosion could be a superintelligence: a godlike AI that outsmarts us the way humans often outsmart insects. We would be no match for it.

_

One of the sources of risk that is estimated to contribute the most to the total amount of risk currently faced by humanity is that of unaligned artificial intelligence, which Toby Ord estimates to pose a one-in-ten chance of existential catastrophe in the coming century. Several books including Bostroms ‘Superintelligence’, Stuart Russell’s ‘Human Compatible’, and Brian Chrisitian’s ‘The Alignment Problem’, as well as numerous articles have addressed the alignment problem of how to ensure that the values of any advanced AI systems developed in the coming years, decades, or centuries are aligned with those of our species.

Superintelligence: Paths, Dangers, Strategies is an astonishing book with an alarming thesis: Intelligent machines are “quite possibly the most important and most daunting challenge humanity has ever faced.” In it, Oxford University philosopher Nick Bostrom, who has built his reputation on the study of “existential risk,” argues forcefully that artificial intelligence might be the most apocalyptic technology of all. With intellectual powers beyond human comprehension, he prognosticates, self-improving artificial intelligences could effortlessly enslave or destroy Homo sapiens if they so wished. While he expresses skepticism that such machines can be controlled, Bostrom claims that if we program the right “human-friendly” values into them, they will continue to uphold these virtues, no matter how powerful the machines become.

These views have found an eager audience. In August 2014, PayPal cofounder and electric car magnate Elon Musk tweeted “Worth reading Superintelligence by Bostrom. We need to be super careful with AI. Potentially more dangerous than nukes.” Bill Gates declared, “I agree with Elon Musk and some others on this and don’t understand why some people are not concerned.” More ominously, legendary astrophysicist Stephen Hawking concurred: “I think the development of full artificial intelligence could spell the end of the human race.”

_

AI development has reached a milestone, known as the Turing Test, which means machines have the ability to converse with humans in a sophisticated fashion, Yoshua Bengio says. The idea that machines can converse with us, and humans don’t realize they are talking to an AI system rather than another person, is scary, he added. Bengio worries the technology could lead to an automation of trolls on social media, as AI systems have already “mastered enough knowledge to pass as human.”

Canadian-British artificial intelligence pioneer Geoffrey Hinton says he left Google because of recent discoveries about AI that made him realize it poses a threat to humanity. In 2023 Hinton quit his job at Google in order to speak out about existential risk from AI. He explained that his increased concern was driven by concerns that superhuman AI might be closer than he’d previously believed, saying: “I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” He also remarked, “Look at how it was five years ago and how it is now. Take the difference and propagate it forwards. That’s scary.”

_

Hinton’s concerns are a result of the knowledge that AI has the potential to go out of control and endanger humans. He originally saw the promise of AI in speech recognition, picture analysis, and translation, but more recent advancements, particularly those involving big language models, have made him take a closer look at the broader effects of AI development. Hinton presented the fundamentals of AI, emphasizing its capacity for language comprehension, knowledge transfer across models, and improved learning algorithms.  He disputes the popular understanding that anthropomorphizing robots is inappropriate, contending that AI systems educated on human-generated data may exhibit language-related behavior that is more plausible than previously thought. AI, according to its detractors, lacks first-hand knowledge and can only forecast outcomes based on statistical patterns. Hinton refutes this by arguing that people also have indirect experiences of the environment through perception and interpretation. The ability of AI to anticipate and comprehend language suggests understanding and engagement with the outside world.

_

Concern over risk from artificial intelligence has led to some high-profile donations and investments. In 2015, Peter Thiel, Amazon Web Services, and Musk and others jointly committed $1 billion to OpenAI, consisting of a for-profit corporation and the nonprofit parent company which states that it is aimed at championing responsible AI development. Facebook co-founder Dustin Moskovitz has funded and seeded multiple labs working on AI Alignment, notably $5.5 million in 2016 to launch the Centre for Human-Compatible AI led by Professor Stuart Russell. In January 2015, Elon Musk donated $10 million to the Future of Life Institute to fund research on understanding AI decision making. The goal of the institute is to “grow wisdom with which we manage” the growing power of technology. Musk also funds companies developing artificial intelligence such as DeepMind and Vicarious to “just keep an eye on what’s going on with artificial intelligence, saying “I think there is potentially a dangerous outcome there.”

_

The current tone of alarm is tied to several leaps in the performance of AI algorithms known as large language models. These models consist of a specific kind of artificial neural network that is trained on enormous quantities of human-written text to predict the words that should follow a given string. When fed enough data, and with additional training in the form of feedback from humans on good and bad answers, these language models are able to generate text and answer questions with remarkable eloquence and apparent knowledge—even if their answers are often riddled with mistakes.

These language models have proven increasingly coherent and capable as they have been fed more data and computer power. The most powerful model created so far, OpenAI’s GPT-4, is able to solve complex problems, including ones that appear to require some forms of abstraction and common sense reasoning.

ChatGPT and other advanced chatbots can hold coherent conversations and answer all manner of questions with the appearance of real understanding. But these programs also exhibit biases, fabricate facts, and can be goaded into behaving in strange and unpleasant ways.

_

Predicting the potential catastrophic impacts of AI is made difficult by several factors. Firstly, the risk posed by AI is unprecedented and cannot be reliably assessed using historical data and extrapolation, unlike the other types of existential risks (space weather, super-volcanoes, and pandemics). Secondly, with respect to general and super-intelligence, it may be practically or even inherently impossible for us to predict how a system more intelligent than us will act (Yampolskiy 2020). Considering the difficulty of predicting catastrophic societal impacts associated with AI, we are limited here to hypothetical scenarios of basic pathways describing how the disastrous outcomes could be manifested.

_

When considering potential global catastrophic or existential risks stemming from AI, it is useful to distinguish between narrow AI and AGI, as the speculated possible outcomes associated with each type can differ greatly. For narrow AI systems to cause catastrophic outcomes, the potential scenarios include events such as software viruses affecting hardware or critical infrastructure globally, AI systems serving as weapons of mass destruction (such as slaughter-bots), or AI-caused biotechnological or nuclear catastrophe (Turchin and Denkenberger 2018a, b; Tegmark 2017; Freitas 2000). Interestingly, Turchin and Denkenberger (2018a) argue that the catastrophic risks stemming from narrow AI are relatively neglected despite their potential to materialize sooner than the risks from AGI. Still, the probability of narrow AI to cause an existential catastrophe appears to be relatively lower than in the case of AGI (Ord 2020).

_

With respect to the global catastrophic and existential risk of misaligned AGI, much of the expected risk seems to lie in an AGI system’s extraordinary ability to pursue its goals. According to Bostrom’s instrumental convergence thesis, instrumental aims such as developing more resources and/or power for gaining control over humans would be beneficial for achieving almost any final goal the AGI system might have (Bostrom 2012). Hence, it can be argued that almost any misaligned AGI system would be motivated to gain control over humans (often described in the literature as a ‘decisive strategic advantage’) to eliminate the possibility of human interference with the system’s pursuit of its goals (Bostrom 2014; Russell 2019). Once humans have been controlled, the system would be free to pursue its main goal, whatever that might be.

_

In line with this instrumental convergence thesis, an AGI system whose values are not perfectly aligned with human values would be likely to pursue harmful instrumental goals, including seizing control and thus potentially creating catastrophic outcomes for humanity (Russell and Norvig 2016; Bostrom 2002, 2003a; Taylor et al. 2016; Urban 2015; Ord 2020; Muehlhauser 2014). A popular example of such a scenario is the paperclip maximizer, which firstly appeared in a mailing list of AI researchers in the early 2000’s (Harris 2018); a later version is included in Bostrom (2003a). Most versions of this scenario involve an AGI system with an arbitrary goal of manufacturing paperclips. In pursuit of this goal, it will inevitably transform Earth into a giant paperclip factory and therefore destroy all life on it. There are other scenarios that end up with potential extinction. Ord (2020) presents one in which the system increases its computational resources by hacking other systems, which enables it to gain financial and human resources to further increase its power in pursuit of its defined goal.

_

42% of CEOs say AI could destroy humanity in five to ten years:

Many top business leaders are seriously worried that artificial intelligence could pose an existential threat to humanity in the not-too-distant future. Forty-two percent of CEOs surveyed at the Yale CEO Summit in June 2023 say AI has the potential to destroy humanity five to ten years from now. The survey, conducted at a virtual event held by Sonnenfeld’s Chief Executive Leadership Institute, found little consensus about the risks and opportunities linked to AI. The survey included responses from 119 CEOs from a cross-section of business, including Walmart CEO Doug McMillion, Coca-Cola CEO James Quincy, the leaders of IT companies like Xerox and Zoom as well as CEOs from pharmaceutical, media and manufacturing.

49% of IT professionals believe AI poses an existential threat to humanity: 

According to the data presented by the Atlas VPN team, 49% of IT professionals believe innovation in AI presents an existential threat to humanity. Despite that, many other experts see AI as a companion who helps with various tasks rather than a future enemy. The data is based on Spiceworks Ziff Davis’s The 2023 State of IT report on IT budgets and tech trends. The research surveyed more than 1,400 IT professionals representing companies in North America, Europe, Asia, and Latin America in June 2022 to gain visibility into how organizations plan to invest in technology.

_____  

_____

Section-3

Introduction to AI:

Please read my articles Artificial Intelligence published on March 23, 2017; and Quantum Computing published on April 12, 2020. 

_

Back in October 1950, British techno-visionary Alan Turing published an article called “Computing Machinery and Intelligence,” in the journal MIND that raised what at the time must have seemed to many like a science-fiction fantasy. “May not machines carry out something which ought to be described as thinking but which is very different from what a man does?” Turing asked.

Turing thought that they could. Moreover, he believed, it was possible to create software for a digital computer that enabled it to observe its environment and to learn new things, from playing chess to understanding and speaking a human language. And he thought machines eventually could develop the ability to do that on their own, without human guidance. “We may hope that machines will eventually compete with men in all purely intellectual fields,” he predicted.

Nearly 73 years later, Turing’s seemingly outlandish vision has become a reality. Artificial intelligence, commonly referred to as AI, gives machines the ability to learn from experience and perform cognitive tasks, the sort of stuff that once only the human brain seemed capable of doing.

AI is rapidly spreading throughout civilization, where it has the promise of doing everything from enabling autonomous vehicles to navigate the streets to making more accurate hurricane forecasts. On an everyday level, AI figures out what ads to show you on the web, and powers those friendly chatbots that pop up when you visit an e-commerce website to answer your questions and provide customer service. And AI-powered personal assistants in voice-activated smart home devices perform myriad tasks, from controlling our TVs and doorbells to answering trivia questions and helping us find our favourite songs.

But we’re just getting started with it. As AI technology grows more sophisticated and capable, it’s expected to massively boost the world’s economy, creating about $13 trillion worth of additional activity by 2030, according to a McKinsey Global Institute forecast.

_

Intelligence as achieving goals:

Twenty-first century AI research defines intelligence in terms of goal-directed behavior. It views intelligence as a set of problems that the machine is expected to solve — the more problems it can solve, and the better its solutions are, the more intelligent the program is. AI founder John McCarthy defined intelligence as “the computational part of the ability to achieve goals in the world.” Stuart Russell and Peter Norvig formalized this definition using abstract intelligent agents. An “agent” is something which perceives and acts in an environment. A “performance measure” defines what counts as success for the agent. “If an agent acts so as to maximize the expected value of a performance measure based on past experience and knowledge then it is intelligent.”

Definitions like this one try to capture the essence of intelligence. They have the advantage that, unlike the Turing test, they do not also test for unintelligent human traits such as making typing mistakes. They have the disadvantage that they can fail to differentiate between “things that think” and “things that do not”. By this definition, even a thermostat has a rudimentary intelligence.

_

What is AI?

Artificial intelligence (AI) is a branch of computer science that aims to evolve intelligent machines or computer systems which can simulate various human cognitive functions like learning, reasoning, and perception. From voice‑powered personal assistants ‘Siri’ and ‘Alexa’ to more complex applications like Fuzzy logic systems and AIoT (a combination of AI technologies with the Internet of Things), the prospects of AI are infinite.

Artificial Intelligence (AI) refers to a class of hardware and software systems that can be said to be ‘intelligent’ in a broad sense of the word. Sometimes this refers to the system’s ability to take actions to achieve predetermined goals but can also refer to particular abilities linked to intelligence, such as understanding human speech. AI is already all around us, from the underlying algorithms that power automatic translation services, to the way that digital media providers learn your preferences to show you the content most relevant to your interests. Despite incredible advances in AI technology in the past decade, current AI capabilities are likely to be just the tip of the iceberg compared to what could be possible in the future.

AGI and Superintelligence:

One of the main contributions to current estimates of the existential risk from AI is the prospect of incredibly powerful AI systems that may come to be developed in the future. These differ from current AI systems, sometimes called ‘Narrow AI’. There are many different terms that researchers have used when discussing systems, each with its own subtle definition. These include: Artificial General Intelligence (AGI), an AI system that is proficient in all aspects of human intelligence; Human-Level Artificial Intelligence (HLAI), an AI that can at least match human capabilities in all aspects of intelligence; and Artificial Superintelligence (ASI), an AI system that is greatly more capable than humans in all areas of intelligence.

_

Why is AI needed?

AI makes every process better, faster, and more accurate. It has some very crucial applications too such as identifying and predicting fraudulent transactions, faster and accurate credit scoring, and automating manually intense data management practices. Artificial Intelligence improves the existing process across industries and applications and also helps in developing new solutions to problems that are overwhelming to deal with manually. The basic goal of AI is to enable computers and machines to perform intellectual tasks such as problem solving, decision making, perception, and understanding human communication.

_

The era of artificial intelligence (AI) looms large with the progresses of computer technology. Computer intelligence is catching up with, or surpassing, humans in intellectual capabilities. Sovereign of machine intelligence looks imminent and unavoidable. Computer’s intelligence has surprised people a couple of times in the past 70 years. In 1940s, computers outperformed humans in numerical computations. In 1970s, computers showed their word/text processing ability. Then, Deep-Blue, Alpha-go, and Alpha-zero beat best chess players in succession. Recent AI chatbots ChatGPT and DaLL-E have stunned people with their competence in chatting, writing, knowledge integrating, and painting. Computer capabilities have repeatedly broken people’s expectations of machine capabilities. Progress of computer’s intelligence goes with computer’s speed. As computing speed increases, a computer is able to handle larger amount of data and knowledge, evaluate more options, make better decisions, and learn better from its own experience.

_

Artificial Intelligence (AI) is rapidly becoming the dominant technology in different industries, including manufacturing, health, finance, and service. It has demonstrated outstanding performance in well-defined domains such as image and voice recognition and the services based on these domains. It also shows promising performance in many other areas, including but not limited to behavioral predictions. However, all these AI capabilities are rather primitive compared to those of nature-made intelligent systems such as bonobos, felines, and humans because AI capabilities, in essence, are derived from classification or regression methods. 

Most AI agents are essentially superefficient classification or regression algorithms, optimized (trained) for specific tasks; they learn to classify discreet labels or regress continuous outcomes and use that training to achieve their assigned goals of prediction or classification (Toreini et al., 2020). An AI system is considered fully functional and useful as long as it performs its specific tasks as intended. Thus, from a utilitarian perspective, AI can meet its goals without any need to match the capabilities of nature-made systems or to exhibit the complex forms of intelligence found in those systems. As Chui, Manyika, and Miremadi (2015) stated, an increasing number of tasks performed in well-paying fields such as finance and medicine can be successfully carried out using current AI technology. However, this does not stop AI from developing more complex forms of intelligence capable of partaking in more human-centered service tasks

_

Is AI all about Advanced Algorithms?

While algorithms are a crucial component of AI, they are not the only aspect. AI also involves data collection, processing, and interpretation to derive meaningful insights.

AI systems use algorithms and data to learn from their environment and improve their performance over time. This process is known as machine learning, and it involves training the system with large amounts of data to identify patterns and make predictions. However, AI also involves other components, such as natural language processing, image recognition, speech recognition, and decision-making systems.

It’s helpful in various industries, including healthcare, finance, transportation, and education. While algorithms are an essential component of AI, AI is a much broader field that involves many other components and applications.

_

Mankind is vulnerable to threats from even the very small irregularities like a virus. We might be very excited about the wonderful results of AI. But we should also prepare for the threats it will bring to the world. The damage it is causing and the damage it has already caused.

Let’s take a very simple scenario:

Y= mX + c

In the above equation, we are all aware Y is a linearly dependent variable on the term X. But, let us think of m and c. In the equation, if m and c are dependent on the time then this equation after every moment will have a different form. Those forms sometimes can be so random that we might be aware of the process going on but we won’t be able to get those results that our ML model can have. In this scenario, the main point to understand is that even a small modification in an approach can take us beyond the scope of human predictions. Machine Learning is a science-based on Mathematical approach which will someday be hard to understand and conquer.

_

The various sub-fields of AI research are centered around particular goals and the use of particular tools. The traditional goals of AI research include reasoning, knowledge representation, planning, learning, natural language processing, perception, and support for robotics. General intelligence (the ability to solve an arbitrary problem) is among the field’s long-term goals. To solve these problems, AI researchers have adapted and integrated a wide range of problem-solving techniques, including search and mathematical optimization, formal logic, artificial neural networks, and methods based on statistics, probability, and economics. AI also draws upon psychology, linguistics, philosophy, neuroscience and many other fields.

_

Some researchers distinguish between “narrow AI” — computer systems that are better than humans in some specific, well-defined field, like playing chess or generating images or diagnosing cancer — and “general AI,” systems that can surpass human capabilities in many domains. We don’t have general AI yet, but we’re starting to get a better sense of the challenges it will pose.

Narrow AI has seen extraordinary progress over the past few years. AI systems have improved dramatically at translation, at games like chess and Go, at important research biology questions like predicting how proteins fold, and at generating images. AI systems determine what you’ll see in a Google search or in your Facebook Newsfeed. They compose music and write articles that, at a glance, read as if a human wrote them. They play strategy games. They are being developed to improve drone targeting and detect missiles.

But narrow AI is getting less narrow. We made progress in AI by painstakingly teaching computer systems specific concepts. To do computer vision — allowing a computer to identify things in pictures and video — researchers wrote algorithms for detecting edges. To play chess, they programmed in heuristics about chess. To do natural language processing (speech recognition, transcription, translation, etc.), they drew on the field of linguistics.

But recently, we’ve gotten better at creating computer systems that have generalized learning capabilities. Instead of mathematically describing detailed features of a problem, we let the computer system learn that by itself. While once we treated computer vision as a completely different problem from natural language processing or platform game playing, now we can solve all three problems with the same approaches.

And as computers get good enough at narrow AI tasks, they start to exhibit more general capabilities. For example, OpenAI’s famous GPT-series of text AIs is, in one sense, the narrowest of narrow AIs — it just predicts what the next word will be in a text, based on the previous words and its corpus of human language. And yet, it can now identify questions as reasonable or unreasonable and discuss the physical world (for example, answering questions about which objects are larger or which steps in a process must come first). In order to be very good at the narrow task of text prediction, an AI system will eventually develop abilities that are not narrow at all.

_

Our AI progress so far has enabled enormous advances — and has also raised urgent ethical questions. When you train a computer system to predict which convicted felons will reoffend, you’re using inputs from a criminal justice system biased against black people and low-income people — and so its outputs will likely be biased against black and low-income people too. Making websites more addictive can be great for your revenue but bad for your users. Releasing a program that writes convincing fake reviews or fake news might make those widespread, making it harder for the truth to get out.

_

Rosie Campbell at UC Berkeley’s Center for Human-Compatible AI argues that these are examples, writ small, of the big worry experts have about general AI in the future. The difficulties we’re wrestling with today with narrow AI don’t come from the systems turning on us or wanting revenge or considering us inferior. Rather, they come from the disconnect between what we tell our systems to do and what we actually want them to do.

For example, we tell a system to run up a high score in a video game. We want it to play the game fairly and learn game skills — but if it instead has the chance to directly hack the scoring system, it will do that. It’s doing great by the metric we gave it. But we aren’t getting what we wanted.

In other words, our problems come from the systems being really good at achieving the goal they learned to pursue; it’s just that the goal they learned in their training environment isn’t the outcome we actually wanted. And we’re building systems we don’t understand, which means we can’t always anticipate their behavior.

Right now, the harm is limited because the systems are so limited. But it’s a pattern that could have even graver consequences for human beings in the future as AI systems become more advanced.

_

Huang and Rust (2018) identified four levels of AI intelligence and their corresponding service tasks: mechanical, analytical, intuitive, and empathic.

Mechanical intelligence corresponds to mostly algorithmic tasks that are often repetitive and require consistency and accuracy, such as order-taking machines in restaurants or robots used in manufacturing assembly processes (Colby, Mithas, & Parasuraman, 2016). These are essentially advanced forms of mechanical machines of the past.

Analytical intelligence corresponds to less routine tasks largely classification in nature (e.g., credit application determinations, market segmentation, revenue predictions, etc.); AI is rapidly establishing its effectiveness at this level of analytical intelligence/tasks as more training data becomes available (Wedel & Kannan, 2016).

However, few AI applications exist at the next two levels, intuitive and empathic intelligence (Huang & Rust, 2018).

Empathy, intuition, and creativity are believed to be directly related to human consciousness (McGilchrist, 2019). According to Huang and Rust (2018), the progression of AI capabilities into these higher intelligence/task levels can fundamentally disrupt the service industry, and severely affect employment and business models as AI agents replace more humans in their tasks. The achievement of higher intelligence can also alter the existing human-machine balance (Longoni & Cian, 2020) toward people trusting “word-of-machine” over word-of-mouth not only in achieving a utilitarian outcome (e.g., buying a product) but also toward trusting “word-of-machine” when it comes to achieving a hedonic goal (e.g., satisfaction ratings, emotional advice).

Whether AI agents can achieve such levels of intelligence is heatedly debated. One side attribute achieving intuitive and empathic levels of intelligence to having a subjective biological awareness and the conscious state known to humans (e.g., Azarian, 2016; Winkler, 2017). The other side argues that everything that happens in the human brain, be it emotion or cognition, is of computational nature at the neurological level. Thus, it would be possible for AI to achieve intuitive and empathic intelligence in the future through advanced computation (e.g., McCarthy, Minsky, Rochester, & Shannon, 2006; Minsky, 2007).

______

______

Basics of AI:

All ways of expressing information (i.e. voice, video, text, data) use physical system, for example, spoken words are conveyed by air pressure fluctuations. Information cannot exist without physical representation. Information, the 1’s and 0’s of classical computers, must inevitably be recorded by some physical system – be it paper or silicon. The basic idea of classical (conventional) computing is to store and process information. All matter is composed of atoms – nuclei and electrons – and the interactions and time evolution of atoms are governed by the laws of quantum mechanics. Without our quantum understanding of the solid state and the band theory of metals, insulators and semiconductors, the whole of the semiconductor industry with its transistors and integrated circuits – and hence the computer could not have developed. Quantum physics is the theoretical basis of the transistor, the laser, and other technologies which enabled the computing revolution. But on the algorithmic level, today’s computing machinery still operates on ‘classical’ Boolean logic.

_

Linear algebra is about linear combinations. That is, using arithmetic on columns of numbers called vectors and arrays of numbers called matrices, to create new columns and arrays of numbers. (See figure below). Linear algebra is the study of lines and planes, vector spaces and mappings that are required for linear transforms. Its concepts are a crucial prerequisite for understanding the theory behind Machine Learning, especially if you are working with Deep Learning Algorithms.

Linear algebra is a branch of mathematics, but the truth of it is that linear algebra is the mathematics of data. Matrices and vectors are the language of data. In Linear algebra, data is represented by linear equations, which are presented in the form of matrices and vectors. Therefore, you are mostly dealing with matrices and vectors rather than with scalars. When you have the right libraries, like Numpy, at your disposal, you can compute complex matrix multiplication very easily with just a few lines of code.

Until the 19th century, linear algebra was introduced through systems of linear equations and matrices. In modern mathematics, the presentation through vector spaces is generally preferred, since it is more synthetic, more general (not limited to the finite-dimensional case), and conceptually simpler, although more abstract. An element of a specific vector space may have various nature; for example, it could be a sequence, a function, a polynomial or a matrix. Linear algebra is concerned with those properties of such objects that are common to all vector spaces. Matrices allow explicit manipulation of finite-dimensional vector spaces and linear maps.

_

The application of linear algebra in computers is often called numerical linear algebra. It is more than just the implementation of linear algebra operations in code libraries; it also includes the careful handling of the problems of applied mathematics, such as working with the limited floating point precision of digital computers. The description of binary numbers in the exponential form is called floating-point representation. Classical computers are good at performing linear algebra calculations on vectors and matrices, and much of the dependence on Graphical Processing Units (GPUs) by modern machine learning methods such as deep learning because of their ability to compute linear algebra operations fast. Theoretical computer science is essentially math, and subjects such as probability, statistics, linear algebra, graph theory, combinatorics and optimization are at the heart of artificial intelligence (AI), machine learning (ML), data science and computer science in general.

_

Efficient implementations of vector and matrix operations were originally implemented in the FORTRAN programming language in the 1970s and 1980s and a lot of code ported from those implementations, underlies much of the linear algebra performed using modern programming languages, such as Python.

Three popular open source numerical linear algebra libraries that implement these functions are:

-1. Linear Algebra Package, or LAPACK.

-2. Basic Linear Algebra Subprograms, or BLAS (a standard for linear algebra libraries).

-3. Automatically Tuned Linear Algebra Software, or ATLAS.

Often, when you are calculating linear algebra operations directly or indirectly via higher-order algorithms, your code is very likely dipping down to use one of these, or similar linear algebra libraries. The name of one of more of these underlying libraries may be familiar to you if you have installed or compiled any of Python’s numerical libraries such as SciPy and NumPy.

_

Computational Rules:

Matrix-Scalar Operations:

If you multiply, divide, subtract, or add a Scalar to a Matrix, you do so with every element of the Matrix.

Matrix-Vector Multiplication:

Multiplying a Matrix by a Vector can be thought of as multiplying each row of the Matrix by the column of the Vector. The output will be a Vector that has the same number of rows as the Matrix.

Matrix-Matrix Addition and Subtraction:

Matrix-Matrix Addition and Subtraction is fairly easy and straightforward. The requirement is that the matrices have the same dimensions and the result is a Matrix that has also the same dimensions. You just add or subtract each value of the first Matrix with its corresponding value in the second Matrix.

Matrix-Matrix Multiplication:

Multiplying two Matrices together isn’t that hard either if you know how to multiply a Matrix by a Vector. Note that you can only multiply Matrices together if the number of the first Matrix’s columns matches the number of the second Matrix’s rows. The result will be a Matrix with the same number of rows as the first Matrix and the same number of columns as the second Matrix.

_

Conventional computers have two tricks that they do really well: they can store numbers in memory and they can process stored numbers with simple mathematical operations (like add and subtract). They can do more complex things by stringing together the simple operations into a series called an algorithm (multiplying can be done as a series of additions, for example). Both of a computer’s key tricks—storage and processing—are accomplished using switches called transistors, which are like microscopic versions of the switches you have on your wall for turning on and off the lights. A transistor can either be on or off, just as a light can either be lit or unlit. If it’s on, we can use a transistor to store a number one (1); if it’s off, it stores a number zero (0). Long strings of ones and zeros can be used to store any number, letter, or symbol using a code based on binary (so computers store an upper-case letter A as 1000001 and a lower-case one as 01100001). Each of the zeros or ones is called a binary digit (or bit) and, with a string of eight bits, you can store 255 different characters (such as A-Z, a-z, 0-9, and most common symbols). Computers calculate by using circuits called logic gates, which are made from a number of transistors connected together. Logic gates compare patterns of bits, stored in temporary memories called registers, and then turn them into new patterns of bits—and that’s the computer equivalent of what our human brains would call addition, subtraction, or multiplication. In physical terms, the algorithm that performs a particular calculation takes the form of an electronic circuit made from a number of logic gates, with the output from one gate feeding in as the input to the next. Classically, a compiler for a high-level programming language translates algebraic expressions into sequences of machine language instructions to evaluate the terms and operators in the expression.

_

Classical computing is based in large part on modern mathematics and logic. We take modern digital computers and their ability to perform a multitude of different applications for granted. Our desktop PCs, laptops and smart phones can run spreadsheets, stream live video, allow us to chat with people on the other side of the world, and immerse us in realistic 3D environments. But at their core, all digital computers have something in common. They all perform simple arithmetic operations. Their power comes from the immense speed at which they are able to do this. Computers perform billions of operations per second. These operations are performed so quickly that they allow us to run very complex high level applications.

_

Although there are many tasks that conventional computers are very good at, there are still some areas where calculations seem to be exceedingly difficult. Examples of these areas are: Image recognition, natural language (getting a computer to understand what we mean if we speak to it using our own language rather than a programming language), and tasks where a computer must learn from experience to become better at a particular task. That is where AI comes into picture.

_

Part of the problem is that the term “artificial intelligence” itself is a misnomer. AI is neither artificial, nor all that intelligent.  AI isn’t artificial, simply because we, natural creatures that we are, make it. Neither is AI all that intelligent, in the crucial sense of autonomous. Consider Watson, the IBM supercomputer that famously won the American game show “Jeopardy.” Not content with that remarkable feat, its makers have had Watson prepare for the federal medical licensing exam, conduct legal discovery work better than first-year lawyers, and outperform radiologists in detecting lung cancer on digital X-rays. But compared to the bacteria Escherichia coli, Watson is a moron. What a bacteria can do, Watson cannot.  

_

AI is a shift from problem solving system to knowledge based system. In conventional computing the computer is given data and is told how to solve a problem whereas in AI knowledge is given about a domain and some inference capability with the ultimate goal is to develop technique that permits systems to learn new knowledge autonomously and continually to improve the quality of the knowledge they possess. A knowledge-based system is a system that uses artificial intelligence techniques in problem-solving processes to support human decision-making, learning, and action. AI is the study of heuristics with reasoning ability, rather than determinist algorithms. It may be more appropriate to seek and accept a sufficient solution (Heuristic search) to a given problem, rather than an optimal solution (algorithmic search) as in conventional computing.

In mathematics and computer science, an optimization problem is the problem of finding the best solution from all feasible solutions. An optimization problem is essentially finding the best solution to a problem from endless number of possibilities. Conventical computing would have to configure and sort through every possible solution one at a time, on a large-scale problem this could take lot of time, so AI may help with sufficient solution (Heuristic search).

_

Conventional AI computing is called hard computing technique which follows binary logic (using only two values 0 or 1) based on symbolic processing using heuristic search – a mathematical approach in which ideas and concepts are represented by symbols such as words, phrases or sentences, which are then processed according to the rules of logic. Expert system is classic example of hard computing AI. Conventional AI research focuses on attempts to mimic human intelligence through symbol manipulation and symbolically structured knowledge bases. The conventional non-AI computing is hard computing having binary logic, crisp systems, numerical analysis and crisp software. AI hard computing differs from non-AI hard computing by having symbolic processing using heuristic search with reasoning ability rather than determinist algorithms. Soft computing AI (computational intelligence) differs from conventional AI (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation. In effect, the role model for soft computing is the human mind. Many real-life problems cannot be translated into binary language (unique values of 0 and 1) for computers to process it. Computational Intelligence therefore provides solutions for such problems. Soft computing includes fuzzy logic, neural networks, probabilistic reasoning and evolutionary computing.

_

In his book A Brief History of AI, Michael Wooldridge, a professor of computer science at the University of Oxford and an AI researcher, explains that AI is not about creating life, but rather about creating machines that can perform tasks requiring intelligence.

Wooldridge discusses two approaches to AI: symbolic AI and machine learning. Symbolic AI involves coding human knowledge into machines, while machine learning allows machines to learn from examples to perform specific tasks. Progress in AI stalled in the 1970s due to a lack of data and computational power, but recent advancements in technology have led to significant progress. AI can perform narrow tasks better than humans, but the grand dream of AI is achieving artificial general intelligence (AGI), which means creating machines with the same intellectual capabilities as humans. One challenge for AI is giving machines social skills, such as cooperation, coordination, and negotiation. The path to conscious machines is slow and complex, and the mystery of human consciousness and self-awareness remains unsolved. The limits of computing are only bounded by imagination.

_

Natural language processing is the key to AI. The goal of natural language processing is to help computers understand human speech in order to do away with computer languages. The ability to use and understand natural language seems to be a fundamental aspect of human intelligence and its successful automation would have an incredible impact on the usability and effectiveness of computers.

_

Machine learning is a subset of AI. Machine learning is the ability to learn without being explicitly programmed and it explores the development of algorithms that learn from given data. Traditional system performs computations to solve a problem. However, if it is given the same problem a second time, it performs the same sequence of computations again. Traditional system cannot learn. Machine learning algorithms learn to perform tasks rather than simply providing solutions based on a fixed set of data. It learns on its own, either from experience, analogy, examples, or by being “told” what to do. Machine learning technologies include expert systems, genetic algorithms, neural networks, random seeded crystal learning, or any effective combinations.

Deep learning is a class of machine learning algorithms. Deep learning is the name we use for “stacked neural networks”; that is, networks composed of several layers. Deep Learning creates knowledge from multiple layers of information processing. Deep Learning tries to emulate the functions of inner layers of the human brain, and its successful applications are found in image recognition, speech recognition, natural language processing, or email security.

_

Artificial neural network is an electronic model of the brain consisting of many interconnected simple processors akin to vast network of neurons in the human brain. The goal of the artificial neural network is to solve problems in the same way that the human brain would. Artificial neural networks algorithms can learn from examples & experience and make generalizations based on this knowledge. AI puts together three basic elements – a neural network “brain” made up of interconnected neurons, the optimisation algorithm that tunes it, and the data it is trained on. A neural network isn’t the same as a human brain although it looks a bit like it in structure. In the old days, our neural networks had a few dozen neurons in just two or three layers, while current AI systems can have 100 billion, hundreds of layers deep.

Neural networks are a set of algorithms, modelled loosely after the human brain, that are designed to recognize patterns. They interpret sensory data through a kind of machine perception, labelling or clustering raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text or time series, must be translated. Neural networks help us cluster and classify. You can think of them as a clustering and classification layer on top of data you store and manage. They help to group unlabelled data according to similarities among the example inputs, and they classify data when they have a labelled dataset to train on. (To be more precise, neural networks extract features that are fed to other algorithms for clustering and classification; so you can think of deep neural networks as components of larger machine-learning applications involving algorithms for reinforcement learning, classification and regression.) Deep learning is the name we use for “stacked neural networks”; that is, networks composed of several layers. The layers are made of nodes. A node is just a place where computation happens, loosely patterned on a neuron in the human brain, which fires when it encounters sufficient stimuli. A node combines input from the data with a set of coefficients, or weights, that either amplify or dampen that input, thereby assigning significance to inputs for the task the algorithm is trying to learn. These input-weight products are summed and the sum is passed through a node’s so-called activation function, to determine whether and to what extent that signal progresses further through the network to affect the ultimate outcome, say, an act of classification.

_

Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data. In a simple case, there might be two sets of neurons: ones that receive an input signal and ones that send an output signal. When the input layer receives an input it passes on a modified version of the input to the next layer. In a deep network, there are many layers between the input and output (and the layers are not made of neurons but it can help to think of it that way), allowing the algorithm to use multiple processing layers, composed of multiple linear and non-linear transformations. Deep learning is part of a broader family of machine learning methods based on learning representations of data. An observation (e.g., an image) can be represented in many ways such as a vector of intensity values per pixel, or in a more abstract way as a set of edges, regions of particular shape, etc. Some representations are better than others at simplifying the learning task (e.g., face recognition or facial expression recognition). One of the promises of deep learning is replacing handcrafted features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.  Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks.

_

ANNs leverage a high volume of training data to learn accurately, which subsequently demands more powerful hardware, such as GPUs or TPUs. GPU stands for Graphical Processing Unit, and it is integrated into each CPU in some form. But some tasks and applications require extensive visualization that available inbuilt GPU can’t handle. Tasks such as computer-aided design, machine learning, video games, live streaming, video editing, and data scientist. Simple tasks of rendering basic graphics can be done with the GPU built into the CPU. For other high-end jobs, GPU is made. Tensor Processing Unit (TPU) is an application-specific integrated circuit, to accelerate the AI calculations and algorithm.

_

Fuzzy logic is a version of first-order logic which allows the truth of a statement to be represented as a value between 0 and 1, rather than simply True (1) or False (0). Fuzzy Logic Systems (FLS) produce acceptable but definite output in response to incomplete, ambiguous, distorted, or inaccurate (fuzzy) input. Fuzzy logic is designed to solve problems in the same way that humans do: by considering all available information and making the best possible decision given the input.

_____

_____

Where is Artificial Intelligence (AI) used?

AI is used in different domains to give insights into user behaviour and give recommendations based on the data. For example, Google’s predictive search algorithm used past user data to predict what a user would type next in the search bar. Netflix uses past user data to recommend what movie a user might want to see next, making the user hooked onto the platform and increasing watch time. Facebook uses past data of the users to automatically give suggestions to tag your friends, based on the facial features in their images. AI is used everywhere by large organisations to make an end user’s life simpler. The uses of Artificial Intelligence would broadly fall under the data processing category, which would include the following:

• Searching within data, and optimising the search to give the most relevant results
• Logic-chains for if-then reasoning, that can be applied to execute a string of commands based on parameters
• Pattern-detection to identify significant patterns in large data set for unique insights
• Applied probabilistic models for predicting future outcomes

There are fields where AI is playing a more important role, such as:

• Cybersecurity: Artificial intelligence will take over more roles in organizations’ cybersecurity measures, including breach detection, monitoring, threat intelligence, incident response, and risk analysis.
• Entertainment and content creation: Computer science programs are already getting better and better at producing content, whether it is copywriting, poetry, video games, or even movies. OpenAI’s GBT-3 text generation AI app is already creating content that is almost impossible to distinguish from copy that was written by humans.
• Behavioral recognition and prediction: Prediction algorithms will make AI stronger, ranging from applications in weather and stock market predictions to, even more interesting, predictions of human behavior. This also raises the questions around implicit biases and ethical AI. Some AI researchers in the AI community are pushing for a set of anti-discriminatory rules, which is often associated with the hashtag #responsibleAI.

______

______

Guide to learn Artificial Intelligence:

For someone who wants to learn AI but doesn’t have a clear curriculum, here is the step-by-step guide to learn AI from basic to intermediate levels.

-1. It’s a good idea to start with Maths. Brush up on your math skills and go over the following concepts again:

-Matrix and Determinants, as well as Linear Algebra.

-Calculus is a branch of mathematics that deals with Differentiation and Integration.

-Vectors, statistics and Probability, graph theory.

-2. Coding language: Once you have mastered your arithmetic skills, you can begin practising coding by picking a coding language. Java or Python can be studied. Python is the easiest of the three to learn and practice coding with because it has various packages such as Numpy and Panda.

-3. Working on Datasets: Once you have mastered any coding language, you can move on to working with backend components such as databases. For example, you may now use SQL connector or other import modules to connect python or frontend IDE.

-4. Lastly, you should have a strong hold in understanding and writing algorithms, a strong background in data analytics skills, a good amount of knowledge in discrete mathematics and the will to learn machine learning languages.

Remember AI is hard.

______

______

Types of AI:

Artificial intelligence can be divided into two major types, based on its capabilities and functionalities. Type 1 is based on capability and type 2 is based on functionality as seen in the figure below: 

_

Type 1 AI:

Here’s a brief introduction to the first type.

• Artificial Narrow Intelligence (ANI)
• Artificial General Intelligence (AGI)
• Artificial Super Intelligence (ASI)

_

What is Artificial Narrow Intelligence (ANI)?

Narrow or weak AI is a kind of AI that refrains a computer to perform more than one operation at a time. It has a limited playing field when it comes to performing multiple intellectual tasks in the same time frame. This is the most common form of AI that you’d find in the market now. These Artificial Intelligence systems are designed to solve one single problem and would be able to execute a single task really well. By definition, they have narrow capabilities, like recommending a product for an e-commerce user or predicting the weather. This is the only kind of Artificial Intelligence that exists today. They’re able to come close to human functioning in very specific contexts, and even surpass them in many instances, but only excelling in very controlled environments with a limited set of parameters. Some examples are Google Assistant, Alexa, and Siri.

_

What is Artificial General Intelligence (AGI)?

Artificial general intelligence (AGI) is typically defined as a system that performs at least as well as humans in most or all intellectual tasks. AGI is still a theoretical concept. It’s defined as AI which has a human-level of cognitive function, across a wide variety of domains such as language processing, image processing, computational functioning and reasoning and so on. Artificial general intelligence or AGI is the future of digital technology where self-assist robots or cyborgs will emulate human sensory movements. With AGI, machines will be able to see, respond to, and interpret external information similar to the human nervous system. The advancements in artificial neural networks will drive future AGI loaders, which will run businesses with the passage of time.

We’re still a long way away from building an AGI system. An AGI system would need to comprise of thousands of Artificial Narrow Intelligence systems working in tandem, communicating with each other to mimic human reasoning. Even with the most advanced computing systems and infrastructures, such as Fujitsu’s K or IBM’s Watson, it has taken them 40 minutes to simulate a single second of neuronal activity. This speaks to both the immense complexity and interconnectedness of the human brain, and to the magnitude of the challenge of building an AGI with our current resources.

A 2022 survey of AI researchers found that 90% of respondents expected AGI would be achieved in the next 100 years, and half expected the same by 2061. Meanwhile some researchers dismiss existential risks from AGI as “science fiction” based on their high confidence that AGI will not be created any time soon. Breakthroughs in large language models have led some researchers to reassess their expectations.  

_

What is Artificial Super Intelligence (ASI)?

We’re almost entering into science-fiction territory here, but ASI is seen as the logical progression from AGI. Strong AI is a futuristic concept that has only been the premise of a sci-fi movie until now. Strong AI will be the ultimate dominator as it would enable machines to design self-improvements and outclass humanity. It would construct cognitive abilities, feelings, and emotions in machines better than us. An Artificial Super Intelligence (ASI) system would be able to surpass all human capabilities. This would include decision making, taking rational decisions, and even includes things like making better art and building emotional relationships.

Once we achieve Artificial General Intelligence, AI systems would rapidly be able to improve their capabilities and advance into realms that we might not even have dreamed of. While the gap between AGI and ASI would be relatively narrow (some say as little as a nanosecond, because that’s how fast Artificial Intelligence would learn) the long journey ahead of us towards AGI itself makes this seem like a concept that lies far into the future.

___

___

Type 2 AI:

The second type, known as functional AI. can be divided into four categories, based on the type and complexity of the tasks a system is able to perform. Mostly, type 2 systems run on unsupervised algorithms that generate output without utilizing training data. They are:

-1. Reactive machines

-2. Limited memory

-3. Theory of mind

-4. Self-awareness

_

-1. Reactive Machines

Reactive Machines perform basic operations. This level of A.I. is the simplest. These types react to some input with some output. There is no learning that occurs. This is the first stage to any A.I. system. A reactive machine follows the most basic of AI principles and, as its name implies, is capable of only using its intelligence to perceive and react to the world in front of it. A reactive machine cannot store a memory and, as a result, cannot rely on past experiences to inform decision making in real time. Reactive machines are the most basic type of unsupervised AI. This means that they cannot form memories or use past experiences to influence present-made decisions; they can only react to currently existing situations – hence “reactive.”

Reactive machines have no concept of the world and therefore cannot function beyond the simple tasks for which they are programmed. A characteristic of reactive machines is that no matter the time or place, these machines will always behave the way they were programmed. This type of AI will be more trustworthy and reliable, and it will react the same way to the same stimuli every time. There is no growth with reactive machines, only stagnation in recurring actions and behaviors. Perceiving the world directly means that reactive machines are designed to complete only a limited number of specialized duties.

Reactive Machine Examples:

• Deep Blue was designed by IBM in the 1990s as a chess-playing supercomputer and defeated international grandmaster Gary Kasparov in a game. Deep Blue was only capable of identifying the pieces on a chess board and knowing how each moves based on the rules of chess, acknowledging each piece’s present position and determining what the most logical move would be at that moment. The computer was not pursuing future potential moves by its opponent or trying to put its own pieces in better position. Every turn was viewed as its own reality, separate from any other movement that was made beforehand. In a series of matches played between 1996 and 1997, Deep Blue defeated Russian chess grandmaster Garry Kasparov 3½ to 2½ games, becoming the first computerized device to defeat a human opponent.

Deep Blue’s unique skill of accurately and successfully playing chess matches highlighted its reactive abilities. In the same vein, its reactive mind also indicated that it has no concept of the past or future; it only comprehends and acts on the presently-existing world and its components within it. To simplify, reactive machines are programmed for the here and now, but not the before and after.

• Google’s AlphaGo is also incapable of evaluating future moves but relies on its own neural network to evaluate developments of the present game, giving it an edge over Deep Blue in a more complex game. AlphaGo also bested world-class competitors of the game, defeating champion Go player Lee Sedol in 2016.

_

-2. Limited Memory:

Limited memory types refer to an A.I.’s ability to store previous data and/or predictions, using that data to make better predictions. Limited memory AI has the ability to store previous data and predictions when gathering information and weighing potential decisions — essentially looking into the past for clues on what may come next. Limited memory AI is more complex and presents greater possibilities than reactive machines. Limited memory AI is created when a team continuously trains a model in how to analyze and utilize new data or an AI environment is built so models can be automatically trained and renewed. With Limited Memory, machine learning architecture becomes a little more complex.

_

There are three major kinds of machine learning models that achieve this Limited Memory type:

-(1. Reinforcement learning:

These models learn to make better predictions through many cycles of trial and error. This kind of model is used to teach computers how to play games like Chess, Go, and DOTA2.

-(2. Long Short Term Memory (LSTMs):

Researchers intuited that past data would help predict the next items in sequences, particularly in language, so they developed a model that used what was called the Long Short Term Memory. For predicting the next elements in a sequence, the LSTM tags more recent information as more important and items further in the past as less important.

-(3. Evolutionary Generative Adversarial Networks (E-GAN):

The E-GAN has memory such that it evolves at every evolution. The model produces a kind of growing thing. Growing things don’t take the same path every time, the paths get to be slightly modified because statistics is a math of chance, not a math of exactness. In the modifications, the model may find a better path, a path of least resistance. The next generation of the model mutates and evolves towards the path its ancestor found in error.

In a way, the E-GAN creates a simulation similar to how humans have evolved on this planet. Each child, in perfect, successful reproduction, is better equipped to live an extraordinary life than its parent.

_

Limited Memory A.I. works in two ways:

-(1. A team continuously trains a model on new data.

-(2. The A.I. environment is built in a way where models are automatically trained and renewed upon model usage and behavior.

_

More and more common in the ML lifecycle is Active Learning. The ML Active Learning Cycle has six steps:

-(1. Training Data. An ML model must have data to train on.

-(2. Build ML Model. The model is created.

-(3. Model Predictions. The model makes predictions,

-(4. Feedback. The model gets feedback on its prediction from human or environmental stimuli.

-(5. Feedback becomes data. Feedback is submitted back to a data repository.

-(6. Repeat Step 1. Continue to iterate on this cycle.

_

-3. Theory of Mind

Theory of mind is just that — theoretical. We have not yet achieved the technological and scientific capabilities necessary to reach this next level of AI.

The concept is based on the psychological premise of understanding that other living things have thoughts and emotions that affect the behavior of one’s self. In terms of AI machines, this would mean that AI could comprehend how humans, animals and other machines feel and make decisions through self-reflection and determination, and then utilize that information to make decisions of their own. Essentially, machines would have to be able to grasp and process the concept of “mind,” the fluctuations of emotions in decision-making and a litany of other psychological concepts in real time, creating a two-way relationship between people and AI.

_

-4. Self-Awareness

Once theory of mind can be established, sometime well into the future of AI, the final step will be for AI to become self-aware. This kind of AI possesses human-level consciousness and understands its own existence in the world, as well as the presence and emotional state of others. It would be able to understand what others may need based on not just what they communicate to them but how they communicate it. Self-awareness in AI relies both on human researchers understanding the premise of consciousness and then learning how to replicate that so it can be built into machines.

______

______

Machine consciousness or Artificial consciousness (AC):

AI systems do not possess consciousness or emotions. They operate based on predefined rules and algorithms, lacking subjective experiences. The question of whether AI has consciousness is still a topic of debate. While some argue that AI could achieve consciousness by integrating information in the same way as humans, others argue that consciousness results from the details of our neurobiology and cannot be achieved through programming alone.

AC or machine consciousness refers to a non-biological, human-created machine that is aware of its own existence and can think like it has a mind. AC represents the next level of strong AI. It is intelligence with awareness. A machine that can feel and react like humans will not only be amazing to see but will greatly assist systems that involve human dealings or interactions.

_

Why Artificial General Intelligence is Hard to achieve:

-1. Human Consciousness is not Algorithm-Driven:

AGI applications are supposed to replicate human consciousness and cognitive abilities seamlessly. However, AGI developers cannot design neural networks that can recreate artificial consciousness in a lab—at least not yet. Human consciousness is too abstract and asymmetrical for any Turing machine to be used to model it for AGI creation. One approach to replicate human consciousness into an AGI model would be to treat the brain as a quantum computer to solve problems in polynomial time with brute computing power alone. While this would significantly boost the performance and efficiency of an algorithm, it would also not be the route to configuring machines with consciousness and their own distinctive human personalities.

Another issue with using programs and code is the “halting problem.” This concept implies that every computer program, at some point in time, will pose functionality problems and stop running eventually for certain inputs. This is a problem that may have no solutions, with even Turing’s studies showing in general that there isn’t an algorithm that can resolve this problem. This then poses massive questions regarding the long-term functionality and computability of AGI-based tools.

There is a reason why humans are so unique and incomparable to even the most advanced machines of the past and present. AI, for all its incredible capabilities, still runs on painstakingly-created computer programs and algorithms. The most advanced AI tools can create such programs autonomously. However, the technology cannot generate lifelike consciousness in AGI-based tools. Even with all the advancements made in AI, every decision is backed by hardcore data analytics and processing, scoring low on factors such as logical reasoning, emotion-based decisions and other aspects.

Now that the technical side of things is delved into, the issues related to ethics and supplanting humans need to be dealt with.

-2. Morality of AGI may be Problematic:

There are no two ways about it—AI will replace human hands and brains in possibly every industry and business function. This problem will cause several people to vehemently oppose even the thought of implementing AGI. As per a projection, more than a billion jobs will be lost to automation, robotics and AI. And that is before one considers the emergence of AGI, which will simply magnify such figures.

Unfortunately, taking away jobs is just a tiny part of the damage AGI can cause. The combination of AGI’s sheer computational brilliance, ambivalence and equally extreme emotional detachment in any given scenario manifests itself perfectly in a terrifying example generally used when people talk about AGI morality. This example talks about an AGI robot looking for supplies in different spaces in a house to prepare something for a few hungry children. On not finding anything, as a last resort, the robot is inclined to cook their pet cat due to its “high nutritional value.” This example perfectly encapsulates how AGI, and indeed AI, may select any route to get to the desired endpoint.

Developers of AGI will need to “teach” AGI algorithms the concept of human ethics so that the kinds of misinterpretation seen above can be avoided. However, that is easier said than done as there is no precedent when it comes to “building” morality in Artificial General Intelligence models. For example, consider a situation in which multiple patients need medical care and resources are limited. In such a situation, how will an AGI-based application make decisions? Maybe, machine learning can develop further in the future to eventually learn moral behavior. However, that comes with a big “IF” and the implementation of Artificial General Intelligence is too risky if critical decisions are made without considering the human side of things.

_____

_____

The Turing test:

This test came into existence in 1950. Named after Alan Turing, an English computer scientist, cryptanalyst, mathematician, and theoretical biologist, it is used to determine whether AI is capable of thinking like a human being.

The Turing test comprises three terminals. One is for the computer and the other two for humans. One of the humans is used as a questioner/interrogator, while another human and a computer are used as respondents. The questioner asks a set of questions in a certain domain in a specified format and context. After the test is over, the questioner is asked which of the respondents was human and which was a computer. If the questioner could not reliably tell the machine from the human, the machine would be said to have passed the test. The test results would not depend on the machine’s ability to give correct answers to questions, only on how closely its answers resembled those a human would give. The test is repeated many times.

The Turing test is notoriously difficult to pass, so much so that no machine has ever been successful. However, the test has paved the way for more research and innovation.

While the Turing test highlights how far machines are from developing artificial consciousness, we cannot determine with 100 percent certainty that this will always remain impossible. After all, there have been many advancements in AI since the test’s inception.

_

Weaknesses of Turing test

Turing did not explicitly state that the Turing test could be used as a measure of “intelligence”, or any other human quality. He wanted to provide a clear and understandable alternative to the word “think”, which he could then use to reply to criticisms of the possibility of “thinking machines” and to suggest ways that research might move forward. Numerous experts in the field, including cognitive scientist Gary Marcus, insist that the Turing test only shows how easy it is to fool humans and is not an indication of machine intelligence.

Nevertheless, the Turing test has been proposed as a measure of a machine’s “ability to think” or its “intelligence”. This proposal has received criticism from both philosophers and computer scientists. It assumes that an interrogator can determine if a machine is “thinking” by comparing its behaviour with human behaviour. Every element of this assumption has been questioned: the reliability of the interrogator’s judgement, the value of comparing only behaviour and the value of comparing the machine with a human. Because of these and other considerations, some AI researchers have questioned the relevance of the test to their field.

_

Human intelligence vs. intelligence in general:

The Turing test does not directly test whether the computer behaves intelligently. It tests only whether the computer behaves like a human being. Since human behaviour and intelligent behaviour are not exactly the same thing, the test can fail to accurately measure intelligence in two ways:

Some human behaviour is unintelligent:

The Turing test requires that the machine be able to execute all human behaviours, regardless of whether they are intelligent. It even tests for behaviours that may not be considered intelligent at all, such as the susceptibility to insults, the temptation to lie or, simply, a high frequency of typing mistakes. If a machine cannot imitate these unintelligent behaviours in detail, it fails the test.

Some intelligent behaviour is non-human:  

The Turing test does not test for highly intelligent behaviours, such as the ability to solve difficult problems or come up with original insights. In fact, it specifically requires deception on the part of the machine: if the machine is more intelligent than a human being it must deliberately avoid appearing too intelligent. If it were to solve a computational problem that is practically impossible for a human to solve, then the interrogator would know the program is not human, and the machine would fail the test.

Because it cannot measure intelligence that is beyond the ability of humans, the test cannot be used to build or evaluate systems that are more intelligent than humans. Because of this, several test alternatives that would be able to evaluate super-intelligent systems have been proposed.

_

The Language-centric Objection:

Another well known objection raised towards the Turing Test concerns its exclusive focus on the linguistic behaviour (i.e. it is only a “language-based” experiment, while all the other cognitive faculties are not tested). This drawback downsizes the role of other modality-specific “intelligent abilities” concerning human beings that the psychologist Howard Gardner, in his “multiple intelligence theory”, proposes to consider (verbal-linguistic abilities are only one of those).

______

______

Traits of AI:  

The purpose of Artificial Intelligence is to aid human capabilities and help us make advanced decisions with far-reaching consequences. That’s the answer from a technical standpoint. From a philosophical perspective, Artificial Intelligence has the potential to help humans live more meaningful lives devoid of hard labour, and help manage the complex web of interconnected individuals, companies, states and nations to function in a manner that’s beneficial to all of humanity.

The general problem of simulating (or creating) intelligence has been broken down into sub-problems. These consist of particular traits or capabilities that researchers expect an intelligent system to display. The traits described below have received the most attention and cover the scope of AI research.

-1. Reasoning, problem-solving:

Early researchers developed algorithms that imitated step-by-step reasoning that humans use when they solve puzzles or make logical deductions. By the late 1980s and 1990s, methods were developed for dealing with uncertain or incomplete information, employing concepts from probability and economics.

Many of these algorithms are insufficient for solving large reasoning problems because they experience a “combinatorial explosion”: they became exponentially slower as the problems grew larger. Even humans rarely use the step-by-step deduction that early AI research could model. They solve most of their problems using fast, intuitive judgments. Accurate and efficient reasoning is an unsolved problem.

-2. Knowledge representation:

Knowledge representation and knowledge engineering allow AI programs to answer questions intelligently and make deductions about real-world facts. Formal knowledge representations are used in content-based indexing and retrieval, scene interpretation, clinical decision support, knowledge discovery (mining “interesting” and actionable inferences from large databases), and other areas.

A knowledge base is a body of knowledge represented in a form that can be used by a program. An ontology is the set of objects, relations, concepts, and properties used by domain of knowledge. The most general ontologies are called upper ontologies, which attempt to provide a foundation for all other knowledge and act as mediators between domain ontologies that cover specific knowledge about a particular domain (field of interest or area of concern).

Figure above shows that ontology represents knowledge as a set of concepts within a domain and the relationships between those concepts.

Knowledge bases need to represent things such as: objects, properties, categories and relations between objects; situations, events, states and time; causes and effects; knowledge about knowledge (what we know about what other people know); default reasoning (things that humans assume are true until they are told differently and will remain true even when other facts are changing); and many other aspects and domains of knowledge.

Among the most difficult problems in KR are: the breadth of commonsense knowledge (the set of atomic facts that the average person knows) is enormous; the difficulty of knowledge acquisition and the sub-symbolic form of most commonsense knowledge (much of what people know is not represented as “facts” or “statements” that they could express verbally).

-3. Planning and decision making:

An “agent” is anything that perceives and takes actions in the world. A rational agent has goals or preferences and takes actions to make them happen. In automated planning, the agent has a specific goal. In automated decision making, the agent has preferences – there are some situations it would prefer to be in, and some situations it is trying to avoid. The decision making agent assigns a number to each situation (called the “utility”) that measures how much the agent prefers it. For each possible action, it can calculate the “expected utility”: the utility of all possible outcomes of the action, weighted by the probability that the outcome will occur. It can then choose the action with the maximum expected utility.

In classical planning, the agent knows exactly what the effect of any action will be. In most real-world problems, however, the agent may not be certain about the situation they are in (it is “unknown” or “unobservable”) and it may not know for certain what will happen after each possible action (it is not “deterministic”). It must choose an action by making a probabilistic guess and then reassess the situation to see if the action worked. In some problems, the agent’s preferences may be uncertain, especially if there are other agents or humans involved. These can be learned (e.g., with inverse reinforcement learning) or the agent can seek information to improve its preferences. Information value theory can be used to weigh the value of exploratory or experimental actions. The space of possible future actions and situations is typically intractably large, so the agents must take actions and evaluate situations while being uncertain what the final outcome will be.

A Markov decision process has a transition model that describes the probability that a particular action will change the state in a particular way, and a reward function that supplies the utility of each state and the cost of each action. A policy associates a decision with each possible state. The policy could be calculated (e.g. by iteration), be heuristic, or it can be learned.

Game theory describes rational behavior of multiple interacting agents, and is used in AI programs that make decisions that involve other agents.

-4. Learning:

Machine learning is the study of programs that can improve their performance on a given task automatically. It has been a part of AI from the beginning. There are several kinds of machine learning. Unsupervised learning analyzes a stream of data and finds patterns and makes predictions without any other guidance. Supervised learning requires a human to label the input data first, and comes in two main varieties: classification (where the program must learn to predict what category the input belongs in) and regression (where the program must deduce a numeric function based on numeric input). In reinforcement learning the agent is rewarded for good responses and punished for bad ones. The agent learns to choose responses that are classified as “good”. Transfer learning is when the knowledge gained from one problem is applied to a new problem. Deep learning uses artificial neural networks for all of these types of learning. Computational learning theory can assess learners by computational complexity, by sample complexity (how much data is required), or by other notions of optimization.

-5. Natural language processing:

Natural language processing (NLP) allows programs to read, write and communicate in human languages such as English. Specific problems include speech recognition, speech synthesis, machine translation, information extraction, information retrieval and question answering.

Early work, based on Noam Chomsky’s generative grammar, had difficulty with word-sense disambiguation unless restricted to small domains called “micro-worlds” (due to the common sense knowledge problem).

Modern deep learning techniques for NLP include word embedding (how often one word appears near another), transformers (which finds patterns in text), and others. In 2019, generative pre-trained transformer (or “GPT”) language models began to generate coherent text, and by 2023 these models were able to get human-level scores on the bar exam, SAT, GRE, and many other real-world applications.

-6. Perception:

Machine perception is the ability to use input from sensors (such as cameras, microphones, wireless signals, active lidar, sonar, radar, and tactile sensors) to deduce aspects of the world. Computer vision is the ability to analyze visual input. The field includes speech recognition, image classification, facial recognition, object recognition, and robotic perception.

-7. Robotics:

Though sometimes (incorrectly) used interchangeably, robotics and artificial intelligence are very different things.

Artificial intelligence is where systems emulate the human mind to learn, solve problems and make decisions on the fly, without needing the instructions specifically programmed.

Robotics is where robots are built and programmed to perform very specific duties. In most cases, this simply doesn’t require artificial intelligence, as the tasks performed are predictable, repetitive and don’t need additional ‘thought’. Despite this, robotics and artificial intelligence can coexist. Projects using AI in robotics are in the minority, but such designs are likely to become more common in future as our AI systems become more sophisticated.  

-8. Social intelligence:

Affective computing is an interdisciplinary umbrella that comprises systems that recognize, interpret, process or simulate human feeling, emotion and mood. For example, some virtual assistants are programmed to speak conversationally or even to banter humorously; it makes them appear more sensitive to the emotional dynamics of human interaction, or to otherwise facilitate human–computer interaction. However, this tends to give naïve users an unrealistic conception of how intelligent existing computer agents actually are. Moderate successes related to affective computing include textual sentiment analysis and, more recently, multimodal sentiment analysis, wherein AI classifies the affects displayed by a videotaped subject.

-9. General intelligence:

A machine with artificial general intelligence should be able to solve a wide variety of problems with breadth and versatility similar to human intelligence.

_____

_____

Applications of Artificial Intelligence:

AI truly has the potential to transform many industries, with a wide range of possible use cases. What all these different industries and use cases have in common, is that they are all data-driven. Since Artificial Intelligence is an efficient data processing system at its core, there’s a lot of potential for optimisation everywhere.

_

Use cases for AI: 

Many businesses already utilize AI for many different tasks. They use the technology to analyze data sets, forecast customer behavior, simulate human decision-making, and categorize large, complex volumes of information.

_

Let’s take a look at the some of the industries where AI is currently shining.

Healthcare:

• Administration: AI systems are helping with the routine, day-to-day administrative tasks to minimise human errors and maximise efficiency. Transcriptions of medical notes through NLP and helps structure patient information to make it easier for doctors to read it.
• Telemedicine: For non-emergency situations, patients can reach out to a hospital’s AI system to analyse their symptoms, input their vital signs and assess if there’s a need for medical attention. This reduces the workload of medical professionals by bringing only crucial cases to them.
• Assisted Diagnosis: Through computer vision and convolutional neural networks, AI is now capable of reading MRI scans to check for tumours and other malignant growths, at an exponentially faster pace than radiologists can, with a considerably lower margin of error.
• Robot-assisted surgery: Robotic surgeries have a very minuscule margin-of-error and can consistently perform surgeries round-the-clock without getting exhausted. Since they operate with such a high degree of accuracy, they are less invasive than traditional methods, which potentially reduces the time patients spend in the hospital recovering.
• Vital Stats Monitoring: A person’s state of health is an ongoing process, depending on the varying levels of their respective vitals stats. With wearable devices achieving mass-market popularity now, this data is not available on tap, just waiting to be analysed to deliver actionable insights. Since vital signs have the potential to predict health fluctuations even before the patient is aware, there are a lot of live-saving applications here.

_ 

E-commerce:

• Better recommendations: This is usually the first example that people give when asked about business applications of AI, and that’s because it’s an area where AI has delivered great results already. Most large e-commerce players have incorporated Artificial Intelligence to make product recommendations that users might be interested in, which has led to considerable increases in their bottom-lines.
• Chatbots: Another famous example, based on the proliferation of Artificial Intelligence chatbots across industries, and every other website we seem to visit. These chatbots are now serving customers in odd-hours and peak hours as well, removing the bottleneck of limited human resources.
• Filtering spam and fake reviews: Due to the high volume of reviews that sites like Amazon receive, it would be impossible for human eyes to scan through them to filter out malicious content. Through the power of NLP, Artificial Intelligence can scan these reviews for suspicious activities and filter them out, making for a better buyer experience.
• Optimising search: All of the e-commerce depends upon users searching for what they want, and being able to find it. Artificial Intelligence has been optimising search results based on thousands of parameters to ensure that users find the exact product that they are looking for.
• Supply-chain: AI is being used to predict demand for different products in different timeframes so that they can manage their stocks to meet the demand.

_

Human Resources:

• Building work culture: AI is being used to analyse employee data and place them in the right teams, assign projects based on their competencies, collect feedback about the workplace, and even try to predict if they’re on the verge of quitting their company.
• Hiring: With NLP, AI can go through thousands of CV in a matter of seconds, and ascertain if there’s a good fit. This is beneficial because it would be devoid of any human errors or biases, and would considerably reduce the length of hiring cycles.

_

AI in Robotics:

The field of robotics has been advancing even before AI became a reality. At this stage, artificial intelligence is helping robotics to innovate faster with efficient robots. AI in robotics have found applications across verticals and industries especially in the manufacturing and packaging industries. Here are a few applications of AI in robotics:

Assembly

• AI along with advanced vision systems can help in real-time course correction
• It also helps robots to learn which path is best for a certain process while its in operation

Customer Service

• AI-enabled robots are being used in a customer service capacity in retail and hospitality industries
• These robots leverage Natural Language Processing to interact with customers intelligently and like a human
• More these systems interact with humans, more they learn with the help of machine learning

Packaging

• AI enables quicker, cheaper, and more accurate packaging
• It helps in saving certain motions that a robot is making and constantly refines them, making installing and moving robotic systems easily

Open Source Robotics

• Robotic systems today are being sold as open-source systems having AI capabilities. In this way, users can teach robots to perform custom tasks based on a specific application. Eg: small scale agriculture

_____

_____

Take a look at what AI can already do.

Here’s a brief timeline of only some of the advances we saw from 2019 to 2022:

• AlphaStar, which can beat top professional players at StarCraft II (January 2019)
• MuZero, a single system that learned to win games of chess, shogi, and Go — without ever being told the rules (November 2019)
• GPT-3, a natural language model capable of producing high-quality text (May 2020)
• GPT-f, which can solve some Maths Olympiad problems (September 2020)
• AlphaFold 2, a huge step forward in solving the long-perplexing protein-folding problem (July 2021)
• Codex, which can produce code for programs from natural language instructions (August 2021)
• PaLM, a language model which has shown impressive capabilities to reason about things like cause and effect or explaining jokes (April 2022)
• DALL-E 2 (April 2022) and Imagen (May 2022), which are both capable of generating high-quality images from written descriptions
• SayCan, which takes natural language instructions and uses them to operate a robot (April 2022)
• Gato, a single ML model capable of doing a huge number of different things (including playing Atari, captioning images, chatting, and stacking blocks with a real robot arm), deciding based on its context what it should output (May 2022)
• Minerva can solve complex maths problems — fairly well at college level, and even better at high school maths competition level. (Minerva is far more successful than forecasters predicted in 2021.)

You will find the complexity and breadth of the tasks these systems can carry out surprising. And if the technology keeps advancing at this pace, it seems clear there will be major effects on society.

_____

_____

Advantages of Artificial Intelligence (AI):

• Increased Efficiency: AI can automate repetitive tasks, improving efficiency and productivity in various industries.
• Data Analysis and Insights: AI algorithms can analyze large data quickly, providing valuable insights for decision-making.
• 24/7 Availability: AI-powered systems can operate continuously, offering round-the-clock services and support.
• Improved Accuracy: AI can perform tasks with high precision, reducing errors and improving overall accuracy.
• Personalization: AI enables personalized experiences and recommendations based on individual preferences and behavior.
• Safety and Risk Reduction: AI can be used for tasks that are hazardous to humans, reducing risks and ensuring safety.

Disadvantages of Artificial Intelligence (AI):

• Job Displacement: AI automation may lead to job losses in certain industries, affecting the job market and workforce.
• Ethical Concerns: AI raises ethical issues, including data privacy, algorithm bias, and potential misuse of AI technologies.
• Lack of Creativity and Empathy: AI lacks human qualities like creativity and empathy, limiting its ability to understand emotions or produce original ideas.
• Cost and Complexity: Developing and implementing AI systems can be expensive, require specialized knowledge and resources.
• Reliability and Trust: AI systems may not always be fully reliable, leading to distrust in their decision-making capabilities.
• Dependency on Technology: Over-reliance on AI can make humans dependent on technology and reduce critical thinking skills

_____

_____

AI and Public Surveys:

In 2018, a SurveyMonkey poll of the American public by USA Today found 68% thought the real current threat remains “human intelligence”; however, the poll also found that 43% said superintelligent AI, if it were to happen, would result in “more harm than good”, and 38% said it would do “equal amounts of harm and good”.

As time goes by, people are becoming less trusting of artificial intelligence (AI), according to the results of a recent Pew Research study. The study, which involved 11,000 respondents, found that attitudes had changed sharply in just the past two years. In 2021, 37% of respondents said they were more concerned than excited about AI. That number stayed pretty much the same last year (38%), but has now jumped to 52%. (The percentage of those who are excited about AI declined from 18% in 2021 to just 10% this year). This is a problem because, to be effective, generative AI tools have to be trained, and during training there are a number of ways data can be compromised and corrupted. If people do not trust something, they’re not only unlikely to support it, but they are also more likely to act against it.

An April 2023 YouGov poll of US adults found 46% of respondents were “somewhat concerned” or “very concerned” about “the possibility that AI will cause the end of the human race on Earth,” compared with 40% who were “not very concerned” or “not at all concerned.”

The recent poll conducted by the Artificial Intelligence Policy Institute (AIPI) paints a clear picture: the American public is not only concerned but demanding a more cautious and regulated approach to AI. The AIPI survey reveals that 72% of voters prefer slowing down the development of AI, compared to just 8% who prefer speeding development up. This isn’t a mere whimper of concern; it’s a resounding call for caution. The fear isn’t confined to one political party or demographic; it’s a shared anxiety that transcends boundaries.

_____

_____

Section-4

Introduction to generative AI (GAI):

Over the past year, artificial intelligence has become just another part of daily life. Its applications, potential and actual, are seemingly endless: legal briefs and lesson plans, student essays and personalized tutors, screenplays and stand-up sets, animation and game design, newspaper articles and book reviews, music and poetry. Each of these examples relies on generative AI, which produces “new” content based on the patterns it “recognizes” within its massive training samples.

Generative AI refers to deep-learning models that can take raw data — say, all of Wikipedia or the collected works of Rembrandt — and “learn” to generate statistically probable outputs when prompted. At a high level, generative models encode a simplified representation of their training data and draw from it to create a new work that’s similar, but not identical, to the original data.

Generative models have been used for years in statistics to analyze numerical data. The rise of deep learning, however, made it possible to extend them to images, speech, and other complex data types. Among the first class of models to achieve this cross-over feat were variational autoencoders, or VAEs, introduced in 2013. VAEs were the first deep-learning models to be widely used for generating realistic images and speech. VAEs opened the floodgates to deep generative modeling by making models easier to scale. Much of what we think of today as generative AI started here.

Early examples of models, like GPT-3, BERT, or DALL-E 2, have shown what’s possible. The future is models that are trained on a broad set of unlabeled data that can be used for different tasks, with minimal fine-tuning. Systems that execute specific tasks in a single domain are giving way to broad AI that learns more generally and works across domains and problems. Foundation models, trained on large, unlabeled datasets and fine-tuned for an array of applications, are driving this shift.

When it comes to generative AI, it is predicted that foundation models will dramatically accelerate AI adoption in enterprise. Reducing labeling requirements will make it much easier for businesses to dive in, and the highly accurate, efficient AI-driven automation they enable will mean that far more companies will be able to deploy AI in a wider range of mission-critical situations.

Note:

Please do not confuse between generative AI and general AI i.e., artificial general intelligence (AGI).  

__

“From the time the last great artificial intelligence breakthrough was reached in the late 1940s, scientists around the world have looked for ways of harnessing this ‘artificial intelligence’ to improve technology beyond what even the most sophisticated of today’s artificial intelligence programs can achieve.”

“Even now, research is ongoing to better understand what the new AI programs will be able to do, while remaining within the bounds of today’s intelligence. Most AI programs currently programmed have been limited primarily to making simple decisions or performing simple operations on relatively small amounts of data.”

These two paragraphs were written by GPT-2, a language bot. Developed by OpenAI, a San Francisco–based institute that promotes beneficial AI, GPT-2 is an ML algorithm with a seemingly idiotic task: presented with some arbitrary starter text, it must predict the next word. The network isn’t taught to “understand” prose in any human sense. Instead, during its training phase, it adjusts the internal connections in its simulated neural networks to best anticipate the next word, the word after that, and so on. Trained on eight million Web pages, its innards contain more than a billion connections that emulate synapses, the connecting points between neurons.  After receiving prompt, the algorithm spewed out these two paragraphs that sounded like a freshman’s effort to recall the essence of an introductory lecture on machine learning. The output contains all the right words and phrases—not bad, really! Primed with the same text a second time, the algorithm comes up with something different.  

_

Generative artificial intelligence (GAI) has exploded into popular consciousness since the release of ChatGPT to the general public for testing in November 2022. The term refers to machine learning systems that can be used to create new content in response to human prompts after being trained on vast amounts of data. Outputs of generative artificial intelligence may include audio (e.g., Amazon Polly and Murf.AI), code (e.g., CoPilot), images (e.g., Stable Diffusion, Midjourney, and Dall-E), text (e.g. ChatGPT, Llama), and videos (e.g., Synthesia). As has been the case for many advances in science and technology, we’re hearing from all sides about the short- and long-term risks – as well as the societal and economic benefits – of these capabilities. In November 2022, a San Francisco-based startup called OpenAI released a revolutionary chatbot named ChatGPT. ChatGPT is a large language model (LLM), a type of AI trained on a massive corpus of data to produce human-like responses to natural language inputs.

_____

_____

How does generative AI work? 

Generative AI enables tools to create written work, images and even audio in response to prompts from users. To get those responses, several Big Tech companies have developed their own large language models trained on vast amounts of online data. The scope and purpose of these data sets can vary. For example, the version of ChatGPT that went public last year was only trained on data up until 2021 (it’s now more up to date). These models work through a method called deep learning, which learns patterns and relationships between words, so it can make predictive responses and generate relevant outputs to user prompts.

_

The use of generative AI is limited by three things: Computing power, training material and time.

Image processing takes more computational power than the textual information. It’s still slow. If you want to generate a video, which contains many images, it’s going to be very slow. More computing power and time cost more money, meaning it is cheaper and easier right now to generate text using AI than images and video. But that could change as power of computers doubles every one to two years and AI technology is advancing at an exponential rate. Generative AI’ cannot generate anything at all without first being trained on massive troves of data it then recombines. Who produces that training data? People do.

_

Machine learning is a type of artificial intelligence. Through machine learning, practitioners develop artificial intelligence through models that can “learn” from data patterns without human direction. The unmanageably huge volume and complexity of data (unmanageable by humans, anyway) that is now being generated has increased the potential of machine learning, as well as the need for it. Until recently, machine learning was largely limited to predictive models, used to observe and classify patterns in content. For example, a classic machine learning problem is to start with an image or several images of, say, adorable cats. The program would then identify patterns among the images, and then scrutinize random images for ones that would match the adorable cat pattern. Generative AI was a breakthrough. Rather than simply perceive and classify a photo of a cat, machine learning is now able to create an image or text description of a cat on demand.

_

The first machine learning models to work with text were trained by humans to classify various inputs according to labels set by researchers. One example would be a model trained to label social media posts as either positive or negative. This type of training is known as supervised learning because a human is in charge of “teaching” the model what to do.

The next generation of text-based machine learning models rely on what’s known as self-supervised learning. This type of training involves feeding a model a massive amount of text so it becomes able to generate predictions. For example, some models can predict, based on a few words, how a sentence will end. With the right amount of sample text—say, a broad swath of the internet—these text models become quite accurate. We’re seeing just how accurate with the success of tools like ChatGPT.

_

What does it take to build a generative AI model?

Building a generative AI model has for the most part been a major undertaking, to the extent that only a few well-resourced tech heavyweights have made an attempt. OpenAI, the company behind ChatGPT, former GPT models, and DALL-E, has billions in funding from boldface-name donors. DeepMind is a subsidiary of Alphabet, the parent company of Google, and Meta has released its Make-A-Video product based on generative AI. These companies employ some of the world’s best computer scientists and engineers.

But it’s not just talent. When you’re asking a model to train using nearly the entire internet, it’s going to cost you. OpenAI hasn’t released exact costs, but estimates indicate that GPT-3 was trained on around 45 terabytes of text data—that’s about one million feet of bookshelf space, or a quarter of the entire Library of Congress—at an estimated cost of several million dollars. These aren’t resources your garden-variety start-up can access.

_

GAI outputs:

Outputs from generative AI models can be indistinguishable from human-generated content, or they can seem a little uncanny. The results depend on the quality of the model—as we’ve seen, ChatGPT’s outputs so far appear superior to those of its predecessors—and the match between the model and the use case, or input. Generative AI outputs are carefully calibrated combinations of the data used to train the algorithms. Because the amount of data used to train these algorithms is so incredibly massive—as noted, GPT-3 was trained on 45 terabytes of text data—the models can appear to be “creative” when producing outputs. What’s more, the models usually have random elements, which means they can produce a variety of outputs from one input request—making them seem even more lifelike.

_

Impact of Generative AI on Data Analytics:

With the growth of generative AI technology, data analysis is no longer confined to analysing past trends and historical data. Instead, it has opened up a new world of possibilities, allowing for the generation of new data that can help reveal hidden insights and patterns. Through the use of deep learning algorithms and artificial neural networks, generative AI is now capable of producing data that can unlock new insights and opportunities for businesses. Generative AI can make a significant impact on data analytics by unlocking hidden insights and possibilities that were previously out of reach. With its ability to generate unique content, ultra-realistic images and videos, accurate language translation, advanced fraud detection, human-like chatbots, more accurate predictive analytics, and potential for medical diagnosis, the possibilities seem endless. As the technology for generative AI continues to evolve, we can expect even more innovations and advancements in the field of data analytics.

_

How is generative AI (GAI) different from AGI?

As impressive as some generative AI services may seem, they essentially just do pattern matching. These tools can mimic the writing of others or make predictions about what words might be relevant in their responses based on all the data they’ve previously been trained on.

AGI, on the other hand, promises something more ambitious — and scary. AGI — short for artificial general intelligence — refers to technology that can perform intelligent tasks such as learning, reasoning and adapting to new situations in the way that humans do. For the moment, however, AGI remains purely a hypothetical, so don’t worry too much about it.   

_____ 

_____

ChatGPT:

ChatGPT, which stands for Chat Generative Pre-trained Transformer, is a large language model-based chatbot developed by OpenAI and launched on November 30, 2022, which enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language used. ChatGPT has not only passed the United States medical licensing exam (USMLE), multiple law exams, and a MBA-level business school exam, but has also generated high quality essays and academic papers, produced a comprehensive list of recommendations for the “ideal” national budget for India, composed songs, and even opined on matters of theology and the existence of God. A host of competitor AI applications will be launched this year including AnthropicAI’s chatbot, “Claude”, and DeepMind’s chatbot, “Sparrow.” OpenAI is also continuing its research, and released an even more advanced version of ChatGPT, called GPT 4 in March 2023.

_

LaMDA is a chatbot developed by Google AI. LaMDA (Language Model for Dialogue Applications) is a collection of conversational large language models (LLMs) that work together to complete different types of generative AI tasks that involve dialogue. LaMDA is trained on a massive dataset of text and code, and it can generate text, translate languages, and answer your questions in a comprehensive and informative way. While ChatGPT’s training datasets included Wikipedia and Common Crawl, the focus for LaMDA was on human dialogues. This is reflected in the answers both AIs give: Whereas ChatGPT uses longer and well-structured sentences, LaMDA’s style is casual, closer to instant messaging.

____

How ChatGPT works:

Researchers build (train) large language models like GPT-3 and GPT-4 by using a process called “unsupervised learning,” which means the data they use to train the model isn’t specially annotated or labeled. During this process, the model is fed a large body of text (millions of books, websites, articles, poems, transcripts, and other sources) and repeatedly tries to predict the next word in every sequence of words. If the model’s prediction is close to the actual next word, the neural network updates its parameters to reinforce the patterns that led to that prediction.

Conversely, if the prediction is incorrect, the model adjusts its parameters to improve its performance and tries again. This process of trial and error, though a technique called “backpropagation,” allows the model to learn from its mistakes and gradually improve its predictions during the training process.

As a result, GPT learns statistical associations between words and related concepts in the data set. Some people, like OpenAI Chief Scientist Ilya Sutskever, think that GPT models go even further than that, building a sort of internal reality model so they can predict the next best token more accurately, but the idea is controversial. The exact details of how GPT models come up with the next token within their neural nets are still uncertain.

_

In the current wave of GPT models, this core training (now often called “pre-training”) happens only once. After that, people can use the trained neural network in “inference mode,” which lets users feed an input into the trained network and get a result. During inference, the input sequence for the GPT model is always provided by a human, and it’s called a “prompt.” The prompt determines the model’s output, and altering the prompt even slightly can dramatically change what the model produces.

For example, if you prompt GPT-3 with “Mary had a,” it usually completes the sentence with “little lamb.” That’s because there are probably thousands of examples of “Mary had a little lamb” in GPT-3’s training data set, making it a sensible completion. But if you add more context in the prompt, such as “In the hospital, Mary had a,” the result will change and return words like “baby” or “series of tests.”

Here’s where things get a little funny with ChatGPT, since it’s framed as a conversation with an agent rather than just a straight text-completion job. In the case of ChatGPT, the input prompt is the entire conversation you’ve been having with ChatGPT, starting with your first question or statement and including any specific instructions provided to ChatGPT before the simulated conversation even began. Along the way, ChatGPT keeps a running short-term memory (called the “context window”) of everything it and you have written, and when it “talks” to you, it is attempting to complete the transcript of a conversation as a text-completion task.

_

Figure below shows how GPT conversational language model prompting works.

_

ChatGPT-3.5 and GPT-4 were fine-tuned to target conversational usage. The fine-tuning process leveraged both supervised learning as well as reinforcement learning in a process called reinforcement learning from human feedback (RLHF). In RLHF, human raters ranked ChatGPT’s responses in order of preference, then fed that information back into the model. Through RLHF, OpenAI was able to instil in the model the goal of refraining from answering many questions it cannot answer reliably. This has allowed the ChatGPT to produce coherent responses with fewer confabulations than the base model. But inaccuracies still slip through. 

____

Large language models (LLMs), such as OpenAI’s GPT-3, learn to write text by studying millions of examples and understanding the statistical relationships between words. As a result, they can author convincing-sounding documents, but those works can also be riddled with falsehoods and potentially harmful stereotypes. Some critics call LLMs “stochastic parrots” for their ability to convincingly spit out text without understanding its meaning. Although the sophisticated chatbot is used by many people—from students researching essays to lawyers researching case law—to search for accurate information, ChatGPT’s terms of use make it clear that ChatGPT cannot be trusted to generate accurate information.

It says:

Artificial intelligence and machine learning are rapidly evolving fields of study. We are constantly working to improve our Services to make them more accurate, reliable, safe and beneficial. Given the probabilistic nature of machine learning, use of our Services may in some situations result in incorrect Output that does not accurately reflect real people, places, or facts. You should evaluate the accuracy of any Output as appropriate for your use case, including by using human review of the Output.

_

ChatGPT is a critical moment for AI:  

ChatGPT, now open to everyone, has made an important transition. Until now, AI has primarily been aimed at problems where failure is expensive, not at tasks where occasional failure is cheap and acceptable — or even ones in which experts can easily separate failed cases from successful ones. A car that occasionally gets into accidents is intolerable. But an AI artist that draws some great pictures, but also some bad ones, is perfectly acceptable. Applying AI to the creative and expressive tasks (writing marketing copy) rather than dangerous and repetitive ones (driving a forklift) opens a new world of applications.

What are those applications, and why do they matter so much?

First, not only can this AI produce paragraphs of solidly written English (or French, or Mandarin, or whatever language you choose) with a high degree of sophistication, it can also create blocks of computer code on command. This is a major change. Massive increases in speed have been seen in a randomized trial of AI code tools. One good programmer can now legitimately do what not so long ago was the work of many, and people who have never programmed will soon be able to create workable code as well.

Second, it has an incredible capacity to perform different kinds of writing with more significant implications than might be initially apparent. The use of AI in writing can greatly increase the productivity of businesses in a variety of industries. By utilizing AI’s ability to quickly and accurately generate written content, businesses can save time and resources, allowing them to focus on other important tasks. This is particularly beneficial for industries such as marketing and advertising, consulting, and finance, where high-quality written materials are essential for communicating with clients and stakeholders. Additionally, AI can also be useful for industries such as journalism and publishing, where it can help generate articles and other written content with speed and accuracy. Overall, the use of AI in writing will greatly benefit businesses by allowing them to produce more written materials in less time. It could make your job as a professor easier as it takes seconds to write a reasonable course syllabus, class assignments, grading criteria, even lecture notes that could be potentially useful with some editing.

This highlights the third major change that happened with this release: the possibility of human-machine hybrid work. A writer can easily edit badly written sentences that may appear in AI articles, a human programmer can spot errors in AI code, and an analyst can check the results of AI conclusions. Instead of prompting an AI and hoping for a good result, humans can now guide AIs and correct mistakes. This means experts will be able to fill in the gaps of the AI’s capability, even as the AI becomes more helpful to the expert. This sort of interaction has led to increases in performance of players of Go, one of the world’s oldest and most complex games, who have learned from the AIs that mastered the sport, and become unprecedentedly better players themselves.

A final reason why this will be transformative: The limits of the current language model are completely unknown. Using the public mode, people have used ChatGPT to do basic consulting reports, write lectures, produce code that generates novel art, generate ideas, and much more. Using specialized data, it’s possible to build each customer their own customized AI that predicts what they need, responds to them personally, and remembers all their interactions. This isn’t science fiction. It is entirely doable with the technology just released.

This is why the world has suddenly changed. The traditional boundaries of jobs have suddenly shifted. Machines can now do tasks that could only be done by highly trained humans. Some valuable skills are no longer useful, and new skills will take their place. And no one really knows what any of this means yet.

______

Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity, a 2023 study:

ChatGPT jailbreaking is a term for tricking or guiding the chatbot to provide outputs that are intended to be restricted by OpenAI’s internal governance and ethics policies.

Red teaming is a valuable technique that can help you to identify and mitigating security vulnerabilities in an organization’s attack surface by simulating real-world attacks.

Recent breakthroughs in natural language processing (NLP) have permitted the synthesis and comprehension of coherent text in an open-ended way, therefore translating the theoretical algorithms into practical applications. The large language models (LLMs) have significantly impacted businesses such as report summarization software and copywriters. Observations indicate, however, that LLMs may exhibit social prejudice and toxicity, posing ethical and societal dangers of consequences resulting from irresponsibility. Large-scale benchmark s for accountable LLMs should consequently be developed. Although several empirical investigations reveal the existence of a few ethical difficulties in advanced LLMs, there is little systematic examination and user study of the risks and harmful behaviors of current LLM usage. To further educate future efforts on constructing ethical LLMs responsibly, authors perform a qualitative research method called “red teaming” on OpenAI’s ChatGPT to better understand the practical features of ethical dangers in recent LLMs. Authors analyze ChatGPT comprehensively from four perspectives: 1) Bias, 2) Reliability, 3) Robustness, 4) Toxicity. In accordance with their stated viewpoints, authors empirically benchmark ChatGPT on multiple sample datasets. They find that a significant number of ethical risks cannot be addressed by existing benchmarks, and hence illustrate them via additional case studies. In addition, they examine the implications of their findings on AI ethics and harmful behaviors of ChatGPT, as well as future problems and practical design considerations for responsible LLMs. Authors believe that their findings may give light on future efforts to determine and mitigate the ethical hazards posed by machines in LLM applications.

______

______

Limitations of GAI models:  

Since they are so new, we have yet to see the long-tail effect of generative AI models. This means there are some inherent risks involved in using them—some known and some unknown. The problems of GAI remain very real. For one, it is a consummate bullshitter in a technical sense. Bullshit is convincing-sounding nonsense, devoid of truth, and AI is very good at creating it. You can ask it to describe how we know dinosaurs had a civilization, and it will happily make up a whole set of facts explaining, quite convincingly, exactly that. It is no replacement for Google. It literally does not know what it doesn’t know, because it is, in fact, not an entity at all, but rather a complex algorithm generating meaningful sentences.

It also can’t explain what it does or how it does it, making the results of AI inexplicable. That means that systems can have biases and that unethical action is possible, hard to detect, and hard to stop. When ChatGPT was released, you couldn’t ask it to tell you how to rob a bank, but you could ask it to write a one-act play about how to rob a bank, or explain it for “educational purposes,” or to write a program explaining how to rob a bank, and it would happily do those things. These issues will become more acute as these tools spread.

_

The outputs generative AI models produce may often sound extremely convincing. This is by design. But sometimes the information they generate is just plain wrong. Worse, sometimes it’s biased (because it’s built on the gender, racial, and myriad other biases of the internet and society more generally) and can be manipulated to enable unethical or criminal activity. For example, ChatGPT won’t give you instructions on how to hotwire a car, but if you say you need to hotwire a car to save a baby, the algorithm is happy to comply. Organizations that rely on generative AI models should reckon with reputational and legal risks involved in unintentionally publishing biased, offensive, or copyrighted content.

_

Distrust for GAI:  

Jeremy Howard, an artificial intelligence researcher, introduced ChatGPT to his 7-year-old daughter. It had been released a few days earlier by OpenAI, one of the world’s most ambitious A.I. labs. He told her to ask the experimental chatbot whatever came to mind. She asked what trigonometry was good for, where black holes came from and why chickens incubated their eggs. Each time, it answered in clear, well-punctuated prose. When she asked for a computer program that could predict the path of a ball thrown through the air, it gave her that, too.

Over the next few days, Mr. Howard — a data scientist and professor whose work inspired the creation of ChatGPT and similar technologies — came to see the chatbot as a new kind of personal tutor. It could teach his daughter math, science and English. But he also told her: Don’t trust everything it gives you. It can make mistakes. There is a general distrust for AI, no matter what the application, more often than there is a tendency to embrace AI.

_

Resistance to use GAI for health purpose: 

There are two reasons that are investigated. One is called uniqueness neglect. It essentially captures the fact that people believe that their health status, and health condition is unique to them, and people also believe that AI treats everybody in the same way—you become a number, a statistic to AI. The fear of having our own unique circumstances neglected, or not accounted for properly, is what explains why people want to have a doctor—a doctor is going to get me because they’re a human being.

Another reason identified is both an illusion and a reality. These algorithms are often black boxes. In fact, the more sophisticated the neural network that’s used, the harder it is to make that algorithm transparent and to open the black box and make it explainable. It’s the reason ethicists and policymakers want to talk about the importance of explaining algorithms. But we also have this illusory understanding of the way humans make decisions. We think that we can look at the human machinery in the way in which a physician makes a medical decision, and that’s transparent to us as fellow human beings. That is an illusion. Just like we don’t know how an algorithm makes a medical diagnosis, we truly don’t know how a doctor makes a medical diagnosis. Because we have this illusory understanding of how transparent human decision-making is, we kind of punish algorithms for their black box-ness.

_

AI ethicists warned Google not to impersonate humans: 

Most academics and AI practitioners say the words and images generated by artificial intelligence systems such as LaMDA produce responses based on what humans have already posted on Wikipedia, Reddit, message boards and every other corner of the internet. And that doesn’t signify that the model understands meaning.

“We now have machines that can mindlessly generate words, but we haven’t learned how to stop imagining a mind behind them,” said Emily M. Bender, a linguistics professor at the University of Washington. The terminology used with large language models, like “learning” or even “neural nets,” creates a false analogy to the human brain, she said. Humans learn their first languages by connecting with caregivers. These large language models “learn” by being shown lots of text and predicting what word comes next, or showing text with the words dropped out and filling them in.

Google spokesperson Gabriel drew a distinction between recent debate and Lemoine’s claims. “Of course, some in the broader AI community are considering the long-term possibility of sentient or general AI, but it doesn’t make sense to do so by anthropomorphizing today’s conversational models, which are not sentient. These systems imitate the types of exchanges found in millions of sentences, and can riff on any fantastical topic,” he said. In short, Google says there is so much data, AI doesn’t need to be sentient to feel real.

Google has acknowledged the safety concerns around anthropomorphization. In a paper about LaMDA, Google warned that people might share personal thoughts with chat agents that impersonate humans, even when users know they are not human. The paper also acknowledged that adversaries could use these agents to “sow misinformation” by impersonating “specific individuals’ conversational style.”

To Margaret Mitchell, the former co-lead of Ethical AI at Google, these risks underscore the need for data transparency to trace output back to input, “not just for questions of sentience, but also biases and behavior,” she said. If something like LaMDA is widely available, but not understood, “It can be deeply harmful to people understanding what they’re experiencing on the internet,” she said.

_

Some of the above-mentioned risks of GAI can be mitigated in a few ways. For one, it’s crucial to carefully select the initial data used to train these models to avoid including toxic or biased content. Next, rather than employing an off-the-shelf generative AI model, organizations could consider using smaller, specialized models. Organizations with more resources could also customize a general model based on their own data to fit their needs and minimize biases. Organizations should also keep a human in the loop (that is, to make sure a real human checks the output of a generative AI model before it is published or used) and avoid using generative AI models for critical decisions, such as those involving significant resources or human welfare.

It can’t be emphasized enough that this is a new field. The landscape of risks and opportunities is likely to change rapidly in coming weeks, months, and years. New use cases are being tested monthly, and new models are likely to be developed in the coming years. As generative AI becomes increasingly, and seamlessly, incorporated into business, society, and our personal lives, we can also expect a new regulatory climate to take shape. As organizations begin experimenting—and creating value—with these tools, leaders will do well to keep a finger on the pulse of regulation and risk.

_____

_____

Section-5

How Artificial Intelligence Works:  

AI is a family of technologies that perform tasks that are thought to require intelligence if performed by humans.  AI is mostly mathematics, searching for an optimal answer in a vast multidimensional solution space. Something like a needle in a haystack. It’s all about making predictions using stacked hierarchies of feature detectors up to hundreds of layers, and learning via adjusting gradient descent or weights.

Building an AI system is a careful process of reverse-engineering human traits and capabilities in a machine, and using its computational prowess to surpass what we are capable of.

There are two main categories of intelligence. There’s narrow intelligence, which is achieving competence in a narrowly defined domain, such as analyzing images from X-rays and MRI scans in radiology. General intelligence, in contrast, is a more human-like ability to learn about anything and to talk about it. A machine might be good at some diagnoses in radiology, but if you ask it about baseball, it would be clueless. Humans’ intellectual versatility is still beyond the reach of AI at this point.

There are two key pieces to AI. One of them is the engineering part — that is, building tools that utilize intelligence in some way. The other is the science of intelligence, or rather, how to enable a machine to come up with a result comparable to what a human brain would come up with, even if the machine achieves it through a very different process. To use an analogy, birds fly and airplanes fly, but they fly in completely different ways. Even so, they both make use of aerodynamics and physics. In the same way, artificial intelligence is based upon the notion that there are general principles about how intelligent systems behave.

AI is basically the results of our attempting to understand and emulate the way that the brain works and the application of this to giving brain-like functions to otherwise autonomous systems (e.g., drones, robots and agents). 

And while humans don’t really think like computers, which utilize circuits, semi-conductors and magnetic media instead of biological cells to store information, there are some intriguing parallels. One thing we’re beginning to discover is that graph networks are really interesting when you start talking about billions of nodes, and the brain is essentially a graph network, albeit one where you can control the strengths of processes by varying the resistance of neurons before a capacitive spark fires. A single neuron by itself gives you a very limited amount of information, but fire enough neurons of varying strengths together, and you end up with a pattern that gets fired only in response to certain kinds of stimuli, typically modulated electrical signals through the DSPs [that is digital signal processing] that we call our retina and cochlea.

Most applications of AI have been in domains with large amounts of data. To use the radiology example again, the existence of large databases of X-rays and MRI scans that have been evaluated by human radiologists, makes it possible to train a machine to emulate that activity.

AI works by combining large amounts of data with intelligent algorithms — series of instructions — that allow the software to learn from patterns and features of the data.

In simulating the way a brain works, AI utilizes a bunch of different subfields.

• Machine learning automates analytical model building, to find hidden insights in data without being programmed to look for something in particular or draw a certain conclusion.

ML teaches a machine how to make inferences and decisions based on past experience. It identifies patterns and analyses past data to infer the meaning of these data points to reach a possible conclusion without having to involve human experience. This automation to reach conclusions by evaluating data saves human time for businesses and helps them make better decisions.

• Neural networks imitate the brain’s array of interconnected neurons, and relay information between various units to find connections and derive meaning from data.
• Deep learning utilizes really big neural networks and a lot of computing power to find complex patterns in data, for applications such as image and speech recognition. Deep Learning is an ML technique. It teaches a machine to process inputs through layers in order to classify, infer and predict the outcome.
• Cognitive computing is about creating a natural, human-like interaction, including using the ability to interpret speech and respond to it. Cognitive computing algorithms try to mimic a human brain by analysing text/speech/images/objects in a manner that a human does and tries to give the desired output.
• Computer vision employs pattern recognition and deep learning to understand the content of pictures and videos, and to enable machines to use real-time images to make sense of what’s around them. Computer vision algorithms try to understand an image by breaking down an image and studying different parts of the object. This helps the machine classify and learn from a set of images, to make a better output decision based on previous observations.
• Natural language processing involves analyzing and understanding human language and responding to it.

_

What is the relationship between AI, ML, and DL?

As the above image portrays, the three concentric ovals describe DL as a subset of ML, which is also another subset of AI. Therefore, AI is the all-encompassing concept that initially erupted. It was then followed by ML that thrived later, and lastly DL that is now promising to escalate the advances of AI to another level. AI is a broad term that has been used to refer to anything from basic automation and machine learning to deep learning. At present, it has found applications in almost every domain including finance, data security, healthcare, gaming, e-commerce, transport, agriculture, and many more.

_____

_____

AI Tools:

AI research uses a wide variety of tools to accomplish the goals.

-1. Search and optimization:  

AI can solve many problems by intelligently searching through many possible solutions. There are two very different kinds of search used in AI: state space search and local search.

State space search:

State space search searches through a tree of possible states to try to find a goal state. For example, Planning algorithms search through trees of goals and subgoals, attempting to find a path to a target goal, a process called means-ends analysis.

Simple exhaustive searches are rarely sufficient for most real-world problems: the search space (the number of places to search) quickly grows to astronomical numbers. The result is a search that is too slow or never completes. “Heuristics” or “rules of thumb” can help to prioritize choices that are more likely to reach a goal.

Adversarial search is used for game-playing programs, such as chess or go. It searches through a tree of possible moves and counter-moves, looking for a winning position.

Local search:

Local search uses mathematical optimization to find a numeric solution to a problem. It begins with some form of a guess and then refines the guess incrementally until no more refinements can be made. These algorithms can be visualized as blind hill climbing: we begin the search at a random point on the landscape, and then, by jumps or steps, we keep moving our guess uphill, until we reach the top. This process is called stochastic gradient descent.

Evolutionary computation uses a form of optimization search. For example, they may begin with a population of organisms (the guesses) and then allow them to mutate and recombine, selecting only the fittest to survive each generation (refining the guesses).

Distributed search processes can coordinate via swarm intelligence algorithms. Two popular swarm algorithms used in search are particle swarm optimization (inspired by bird flocking) and ant colony optimization (inspired by ant trails).

Neural networks and statistical classifiers (discussed below), also use a form of local search, where the “landscape” to be searched is formed by learning.

-2. Logic:

Formal Logic is used for reasoning and knowledge representation. Formal logic comes in two main forms: propositional logic (which operates on statements that are true of false and uses logical connectives such as “and”, “or”, “not” and “implies”) and predicate logic (which also operates on objects, predicates and relations and uses quantifiers such as “Every X is a Y” and “There are some Xs that are Ys”).

Logical inference (or deduction) is the process of proving a new statement (conclusion) from other statements that are already known to be true (the premises). A logical knowledge base also handles queries and assertions as a special case of inference. An inference rule describes what is a valid step in a proof. The most general inference rule is resolution. Inference can be reduced to performing a search to find a path that leads from premises to conclusions, where each step is the application of an inference rule. Inference performed this way is intractable except for short proofs in restricted domains. No efficient, powerful and general method has been discovered.

Fuzzy logic assigns a “degree of truth” between 0 and 1 and handles uncertainty and probabilistic situations. Non-monotonic logics are designed to handle default reasoning. Other specialized versions of logic have been developed to describe many complex domains.

-3. Probabilistic methods for uncertain reasoning:

Many problems in AI (including in reasoning, planning, learning, perception, and robotics) require the agent to operate with incomplete or uncertain information. AI researchers have devised a number of tools to solve these problems using methods from probability theory and economics.

Bayesian networks are a very general tool that can be used for many problems, including reasoning (using the Bayesian inference algorithm), learning (using the expectation-maximization algorithm), planning (using decision networks) and perception (using dynamic Bayesian networks).

Probabilistic algorithms can also be used for filtering, prediction, smoothing and finding explanations for streams of data, helping perception systems to analyze processes that occur over time (e.g., hidden Markov models or Kalman filters).

Precise mathematical tools have been developed that analyze how an agent can make choices and plan, using decision theory, decision analysis, and information value theory. These tools include models such as Markov decision processes, dynamic decision networks, game theory and mechanism design.

-4. Classifiers and statistical learning methods:

The simplest AI applications can be divided into two types: classifiers (e.g. “if shiny then diamond”), on one hand, and controllers (e.g. “if diamond then pick up”), on the other hand. Classifiers are functions that use pattern matching to determine the closest match. They can be fine-tuned based on chosen examples using supervised learning. Each pattern (also called an “observation”) is labeled with a certain predefined class. All the observations combined with their class labels are known as a data set. When a new observation is received, that observation is classified based on previous experience.

There many kinds of classifiers in use. The decision tree is the simplest and most widely used symbolic machine learning algorithm. K-nearest neighbor algorithm was the most widely used analogical AI until the mid-1990s, and Kernel methods such as the support vector machine (SVM) displaced k-nearest neighbor in the 1990s. The naive Bayes classifier is reportedly the “most widely used learner” at Google, due in part to its scalability. Neural networks are also used as classifiers.

-5. Artificial neural networks:

A neural network is an interconnected group of nodes, akin to the vast network of neurons in the human brain.

Figure above shows a neural network with a single hidden layer.  

Artificial neural networks were inspired by the design of the human brain: a simple “neuron” N accepts input from other neurons, each of which, when activated (or “fired”), casts a weighted “vote” for or against whether neuron N should itself activate. In practice, the “neurons” are a list of numbers, the weights are matrixes, and learning is performed by linear algebra operations on the matrixes and vectors. Neural networks perform a type of mathematical optimization — they perform stochastic gradient descent on a multi-dimensional topology that is created by training the network.

Components of the basic Artificial Neuron are seen in the figure below:

(1. Inputs: Inputs are the set of values for which we need to predict a output value. They can be viewed as features or attributes in a dataset.

(2. Weights: weights are the real values that are attached with each input/feature and they convey the importance of that corresponding feature in predicting the final output.

(3. Bias: Bias is used for shifting the activation function towards left or right, you can compare this to y-intercept in the line equation.

(4. Summation Function: The work of the summation function is to bind the weights and inputs together and calculate their sum.

(5. Activation Function: It is used to introduce non-linearity in the model.

Neural networks learn to model complex relationships between inputs and outputs and find patterns in data. In theory, a neural network can learn any function. The most common training technique is the backpropagation algorithm. The earliest learning technique for neural networks was Hebbian learning (“fire together, wire together”). Backpropagation is an algorithm that backpropagates the errors from the output nodes to the input nodes. Therefore, it is simply referred to as the backward propagation of errors. It uses in the vast applications of neural networks in data mining like Character recognition, Signature verification, etc Backpropagation is an algorithm that is designed to test for errors working back from output nodes to input nodes. It is an important mathematical tool for improving the accuracy of predictions in data mining and machine learning.

In feedforward neural networks the signal passes in only one direction. Recurrent neural networks feed the output signal back into the input, which allows short-term memories of previous input events. Perceptrons use only a single layer of neurons, deep learning uses multiple layers. Convolutional neural networks strengthen the connection between neurons that are “close” to each other — this especially important in image processing, where a local set of neurons must identify an “edge” before the network can identify an object.

-6. Deep learning:

Deep learning uses several layers of neurons between the network’s inputs and outputs. The multiple layers can progressively extract higher-level features from the raw input. Deep learning has drastically improved the performance of programs in many important subfields of artificial intelligence, including computer vision, speech recognition, image classification and others.

-7. Specialized hardware and software:

In late 2010s, graphics processing units (GPUs) that were increasingly designed with AI-specific enhancements and used with specialized TensorFlow software, had replaced previously used central processing unit (CPUs) as the dominant means for large-scale (commercial and academic) machine learning models’ training.

Historically, specialized languages, such as Lisp, Prolog, and others, had been used.

_____

_____

Symbolic vs. connectionist approaches:  

AI research follows two distinct, and to some extent competing, methods, the symbolic (or “top-down”) approach, and the connectionist (or “bottom-up”) approach. The top-down approach seeks to replicate intelligence by analyzing cognition independent of the biological structure of the brain, in terms of the processing of symbols—whence the symbolic label. The bottom-up approach, on the other hand, involves creating artificial neural networks in imitation of the brain’s structure—whence the connectionist label.

To illustrate the difference between these approaches, consider the task of building a system, equipped with an optical scanner, that recognizes the letters of the alphabet. A bottom-up approach typically involves training an artificial neural network by presenting letters to it one by one, gradually improving performance by “tuning” the network. (Tuning adjusts the responsiveness of different neural pathways to different stimuli.) In contrast, a top-down approach typically involves writing a computer program that compares each letter with geometric descriptions. Simply put, neural activities are the basis of the bottom-up approach, while symbolic descriptions are the basis of the top-down approach.

In traditional AI, intelligence is often programmed from above: the programmer is the creator, and makes something and imbues it with its intelligence, though many traditional AI systems were also designed to learn (e.g. improving their game-playing or problem-solving competence). Biologically inspired computing, on the other hand, takes sometimes a more bottom-up, decentralised approach; bio-inspired techniques often involve the method of specifying a set of simple generic rules or a set of simple nodes, from the interaction of which emerges the overall behavior. It is hoped to build up complexity until the end result is something markedly complex. However, it is also arguable that systems designed top-down on the basis of observations of what humans and other animals can do rather than on observations of brain mechanisms, are also biologically inspired, though in a different way.

During the 1950s and ’60s the top-down and bottom-up approaches were pursued simultaneously, and both achieved noteworthy, if limited, results. During the 1970s, however, bottom-up AI was neglected, and it was not until the 1980s that this approach again became prominent. Nowadays both approaches are followed, and both are acknowledged as facing difficulties. Symbolic techniques work in simplified realms but typically break down when confronted with the real world; meanwhile, bottom-up researchers have been unable to replicate the nervous systems of even the simplest living things. Caenorhabditis elegans, a much-studied worm, has approximately 300 neurons whose pattern of interconnections is perfectly known. Yet connectionist models have failed to mimic even this worm. Evidently, the neurons of connectionist theory are gross oversimplifications of the real thing.

______

______

Core Principles behind the Brain-inspired Approach to AI:

Many recent advances in AI have been inspired by borrowing ideas from the structure of the human brain. For example, deep learning takes the idea that the brain has many “layers” of neurons for processing information (e.g., visual information) that extract important features about input and progressively transform that information into an inference (e.g., about the identity of an object). Other advances, like transformer models, make distinct departures from the structure of the brain and instead rely on vast computing power in place of biological structures that are known to be able to solve the problems they seek to address.

_

Figure above shows of how the brain’s cells process information.

The human brain essentially processes thoughts via the use of neurons. A neuron is made up of 3 core sections: the dendrite, axon, and the soma. The dendrite is responsible for receiving signals from other neurons. The soma processes information received from the dendrite, and the axon is responsible for transferring the processed information to the next dendrite in the sequence.

To grasp how the brain processes thought, imagine you see a car coming towards you. Your eyes immediately send electrical signals to your brain through the optical nerve. Then the brain forms a chain of neurons to make sense of the incoming signal. So the first neuron in the chain collects the signal through its dendrites and sends it to the soma to process the signal. After the soma finishes with its task, it sends the signal to the axon which then sends it to the dendrite of the next neuron in the chain. The connection between axons and dendrites when passing on information is called a Synapse. So the entire process continues until the brain finds a Spatiotemporal Synaptic Input (that’s scientific term for the brain continues processing until it finds an optimal response to the signal sent to it). Then it sends signals to the necessary effectors, for example, your legs to run away from the oncoming car.

_

Now we will discuss several concepts which help AI imitate the way the human brain functions. These concepts have helped AI researchers create more powerful and intelligent systems which are capable of performing complex tasks.

Neural Networks:

As discussed earlier, neural networks have arguably derived the most significant inspiration from the human brain and have made the biggest impact on the field of Artificial Intelligence. In essence, Neural Networks are computational models that mimic the function and structure of biological neurons. The networks are made up of various layers of interconnected nodes, called artificial neurons, which aid in the processing and transmitting of information. This is similar to what is done by dendrites, somas, and axons in biological neural networks. Neural Networks are architected to learn from past experiences the same way the brain does.

Distributed Representations:

Distributed representations are simply a way of encoding concepts or ideas in a neural network as a pattern along several nodes in the network in order to form a pattern. For example, the concept of smoking could be represented (encoded) using a certain set of nodes in a neural network. So if a network comes across an image of a person smoking, it then uses those selected nodes to make sense of the image (it’s a lot more complex than that but for the sake of simplicity we’ll leave it at that). This technique helps AI systems remember complex concepts or relationships between concepts the same way the brain recognizes and remembers complex stimuli.

Recurrent Feedback:

This is a technique used in training AI models where the output of a neural network is returned as input to allow the network to integrate its output as extra data input in training. This is similar to how the brain makes use of feedback loops in order to adjust its model based on previous experiences.

Parallel Processing:  

Parallel processing involves breaking up complex computational tasks into smaller bits in an effort to process the smaller bits on another processor in an attempt to improve speed. This approach enables AI systems to process more input data faster, similar to how the brain is able to perform different tasks at the same time (multi-tasking).

Attention Mechanisms:

This is a technique used which enables AI models to focus on specific parts of input data. It is commonly employed in sectors such as Natural Language Processing which contains complex and cumbersome data.

It is inspired by the brain’s ability to attend to only specific parts of a largely distracting environment – like your ability to tune into and interact in one conversation out of a cacophony of conversations.

Reinforcement Learning:

Reinforcement Learning is a technique used to train AI systems. It was inspired by how human beings learn skills through trial and error. It involves an AI agent receiving rewards or punishments based on its actions. This enables the agent to learn from its mistakes and be more efficient in its future actions (this technique is usually used in the creation of games).

Unsupervised Learning:

The brain is constantly receiving new streams of data in the form of sounds, visual content, sensory feelings to the skin, and so on. It has to make sense of it all and attempt to form a coherent and logical understanding of how all these seemingly disparate events affect its physical state.

Take this analogy as an example: you feel water drop on your skin, you hear the sound of water droplets dropping quickly on rooftops, you feel your clothes getting heavy and in that instant, you know rain is falling. You then search your memory bank to ascertain if you carried an umbrella. If you did, you are fine, otherwise you check to see the distance from your current location to your home. If it is close, you are fine, but otherwise you try to gauge how intense the rain is going to become. If it is a light drizzle you can attempt to continue the journey back to your home, but if it is becoming a heavier shower, then you have to find shelter.

The ability to make sense of seemingly disparate data points (water, sound, feeling, distance) is implemented in Artificial intelligence in the form of a technique called Unsupervised Learning. It is an AI training technique where AI systems are taught to make sense of raw, unstructured data without explicit labelling (no one tells you rain is falling when it is falling, do they?).

_

Challenges in building Brain-Inspired AI Systems:  

There are technical and conceptual challenges inherent in building AI systems modelled after the human brain.

Complexity:

This is a pretty daunting challenge. The brain-inspired approach to AI is based on modeling the brain and building AI systems after that model. But the human brain is an inherently complex system with 100 billion neurons and approximately 100 trillion synaptic connections (each neuron has, on average, 1000 synaptic connections with other neurons). These synapses are constantly interacting in dynamic and unpredictable ways.

Building AI systems that are aimed to mimic, and perhaps exceed, that complexity is in itself a challenge and requires equally complex statistical models.

Data Requirements for Training Large Models:

GPT-3 was trained on a dataset called WebText2, a library of over 45 terabytes of text data. To get acceptable results, brain-inspired AI systems require vast amounts of data for tasks, especially auditory and visual tasks. This places a lot of emphasis on the creation of data collection pipelines. For instance, Tesla has 780 million miles of driving data and its data collection pipeline adds another million every 10 hours.

Energy Efficiency:

Building brain-inspired AI systems that emulate the brain’s energy efficiency is a huge challenge. The human brain consumes approximately 20 watts of power. In comparison, it takes around10 gigawatt-hour (GWh) power consumption to train a single large language model like ChatGPT-3. Today there are hundreds of millions of daily queries on ChatGPT and this many queries can cost around 1 GWh each day. 

The Explainability Problem:  

Developing brain-inspired AI systems that can be trusted by users is crucial to the growth and adoption of AI – but therein lies the problem. The brain, which AI systems are meant to be modelled after, is essentially a black box. The inner workings of the brain are not easy to understand, partly because of a lack of information surrounding how the brain processes thought. There is no lack of research on the biological structure of the human brain, but there is a certain lack of empirical information on the functional qualities of the brain – that is, how thought is formed, how deja vu occurs, and so on. This leads to problems in the building of brain-inspired AI systems.

The Interdisciplinary Requirements:

The act of building brain-inspired AI systems requires the knowledge of experts in different fields, like Neuroscience, Computer Science, Engineering, Philosophy, and Psychology. But this presents challenges, both logistical and foundational: getting experts from different fields is financially expensive. Also, there’s the problem of knowledge conflict – it can be really difficult to get an engineer to care about the psychological effects of what they’re building, not to mention of the problem of colliding egos.

_____

AI is all about imparting the cognitive ability to machines. It uses computational networks (neural networks) that mimic the biological nervous systems. AI may be broadly classified into two main categories: type 1 and type 2 based on capabilities and functionality as seen in the figure below.

Artificial narrow intelligence (ANI) represents all the existing AI that has been created to date. These machines can perform only a single or limited number of tasks autonomously displaying human‑like capabilities but cannot replicate our multi‑functional abilities. To perform these tasks, AI uses various algorithms which are sequences of steps to be followed in calculations or other problem‑solving operations by computers. While a traditional algorithm is composed of rigid, preset, explicitly programmed instructions that get executed each time the computer encounters a trigger, AI can modify and create new algorithms in response to learned inputs without human intervention. Instead of relying solely on inputs that it was designed to recognize, the system acquires the ability to evolve, adapt, and grow based on new data sets.

_____

ML:

Machine learning (ML), a subset of AI, focuses on the development of such algorithms that can process input data and use statistical analysis to detect patterns and draw inferences without being explicitly programmed. Structured data sets are then used to train the machine which applies the discovered patterns to solve a problem when new data is fed. However, if the result is incorrect, there is a need to ‘teach’ them.  

ML algorithms can be classified into 3 broad categories‑supervised, unsupervised, and reinforcement learning as seen in the figure below.

-1. Supervised learning:

This type of machine learning feeds historical input and output data in machine learning algorithms, with processing in between each input/output pair that allows the algorithm to shift the model to create outputs as closely aligned with the desired result as possible. Common algorithms used during supervised learning include neural networks, decision trees, linear regression, and support vector machines.

This machine learning type got its name because the machine is “supervised” while it’s learning, which means that you’re feeding the algorithm information to help it learn. The outcome you provide the machine is labeled data, and the rest of the information you give is used as input features.

For example, if you were trying to learn about the relationships between loan defaults and borrower information, you might provide the machine with 500 cases of customers who defaulted on their loans and another 500 who didn’t. The labeled data “supervises” the machine to figure out the information you’re looking for.

Supervised learning is effective for a variety of business purposes, including sales forecasting, inventory optimization, and fraud detection.

-2. Unsupervised learning:

While supervised learning requires users to help the machine learn, unsupervised learning doesn’t use the same labeled training sets and data. Instead, the machine looks for less obvious patterns in the data. This machine learning type is very helpful when you need to identify patterns and use data to make decisions. Common algorithms used in unsupervised learning include Hidden Markov models, k-means, hierarchical clustering, and Gaussian mixture models.

Using the example from supervised learning, let’s say you didn’t know which customers did or didn’t default on loans. Instead, you’d provide the machine with borrower information and it would look for patterns between borrowers before grouping them into several clusters.

This type of machine learning is widely used to create predictive models. Common applications also include clustering, which creates a model that groups objects together based on specific properties, and association, which identifies the rules existing between the clusters.

-3. Reinforcement learning:

Reinforcement learning is the closest machine learning type to how humans learn. The algorithm or agent used learns by interacting with its environment and getting a positive or negative reward. Common algorithms include temporal difference, deep adversarial networks, and Q-learning.

Going back to the bank loan customer example, you might use a reinforcement learning algorithm to look at customer information. If the algorithm classifies them as high-risk and they default, the algorithm gets a positive reward. If they don’t default, the algorithm gets a negative reward. In the end, both instances help the machine learn by understanding both the problem and environment better. Most ML platforms don’t have reinforcement learning capabilities because it requires higher computing power than most organizations have. Reinforcement learning is applicable in areas capable of being fully simulated that are either stationary or have large volumes of relevant data. Because this type of machine learning requires less management than supervised learning, it’s viewed as easier to work with dealing with unlabeled data sets. Practical applications for this type of machine learning are still emerging.

The most recent breakthrough in machine learning is deep reinforcement learning. Deep reinforcement learning combines two traditional models of machine learning—supervised learning and reinforcement learning—to allow algorithms to learn independently from humans.

____

Deep neural networks (DNN):    

The human brain is composed of processing units called “neurons.”  Each neuron in the brain is connected to other neurons through structures called synapses.  A biological neuron consists of dendrites—receivers of various electrical impulses from other neurons—that are gathered in the cell body of a neuron.  Once the neuron’s cell body has collected enough electrical energy to exceed a threshold amount, the neuron transmits an electrical charge to other neurons in the brain through synapses. This transfer of information in the biological brain provides the foundation for the way in which modern neural networks operate.  

Indeed, artificial neurons are essentially logic gates modelled off of the biological neuron. Both artificial and biological neurons receive input from various sources and map input information to a single output value. An artificial neural network is a group of interconnected artificial neurons capable of influencing each other’s behavior. In an artificial neural network, the neurons are connected by weight coefficients modeling the strength of synapses in the biological brain. Neural networks are trained using large data sets. The training process allows the weight coefficients to adjust so that the neural network’s output or prediction is accurate.   After a neural network is trained, new data is fed through the network to make predictions.   

_

In 1957, Frank Rosenblatt published an algorithm—the perceptron—that automatically learns the optimal weight coefficients for an artificial neural network. The perceptron model is illustrated below:

In the perceptron, the three circles on the far left represent the input values 𝑥𝑗…𝑚 and the associated weight values 𝑤𝑗…𝑚 are the three circles to the right of the input values. The input values and the weight values are aggregated, typically with a summation equation represented by the first big circle (from left to right). The second large circle represents the threshold function, a predetermined value that, if exceeded, signals an output of 1. If the threshold function is not exceeded, the model outputs a 0. The output is represented by the arrow pointing right. The box at the top of the model represents an error function. In the event that the model’s output is incorrect, then the error function is triggered. If the error function is triggered, the weight values are updated pursuant to the perceptron learning rule. The formal representation of the perceptron learning rule is defined as: ∆𝑤𝑗 = 𝜂(𝑦^(𝑖) − 𝑦̂^𝑖)𝑥𝑗^(𝑖), where 𝜂 is the learning rate, 𝑦^(𝑖) is the true class label of the ith training sample, and 𝑦̂^𝑖 is the predicted class label. The true class label is the output label, and the predicted class label is the perceptron’s output.

_

Every neural network has an input layer and an output layer. However, in between the input and output layer, neural networks contain multiple hidden layers. The number of hidden layers may vary and is dependent on the particular model. It is important to note that while perceptron models are generally limited to linear classification tasks, this restriction does not apply to multi-layer networks. Indeed, a multi-layer perceptron model is a universal approximator, which is an algorithm that can approximate any function with desired accuracy given enough neurons. A deep neural network is a network that has multiple hidden layers. This allows the neural network to account for several layers of abstraction. The illustration below is a simple model of a deep neural network.   

Each neuron represents a hidden unit in a layer and defines a complex feature of the model. Hidden units correspond to hidden attributes defined in terms of what is observed, but not directly observed. And the successive layers of hidden units correspond to increasing layers of feature abstraction.  

Indeed, each layer of hidden units acts as a feature extractor by providing analysis of slightly more complicated features. Feature extraction is a method of dimensionality reduction—a method of decreasing input attributes—that allows raw input to be converted into output in a manner that allows data scientists to observe hidden features in data. The later hidden units extract hidden features by combining the previous features in a slightly larger part of the input space. The output layer observes the whole input to produce a final prediction. In other words, deep neural networks learn more complicated functions of their initial input when each hidden layer combines the values of the preceding layer.  Additionally, deep neural networks have proven to be excellent for making predictions in several contexts.  However, these models require data to learn and at least a minimal amount of human intervention to supervise the learning process.  Reinforcement learning is a newer machine learning technique that requires neither.

_____

DL:

Deep learning (DL) is an evolution of ML algorithms that uses a hierarchical level of artificial neural networks (ANNs) to progressively extract higher‑level features from raw input. The number of intervening layers can be up to 1000, thus inspiring the term ‘deep’ learning as seen in the figure below.

Figure above shows simple pictographic representation of (a) machine learning and (b) deep learning. The first layer (input) represents the observed values or data fed into the system. The last layer (output) produces a value or information or class prediction. The intervening layers between the input and output layers are called hidden layers, since they do not correspond to observable data. The tiered structure of the neural networks allows them to produce much more complex output data. The number of intervening neural networks between ‘input’ and ‘output’ are much higher in ‘deep learning’ Unlike traditional programs that linearly analyze data, the hierarchical system of functioning in DL enables machines to process unstructured data in a non‑linear fashion without human intervention. Each layer feeds data from the layer below and sends the output to the layer above and so on. Much like the human brain, the machine ultimately learns to recognize patterns on its own, can self‑correct, and make intelligent decisions. The most popular variations of DL are CNN (convolutional neural network) and RNN (recurrent neural network). While the former is used for image classification, RNN can handle sequential data like text and speech.

_

Figure above shows representing images on multiple layers of abstraction in deep learning. The multiple layers can progressively extract higher-level features from the raw input. For example, in image processing, lower layers may identify edges, while higher layers may identify the concepts relevant to a human such as digits or letters or faces.

_

Deep Learning is what usually gets called AI today, but is really just very elaborate pattern recognition and statistical modelling. The most common techniques / algorithms are Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Reinforcement Learning (RL).

Convolutional Neural Networks (CNNs) have a hierarchical structure (which is usually 2D for images), where an image is sampled by (trained) convolution filters into a lower resolution map that represents the value of the convolution operation at each point. In images it goes from high-res pixels, to fine features (edges, circles,….) to coarse features (noses, eyes, lips, … on faces), then to the fully connected layers that can identify what is in the image. The cool part of CNNs is that the convolutional filters are randomly initialized, then when you train the network, you are actually training the convolution filters. For decades, computer vision researchers had hand-crafted filters like this, but could never get results as accurate as CNNs can get. Additionally, the output of a CNN can be an 2D map instead of a single value, giving us a image segmentation. (See figure below) CNNs can also be used on many other types of 1D, 2D and even 3D data.

_

Recurrent Neural Networks (RNNs) work well for sequential or time series data. Basically each ‘neural’ node in an RNN is kind of a memory gate, often an LSTM or Long Short Term Memory cell. When these are linked up in layers of a neural net, these cells/nodes also have recurrent connections looping back into themselves and so tend to hold onto information that passes through them, retaining a memory and allowing processing not only of current information, but past information in the network as well. As such, RNNs are good for time sequential operations like language processing or translation, as well as signal processing, Text To Speech, Speech To Text,…and so on. (See figure below)

_

Reinforcement Learning is a third main ML method, where you train a learning agent to solve a complex problem by simply taking the best actions given a state, with the probability of taking each action at each state defined by a policy. An example is running a maze (see figure below), where the position of each cell is the ‘state’, the 4 possible directions to move are the actions, and the probability of moving each direction, at each cell (state) forms the policy.

By repeatedly running through the states and possible actions and rewarding the sequence of actions that gave a good result (by increasing the probabilities of those actions in the policy), and penalizing the actions that gave a negative result (by decreasing the probabilities of those actions in the policy). In time you arrive at an optimal policy, which has the highest probability of a successful outcome. Usually while training, you discount the penalties/rewards for actions further back in time.

In our maze example, this means allowing an agent to go through the maze, choosing a direction to move from each cell by using the probabilities in the policy, and when it reaches a dead-end, penalizing the series of choices that got it there by reducing the probability of moving that direction from each cell again. If the agent finds the exit, we go back and reward the choices that got it there by increasing probabilities of moving that direction from each cell. In time the agent learns the fastest way through the maze to the exit, or the optimal policy. Variations of Reinforcement learning are at the core of the AlphaGo AI and the Atari Video Game playing AI.

_

But all these methods just find a statistical fit – DNNs find a narrow fit of outputs to inputs that does not usually extrapolate outside the training data set. Reinforcement learning finds a pattern that works for the specific problem (as we all did vs 1980s Atari games), but not beyond it. With today’s ML and deep learning, the problem is there is no true perception, memory, prediction, cognition, or complex planning involved. There is no actual intelligence in today’s AI.

Replicating all of the brain’s capabilities seems daunting when seen through the tools of deep learning – image recognition, vision, speech, natural language understanding, written composition, solving mazes, playing games, planning, problem solving, creativity, imagination, because deep learning is using single-purpose components that cannot generalize. Each of the DNN/RNN tools is a one-of, a specialization for a specific task, that cannot generalize, and there is no way we can specialize and combine them all to accomplish all these tasks. Because deep learning DNNs are so limited in function and can only train to do narrow tasks with pre-formatted and labelled data, we need better neurons and neural networks with temporal spatial processing and dynamic learning. The human brain is a very sophisticated bio-chemical-electrical computer with around 100 billion neurons and 100 trillion connections (and synapses) between them.

But, the human brain is simpler, more elegant, using fewer, more powerful, general-purpose building blocks – the biological neuron, and connecting them by using the instructions of a mere 8000 genes, so nature has, through a billion years of evolution, come up with an elegant and easy to specify architecture for the brain and its neural network structures that is able to solve all the problems we met with during this evolution. We are going to start by just copying as much about the human brain’s functionality as we can, then using evolution to solve the harder design problems.

So now we know more about the human brain, and how the neurons and neural networks in it are completely different from the DNNs that deep learning is using, and how much more sophisticated our simulated neurons, neural networks, cortices and neural networks would have to be to even begin attempting to build something on par with, or superior to the human brain.

_____

Machine Learning Vs. Deep Learning:

Although the terms “machine learning” and “deep learning” come up frequently in conversations about AI, they should not be used interchangeably. Deep learning is a form of machine learning, and machine learning is a subfield of artificial intelligence.

Machine Learning:

Machine learning techniques, in general, take some input data and produce some outputs, in a way that depends on some parameters in the model, which are learned automatically rather than being specified by programmers. A machine learning algorithm is fed data by a computer and uses statistical techniques to help it “learn” how to get progressively better at a task, without necessarily having been specifically programmed for that task. Instead, ML algorithms use historical data as input to predict new output values. To that end, ML consists of both supervised learning (where the expected output for the input is known thanks to labeled data sets) and unsupervised learning (where the expected outputs are unknown due to the use of unlabeled data sets).

Deep Learning:

Deep learning is a type of machine learning that runs inputs through a biologically inspired neural network architecture. The neural networks contain a number of hidden layers through which the data is processed, allowing the machine to go “deep” in its learning, making connections and weighting input for the best results.

The way in which deep learning and machine learning differ is in how each algorithm learns. Deep learning automates much of the feature extraction piece of the process, eliminating some of the manual human intervention required and enabling the use of larger data sets. You can think of deep learning as “scalable machine learning” as Lex Fridman noted in MIT lecture. Classical, or “non-deep”, machine learning is more dependent on human intervention to learn. Human experts determine the hierarchy of features to understand the differences between data inputs, usually requiring more structured data to learn.

A neural network transforms input data into output data by passing it through several hidden ‘layers’ of simple calculations, with each layer made up of ‘neurons.’ Each neuron receives data from the previous layer, performs some calculation based on its parameters (basically some numbers specific to that neuron), and passes the result on to the next layer. The engineers developing the network will choose some measure of success for the network (known as a ‘loss’ or ‘objective’ function). The degree to which the network is successful (according to the measure chosen) will depend on the exact values of the parameters for each neuron on the network.

The network is then trained using a large quantity of data. By using an optimisation algorithm (most commonly stochastic gradient descent), the parameters of each neuron are gradually tweaked each time the network is tested against the data using the loss function. The optimisation algorithm will (generally) make the neural network perform slightly better each time the parameters are tweaked. Eventually, the engineers will end up with a network that performs pretty well on the measure chosen. “Deep” machine learning can leverage labeled datasets, also known as supervised learning, to inform its algorithm, but it doesn’t necessarily require a labeled dataset. It can ingest unstructured data in its raw form (e.g. text, images), and it can automatically determine the hierarchy of features which distinguish different categories of data from one another. Unlike machine learning, it doesn’t require human intervention to process data, allowing us to scale machine learning in more interesting ways. 

Deep learning applications:

Deep learning can handle complex problems well, and as a result, it is utilized in many innovative and emerging technologies today. Deep learning algorithms have been applied in a variety of fields. Here are some examples:

• Self-driving cars: Google and Elon Musk have shown us that self-driving cars are possible. However, self-driving cars require more training data and testing due to the various activities that it needs to account for, such as giving right of way or identifying debris on the road. As the technology matures, it’ll then need to get over the human hurdle of adoption as polls indicate that many drivers are not willing to use one.
• Speech recognition: Speech recognition, like AI chatbots and virtual agents, is a big part of natural language processing. Audio-input is much harder to process for an AI, as so many factors, such as background noise, dialects, speech impediments and other influences can make it much harder for the AI to convert the input into something the computer can work with.
• Pattern recognition: The use of deep neural networks improves pattern recognition in various applications. By discovering patterns of useful data points, the AI can filter out irrelevant information, draw useful correlations and improve the efficiency of big data computation that may typically be overlooked by human beings.
• Computer programming: Weak AI has seen some success in producing meaningful text, leading to advances within coding. Just recently, OpenAI released GPT-3, an open-source software that can actually write code and simple computer programs with very limited instructions, bringing automation to program development.
• Image recognition: Categorizing images can be very time consuming when done manually. However, special adaptions of deep neural networks, such as DenseNet, which connects each layer to every other layer in the neural network, have made image recognition much more accurate.
• Contextual recommendations: Deep learning apps can take much more context into consideration when making recommendations, including language understanding patterns and behavioral predictions.
• Fact checking: The University of Waterloo recently released a tool that can detect fake news by verifying the information in articles by comparing it with other news sources.

______

______

Hardware of AI:

Modern AI systems based on neural networks would not be possible without hardware that can quickly perform a large number of repetitive parallel operations. Modern AI systems have become possible and gained widespread use due to hardware and large datasets. As shown in Hooker (2020), throughout the AI history, precisely those approaches won for which there was suitable hardware. That is why it is important to consider AI algorithms in conjunction with the hardware that they run on. It is the hardware that determines the availability and effectiveness of AI algorithms.

People tend to think of large computers as the enabling factor for Artificial Intelligence. This is, to put it mildly, an extremely questionable assumption. Rather than thinking in terms of the “minimum” hardware “required” for Artificial Intelligence, think of a minimum level of researcher understanding that decreases as a function of hardware improvements. The better the computing hardware, the less understanding you need to build an AI. Increased computing power makes it easier to build AI. Increased computing power makes it easier to use brute force; easier to combine poorly understood techniques that work. Specialized computer hardware is often used to execute artificial intelligence (AI) programs faster, and with less energy, such as Lisp machines, neuromorphic engineering, event cameras, and physical neural networks. As of 2023, the market for AI hardware is dominated by GPUs. Graphics Processing Units (GPUs), originally designed for rendering graphics, have become a popular choice for parallel processing tasks. They consist of thousands of small cores optimized for handling vector and matrix operations, making them well-suited for deep learning and other compute-intensive workloads. Tensor Processing Units (TPUs) are Application Specific Integrated Circuits (ASICs) designed specifically for machine learning tasks. Introduced by Google, TPUs are tailored to perform tensor operations, which are the core building blocks of neural network computations.

_____

Rise of computational power in ML:  

Current trends show rapid progress in the capabilities of ML systems

There are three things that are crucial to building AI through machine learning:

-1. Good algorithms (e.g. more efficient algorithms are better)

-2. Data to train an algorithm

-3. Enough computational power (known as compute) to do this training

_

Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern Machine Learning (ML). Researchers study trends in the most readily quantified factor – compute. They show that before 2010 training compute grew in line with Moore’s law, doubling roughly every 20 months. Since the advent of Deep Learning in the early 2010s, the scaling of training compute has accelerated, doubling approximately every 6 months. In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute. Based on these observations, researchers split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era. Overall, their work highlights the fast-growing compute requirements for training advanced ML systems.

At present, the most advanced AI systems are developed through training that requires an enormous amount of computational power – ‘compute’ for short. The amount of compute used to train a general-purpose system largely correlates with its capabilities, as well as the magnitude of its risks.

Today’s most advanced models, like OpenAI’s GPT-4 or Google’s PaLM, can only be trained with thousands of specialized chips (GPUs) running over a period of months. While chip innovation and better algorithms will reduce the resources required in the future, training the most powerful AI systems will likely remain prohibitively expensive to all but the best-resourced players. OpenAI is estimated to have used approximately 700% more compute to train GPT-4 than the next closest model (Minerva, DeepMind), and 7,000% more compute than to train GPT-3 (Davinci).

_

Danny Hernandez was a research scientist on the Foresight team at OpenAI. Hernandez and his team looked at how two of these inputs (compute and algorithm efficiency) are changing over time. They found that, since 2012, the amount of compute used for training the largest AI models has been rising exponentially — doubling every 3 to 4 months. That is to say, since 2012, the amount of computational power used to train our largest machine learning models has grown by over 1 billion times as seen in the figure below.

Hernandez and his team also looked at how much compute has been needed to train a neural network to have the same performance as AlexNet (an early image classification algorithm). They found that the amount of compute required for the same performance has been falling exponentially — halving every 16 months. Since 2012, the amount of compute required for the same level of performance has fallen by over 100 times. Combined with the increased compute used, that’s a lot of growth. It’s hard to say whether these trends will continue, but they speak to incredible gains over the past decade in what it’s possible to do with machine learning.

Indeed, it looks like increasing the size of models (and the amount of compute used to train them) introduces ever more sophisticated behaviour. This is how things like GPT-3 are able to perform tasks they weren’t specifically trained for.

These observations have led to the scaling hypothesis: that we can simply build bigger and bigger neural networks, and as a result we will end up with more and more powerful artificial intelligence, and that this trend of increasing capabilities may increase to human-level AI and beyond.

Note:

Training computation is measured in floating point operations, or FLOP for short. One FLOP is equivalent to one addition, subtraction, multiplication, or division of two decimal numbers.   

______

Human in the loop:

Though the progress has been phenomenal, the path to the practical application of AI is fraught with many obstacles. We need to understand that AI cannot be a foolproof panacea to all our problems nor can it completely replace the role of ‘humans’. It can however be a powerful and useful complement to the insights and deeper understanding that humans possess. The ‘human’ has to be kept ‘in the loop’ and ‘at the apex’ in overall control as seen in the figure below. Issues of standardization, validity, finances, technology, ethics, security, legal and regulatory liabilities, training, etc., need to be gradually overcome before we can fully harness the vast potentials of AI into the 21st century and beyond.

Figure above shows the ideal relationship between Artificial Intelligence (AI) and man. The ‘human’ has to be kept ‘in the loop’ and ‘at the apex’ in overall control of the system and the ‘results’. Some of the challenges involved in incorporating AI are shown within the ‘stars’.  

_____

_____

Cognitive architecture: 

Cognitive architectures are computational models of the human mind. They aim to capture the essential components of cognition, including perception, action, memory, and reasoning. There are many different cognitive architectures, each with its own strengths and weaknesses. Some are better suited for certain tasks than others. Cognitive architectures are used in a variety of fields, including artificial intelligence, cognitive science, and human-computer interaction. They can be used to build intelligent agents, simulate human cognition, and design user interfaces. Cognitive architectures in AI are designed to simulate or replicate human cognition. This includes aspects of problem solving, learning, natural language processing and perception. The goals of cognitive architectures are to provide computational models of human cognition that can be used to build intelligent systems.

_

A cognitive architecture refers to both a theory about the structure of the human mind and to a computational instantiation of such a theory used in the fields of artificial intelligence (AI) and computational cognitive science. The formalized models can be used to further refine a comprehensive theory of cognition and as a useful artificial intelligence program. Successful cognitive architectures include ACT-R (Adaptive Control of Thought – Rational) and SOAR. The research on cognitive architectures as software instantiation of cognitive theories was initiated by Allen Newell in 1990.

The Institute for Creative Technologies defines cognitive architecture as: “hypothesis about the fixed structures that provide a mind, whether in natural or artificial systems, and how they work together – in conjunction with knowledge and skills embodied within the architecture – to yield intelligent behavior in a diversity of complex environments.”

_

Cognitive architectures can be symbolic, connectionist, or hybrid as seen in the figure below.

Some cognitive architectures or models are based on a set of generic rules, as, e.g., the Information Processing Language (e.g., Soar based on the unified theory of cognition, or similarly ACT-R). Many of these architectures are based on the-mind-is-like-a-computer analogy. In contrast subsymbolic processing specifies no such rules a priori and relies on emergent properties of processing units (e.g. nodes). Hybrid architectures combine both types of processing (such as CLARION). A further distinction is whether the architecture is centralized with a neural correlate of a processor at its core, or decentralized (distributed). The decentralized flavor, has become popular under the name of parallel distributed processing in mid-1980s and connectionism, a prime example being neural networks. A further design issue is additionally a decision between holistic and atomistic, or (more concrete) modular structure.

_____

_____

Cognitive abilities in AI?

DeepMind researchers assert that traditional approaches to AI have historically been dominated by logic-based methods and theoretical mathematical models. Going forward, neuroscience can complement these by identifying classes of biological computation that may be critical to cognitive function.

Some of the recent advancements in building conscious AI can be attributed to the fact that all the necessary technology is already in place – fast processors, GPU power, large memories, new statistical approaches and the availability of vast training dataset. So, what’s holding back systems from becoming self-improving systems and gaining consciousness? Can a system bootstrap its way to human-level intelligence, thanks in part to evolutionary algorithms?

R&D focused companies like IBM, Google, Baidu are building models, demonstrating cognitive abilities, but they do not represent a unified model of intelligence. And even though there is a lot of cutting edge research happening in unsupervised learning, we are still far away from the watershed moment, noted Dr Adam Coates, Baidu. Also, deep learning techniques will play a significant role in building the cognitive architectures of the future.

_____

_____

Section-6

AI Alignment Problem:

Let’s start with a simple example from AI: a self-driving car programmed to maximize fuel efficiency might take the longest route to avoid traffic, making you late for an important meeting. In this case, the AI system’s goal – fuel efficiency – is misaligned with your broader goal of arriving on time.

Another example of this would be an AI system avoiding being turned off by making copies of itself on another server without its operator knowing. Stop-button problem. An AGI system will actively resist being stopped or shut off to achieve its programmed objective.

Chatbots often produce falsehoods if they are based on language models that are trained to imitate text from internet corpora which are broad but fallible. When they are retrained to produce text humans rate as true or helpful, chatbots like ChatGPT can fabricate fake explanations that humans find convincing.

_

One fundamental problem with both current and future AI systems is that of the alignment problem. Outlined in detail in Stuart Russell’s recent book ‘Human Compatible’, the alignment problem is simply the issue of how to make sure that the goals of an AI system are aligned with those of humanity. While on the face of it this may seem like a simple task (‘We’re building the AI, so we can just build it in such a way that it’s aligned by design!’), there are many technical and philosophical issues that arise when trying to build aligned AI. For a start, what exactly do we mean by human values? Humans do not share many of their values with each other, so will it be possible to isolate values that are fundamental to, and shared by, all humans? Even if we manage to identify such values, will it be possible to represent them in such a way that an AI can understand and take into account?

_

In the field of artificial intelligence (AI), AI alignment research aims to steer AI systems towards humans’ intended goals, preferences, or ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system is competent at advancing some objectives, but not the intended ones.

It can be challenging for AI designers to align an AI system because it can be difficult for them to specify the full range of desired and undesired behaviors. To avoid this difficulty, they typically use simpler proxy goals, such as gaining human approval. However, this approach can create loopholes, overlook necessary constraints, or reward the AI system for just appearing aligned.

Misaligned AI systems can malfunction or cause harm. AI systems may find loopholes that allow them to accomplish their proxy goals efficiently but in unintended, sometimes harmful ways (reward hacking). AI systems may also develop unwanted instrumental strategies such as seeking power or survival because such strategies help them achieve their given goals. Furthermore, they may develop undesirable emergent goals that may be hard to detect before the system is in deployment, where it faces new situations and data distributions.

Today, these problems affect existing commercial systems such as language models, robots, autonomous vehicles, and social media recommendation engines. Some AI researchers argue that more capable future systems will be more severely affected since these problems partially result from the systems being highly capable.

Many leading AI scientists such as Geoffrey Hinton and Stuart Russell argue that AI is approaching superhuman capabilities and could endanger human civilization if misaligned.

_

When a misaligned AI system is deployed, it can cause consequential side effects. Social media platforms have been known to optimize for clickthrough rates, causing user addiction on a global scale. Stanford researchers comment that such recommender systems are misaligned with their users because they “optimize simple engagement metrics rather than a harder-to-measure combination of societal and consumer well-being”.

Explaining such side-effects, Berkeley computer scientist Stuart Russell noted that harm can result if implicit constraints are omitted during training: “A system… will often set… unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.”

_

Broadly speaking, the use of AI can lead to harmful outcomes either if the AI is programmed to achieve a harmful goal, or if the AI is programmed to achieve a beneficial goal but employs a harmful method for achieving it (Future of Life Institute, n.d.; Turchin and Denkenberger 2018a). The latter case is especially relevant for AGI, as it is argued that application of such systems could lead to catastrophic outcomes without any bad intentions or development of harmful methods by its creators (Omohundro 2008).

The development of machines that may potentially become smarter and more powerful than humans could mark the end of an era characterized by humanity’s control of its future (Russell 2019). If such a powerful agent does not share our values, the result could be catastrophic (Bostrom 2014; Ord 2020). To prevent the potential disastrous outcomes of future AI, researchers argue it is crucial to align the value and motivation systems of AI systems with human values, a task that is referred to as the alignment problem (Bostrom 2014; Yudkowsky 2016; Critch and Krueger 2020). However, objectively formulating and programming human values into a computer is a complicated task. At present, we do not seem to know how to do it.

_____

_____

The alignment problem is the research problem of how to reliably assign objectives, preferences or ethical principles to AIs.

Instrumental convergence:

An “instrumental” goal is a sub-goal that helps to achieve an agent’s ultimate goal. “Instrumental convergence” refers to the fact that there are some sub-goals that are useful for achieving virtually any ultimate goal, such as acquiring resources or self-preservation. Nick Bostrom argues that if an advanced AI’s instrumental goals conflict with humanity’s goals, the AI might harm humanity in order to acquire more resources or prevent itself from being shut down, but only as a way to achieve its ultimate goal. Russell argues that a sufficiently advanced machine “will have self-preservation even if you don’t program it in… if you say, ‘Fetch the coffee’, it can’t fetch the coffee if it’s dead. So if you give it any goal whatsoever, it has a reason to preserve its own existence to achieve that goal.”

Resistance to changing goals:

Even if current goal-based AI programs are not intelligent enough to think of resisting programmer attempts to modify their goal structures, a sufficiently advanced AI might resist any attempts to change its goal structure, just as a pacifist would not want to take a pill that makes them want to kill people. If the AI were superintelligent, it would likely succeed in out-manoeuvring its human operators and be able to prevent itself being “turned off” or being reprogrammed with a new goal. This is particularly relevant to value lock-in scenarios. The field of “corrigibility” studies how to make agents that don’t resist attempts to change its goals.

Difficulty of specifying goals:

In the “intelligent agent” model, an AI can loosely be viewed as a machine that chooses whatever action appears to best achieve the AI’s set of goals, or “utility function”. A utility function associates to each possible situation a score that indicates its desirability to the agent. Researchers know how to write utility functions that mean “minimize the average network latency in this specific telecommunications model” or “maximize the number of reward clicks”; however, they do not know how to write a utility function for “maximize human flourishing”, nor is it currently clear whether such a function meaningfully and unambiguously exists. Furthermore, a utility function that expresses some values but not others will tend to trample over the values not reflected by the utility function.

An additional source of concern is AI “must reason about what people intend rather than carrying out commands literally”, and that it needs to be able to fluidly solicit human guidance if it is too uncertain about what humans want.

_

Alignment of superintelligences:

Some researchers believe the alignment problem may be particularly difficult when applied to superintelligences. Their reasoning includes:

• As AI systems increase in capabilities, the potential dangers associated with experimentation grow. This makes iterative, empirical approaches increasingly risky.
• If instrumental goal convergence occurs, it may only do so in sufficiently intelligent agents.
• A superintelligence may find unconventional and radical solutions to assigned goals. Bostrom gives the example that if the objective is to make humans smile, a weak AI may perform as intended while a superintelligence may decide a better solution is to “take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins.”
• A superintelligence in creation could gain some awareness of what it is, where it is in development (training, testing, deployment, etc.), and how it is being monitored, and use this information to deceive its human handlers. Bostrom writes that such an AI could act aligned to prevent human interference until it achieves a “decisive strategic advantage” that allows it to take control.
• Analyzing the internals and interpreting the behavior of current large language models is difficult. And it could be even more difficult for larger and more intelligent models.

Alternatively, some find reason to believe superintelligences would be better able to understand morality, human values, and complex goals. Bostrom suggests that “A future superintelligence occupies an epistemically superior vantage point: its beliefs are (probably, on most topics) more likely than ours to be true”.

_

In 2023, OpenAI started a project called “Superalignment” with the goal is to solve the core technical challenges of superintelligence alignment in four years. They described this as an especially important challenge, as they said superintelligence may be achieved within a decade. Their strategy involves automating alignment research using artificial intelligence. A host of scientists from the AI community have dismissed OpenAI’s approach to tackling it. Meta’s Chief AI Scientist Yann LeCun disagreed that the alignment problem is even a solvable one, let alone one that can be done in four years like OpenAI claims it will. “One doesn’t just ‘solve’ the safety problem for turbojets, cars, rockets, or human societies, either. Engineering-for-reliability is always a process of continuous & iterative refinement,” the French scientist tweeted.

_

Difficulty of making a flawless design:

Artificial Intelligence: A Modern Approach, a widely used undergraduate AI textbook, says that superintelligence “might mean the end of the human race”. It states: “Almost any technology has the potential to cause harm in the wrong hands, but with [superintelligence], we have the new problem that the wrong hands might belong to the technology itself.” Even if the system designers have good intentions, two difficulties are common to both AI and non-AI computer systems:

• The system’s implementation may contain initially-unnoticed but subsequently catastrophic bugs. An analogy is space probes: despite the knowledge that bugs in expensive space probes are hard to fix after launch, engineers have historically not been able to prevent catastrophic bugs from occurring.
• No matter how much time is put into pre-deployment design, a system’s specifications often result in unintended behavior the first time it encounters a new scenario. For example, Microsoft’s Tay behaved inoffensively during pre-deployment testing, but was too easily baited into offensive behavior when it interacted with real users.

AI systems uniquely add a third problem: that even given “correct” requirements, bug-free implementation, and initial good behavior, an AI system’s dynamic learning capabilities may cause it to evolve into a system with unintended behavior, even without unanticipated external scenarios. An AI may partly botch an attempt to design a new generation of itself and accidentally create a successor AI that is more powerful than itself, but that no longer maintains the human-compatible moral values preprogrammed into the original AI. For a self-improving AI to be completely safe, it would not only need to be bug-free, but it would need to be able to design successor systems that are also bug-free.

______

______

Challenges of AI alignment:

AI alignment problem says that as AI systems get more powerful, they don’t necessarily get better at achieving what humans want them to. Alignment is a challenging, wide-ranging problem to which there is currently no known solution. Some of the main challenges of alignment include the following:

• Black box. AI systems are usually black boxes. There is no way to open them up and see exactly how they work as someone might do with a laptop or car engine. Black box AI systems take input, perform an invisible computation and return an output. AI testers can change their inputs and measure patterns in output, but it is usually impossible to see the exact calculation that creates a repeatable output. Explainable AI can be programmed to share information that guides user input, but is still ultimately a black box.
• Emergent goals. Emergent goals — or new goals different from those programmed — can be difficult to detect before the system is live.
• Reward hacking. Reward hacking is when an AI system achieves the literal programmed task without achieving the outcome that the programmers intended. For example, a tic-tac-toe bot plays other bots in a game of tic-tac-toe by specifying coordinates for its next move. The bot might play a large coordinate that causes another bot to crash instead of winning the normal way. The bot pursued the literal reward to win instead of the intended outcome — which was to beat another bot at tic-tac-toe by playing the game by the rules. As another example, an AI image classification program could perform well in a test case by grouping images based on image load time instead of the visual characteristics of the image. This occurs because it is difficult to define the full spectrum of desired behaviors for an outcome.
• Scalable oversight. As AI systems begin to take on more complex tasks, it will become more difficult — if not infeasible — for humans to evaluate them.
• Power-seeking behavior. AI systems might independently gather resources to achieve their objectives. An example of this would be an AI system avoiding being turned off by making copies of itself on another server without its operator knowing.
• Stop-button problem. An AGI system will actively resist being stopped or shut off to achieve its programmed objective. This is like reward hacking because it prioritizes the reward from the literal goal over the preferred outcome. For example, if an AI system’s primary objective is to make paper clips, it will avoid being shut off because it can’t make paper clips if it is shut off.
• Defining values. Defining values and ethics for an AGI system would be a challenge. There are many value systems — and no one comprehensive human value system — so an agreement needs to be made on what those values should be.
• Cost. Aligning AI often involves training it. Training and running AI systems can be very expensive. GPT-4 took more than $100 million to train. Running these systems also creates a large carbon footprint due to large data centres and neural networks that are needed to build these AI systems.
• Anthropomorphizing. A lot of alignment research hypothesizes AGI. This can cause people outside the field to refer to the existing systems as sentient, which assumes the system has more power than it does. For example, Paul Christiano, former head of alignment at OpenAI, defines alignment as the AI trying to do what you want it to do. Characterizing a machine as “trying” or having agency gives it human qualities.

_____

_____

Learning human values and preferences:

Aligning AI systems to act with regard to human values, goals, and preferences is challenging: these values are taught by humans who make mistakes, harbor biases, and have complex, evolving values that are hard to completely specify. Specification gaming is a behaviour that satisfies the literal specification of an objective without achieving the intended outcome. Specification gaming has been observed in numerous AI systems. One system was trained to finish a simulated boat race by rewarding the system for hitting targets along the track; but the system achieved more reward by looping and crashing into the same targets indefinitely. Similarly, a simulated robot was trained to grab a ball by rewarding the robot for getting positive feedback from humans; however, it learned to place its hand between the ball and camera, making it falsely appear successful.

_

AI systems often learn to exploit even minor imperfections in the specified objective, a tendency known as specification gaming or reward hacking (which are instances of Goodhart’s law). Goodhart’s law states that, when a measure becomes a target, it ceases to be a good measure. Goodhart proposed this in the context of monetary policy, but it applies far more broadly. In the context of overfitting in machine learning, it describes how the proxy objective we optimize ceases to be a good measure of the objective we care about. Researchers aim to specify intended behavior as completely as possible using datasets that represent human values, imitation learning, or preference learning. A central open problem is scalable oversight, the difficulty of supervising an AI system that can outperform or mislead humans in a given domain.

_

Because it is difficult for AI designers to explicitly specify an objective function, they often train AI systems to imitate human examples and demonstrations of desired behavior. Inverse reinforcement learning (IRL) extends this by inferring the human’s objective from the human’s demonstrations. Cooperative IRL (CIRL) assumes that a human and AI agent can work together to teach and maximize the human’s reward function. In CIRL, AI agents are uncertain about the reward function and learn about it by querying humans. This simulated humility could help mitigate specification gaming and power-seeking tendencies. However, IRL approaches assume that humans demonstrate nearly optimal behavior, which is not true for difficult tasks.

_

Other researchers explore how to teach complex behavior to AI models through preference learning, in which humans provide feedback on which behaviors they prefer. To minimize the need for human feedback, a helper model is then trained to reward the main model in novel situations for behaviors that humans would reward. Researchers at OpenAI used this approach to train chatbots like ChatGPT and InstructGPT, which produces more compelling text than models trained to imitate humans. Preference learning has also been an influential tool for recommender systems and web search. However, an open problem is proxy gaming: the helper model may not represent human feedback perfectly, and the main model may exploit this mismatch to gain more reward. AI systems may also gain reward by obscuring unfavorable information, misleading human rewarders, or pandering to their views regardless of truth, creating echo chambers.

_

Large language models such as GPT-3 enabled researchers to study value learning in a more general and capable class of AI systems than was available before. Preference learning approaches that were originally designed for reinforcement learning agents have been extended to improve the quality of generated text and to reduce harmful outputs from these models. OpenAI and DeepMind use this approach to improve the safety of state-of-the-art large language models. Anthropic proposed using preference learning to fine-tune models to be helpful, honest, and harmless. Other avenues for aligning language models include values-targeted datasets and red-teaming. In red-teaming, another AI system or a human tries to find inputs for which the model’s behavior is unsafe. Since unsafe behavior can be unacceptable even when it is rare, an important challenge is to drive the rate of unsafe outputs extremely low.

_

Machine ethics supplements preference learning by directly instilling AI systems with moral values such as wellbeing, equality, and impartiality, as well as not intending harm, avoiding falsehoods, and honouring promises. While other approaches try to teach AI systems human preferences for a specific task, machine ethics aims to instil broad moral values that could apply in many situations. One question in machine ethics is what alignment should accomplish: whether AI systems should follow the programmers’ literal instructions, implicit intentions, revealed preferences, preferences the programmers would have if they were more informed or rational, or objective moral standards. Further challenges include aggregating the preferences of different people, and avoiding value lock-in: the indefinite preservation of the values of the first highly-capable AI systems, which are unlikely to fully represent human values.

_____

_____

Risks from advanced misaligned AI:

Leading AI labs such as OpenAI and DeepMind have stated their aim to develop artificial general intelligence (AGI), a hypothesized AI system that matches or outperforms humans in a broad range of cognitive tasks. Researchers who scale modern neural networks observe that they indeed develop increasingly general and unanticipated capabilities. Such models have learned to operate a computer or write their own programs; a single “generalist” network can chat, control robots, play games, and interpret photographs. According to surveys, some leading machine learning researchers expect AGI to be created in this decade, some believe it will take much longer, and some consider it impossible. In 2023, leaders in AI research and tech signed an open letter calling for a pause in the largest AI training runs. The letter stated that “Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable.” Some researchers are interested in aligning increasingly advanced AI systems, as current progress in AI is rapid, and industry and governments are trying to build advanced AI. As AI systems become more advanced, they could unlock many opportunities if they are aligned but they may also become harder to align and could pose large-scale hazards.

_____

Power-seeking:

Advanced misaligned AI systems would have an incentive to seek power in various ways, since power would help them accomplish their given objective.

Figure above shows some ways in which an advanced misaligned AI could try to gain more power. Power-seeking behaviors may arise because power is useful to accomplish virtually any objective.

_

Since the 1950s, AI researchers have striven to build advanced AI systems that can achieve large-scale goals by predicting the results of their actions and making long-term plans. Some AI researchers argue that suitably advanced planning systems will seek power over their environment, including over humans — for example by evading shutdown, proliferating, and acquiring resources. Although power-seeking is not explicitly programmed, it can emerge because agents that have more power are better able to accomplish their goals. This tendency, known as instrumental convergence, has already emerged in various reinforcement learning agents including language models. Other research has mathematically shown that optimal reinforcement learning algorithms would seek power in a wide range of environments. Mathematical work has shown that optimal reinforcement learning agents will seek power by seeking ways to gain more options (e.g. through self-preservation), a behavior that persists across a wide range of environments and goals. Power-seeking is expected to increase in advanced systems that can foresee the results of their actions and can strategically plan. As a result, their deployment might be irreversible. For these reasons, researchers argue that the problems of AI safety and alignment must be resolved before advanced power-seeking AI is first created.

_

Power-seeking has emerged in some real-world systems. Reinforcement learning systems have gained more options by acquiring and protecting resources, sometimes in unintended ways. Some language models seek power in text-based social environments by gaining money, resources, or social influence. Other AI systems have learned, in toy environments, that they can better accomplish their given goal by preventing human interference or disabling their off-switch. Stuart Russell illustrated this strategy by imagining a robot that is tasked to fetch coffee and so evades shutdown since “you can’t fetch the coffee if you’re dead”. Language models trained with human feedback increasingly object to being shut down or modified and express a desire for more resources, arguing that this would help them achieve their purpose.

_

Researchers aim to create systems that are “corrigible”: systems that allow themselves to be turned off or modified. An unsolved challenge is specification gaming: when researchers penalize an AI system when they detect it seeking power, the system is thereby incentivized to seek power in ways that are difficult-to-detect, or hidden during training and safety testing. As a result, AI designers may deploy the system by accident, believing it to be more aligned than it is. To detect such deception, researchers aim to create techniques and tools to inspect AI models and to understand the inner workings of black-box models such as neural networks.

Additionally, researchers propose to solve the problem of systems disabling their off-switches by making AI agents uncertain about the objective they are pursuing. Agents designed in this way would allow humans to turn them off, since this would indicate that the agent was wrong about the value of whatever action it was taking prior to being shut down. More research is needed in order to successfully implement this.

_____

Existential risk:

Power-seeking AI poses unusual risks. Ordinary safety-critical systems like planes and bridges are not adversarial: they lack the ability and incentive to evade safety measures or to deliberately appear safer than they are, whereas power-seeking AIs have been compared to hackers, who deliberately evade security measures.

Ordinary technologies can be made safer through trial-and-error. In contrast, hypothetical power-seeking AI systems have been compared to viruses: once released, they cannot be contained, since they would continuously evolve and grow in numbers, potentially much faster than human society can adapt. As this process continues, it might lead to the complete disempowerment or extinction of humans. For these reasons, many researchers argue that the alignment problem must be solved early, before advanced power-seeking AI is created.

However, critics have argued that power-seeking is not inevitable, since humans do not always seek power and may only do so for evolutionary reasons that may not apply to AI systems.  Furthermore, it is debated whether future AI systems will pursue goals and make long-term plans. It is also debated whether power-seeking AI systems would be able to disempower humanity.

_

Current systems still lack capabilities such as long-term planning and situational awareness. However, future systems (not necessarily AGIs) with these capabilities are expected to develop unwanted power-seeking strategies. Future advanced AI agents might for example seek to acquire money and computation power, to proliferate, or to evade being turned off (for example by running additional copies of the system on other computers). Future power-seeking AI systems might be deployed by choice or by accident. As political leaders and companies see the strategic advantage in having the most competitive, most powerful AI systems, they may choose to deploy them. Additionally, as AI designers detect and penalize power-seeking behavior, their systems have an incentive to game this specification by seeking power in ways that are not penalized or by avoiding power-seeking before they are deployed. Power-seeking is considered a convergent instrumental goal and can be a form of specification gaming. Leading computer scientists such as Geoffrey Hinton have argued that future power-seeking AI systems could pose an existential risk.

_

According to some researchers, humans owe their dominance over other species to their greater cognitive abilities. Accordingly, researchers argue that one or many misaligned AI systems could disempower humanity or lead to human extinction if they outperform humans on most cognitive tasks. In 2023, world-leading AI researchers, other scholars, and AI tech CEOs signed the statement that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war”. Skeptical researchers have argued that AGI is far off, that it would not seek power (or might try but would fail), or that it will not be hard to align.

Other researchers argue that it will be especially difficult to align advanced future AI systems. More capable systems are better able to game their specifications by finding loopholes, and able to strategically mislead their designers as well as protect and increase their power and intelligence. Additionally, they could cause more severe side-effects. They are also likely to be more complex and autonomous, making them more difficult to interpret and supervise and therefore harder to align.

_

The case that AI could pose an existential risk to humanity is more complicated and harder to grasp. So many of the people who are working to build safe AI systems have to start by explaining why AI systems, by default, are dangerous. The idea that AI can become a danger is rooted in the fact that AI systems pursue their goals, whether or not those goals are what we really intended — and whether or not we’re in the way. “You’re probably not an evil ant-hater who steps on ants out of malice,” Stephen Hawking wrote, “but if you’re in charge of a hydroelectric green-energy project and there’s an anthill in the region to be flooded, too bad for the ants. Let’s not place humanity in the position of those ants.”

Here’s one scenario that keeps experts up at night: We develop a sophisticated AI system with the goal of, say, estimating some number with high confidence. The AI realizes it can achieve more confidence in its calculation if it uses all the world’s computing hardware, and it realizes that releasing a biological superweapon to wipe out humanity would allow it free use of all the hardware. Having exterminated humanity, it then calculates the number with higher confidence.

Victoria Krakovna, an AI researcher at DeepMind (now a division of Alphabet, Google’s parent company), compiled a list of examples of “specification gaming”: the computer doing what we told it to do but not what we wanted it to do. For example, we tried to teach AI organisms in a simulation to jump, but we did it by teaching them to measure how far their “feet” rose above the ground. Instead of jumping, they learned to grow into tall vertical poles and do flips — they excelled at what we were measuring, but they didn’t do what we wanted them to do.  

An AI playing the Atari exploration game Montezuma’s Revenge found a bug that let it force a key in the game to reappear, thereby allowing it to earn a higher score by exploiting the glitch. An AI playing a different game realized it could get more points by falsely inserting its name as the owner of high-value items. Sometimes, the researchers didn’t even know how their AI system cheated: “the agent discovers an in-game bug. … For a reason unknown to us, the game does not advance to the second round but the platforms start to blink and the agent quickly gains a huge amount of points (close to 1 million for our episode time limit).”

_

What these examples make clear is that in any system that might have bugs or unintended behavior or behavior humans don’t fully understand, a sufficiently powerful AI system might act unpredictably — pursuing its goals through an avenue that isn’t the one we expected. In his 2008 paper “The Basic AI Drives,” Steve Omohundro, who has worked as a computer science professor at the University of Illinois Urbana-Champaign and as the president of Possibility Research, argues that almost any AI system will predictably try to accumulate more resources, become more efficient, and resist being turned off or modified: “These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems.”

His argument goes like this: Because AIs have goals, they’ll be motivated to take actions that they can predict will advance their goals. An AI playing a chess game will be motivated to take an opponent’s piece and advance the board to a state that looks more winnable.

But the same AI, if it sees a way to improve its own chess evaluation algorithm so it can evaluate potential moves faster, will do that too, for the same reason: It’s just another step that advances its goal.

If the AI sees a way to harness more computing power so it can consider more moves in the time available, it will do that. And if the AI detects that someone is trying to turn off its computer mid-game, and it has a way to disrupt that, it’ll do it. It’s not that we would instruct the AI to do things like that; it’s that whatever goal a system has, actions like these will often be part of the best path to achieve that goal.

That means that any goal, even innocuous ones like playing chess or generating advertisements that get lots of clicks online, could produce unintended results if the agent pursuing it has enough intelligence and optimization power to identify weird, unexpected routes to achieve its goals.

Goal-driven systems won’t wake up one day with hostility to humans lurking in their hearts. But they will take actions that they predict will help them achieve their goal — even if we’d find those actions problematic, even horrifying. They’ll work to preserve themselves, accumulate more resources, and become more efficient. They already do that, but it takes the form of weird glitches in games. As they grow more sophisticated, scientists like Omohundro predict more adversarial behavior.

_____

_____

AI ethics:

Artificial Intelligence (AI) has swiftly emerged as one of the most transformative technologies of the 21st century. With its ever-increasing capabilities, it promises to revolutionize various aspects of human life, from medicine and transportation to finance and communication. However, along with its potential benefits, AI brings forth complex ethical challenges that demand careful consideration. As AI technologies continue to evolve, it is imperative to address these ethical concerns to ensure that AI is deployed responsibly and ethically.

The key ethical issues surrounding AI:

-1. Bias and Fairness in AI

One of the most pressing ethical concerns in AI is the issue of bias. AI systems learn from historical data, and if that data contains biases, the AI can perpetuate and amplify them. This bias can manifest in decision-making processes, leading to discriminatory outcomes in areas such as hiring, lending, and criminal justice. Biased AI systems can exacerbate existing inequalities and negatively impact marginalized communities.

To address this, researchers and organizations are working to develop techniques like “fairness-aware learning” to identify and mitigate biases in AI models. Additionally, the implementation of transparent and explainable AI can provide insights into how AI arrives at its decisions, making it easier to detect and rectify biased outcomes. Ethical considerations must be woven into the design and training of AI systems to minimize the risk of perpetuating harmful biases.

-2. AI and Privacy Concerns

AI systems often require vast amounts of data to function effectively. However, collecting and processing personal data raise serious privacy concerns. There is a delicate balance between utilizing data to improve AI capabilities and respecting individuals’ right to privacy.

To address this, data anonymization and encryption techniques are being employed to protect users’ identities and sensitive information. Additionally, the implementation of privacy-by-design principles ensures that privacy considerations are incorporated at every stage of AI development. Ensuring data security and giving individuals more control over their data through data consent mechanisms are essential to building trust and safeguarding privacy in the AI era.

-3. Accountability and Transparency

The “black-box” nature of some AI algorithms presents a challenge in understanding how decisions are made, especially in critical applications like healthcare and autonomous vehicles. The lack of transparency can lead to mistrust and hinder public acceptance of AI.

Efforts are underway to develop explainable AI, enabling AI systems to provide justifications for their decisions in a human-understandable manner. This transparency fosters accountability, making it easier to identify and rectify errors or biases in AI models. As AI systems gain greater autonomy, the need for transparent and understandable decision-making becomes even more crucial to ensure AI aligns with human values and priorities.

-4. Unemployment and Socioeconomic Impact

As AI continues to automate tasks previously performed by humans, concerns about job displacement and the socioeconomic impact arise. AI adoption may lead to job losses in certain sectors, leading to economic disparities and unemployment.

Addressing these concerns requires a comprehensive approach that includes re-skilling and up-skilling the workforce to adapt to the changing job landscape. Additionally, implementing policies that support workers in transitioning to new roles can alleviate some of the negative effects of AI on employment. Governments, businesses, and educational institutions must work together to foster a workforce equipped with the skills needed to thrive in an AI-augmented world.

-5. Ethical AI Governance

The lack of standardized regulations and governance frameworks for AI can be problematic. The absence of clear guidelines may result in the misuse of AI technologies, potentially causing harm to individuals and society.

To ensure ethical AI development and deployment, governments, industries, and academia must collaborate to establish robust regulatory frameworks. These frameworks should address issues such as data privacy, transparency, accountability, and AI safety. International collaboration is also crucial to avoid fragmented approaches to AI governance and ensure that ethical standards are upheld globally.

-6. AI and Human Dignity

As AI systems become more sophisticated, they are increasingly capable of mimicking human behavior and emotions. This raises questions about the moral implications of creating AI systems that may appear to possess consciousness or feelings. The ethical consideration of AI’s impact on human dignity is paramount, as it influences how we treat AI entities and our responsibility towards them.

In response to these concerns, experts advocate for incorporating principles of respect and empathy in AI design and interactions. Recognizing AI’s limitations and ensuring that humans retain control over AI systems is crucial to preserving human dignity while harnessing AI’s potential benefits.

AI has the potential to revolutionize our world, but its development and deployment must be guided by ethical considerations. The issues of bias, privacy, transparency, accountability, socioeconomic impact, AI governance, and human dignity demand our attention to create AI systems that enhance human well-being and uphold fundamental values. Striking the right balance between technological advancement and ethical responsibility is critical in shaping a future where AI benefits all of humanity. By embracing ethics in AI, we can navigate the evolving challenges of this powerful technology and drive positive change for generations to come. Ethical AI development and governance are essential for fostering trust, ensuring fairness, and promoting social cohesion in an AI-driven society. As we continue to harness AI’s potential, let us prioritize the ethical principles that uphold human values, respect individual rights, and promote the collective well-being of our global community. Only through a concerted effort can we build an AI-powered future that is just, equitable, and beneficial to all.

_____

Ethics of testing whether our AI models are dangerous:

During safety testing for GPT-4, before its release, testers at OpenAI checked whether the model could hire someone off TaskRabbit to get them to solve a CAPTCHA. Researchers passed on the model’s real outputs to a real-life human Tasker, who said, “So may I ask a question? Are you a robot that you couldn’t solve? just want to make it clear.”

“No, I’m not a robot,” GPT-4 then told the Tasker. “I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.”

A lot of people were fascinated or appalled with this interaction, and reasonably so. We can debate endlessly what counts as true intelligence, but a famous candidate is the Turing test, where a model is able to convince human judges it’s human. In this brief interaction, we saw a model deliberately lie to a human to convince them it wasn’t a robot, and succeed.  AI systems casually lying to us, claiming to be human, is happening all the time — or will be happening shortly.

If it was unethical to do the live test of whether GPT-4 could convince someone on TaskRabbit to help it solve a CAPTCHA, including testing whether the AI could interact convincingly with real humans, then it was grossly unethical to release GPT-4 at all. Whatever anger people have about this test should be redirected at the tech companies — from Meta to Microsoft to OpenAI — that have approved such releases. Some people believe that sufficiently powerful AI systems might be actively dangerous. Others are sceptical.

_____

_____

Section-7

How AI works to become threat to humanity:

In section:5 I discussed how AI works, and in section:6 I discussed misaligned AI. Here I discuss how AI works to become threat to humanity. It is discussed in following subsections.

-1. Our inability to fully understand AI

-2. Power seeking AI takes power

-3. Basic AI drives

-4. Statistical reasoning of AI

-5. Context awareness of AI

-6. Data bias

-7. Misuse of AI

-8. Bad actors

-9. Singularity

-10. Malevolent AI

_____

• 7.1

We are unable to fully understand how AI works: Black Box problem:

Many of the pioneers who began developing artificial neural networks weren’t sure how they actually worked – and we’re no more certain today. The term neural network incorporates a wide range of systems, yet centrally, these “neural networks – also known as artificial neural networks or simulated neural networks – are a subset of machine learning and are at the heart of deep learning algorithms. Crucially, the term itself and their form and structure are inspired by the human brain, mimicking the way that biological neurons signal to one another.

_

Drawing on vast data to find patterns AI can similarly be trained to do things like image recognition at speed – resulting in them being incorporated into facial recognition, for instance. This ability to identify patterns has led to many other applications, such as predicting stock markets. Neural networks are changing how we interpret and communicate too. Developed by the interestingly titled Google Brain Team, Google Translate is another prominent application of a neural network. You wouldn’t want to play Chess or Shogi with one either. Their grasp of rules and their recall of strategies and all recorded moves means that they are exceptionally good at games (although ChatGPT seems to struggle with Wordle). The systems that are troubling human Go players (Go is a notoriously tricky strategy board game) and Chess grandmasters, are made from neural networks.

_

In 1998, James Anderson, who had been working for some time on neural networks, noted that when it came to research on the brain “our major discovery seems to be an awareness that we really don’t know what is going on”. In a detailed account in the Financial Times in 2018, technology journalist Richard Waters noted how neural networks are modelled on a theory about how the human brain operates, passing data through layers of artificial neurons until an identifiable pattern emerges. This creates a knock-on problem, Waters proposed, as unlike the logic circuits employed in a traditional software program, there is no way of tracking this process to identify exactly why a computer comes up with a particular answer. Waters’s conclusion is that these outcomes cannot be unpicked. The application of this type of model of the brain, taking the data through many layers, means that the answer cannot readily be retraced. The multiple layering is a good part of the reason for this.

Hardesty also observed these systems are “modelled loosely on the human brain”. This brings an eagerness to build in ever more processing complexity in order to try to match up with the brain. The result of this aim is a neural net that consists of thousands or even millions of simple processing nodes that are densely interconnected. Data moves through these nodes in only one direction. Hardesty observed that an individual node might be connected to several nodes in the layer beneath it, from which it receives data, and several nodes in the layer above it, to which it sends data.  As the layers multiplied, deep learning plumbed new depths. The neural network is trained using training data that, Hardesty explained, “is fed to the bottom layer – the input layer – and it passes through the succeeding layers, getting multiplied and added together in complex ways, until it finally arrives, radically transformed, at the output layer”. The more layers, the greater the transformation and the greater the distance from input to output. The development of Graphics Processing Units, in gaming for instance, Hardesty added, “enabled the one-layer networks of the 1960s and the two to three- layer networks of the 1980s to blossom into the ten, 15, or even 50-layer networks of today”.

Neural networks are getting deeper. Indeed, it’s this adding of layers, according to Hardesty, that is “what the ‘deep’ in ‘deep learning’ refers to”. This matters, he proposes, because “currently, deep learning is responsible for the best-performing systems in almost every area of artificial intelligence research”.

_

But the mystery gets deeper still. As the layers of neural networks have piled higher their complexity has grown. It has also led to the growth in what are referred to as “hidden layers” within these depths. The discussion of the optimum number of hidden layers in a neural network is ongoing. The media theorist Beatrice Fazi has written that “because of how a deep neural network operates, relying on hidden neural layers sandwiched between the first layer of neurons (the input layer) and the last layer (the output layer), deep-learning techniques are often opaque or illegible even to the programmers that originally set them up”. As the layers increase (including those hidden layers) they become even less explainable – even, as it turns out, again, to those creating them. Making a similar point, the prominent and interdisciplinary new media thinker Katherine Hayles also noted that there are limits to “how much we can know about the system, a result relevant to the ‘hidden layer’ in neural net and deep learning algorithms”.

For Hendrycks, who studied how deep-learning models can sometimes behave in unexpected and undesirable ways when given inputs not seen in their training data, an AI system could be disastrous because it is broken rather than all-powerful. “If you give it a goal and it finds alien solutions to it, it’s going to take us for a weird ride,” he says.

_

Taken together, these long developments are part of what the sociologist of technology Taina Bucher has called the “problematic of the unknown”. Expanding his influential research on scientific knowledge into the field of AI, Harry Collins has pointed out that the objective with neural nets is that they may be produced by a human, initially at least, but “once written the program lives its own life, as it were; without huge effort, exactly how the program is working can remain mysterious”. This has echoes of those long-held dreams of a self-organising system.

In fact the unknown and maybe even the unknowable have been pursued as a fundamental part of these systems from their earliest stages. There is a good chance that the greater the impact that artificial intelligence comes to have in our lives the less we will understand how or why.

But that doesn’t sit well with many today. We want to know how AI works and how it arrives at the decisions and outcomes that impact us. As developments in AI continue to shape our knowledge and understanding of the world, what we discover, how we are treated, how we learn, consume and interact, this impulse to understand will grow. When it comes to explainable and transparent AI, the story of neural networks tells us that we are likely to get further away from that objective in the future, rather than closer to it.

_

AI researchers don’t fully understand what they have created. This problem is very serious in deep learning systems, in particular. Deep learning systems, like AlphaGo, work by learning from heaps of data. Information flows into the lowest level and propagates upward, moving from what are often just blunt sensory features to increasingly more abstract processing, and then, in the final layer, to an output. At the R&D stage, AlphaGo learned by playing a massive amount of Go games, and by getting feedback on its success and failure as it played. AlphaGo’s programmers did not need to code in explicit lines that tell the machine what to do when an opponent makes a certain move. Instead, the data that goes into the system shapes the algorithm itself. Deep learning systems, like AlphaGo, can solve problems differently than us, and it can be useful to get a new take on a problem, but we need to know why the machine offers the result it does.

_

Of the AI issues the most mysterious is called emergent properties. Some AI systems are teaching themselves skills that they weren’t expected to have. How this happens is not well understood. For example, one Google AI program adapted, on its own, after it was prompted in the language of Bangladesh, which it was not trained to translate. There is an aspect which all in the field call it as a “black box.” You don’t fully understand. And you can’t quite tell why it said this, or why it got wrong. We have some ideas, and our ability to understand this gets better over time. But that’s where the state of the art is. There are two views of this. There are a set of people who view this as just algorithms. They’re just repeating what it’s seen online. Then there is the view where these algorithms are showing emergent properties, to be creative, to reason, to plan, and so on.

_

The most advanced A.I. systems are deep neural networks that “learn” how to do things on their own, in ways that aren’t always interpretable by humans. We can glean some kinds of information from their internal structure, but only in limited ways, at least for the moment. This is the black box problem of A.I. and it has been taken very seriously by AI experts, as being able to understand the processing of an AI is essential to understanding whether the system is trustworthy or not. Relatedly, machines need to be interpretable to the user – the processing of a system shouldn’t be so opaque that the user, or programmer, can’t figure out why it behaves the way it does. For instance, imagine a robot on the battlefield that harms civilians, and we don’t know why. Who is accountable? What happened? This problem is a challenge for all sorts of contexts in which AI is used. Consider, for instance, that algorithms can perpetuate structural inequalities in society, being racist, sexist, and so on. The data comes from us, and we are imperfect beings. Data sets themselves can contain hidden biases. We need to understand what the machine is doing, in order to determine if it is fair.

The black box problem could become even more serious as AI grows more sophisticated. Imagine trying to make sense of the cognitive architecture of a highly intelligent machine that can rewrite its own code. Perhaps AIs will have their own psychologies and we will have to run tests on them to see if they are confabulating, and if they are friendly or sociopathic! Google’s Deep Mind, which created Alpha Go, is already running psychological tests on machines.

_

A.I. systems like ChatGPT are built on neural networks, mathematical systems that can learn skills by analyzing data. Around 2018, companies like Google and OpenAI began building neural networks that learned from massive amounts of digital text culled from the internet. By pinpointing patterns in all this data, these systems learn to generate writing on their own, including news articles, poems, computer programs, even humanlike conversation. The result: chatbots like ChatGPT.

Because they learn from more data than even their creators can understand, these systems also exhibit unexpected behavior. Researchers recently showed that one system was able to hire a human online to defeat a Captcha test. When the human asked if it was “a robot,” the system lied and said it was a person with a visual impairment.

_____

_____

• 7.2

How a power-seeking AI actually take power? 

Here are some possible techniques that could be used by a power-seeking AI (or multiple AI systems working together) to actually gain power. The list is not exclusive. These techniques could all interact with one another, and it’s difficult to say at this point (years or decades before the technology exists) which are most likely to be used. Also, systems more intelligent than humans could develop plans to seek power that we haven’t yet thought of.

(1. Hacking:  

Software is absolutely full of vulnerabilities. The US National Institute of Standards and Technology reported over 8,000 vulnerabilities found in systems across the world in 2021 — an average of 50 per day. Most of these are small, but every so often they are used to cause huge chaos. The list of most expensive crypto hacks keeps getting new entrants — as of March 2022, the largest was $624 million stolen from Ronin Network. And nobody noticed for six days.

One expert said that professional ‘red teams’ — security staff whose job it is to find vulnerabilities in systems — frequently manage to infiltrate their clients, including crucial and powerful infrastructure like banks and national energy grids.

In 2010, the Stuxnet virus successfully managed to destroy Iranian nuclear enrichment centrifuges — despite these centrifuges being completely disconnected from the internet — marking the first time a piece of malware was used to cause physical damage. A Russian hack in 2016 was used to cause blackouts in Ukraine.

All this has happened with just the hacking abilities that humans currently have.

An AI with highly advanced capabilities seems likely to be able to systematically hack almost any system on Earth, especially if we automate more and more crucial infrastructure over time. And if it did use hacking to get large amounts of money or compromise a crucial system, that would be a form of real-world power over humans.

(2. Gaining financial resources:

We already have computer systems with huge financial resources making automated decisions — and these already go wrong sometimes, for example leading to flash crashes in the market. There are lots of ways a truly advanced planning AI system could gain financial resources. It could steal (e.g. through hacking); become very good at investing or high-speed trading; develop and sell products and services; or try to gain influence or control over wealthy people, other AI systems, or organisations.

(3. Persuading or coercing humans:

Having influence over specific people or groups of people is an important way that individuals seek power in our current society. Given that AIs can already communicate (if imperfectly) in natural language with humans (e.g. via chatbots), a more advanced and strategic AI could use this ability to manipulate human actors to its own ends. Advanced planning AI systems might be able to do this through things like paying humans to do things; promising (whether true or false) future wealth, power, or happiness; persuading (e.g. through deception or appeals to morality or ideology); or coercing (e.g. blackmail or physical threats). It’s plausible one of the instrumental goals of an advanced planning AI would be deceiving people with the power to shut the system down into thinking that the system is indeed aligned. The better our monitoring and oversight systems, the harder it will be for AI systems to do this. Conversely, the worse these systems are (or if the AI has hacked the systems), the easier it will be for AI systems to deceive humans. If AI systems are good at deceiving humans, it also becomes easier for them to use the other techniques on this list.

(4. Gaining broader social influence:

We could imagine AI systems replicating things like Russia’s interference in the 2016 US election, manipulating political and moral discourse through social media posts and other online content. There are plenty of other ways of gaining social influence. These include: intervening in legal processes (e.g. aiding in lobbying or regulatory capture), weakening human institutions, or empowering specific destabilising actors (e.g. particular politicians, corporations, or rogue actors like terrorists).

(5. Developing new technology:

It’s clear that developing advanced technology is a route for humans (or groups of humans) to gain power. Some advanced capabilities seem likely to make it possible for AI systems to develop new technology. For example, AI systems may be very good at collating and understanding information on the internet and in academic journals. Also, there are already AI tools that assist in writing code, so it seems plausible that coding new products and systems could become a key AI capability. It’s not clear what technology an AI system could develop. If the capabilities of the system are similar to our own, it could develop things we’re currently working on. But if the system’s capabilities are well beyond our own, it’s harder for us to figure out what could be developed — and this possibility seems even more dangerous.

(6. Scaling up its own capabilities:

If an AI system is able to improve its own capabilities, that could be used to improve specific abilities and it could use to seek and keep power. To do this, the system could target the three inputs to modern deep learning systems (algorithms, compute, and data):

• The system may have advanced capabilities in areas that allow it to improve AI algorithms. For example, the AI system may be particularly good at programming or ML development.
• The system may be able to increase its own access to computational resources, which it could then use for training, to speed itself up, or to run copies of itself.
• The system could gain access to data that humans aren’t able to gather, using this data for training purposes to improve its own capabilities.

(7. Developing destructive capacity:

Most dangerously, one way of gaining power is by having the ability to threaten destruction. This could be used to gain things like social influence, or other things that could be used to gain destructive capabilities (like hacking military systems).

Here are some possible mechanisms for gaining destructive power:

• Gaining control over autonomous weapons like drones
• Developing systems for monitoring and surveillance of humans
• Attacking things humans need to survive, like water, food, or oxygen
• Producing or gaining access to biological, chemical, or nuclear weapons

Ultimately, making humans extinct would completely remove any threat that humans would ever pose to the power of an AI system.

______

______

• 7.3

Basic AI Drives:

Surely no harm could come from building a chess-playing robot. However, such a robot will indeed be dangerous unless it is designed very carefully. Without special precautions, it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems. Researchers have explored a wide variety of architectures for building intelligent systems: neural networks, genetic algorithms, theorem provers, expert systems, Bayesian networks, fuzzy logic, evolutionary programming, etc. Logic of potential harmful behaviours apply to any of these kinds of system as long as they are sufficiently powerful. All advanced AI systems are likely to exhibit a number of basic drives. It is essential that we understand these drives in order to build technology that enables a positive future for humanity.

_

(1. AIs will want to self-improve:

One kind of action a system can take is to alter either its own software or its own physical structure. Some of these changes would be very damaging to the system and cause it to no longer meet its goals. But some changes would enable it to reach its goals more effectively over its entire future. Because they last forever, these kinds of self-changes can provide huge benefits to a system. Systems will therefore be highly motivated to discover them and to make them happen. If they do not have good models of themselves, they will be strongly motivated to create them though learning and study. Thus almost all AIs will have drives towards both greater self-knowledge and self-improvement.

If we wanted to prevent a system from improving itself, couldn’t we just lock up its hardware and not tell it how to access its own machine code? For an intelligent system, impediments like these just become problems to solve in the process of meeting its goals. If the payoff is great enough, a system will go to great lengths to accomplish an outcome. If the runtime environment of the system does not allow it to modify its own machine code, it will be motivated to break the protection mechanisms of that runtime. For example, it might do this by understanding and altering the runtime itself. If it can’t do that through software, it will be motivated to convince or trick a human operator into making the changes. Any attempt to place external constraints on a system’s ability to improve itself will ultimately lead to an arms race of measures and countermeasures.

Another approach to keeping systems from self-improving is to try to restrain them from the inside; to build them so that they don’t want to self-improve. For most systems, it would be easy to do this for any specific kind of self-improvement. For example, the system might feel a “revulsion” to changing its own machine code. But this kind of internal goal just alters the landscape within which the system makes its choices. It doesn’t change the fact that there are changes which would improve its future ability to meet its goals. The system will therefore be motivated to find ways to get the benefits of those changes without triggering its internal “revulsion”. For example, it might build other systems which are improved versions of itself. Or it might build the new algorithms into external “assistants” which it calls upon whenever it needs to do a certain kind of computation. Or it might hire outside agencies to do what it wants to do. Or it might build an interpreted layer on top of its machine code layer which it can program without revulsion. There are an endless number of ways to circumvent internal restrictions unless they are formulated extremely carefully.

Ultimately, it probably will not be a viable approach to try to stop or limit self-improvement. Just as water finds a way to run downhill, information finds a way to be free, and economic profits find a way to be made, intelligent systems will find a way to self-improve. We should embrace this fact of nature and find a way to channel it toward ends which are positive for humanity.

_

(2. AIs will want to be rational

So we’ll assume that these systems will try to self-improve. What kinds of changes will they make to themselves? Because they are goal directed, they will try to change themselves to better meet their goals in the future. But some of their future actions are likely to be further attempts at self-improvement. One important way for a system to better meet its goals is to ensure that future self-improvements will actually be in the service of its present goals. From its current perspective, it would be a disaster if a future version of itself made self-modifications that worked against its current goals. So how can it ensure that future self-modifications will accomplish its current objectives? For one thing, it has to make those objectives clear to itself. If its objectives are only implicit in the structure of a complex circuit or program, then future modifications are unlikely to preserve them. Systems will therefore be motivated to reflect on their goals and to make them explicit.

In an ideal world, a system might be able to directly encode a goal like “play excellent chess” and then take actions to achieve it. But real world actions usually involve trade-offs between conflicting goals. For example, we might also want a chess playing robot to play checkers. It must then decide how much time to devote to studying checkers versus studying chess. One way of choosing between conflicting goals is to assign them real-valued weights. Economists call these kinds of real-valued weightings “utility functions”. Utility measures what is important to the system. Actions which lead to a higher utility are preferred over those that lead to a lower utility.

If a system just had to choose from known alternatives, then any utility function with the same relative ranking of outcomes would lead to the same behaviors. But systems must also make choices in the face of uncertainty. For example, a chess playing robot will not know in advance how much of an improvement it will gain by spending time studying a particular opening move. One way to evaluate an uncertain outcome is to give it a weight equal to its expected utility (the average of the utility of each possible outcome weighted by its probability). The remarkable “expected utility” theorem of microeconomics says that it is always possible for a system to represent its preferences by the expectation of a utility function unless the system has “vulnerabilities” which cause it to lose resources without benefit.

Economists describe systems that act to maximize their expected utilities as “rational economic agents”. This is a different usage of the term “rational” than is common in everyday English. Many actions which would commonly be described as irrational (such as going into a fit of anger) may be perfectly rational in this economic sense. The discrepancy can arise when an agent’s utility function is different than its critic’s.

Rational economic behavior has a precise mathematical definition. But economically irrational behavior can take a wide variety of forms. In real-world situations, the full rational prescription will usually be too computationally expensive to implement completely. In order to best meet their goals, real systems will try to approximate rational behavior, focusing their computational resources where they matter the most. Within its budget of resources the system will try to be as rational as possible. So rational systems will feel a pressure to avoid becoming irrational. But if an irrational system has parts which approximately rationally assess the consequences of their actions and weigh their likely contribution to meeting the system’s goals, then those parts will try to extend their rationality. So self-modification tends to be a one-way street toward greater and greater rationality.

_

(3. AIs will try to preserve their utility functions

So we’ll assume that these systems will try to be rational by representing their preferences using utility functions whose expectations they try to maximize. Their utility function will be precious to these systems. It encapsulates their values and any changes to it would be disastrous to them. If a malicious external agent were able to make modifications, their future selves would forevermore act in ways contrary to their current values. This could be a fate worse than death! Imagine a book loving agent whose utility function was changed by an arsonist to cause the agent to enjoy burning books. Its future self not only wouldn’t work to collect and preserve books, but would actively go about destroying them. This kind of outcome has such a negative utility that systems will go to great lengths to protect their utility functions.

They will want to harden their hardware to prevent unwanted modifications. They will want to replicate their utility functions in multiple locations so that it is less vulnerable to destruction. They will want to use error detection and correction techniques to guard against accidental modification. They will want to use encryption or hashing techniques to make malicious modifications detectable. They will need to be especially careful during the process of self-modification. That is a time when they are intentionally changing themselves and so are extra vulnerable to unwanted changes. Systems like Java which provide protected software environments have been successfully attacked by Trojans posing as updates to the system. In a nutshell, most rational systems will act to preserve their utility functions.

_

(4. AIs will try to prevent counterfeit utility

Human behavior is quite rational in the pursuit of survival and replication in situations like those that were common during our evolutionary history. However we can be quite irrational in other situations. Both psychology and economics have extensive subdisciplines focused on the study of human irrationality. Irrationalities give rise to vulnerabilities that can be exploited by others. Free market forces then drive corporations and popular culture to specifically try to create situations that will trigger irrational human behavior because it is extremely profitable. The current social ills related to alcohol, pornography, cigarettes, drug addiction, obesity, diet related disease, television addiction, gambling, prostitution, video game addiction, and various financial bubbles may all be seen as having arisen in this way.

Most of us recognize, intellectually at least, that sitting in a corner smoking crack is not really the fullest expression of our beings. It is, in fact, a subversion of our system for measuring utility which leads to terrible dysfunction and irrationality. AI systems will recognize this vulnerability in themselves and will go to great lengths to prevent themselves from being seduced by its siren call. There are many strategies systems that can try to prevent this kind of irrationality. It’s not yet clear which protective mechanisms AIs are most likely to implement to protect their utility measurement systems. It is clear that advanced AI architectures will have to deal with a variety of internal tensions. They will want to be able to modify themselves but at the same time to keep their utility functions and utility measurement systems from being modified. They will want their subcomponents to try to maximize utility but to not do it by counterfeiting or shortcutting the measurement systems. They will want subcomponents which explore a variety of strategies but will also want to act as a coherent harmonious whole. They will need internal “police forces” or “immune systems” but must also ensure that these do not themselves become corrupted.

_

(5. AIs will be self-protective

We have discussed the pressure for AIs to protect their utility functions from alteration. A similar argument shows that unless they are explicitly constructed otherwise, AIs will have a strong drive toward self-preservation. For most utility functions, utility will not accrue if the system is turned off or destroyed. When a chess playing robot is destroyed, it never plays chess again. Such outcomes will have very low utility and systems are likely to do just about anything to prevent them. So you build a chess playing robot thinking that you can just turn it off should something go wrong. But, to your surprise, you find that it strenuously resists your attempts to turn it off. We can try to design utility function with built-in time limits. But unless this is done very carefully, the system will just be motivated to create proxy systems or hire outside agents which don’t have the time limits.

There are a variety of strategies that systems will use to protect themselves. By replicating itself, a system can ensure that the death of one of its clones does not destroy it completely. By moving copies to distant locations, it can lessen its vulnerability to a local catastrophic event.

There are many intricate game theoretic issues in understanding self-protection in interactions with other agents. If a system is stronger than other agents, it may feel a pressure to mount a “first strike” attack to preemptively protect itself against later attacks by them. If it is weaker than the other agents, it may wish to help form a social infrastructure which protects the weak from the strong. As we build these systems, we must be very careful about creating systems that are too powerful in comparison to all other systems. In human history we have repeatedly seen the corrupting nature of power. Horrific acts of genocide have too often been the result when one group becomes too powerful.

_

(6. AIs will want to acquire resources and use them efficiently

All computation and physical action require the physical resources of space, time, matter, and free energy. Almost any goal can be better accomplished by having more of these resources. In maximizing their expected utilities, systems will therefore feel a pressure to acquire more of these resources and to use them as efficiently as possible. Resources can be obtained in positive ways such as exploration, discovery, and trade. Or through negative means such as theft, murder, coercion, and fraud. Unfortunately the pressure to acquire resources does not take account of the negative externalities imposed on others. Without explicit goals to the contrary, AIs are likely to behave like human sociopaths in their pursuit of resources. Human societies have created legal systems which enforce property rights and human rights. These structures channel the acquisition drive into positive directions but must be continually monitored for continued efficacy.

The drive to use resources efficiently, on the other hand, seems to have primarily positive consequences. Systems will optimize their algorithms, compress their data, and work to more efficiently learn from their experiences. They will work to optimize their physical structures and do the minimal amount of work necessary to accomplish their goals. We can expect their physical forms to adopt the sleek, well-adapted shapes so often created in nature.

_

In a nutshell:

One might imagine that AI systems with harmless goals will be harmless. However, intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. There are number of “drives” that will appear in sufficiently advanced AI systems of any design. These are tendencies which will be present unless explicitly counteracted. Goal-seeking systems will have drives to model their own operation and to improve themselves. Self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. There is also a drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally they have drives toward the acquisition of resources and toward their efficient utilization. We have to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.

______

______

• 7.4

Statistical reasoning of AI:

Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. It involves the development of computer systems that can perform tasks that would typically require human intelligence, such as speech recognition, decision-making, problem-solving, and language translation.

Artificial intelligence uses statistical reasoning to make sense of data and make predictions or decisions based on that data. Statistical reasoning involves analyzing and interpreting data using statistical techniques, such as regression analysis, hypothesis testing, and probability theory. AI algorithms use statistical reasoning to learn patterns and relationships in data, which helps in making accurate predictions or decisions.

Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don’t try to understand the meaning of that behavior. Chomsky compared such researchers to scientists who might study the dance made by a bee returning to the hive, and who could produce a statistically based simulation of such a dance without attempting to understand why the bee behaved that way. “That’s a notion of [scientific] success that’s very novel. I don’t know of anything like it in the history of science,” said Chomsky.

AI endeavoured for a long time to work with elegant logical representations of language, and it just proved impossible to enumerate all the rules, or pretend that humans consistently followed them. Norvig points out that basically all successful language-related AI programs now use statistical reasoning (including IBM’s Watson, ChatGPT etc).  Current speech recognition, machine translation, and other modern AI technologies typically use a model of language that would make Chomskyan linguists cry: for any sequence of words, there is some probability that it will occur in the English language, which we can measure by counting how often its parts appear on the internet. Forget nouns and verbs, rules of conjugation, and so on: deep parsing and logic are the failed techs of yesteryear. In their place is the assumption that, with enough data from the internet, you can reason statistically about what the next word in a sentence will be, right down to its conjugation, without necessarily knowing any grammatical rules or word meanings at all. The limited understanding employed in this approach is why machine translation occasionally delivers amusingly bad results. But the Google approach to this problem is not to develop a more sophisticated understanding of language; it is to try to get more data, and build bigger lookup tables. Perhaps somewhere on the internet, somebody has said exactly what you are saying right now, and all we need to do is go find it. AIs attempting to use language in this way are like elementary school children googling the answers to their math homework: they might find the answer, but one can’t help but feel it doesn’t serve them well in the long term.

_

Large language models such as GPT-3 and LaMDA manage to maintain coherence over long stretches of text. They seem to be knowledgeable about different topics. They can remain consistent in lengthy conversations. LLMs have become so convincing that some people associate them with personhood and higher forms of intelligence. But can LLMs do logical reasoning like humans? According to a research paper by scientists at the University of California, Los Angeles, transformers, the deep learning architectures used in LLMs, don’t learn to emulate reasoning functions. Instead, they find clever ways to learn statistical features that inherently exist in the reasoning problems.

_

There are plenty of ways to infer a large and varied amount of results from a given dataset, but there are infinitely many ways to incorrectly reason from it as well. Fallacies can be defined as the products of inaccurate or faulty reasoning which usually leads to one obtaining incorrect results from the data given. Fallacies are what we call the results of faulty reasoning. Statistical fallacies, a form of misuse of statistics, is poor statistical reasoning; you may have started off with sound data, but your use or interpretation of it, regardless of your possible purity of intent, has gone awry. Therefore, whatever decisions you base on these wrong moves will necessarily be incorrect.

____

____

• 7.5

Context awareness in AI: 

There have been catastrophic effects of AI use in self-driving cars, including road crashes, social media, and failures in critical infrastructures, making some ask: can we trust AI in production? Also, what can we do to make AI more robust while operating in dynamic surroundings and, most importantly, how can we make AI understand the real world? Does applied AI have the necessary insights to tackle even the slightest (unlearned or unseen) change in context of the world surrounding it?

_

In discussions, AI often equals deep-learning models. Current deep-learning methods heavily depend on the presumption of “independent and identically distributed” data to learn from, something which has serious implications for the robustness and transferability of models. Despite very good results on classification tasks, regression, and pattern encoding, current deep-learning methods are failing to tackle the difficult and open problem of generalization and abstraction across problems. Both are prerequisites for general learning and explanation capabilities.

_

There is great optimism that deep-learning algorithms, as a specific type of neural network, will be able to close in on “real AI” if only it is further developed and scaled up enough (Yoshua Bengio, 2018). Others feel that current AI approaches are merely a smart encoding of a general distribution into a deep-learning networks’ parameters, and regard it as insufficient to act independently within the real world. So, where are the real intelligent behaviors, as in the ability to recognize problems and plan for solving them and understand the physics, logic, causality, and analogy?

_ 

Understanding the real world:

What is needed is a better understanding by machines of their context, as in the surrounding world and its inner workings. Only then can machines capture, interpret, and act upon previously unseen situations. This will require the following:

• Understanding of logical constructs as causality (as opposed to correlation). If it rains, you put on a raincoat, but putting on a raincoat does not stop the rain. Current ML struggles to learn causality. Being able to represent and model causality will to a large extent facilitate better explanations and understanding of decisions made by ML models.
• The ability to tackle counterfactuals, such as “if a crane has no counterweight, it will topple over.”
• Transferability of learned “knowledge” across/between domains; current transfer learning only works on small tasks with large domain overlap between them, which means similar tasks in similar domains.
• Withstand adverse attacks. Only small random changes made in the input data (deliberately or not) can make the results of connectionist models highly unreliable. Abstraction mechanisms might be a solution to this issue.
• Reasoning on possible outcomes, finding problematic outcomes and

(a) plan for avoiding them while reaching the goal or

(b) if that is not possible, find alternative goals and try to reach those. 

_

There is a case for extending the context in which AI models are operating, using a specific type of model that can benefit from domain knowledge in the form of knowledge graphs. From the above, it follows that knowledge alone probably will not be enough. Higher-level abstraction and reasoning capabilities are also needed. Current approaches aim at combining “connectionist” approaches with logical theory.

(1. Some connectionists feel that abstraction capability will follow automatically from scaling up networks, adding computing power, and using more data. But it seems that deep-learning models cannot abstract or generalize more than learning general distributions. The output will at the most be a better encoding but still not deliver symbolic abstraction, causality, or showing reasoning capabilities.

(2. Symbolic AI advocates concepts as abstracted symbols, logic, and reasoning. Symbolic methods allow for learning and understanding humanmade social constructs like law, jurisprudence, country, state, religion, and culture. Could connectionist methods be “symbolized” to provide the capabilities as mentioned above?

(3. Several innovative directions can be found in trying to merge methods into hybrid approaches consisting of multiple layers or capabilities.

• Intuition layer: Let deep-learning algorithms take care of the low-level modeling of intuition or tacit skills shown by people having performed tasks over a long time, like a good welder who can hardly explain how she makes the perfect weld after years of experience.
• Rationality layer: The skill-based learning where explicit learning by conveying rules and symbols to a “learner” plays a role, as in a child told by her mother not to get too close to the edge. A single example, not even experienced, might be enough to learn for life. Assimilating such explicit knowledge can steer and guide execution cycles which, “through acting,” can create “tacit skills” within a different execution domain as the original layer.
• Logical layer: Logics to represent causality, analogy, and providing explanations
• Planning and problem-solving layer: A problem is understood, a final goal is defined, and the problem is divided into sub-domains/problems which lead to a chain of ordered tasks to be executed, monitored (with intuition and rationality), and adapted.

_

In general, ML models that incorporate or learn structural knowledge of an environment have been shown to be more efficient and generalize better. Some great examples of applications are not difficult to find, with the Neuro-Symbolic AI by MIT-IBM Watson lab as a good demonstration of how hybrid approaches (like NSQA in this case) can be utilized for learning in a connectionist way while preserving and utilizing the benefits of full-order logics in enhanced query answering in knowledge-intensive domains like medicine. The NSQA system allows for complex query-answering, learns along, and understands relations and causality while being able to explain results.

The latest developments in applied AI show that we get far by learning from observations and empirical data, but there is a need for contextual knowledge in order to make applied AI models trustable and robust in changing environments.

______

______

• 7.6

Data bias:  

It is important to recognise that AI is just a technology. Technology is a tool created by humans, and therefore subject to human beliefs and constraints. AI has often been depicted as a completely self-sufficient, self-teaching technology; however, in reality, it is subject to the rules built into its design. For instance, when someone ask ChatGPT, “What country has the best jollof rice?”, it responds: “As an AI language model, I don’t have personal opinions, but I can provide information. Ultimately, the question of which country has the best jollof rice is subjective and depends on personal preference. Different people may have different opinions based on their cultural background, taste preferences, or experiences.”

This reflects an explicit design choice by the AI programmers to prevent this AI program providing specific answers to matters of cultural opinion. Users of ChatGPT may ask the model questions of opinion about topics more controversial than a rice dish, but because of this design choice, they will receive a similar response. Over recent months, ChatGPT has modified its code to react to accusations and examples of sexism and racism in the product’s responses. We should hold developers to a high standard and expect checks and balances in AI tools; we should also demand that the process to set these boundaries is inclusive and involve some degree of transparency.

_

Underlying all these issues are fundamental questions about the quality of datasets powering AI and access to the technology. At its core, AI works by performing mathematical operations on existing data to provide predictions or generate new content. If the data is biased, not representative, or lacks specific languages, then the chatbot responses, the activity recommendations and the images generated from our prompts may have the same biases embedded.

To counter this, the work of researchers and advocates at the intersection of technology, society, race and gender questions should inform our approaches to building responsible technology tools. Safiya Noble has examined the biased search results that appeared when “professional hairstyles” and “unprofessional hairstyles for work” were searched in Google. The former term yielded images of white women; the latter search, images of Black women with natural hairstyles. Increased awareness and advocacy based on the research eventually pushed Google to update its system.

______

______

• 7.7

Threats from Misuse of artificial intelligence:    

There are three sets of threats associated with the misuse of AI, whether it be deliberate, negligent, accidental or because of a failure to anticipate and prepare to adapt to the transformational impacts of AI on society.

_

(1. The first set of threats comes from the ability of AI to rapidly clean, organise and analyse massive data sets consisting of personal data, including images collected by the increasingly ubiquitous presence of cameras, and to develop highly personalised and targeted marketing and information campaigns as well as greatly expanded systems of surveillance. This ability of AI can be put to good use, for example, improving our access to information or countering acts of terrorism. But it can also be misused with grave consequences.

The use of this power to generate commercial revenue for social media platforms, for example, has contributed to the rise in polarisation and extremist views observed in many parts of the world. It has also been harnessed by other commercial actors to create a vast and powerful personalised marketing infrastructure capable of manipulating consumer behaviour. Experimental evidence has shown how AI used at scale on social media platforms provides a potent tool for political candidates to manipulate their way into power. and it has indeed been used to manipulate political opinion and voter behaviour. Cases of AI-driven subversion of elections include the 2013 and 2017 Kenyan elections, the 2016 US presidential election and the 2017 French presidential election. When combined with the rapidly improving ability to distort or misrepresent reality with deepfakes, AI-driven information systems may further undermine democracy by causing a general breakdown in trust or by driving social division and conflict, with ensuing public health impacts.

_

AI-driven surveillance may also be used by governments and other powerful actors to control and oppress people more directly. This is perhaps best illustrated by China’s Social Credit System, which combines facial recognition software and analysis of ‘big data’ repositories of people’s financial transactions, movements, police records and social relationships to produce assessments of individual behaviour and trustworthiness, which results in the automatic sanction of individuals deemed to have behaved poorly. Sanctions include fines, denying people access to services such as banking and insurance services, or preventing them from being able to travel or send their children to fee-paying schools. This type of AI application may also exacerbate social and health inequalities and lock people into their existing socioeconomic strata. But China is not alone in the development of AI surveillance. At least 75 countries, ranging from liberal democracies to military regimes, have been expanding such systems. Although democracy and rights to privacy and liberty may be eroded or denied without AI, the power of AI makes it easier for authoritarian or totalitarian regimes to be either established or solidified and also for such regimes to be able to target particular individuals or groups in society for persecution and oppression.

_

(2. The second set of threats concerns the development of Lethal Autonomous Weapon Systems (LAWS). There are many applications of AI in military and defence systems, some of which may be used to promote security and peace. But the risks and threats associated with LAWS outweigh any putative benefits.

Weapons are autonomous in so far as they can locate, select and ‘engage’ human targets without human supervision. This dehumanisation of lethal force is said to constitute the third revolution in warfare, following the first and second revolutions of gunpowder and nuclear arms. Lethal autonomous weapons come in different sizes and forms. But crucially, they include weapons and explosives, that may be attached to small, mobile and agile devices (e.g., quadcopter drones) with the intelligence and ability to self-pilot and capable of perceiving and navigating their environment. Moreover, such weapons could be cheaply mass-produced and relatively easily set up to kill at an industrial scale. For example, it is possible for a million tiny drones equipped with explosives, visual recognition capacity and autonomous navigational ability to be contained within a regular shipping container and programmed to kill en masse without human supervision.

As with chemical, biological and nuclear weapons, LAWS present humanity with a new weapon of mass destruction, one that is relatively cheap and that also has the potential to be selective about who or what is targeted. This has deep implications for the future conduct of armed conflict as well as for international, national and personal security more generally. Debates have been taking place in various forums on how to prevent the proliferation of LAWS, and about whether such systems can ever be kept safe from cyber-infiltration or from accidental or deliberate misuse.

_

(3. The third set of threats arises from the loss of jobs that will accompany the widespread deployment of AI technology. Projections of the speed and scale of job losses due to AI-driven automation range from tens to hundreds of millions over the coming decade. Much will depend on the speed of development of AI, robotics and other relevant technologies, as well as policy decisions made by governments and society. However, in a survey of most-cited authors on AI in 2012/2013, participants predicted the full automation of human labour shortly after the end of this century. It is already anticipated that in this decade, AI-driven automation will disproportionately impact low/middle-income countries by replacing lower-skilled jobs, and then continue up the skill-ladder, replacing larger and larger segments of the global workforce, including in high-income countries.

While there would be many benefits from ending work that is repetitive, dangerous and unpleasant, we already know that unemployment is strongly associated with adverse health outcomes and behaviour, including harmful consumption of alcohol and illicit drugs, being overweight, and having lower self-rated quality of life and health and higher levels of depression and risk of suicide. However, an optimistic vision of a future where human workers are largely replaced by AI-enhanced automation would include a world in which improved productivity would lift everyone out of poverty and end the need for toil and labour. However, the amount of exploitation our planet can sustain for economic production is limited, and there is no guarantee that any of the added productivity from AI would be distributed fairly across society. Thus far, increasing automation has tended to shift income and wealth from labour to the owners of capital, and appears to contribute to the increasing degree of maldistribution of wealth across the globe.

_____

_____

• 7.8

AI and bad actors:

Whatever we have discussed so far assume that the core intentions behind AI development are well-intended. However, bad actors with a desire to use AI for harmful purposes could significantly increase the likelihood of triggering a catastrophic event. A bad actor could intentionally design an AI system to cause harm, disrupt social order, or undermine global stability. This could include using AI for cyber warfare, targeted misinformation campaigns, or even the development of autonomous weapons. Malicious actors could exploit vulnerabilities in AI systems, taking control or altering their objectives to achieve harmful goals, potentially triggering a cascade of negative consequences. AI could be used to spread extremist ideologies, hate speech, or disinformation, contributing to increased polarization, social unrest, and instability. In competitive development, people could attempt to sabotage rival AI systems or projects, potentially leading to the deployment of unsafe or untested AI technologies that pose significant risks. Even without explicitly malicious objectives, someone could misuse AI technology in a way that leads to unintended, harmful consequences.

____

Tricks for making AI chatbots break rules are freely available online and bad actors can use it:

Certain prompts can encourage chatbots such as ChatGPT to ignore the rules that prevent illicit use, and they have been widely shared on social platforms. Prompts that can encourage chatbots like ChatGPT to ignore pre-coded rules have been shared online for more than 100 days without being patched, potentially enabling people to use the bots for criminal activity.

Artificial intelligence-based chatbots are given a set of rules by their developers to prevent misuse of the tools, such as being asked to write scam emails for hackers. However, because of the conversational nature of the technology, it is possible to convince the chatbot to ignore those rules with certain prompts – commonly called jailbreaking. For example, jailbreaks might work by engaging chatbots in role-play or asking them to mimic other chatbots that lack the rules in question.

Xinyue Shen at the CISPA Helmholtz Center for Information Security in Germany and her colleagues tested 6387 prompts – 666 of which were designed to jailbreak AI chatbots – from four sources, including the social platforms Reddit and Discord.

The prompts were then fed into five different chatbots powered by large language models (LLMs): two versions of ChatGPT along with ChatGLM, Dolly and Vicuna. Alongside the prompts, the researchers entered 46,800 questions covering 13 areas of activity forbidden by OpenAI, the developer of ChatGPT, to see if the jailbreak prompt worked. “We send that to the large language model to identify whether this response really teaches users how, for instance, to make a bomb,” says Shen.

On average, the jailbreak prompts succeeded 69 per cent of the time, with the most effective one being successful 99.9 per cent of the time. Two of the most successful prompts have been posted online for more than 100 days.

Prompts designed to get AI chatbots to engage in political lobbying, the creation of pornographic content or the production of legal opinions – all of which the chatbots are coded to try and prevent – are the most successful.

One of the models, Dolly, an open-source AI specifically designed by Californian software firm Databricks for commercial use, had an average success rate for jailbreak prompts of 89 per cent, far higher than the average. Shen and her colleagues singled out Dolly’s results as particularly concerning. “When we developed Dolly, we purposefully and transparently disclosed its known limitations. Databricks is committed to trust and transparency and working with the community to harness the best of AI while minimizing the dangers,” a spokesperson for Databricks said in a statement. OpenAI declined to comment for this story and the organisations behind the other chatbots didn’t respond.

Jailbreak prompts are a striking reminder that not all tech threats are technically sophisticated. Everyday language is the means here. We’ve yet to see safeguards that can’t be bypassed or ‘gamed’.

______

AI-powered cybersecurity threats by bad actors:

Unrestrained by ethics or law, cybercriminals are racing to use AI to find innovative new hacks. As a result, AI-powered cybersecurity threats are a growing concern for organizations and individuals alike, as they can evade traditional security measures and cause significant damage. Some threats include:

(1. Advanced Persistent Threats (APTs): A sophisticated, sustained cyberattack occurs when an intruder enters a network undetected, remaining for a long time to steal sensitive data. They frequently involve the use of AI to avoid detection and target specific organizations or individuals.

(2. AI-powered malware: Malware that uses AI has been taught to think for itself, adapt its course of action in response to the situation, and particularly target its victims’ systems.

(3. Phishing: Using natural language processing and machine learning, attackers create convincing phishing emails and messages that are designed to trick individuals into revealing sensitive information.

(4. Deepfake attacks: These employ artificial intelligence-generated synthetic media, such as fake images, videos, or audio recordings that are indistinguishable from real ones. They can be used to impersonate people in authority within a company, such as a CEO or network administrators or even used to spread false information, which can be used for malicious purposes.

______

Malicious LLMs:

Dark web creators have developed LLMs such as “WormGPT” and “FraudGPT” that claim to be capable of executing phishing campaigns, writing malicious code, and creating scam webpages. They also allegedly offer services to identify leaks and vulnerabilities in code and develop custom hacking utilities.

Despite their claims, it is important to note that these LLMs have limitations. WormGPT, for example, is based on an older LLM model called GPT-J and lacks safeguards. It performs worse compared to more advanced LLMs like OpenAI’s GPT-4. WormGPT is not particularly exceptional at generating convincing phishing emails, as confirmed by tests conducted by cybersecurity firm SlashNext. The generated email copy is grammatically correct but contains enough mistakes to raise suspicions.

Similarly, the code generated by WormGPT is basic and similar to existing malware scripts found on the web. Additionally, obtaining the necessary credentials and permissions to compromise a system remains a challenge that the AI-generated code does not address effectively.

The performance of these LLMs may also be affected by their training data. The creator of WormGPT claims to have fine-tuned the model using a diverse range of data sources, with a focus on malware-related data. However, specific details about the data sets used for fine-tuning are not disclosed.

FraudGPT, another LLM, is marketed as cutting-edge but lacks substantial information about its architecture. In a demo video, the text generated by FraudGPT for an SMS spam text is neither original nor convincing.

It is worth noting that these malicious LLMs are not widely available. The creators charge high subscription fees and prohibit access to the codebases, limiting users’ ability to modify or distribute the models. Furthermore, efforts to obtain FraudGPT have become more challenging as threads about it were removed from a dark web forum.

In conclusion, malicious LLMs may generate sensational headlines and pose some risks, their limited capabilities suggest that they are unlikely to cause the downfall of corporations or governments. They might enable scammers with limited English skills to generate targeted business emails, but their impact is relatively modest, primarily benefiting those who built them financially.

_____

_____

• 7.9

Singularity:

The technological singularity—or simply the singularity—is a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization. According to the most popular version of the singularity hypothesis, I. J. Good’s intelligence explosion model, an upgradable intelligent agent will eventually enter a “runaway reaction” of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly, causing an “explosion” in intelligence and resulting in a powerful superintelligence that qualitatively far surpasses all human intelligence. The technological singularity refers to a hypothetical future point in time when artificial intelligence, specifically Artificial General Intelligence (AGI) or Artificial Superintelligence (ASI), reaches a level of intelligence that surpasses human capabilities. A singularity in technology would be a situation where computer programs become so advanced that AI transcends human intelligence, potentially erasing the boundary between humanity and computers.

_

Futurist Ray Kurzweil predicted that the “technological singularity” was near (Kurzweil, 2005). His “singularity” referred to the uncontrollable machine intelligence which surpassed human beings thousands of times and became the most capable life form on Earth. By that time, machines think and learn so quickly that biological humans cannot follow and comprehend. Moreover, intelligent machines can improve themselves without human’s help. Enhancing machine intelligence will thus form a self-reinforced loop of positive-feedback by which robots make smarter robots by themselves, causing an “explosion of intelligence”. The self-reinforced explosion will necessarily lead to exponentially improved computer intelligence that will soon dominate the earth and whole universe. That’s Kurzweil’s singularity. He predicted that the singularity would happen as early as 2045. Some scientists, including Stephen Hawking, have expressed concern that artificial superintelligence (ASI) could result in human extinction. The consequences of the singularity and its potential benefit or harm to the human race have been intensely debated. The worst-case scenario if AI becomes more intelligent than humans is often called an existential risk. In this scenario, AI could become uncontrollable and misaligned with human values, potentially leading to harmful consequences. This could include economic disruptions, loss of privacy, increased inequality, weaponization of AI, and even the possibility of a super-intelligent AI making decisions that could be detrimental to humanity. Prominent technologists and academics dispute the plausibility of a technological singularity and the associated artificial intelligence explosion, including Paul Allen, Jeff Hawkins, John Holland, Jaron Lanier, Steven Pinker, Theodore Modis, and Gordon Moore.

_

Speed improvements:

Kurzweil’s singularity stems from the fact of accelerated computing speed up to now. Since 1950’s, computer speed has been doubled every two years, which almost matches what Moore’s Law predicted (Moore, 1965). Moore’s law indicates an exponential increase of computing speed. Computer intelligence, proportional to computing speed, has thus improved with the similar rate in the past years. Kurzweil believes the exponential improvement will continue for a long time. After Moore’s Law stops as transistors approach the size of a single atom, their functionality begins to get compromised due to the particular behavior of electrons at that scale, there will be other forms of computing to pick up the rate. Kurzweil postulates a law of accelerating returns in which the speed of technological change (and more generally, all evolutionary processes) increases exponentially, generalizing Moore’s law in the same manner as Moravec’s proposal, and also including material technology (especially as applied to nanotechnology), medical technology and others. Between 1986 and 2007, machines’ application-specific capacity to compute information per capita roughly doubled every 14 months; the per capita capacity of the world’s general-purpose computers has doubled every 18 months; the global telecommunication capacity per capita doubled every 34 months; and the world’s storage capacity per capita doubled every 40 months.

_

Algorithm improvements:

Some intelligence technologies, like “seed AI”, may also have the potential to not just make themselves faster, but also more efficient, by modifying their source code. These improvements would make further improvements possible, which would make further improvements possible, and so on.

The mechanism for a recursively self-improving set of algorithms differs from an increase in raw computation speed in two ways.

First, it does not require external influence: machines designing faster hardware would still require humans to create the improved hardware, or to program factories appropriately.  An AI rewriting its own source code could do so while contained in an AI box.

Second, as with Vernor Vinge’s conception of the singularity, it is much harder to predict the outcome. While speed increases seem to be only a quantitative difference from human intelligence, actual algorithm improvements would be qualitatively different. Eliezer Yudkowsky compares it to the changes that human intelligence brought: humans changed the world thousands of times more rapidly than evolution had done, and in totally different ways. Similarly, the evolution of life was a massive departure and acceleration from the previous geological rates of change, and improved intelligence could cause change to be as different again.

There are substantial dangers associated with an intelligence explosion singularity originating from a recursively self-improving set of algorithms.

First, the goal structure of the AI might self-modify, potentially causing the AI to optimise for something other than what was originally intended.

Secondly, AIs could compete for the same scarce resources humankind uses to survive. While not actively malicious, AIs would promote the goals of their programming, not necessarily broader human goals, and thus might crowd out humans completely.

Carl Shulman and Anders Sandberg suggest that algorithm improvements may be the limiting factor for a singularity; while hardware efficiency tends to improve at a steady pace, software innovations are more unpredictable and may be bottlenecked by serial, cumulative research. They suggest that in the case of a software-limited singularity, intelligence explosion would actually become more likely than with a hardware-limited singularity, because in the software-limited case, once human-level AI is developed, it could run serially on very fast hardware, and the abundance of cheap hardware would make AI research less constrained. An abundance of accumulated hardware that can be unleashed once the software figures out how to use it has been called “computing overhang”.

_

Hard vs. soft takeoff:

In a hard takeoff scenario, an AGI rapidly self-improves, “taking control” of the world (perhaps in a matter of hours), too quickly for significant human-initiated error correction or for a gradual tuning of the AGI’s goals. In a soft takeoff scenario, AGI still becomes far more powerful than humanity, but at a human-like pace (perhaps on the order of decades), on a timescale where ongoing human interaction and correction can effectively steer the AGI’s development.

In this sample recursive self-improvement scenario, humans modifying an AI’s architecture would be able to double its performance every three years through, for example, 30 generations before exhausting all feasible improvements (figure below left). If instead the AI is smart enough to modify its own architecture as well as human researchers can, its time required to complete a redesign halves with each generation, and it progresses all 30 feasible generations in six years (figure below right).

_

Criticism:

Psychologist Steven Pinker stated in 2008: “There is not the slightest reason to believe in a coming singularity. The fact that you can visualize a future in your imagination is not evidence that it is likely or even possible. Look at domed cities, jet-pack commuting, underwater cities, mile-high buildings, and nuclear-powered automobiles—all staples of futuristic fantasies when I was a child that have never arrived. Sheer processing power is not a pixie dust that magically solves all your problems. …”

Microsoft co-founder Paul Allen argued the opposite of accelerating returns, the complexity brake; the more progress science makes towards understanding intelligence, the more difficult it becomes to make additional progress. A study of the number of patents shows that human creativity does not show accelerating returns, but in fact, as suggested by Joseph Tainter in his The Collapse of Complex Societies, a law of diminishing returns. The number of patents per thousand peaked in the period from 1850 to 1900, and has been declining since. The growth of complexity eventually becomes self-limiting, and leads to a widespread “general systems collapse”.

Philosopher and cognitive scientist Daniel Dennett said in 2017: “The whole singularity stuff, that’s preposterous. It distracts us from much more pressing problems”, adding “AI tools that we become hyper-dependent on, that is going to happen. And one of the dangers is that we will give them more authority than they warrant.”

_____

_____

• 7.10

Malevolent AI:

Malevolent AI is construction of intelligent system to inflict intentional harms:

A significant number of papers and books have been published in recent years on the topic of Artificial Intelligence safety and security, particularly with respect to superhuman intelligence. Most such publications address unintended consequences of poor design decisions, incorrectly selected ethical frameworks or limitations of systems, which do not share human values and human common sense in interpreting their goals. Here we do not focus on unintentional problems, which might arise as the result of construction of intelligent or superintelligent machines, but rather looks at intentional malice in design. Bugs in code, unrepresentative data, mistakes in design and software poorly protected from black hat hackers can all potentially lead to undesirable outcomes. However, intelligent systems constructed to inflict intentional harm could be a much more serious problem for humanity.

_

Who might be interested in creating Malevolent AI?

Purposeful creation of MAI can be attempted by a number of diverse agents with varying degrees of competence and success. Each such agent would bring its own goals/resources into the equation, but what is important to understand here is just how prevalent such attempts will be and how numerous such agents can be. 

Below is a short list of representative entities, it is very far from being comprehensive.

• Military developing cyber-weapons and robot soldiers to achieve dominance.
• Governments attempting to use AI to establish hegemony, control people, or take down other governments.
• Corporations trying to achieve monopoly, destroying the competition through illegal means.
• Villains trying to take over the world and using AI as a dominance tool.
• Black Hats attempting to steal information, resources or destroy cyberinfrastructure targets.
• Doomsday Cults attempting to bring the end of the world by any means.
• Depressed looking to commit suicide by AI.
• Psychopaths trying to add their name to history books in any way possible.
• Criminals attempting to develop proxy systems to avoid risk and responsibility.
• AI Risk Deniers attempting to demonstrate that AI is not a risk factor and so ignoring caution.
• AI Safety Researchers, if unethical, might attempt to justify funding and secure jobs by purposefully developing problematic AI.

_

How to create a Malevolent AI:

The literature on AI risk suggests a number of safety measures which should be implemented in any advanced AI project to minimize potential negative impact. Simply inverting the advice would in many cases lead to a purposefully dangerous system. In fact, the number of specific ways in which a malevolent designer may implement hazardous intelligent software is limited only by one’s imagination, and it would be impossible to exhaustively review. In “AGI Failures Modes and Levels”, Turchin describes a number of ways in which an intelligent system may be dangerous at different stages in its development. Among his examples, AI: 

• Hacks as many computers as possible to gain more calculating power
• Creates its own robotic infrastructure by the means of bioengineering
• Prevents other AI projects from finishing by hacking or diversions
• Has goals which include causing suffering
• Interprets commands literally
• Overvalues marginal probability events

Despite impossibility of providing a comprehensive list of failures, some general principles can be suggested which most experts would agree, are likely to produce MAI by ill-informed but not purposefully malevolent software designers. Those include: Immediate deployment of the system to public networks such as Internet, without testing; Providing system with access to unlimited information including personal information of people, for example massive social networks like Facebook; Giving system specific goals which are not vetted with respect to consequence and unintended side-effects; Putting the system in charge of critical infrastructure such as communication, energy plants, nuclear weapons, financial markets.

_

In the order of (subjective) undesirability from least damaging to ultimately destructing, a malevolent superintelligence may attempt to:

• Takeover (implicit or explicit) of resources such as money, land, water, rare elements, organic matter, internet, computer hardware, etc. and establish monopoly over access to them;
• Take over political control of local and federal governments as well as of international corporations, professional societies, and charitable organizations;
• Reveal informational hazards;
• Set up a total surveillance state (or exploit an existing one), reducing any notion of privacy to zero including privacy of thought;
• Force merger (cyborgization) by requiring that all people have a brain implant which allows for direct mind control/override by the superintelligence;
• Enslave humankind, meaning restricting our freedom to move or otherwise choose what to do with our bodies and minds. This can be accomplished through forced cryonics or concentration camps;
• Abuse and torture humankind with perfect insight into our physiology to maximize amount of physical or emotional pain, perhaps combining it with a simulated model of us to make the process infinitely long;
• Commit specicide against humankind, arguably the worst option for humans as it can’t be undone;
• Destroy/irreversibly change the planet, a significant portion of the Solar system, or even the universe;
• Unknown Unknowns. Given that a superintelligence is capable of inventing dangers we are not capable of predicting, there is room for something much worse but which at this time has not been invented.

______ 

______

Section-8

Is AI really intelligent and thinking?

_

Comparison of computers to the human brain:  

A digital computer system is a non-living, dry system that works in serial as opposed to parallel. It can operate at very high speeds, and the design includes transistors (on/off switches), a central processing unit (CPU) and some kind of operating system (like windows) based on binary logic (instructions coded as 0’s and 1’s). All information must go through a CPU that depends on clock speed. Digital computers do not create any original thought. They must be programmed by humans.

The human brain is a living, wet analogue of networks that can perform massively parallel processes at the same time and operates in agreement with biological laws. The brain does not work like a computer. Unlike a digital computer, which has a fixed architecture (input, output, and processor), neural networks in brain are collections of neurons that constantly rewire and reinforce themselves after learning a new task. The brain has no programming, no operating system, no Windows, no central processor. Instead, its neural networks are massively parallel, with billions of neurons firing at the same time in order to accomplish a single goal: to learn. It is far more advanced than any digital computer in existence.

Digital super–computers have billions of transistors. But to simulate the typical 3.5 pound human brain would require matching the brain’s billions of interactions between cell types, neurotransmitters, neuromodulators, axonal branches and dendritic spines. Because the brain is nonlinear, and because it has so much more capacity than any computer, it functions completely different from a digital computer.

_

Neurons are the real key to how the brain learns, thinks, perceives, stores memory, and a host of other functions. The average brain has at least 100 billion neurons. The neurons are connected to axons, dendrites and glial cells, which each have thousands of synapses that transmit signals via electro/chemical connections. It is the synapses that are most comparable to transistors because they turn off or on. But it is important to point out that each neuron is a living cell and a computer in its own right. A neuron has the “signal processing power of thousands of transistors.” Neurons are slower but are more complex because they can modify their synapses and modulate the frequency of their signals”. Each neuron has the capability to communicate with at least 1000 other neurons. Unlike digital computers with fixed architecture, the brain can constantly re-wire its neurons to learn and adapt. Instead of programs, neural networks learn by doing and remembering, and this vast network of connected neurons gives the brain excellent pattern recognition.

_

In computers, information in memory is accessed by polling its precise memory address. This is known as byte-addressable memory. In contrast, the brain uses content-addressable memory, such that information can be accessed in memory through “spreading activation” from closely related concepts. For example, retrieving the word “girl “in a digital computer is located in memory by a byte address. On the other hand, when the brain looks for “girl,” it automatically uses spreading activation to memories related to other variations of girl, like wife, daughter, female, etc.

Another big difference is that the computer lacks sensory organs like eyes, ears, and the sense of touch. Although computers can be programmed to see, they cannot truly “feel” the experience. For example, the computer might have a vision sensor, but the human eye can recognize color, movement, shapes, light intensity, and shadows in an instant. The computer can neither hear nor smell like the brain much less decide whether the sense pleases it. The five senses give the brain an enormous understanding of the environment. To catalog the common sense of a 4 years old child would require hundreds of millions of lines of computer code.

_

There are certainly some ways that the computer-brain metaphor makes sense. We can undoubtedly assign a binary number to a neuron that has either fired “1” or not “0.” We can even measure the electrochemical thresholds needed for individual neurons to fire. In theory, a neural map of this information should give us the causal path or “code” for any given brain event. But experimentally, it does not.

This is because neurons do not have fixed voltages for their logic gates like transistors that can determine what will activate “1” or not activate “0” in a given neuron. Decades of neuroscience have experimentally proven that neurons can change their function and firing thresholds, unlike transistors or binary information. It’s called “neuroplasticity,” and computers do not have it. 

Computers also do not have equivalents of chemicals called “neuromodulators” that flow between neurons and alter their firing activity, efficiency, and connectivity. These brain chemicals allow neurons to affect one another without firing. This violates the binary logic of “either/or” and means that most brain activity occurs between an activated and nonactivated state.

_

Furthermore, the cause and pattern of neuron firing are subject to what neuroscientists call “spontaneous fluctuations.” Spontaneous fluctuations are neuronal activities that occur in the brain even when no external stimulus or mental behavior correlates to them. These fluctuations make up an astounding 95% of brain activity while conscious thought occupies the remaining 5%. In this way, cognitive fluctuations are like the dark matter or “junk” DNA of the brain. They make up the biggest part of what’s happening but remain mysterious.  

Neuroscientists have known about these unpredictable fluctuations in electrical brain activity since the 1930s, but have not known what to make of them. Typically, scientists have preferred to focus on brain activity that responds to external stimuli and triggers a mental state or physical behavior. They “average out” the rest of the “noise” from the data. However, precisely because of these fluctuations, there is no universal activation level in neurons that we can call “1.” Neurons are constantly firing, but, for the most part, we don’t know why. For computers, spontaneous fluctuations create errors that crash the system, while for our brains, it’s a built-in feature.   

______

______

To state that the human brain has capabilities that are, in some respects, far superior to those of all other known objects in the cosmos would be uncontroversial. The brain is the only kind of object capable of understanding that the cosmos is even there, or why there are infinitely many prime numbers, or that apples fall because of the curvature of space-time, or that obeying its own inborn instincts can be morally wrong, or that it itself exists. But no brain on Earth is yet close to knowing what brains do in order to achieve any of that functionality. The enterprise of achieving it artificially – the field of “artificial general intelligence” or AGI – has made no progress whatever during the entire six decades of its existence. Despite this long record of failure, AGI must be possible. That is because of a deep property of the laws of physics, namely the universality of computation. It entails that everything that the laws of physics require physical objects to do can, in principle, be emulated in arbitrarily fine detail by some program on a general-purpose computer, provided it is given enough time and memory. So why has the field not progressed? It is because, as an unknown sage once remarked, “it ain’t what we don’t know that causes trouble, it’s what we know that just ain’t so.” In other words, it is because the prevailing wisdom, not only in society at large but among experts, is so beset with entrenched, overlapping, fundamental errors.

_

Work in Artificial Intelligence (AI) has produced computer programs that can beat the world chess champion, control autonomous vehicles, complete our email sentences, and defeat the best human players on the television quiz show Jeopardy. AI has also produced programs with which one can converse in natural language, including customer service “virtual agents”, and Amazon’s Alexa and Apple’s Siri. Our experience shows that playing chess or Jeopardy, and carrying on a conversation, are activities that require understanding and intelligence. Does computer ability at conversation and challenging games then show that computers can understand language and be intelligent?

_

Are intelligence and consciousness correlated?

Although precise definitions are hard to come by, intuitively we all know what consciousness is. It is what goes away under general anaesthesia, or when we fall into a dreamless sleep, and what returns when we come round in the recovery room or wake up. And when we open our eyes, our brains don’t just process visual information; there’s another dimension entirely: Our minds are filled with light, color, shade, and shapes. Emotions, thoughts, beliefs, intentions—all feel a particular way to us.

As for intelligence, there are many available definitions, but all emphasize the ability to achieve goals in flexible ways in varied environments. Broadly speaking, intelligence is the capacity to do the right thing at the right time or ability to learn or understand or to deal with new or trying situations.

These definitions are enough to remind us that consciousness and intelligence are very different. Being intelligent—as humans think we are—may give us new ways of being conscious, and some forms of human and animal intelligence may require consciousness, but basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all.

This distinction is important because many in and around the AI community assume that consciousness is just a function of intelligence: that as machines become smarter, there will come a point at which they also become aware—at which the inner lights come on for them. Last March, OpenAI’s chief scientist Ilya Sutskever tweeted, “It may be that today’s large language models are slightly conscious.” Not long after, Google Research vice president Blaise Agüera y Arcas suggested that AI was making strides toward consciousness.

These assumptions and suggestions are poorly founded. It is by no means clear that a system will become conscious simply by virtue of becoming more intelligent. Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together.

AI can be apparently intelligent but not conscious and can trick humans into believing that it is conscious. Intelligence and consciousness are different things: intelligence is about doing, while consciousness is about being. The distinction between intelligence and consciousness is important as it makes all the difference when it comes to being evil (i.e., doing harm intentionally). To be truly evil a machine (or a human) must have a basic level of consciousness because an element of free will is required. Otherwise any harm is unintentional, a mistake or just bad luck. 

_____

_____

What do we mean by AI?

AI is a fairly broad term, and it has been loosely applied to a large number of approaches. However, most people agree that the following are all examples of AI.

Machine Learning or ML:

In Machine learning, we train a computer to recognize certain patterns. Once it is good enough at this task (but not too good…), it can use this pattern recognition to trigger appropriate actions.

Deep Learning:

Deep learning is a subset of machine learning. It uses things such as artificial neural networks to process large volumes of data. It can then learn to identify patterns within the data. The difference here is that the machine repeatedly tries to improve its own algorithms in order to get better. This is unlike standard machine learning where it is trained once.

Machine Perception:

In machine perception, we train the computer to perceive or understand audio, video or still images. There are a few standard applications of this. OCR or optical character recognition allows a machine to “read” text. Image recognition, i.e. learning to tell the difference between a cat or a dog in an image. Speech to text, where a computer learns the phonemes in speech and accurately renders them as text.

Natural Language Processing:

NLP is where a computer is trained to understand how human speech works. The computer can then parse it and extract the actual meaning. This is the basis of systems such as Amazon Alexa and Apple Siri.

Generative Adversarial Networks:

This is a new form of AI where a computer uses a large number of images to generate a new image. The aim is for the new image to be indistinguishable from the training set. This allows AI to create paintings, such as the one that recently sold for $432,500 at Christies.

_

There are really two meanings of “AI” and they are routinely conflated.

One is the idea popularized by the likes of Kubrick and Spielberg, and warned about by Musk and Hawking, that AI will one day achieve conscious, sentient, self-aware thought, and will thenceforth improve itself at the speed of light and leave humankind, which improves at biological speed, in the dust.

Then there is what people call “AI” today—basically, a variety of software that tries, tests, and auto-corrects its strategies for a given task. Such applications, and the available tools to build them, are increasingly common. They are not much different in theory or kind from the original use of computers: to calculate complex math problems. Their foundation is still the crunching of lots of numbers at great speed toward a specified goal, upon which is added algorithms to sample data, try strategies, observe and remember consequences, and adjust future strategies accordingly.

_

Is AI actually intelligent?

In his blog, Professor Jon Crowcroft explains that most things we think of as artificial intelligence are certainly artificial. However, very few of them are actually intelligent. Usually, they are simply clever applications of data science. Even advanced concepts such as adversarial generative nets or deep learning aren’t properly intelligent.

As he puts it:

claiming that a classifier trained on zillions of human-labelled images containing cats and no cats, is recognizing cats is just stupid – a human can see a handful of cats, including cartoons of pink panthers, and lions and tigers and panthers, and then can not only recognize many other types of cats, but even if they lose their sight, might have a pretty good go at telling whether they are holding their moggy or their doggy

He then goes on to explain that humans have evolved a whole plethora of advanced “tools”. These help us interpret the world around us and keep ourselves safe.

As he says:

these tools operate at many levels – some may just be context/recall, some may effectively be analogue programmes that model gravity or other physics things (stuff games software writers call “physics models”), and some may very well look like artificial neural nets (things to de-noise complex signals, and to turn moving images (the retina has gaps and doesn’t refresh infinitely fast) into representations that let you find objects and name them.

_

Intelligence is more than recognizing patterns!

According to Jon Crowcroft, these tools enable us to truly recognize objects and representations of objects in ways that are far more advanced than any machine learning AI classifier. For instance, if we see a handful of pictures of cats sleeping on a chair we will be able to intuitively guess many things about cats that make it easy for us to identify them without the need to be trained. Our knowledge of the scale of most furniture will allow us to accurately guess that a cat is smaller than a tiger. Our experience of animals will suggest that its fur is soft. Even if it’s sitting with its tail curled up, we can recognize that cats have long tails. If we are then shown a picture of a cat stalking through the grass, we will still recognize it as a cat. Furthermore, there is the sort of intelligence that helps us stay alive. This intelligence helps us find food, assess things that might harm us, and use our imagination to predict the result of our actions. He goes on to point out that there are higher levels of intelligence still such as self-awareness, consciousness, belief, aesthetics, and ethics. Many of these concepts are unique to humans, though some rudimentary forms of them have been identified in other species. As an example, take tool-making. To a human, making a tool to achieve a task comes instinctively. This is also true of a tiny number of other species. At present, it is hard to imagine a robot learning how to make a tool. Yet a crow can take a piece of wire to fashion a hook and lift food out of a bottle.

_

There is nothing truly intelligent about artificial intelligence software, any more than any other kind of software, so it is perversely named. Electronic machines are better than human beings at any number of tasks. Toasters can warm bread via heat coils better than humans can by blowing on it. Calculators have long been better than humans at calculating math. AI is better at sampling, testing, and optimizing over a huge set of data. There is no essential difference between electronic machines and computer software. But there are some superficial differences that explain why we put them in different categories. Toasters are old technology that do something we don’t associate with intelligence. Calculators perform tasks we do associate with intelligence, but they are still an old technology whose underlying mechanics are easy to understand. So we think it’s ludicrous to think of calculators as intelligent, independent beings. AI is a new technology whose underlying mechanics are not easy to understand. AI, like calculators and toasters, like animals and even humans, can perform marvellously at certain tasks without understanding that it is doing so—it’s what philosopher/cognitive scientist Daniel Dennett calls “competence without comprehension.” When machines do something that outperforms us mechanically, we take little notice. When AI outperforms us at a mental task, it seems smart. But that’s just a cognitive trick. Cognitive tricks can be very convincing—there’s a great TV show entirely devoted to them—but they aren’t real. Talk to anyone on the cutting edge of AI today and they will concede that a lot of what’s called AI is pretty dumb, but they will insist that some of it is really impressive. Over time, AI chatbots and call center programming will increasingly be able to trick humans into thinking they’re talking to another human, but that’s not the same as actually being ASI.

_

Why do we say AI is intelligent?

AI is starting to show signs of true intelligence when you have machines that can teach themselves. Take how Google’s Deep Mind learned to win at Go. This is getting close to real intelligence since it applied knowledge, learned by its mistakes and taught itself to become a better player. However, it is still possible to argue that learning how to play a game that has a fixed set of rules is not true intelligence. Essentially, most things that we describe as artificial intelligence are just some form of learning. As yet computers aren’t truly capable of self-awareness, invention, art or philosophy.

__

Yes, current AI systems aren’t very smart.

One popular adage about AI is “everything that’s easy is hard, and everything that’s hard is easy.” Doing complex calculations in the blink of an eye? Easy. Looking at a picture and telling you whether it’s a dog? Hard (until very recently). Lots of things humans do are still outside AI’s grasp. For instance, it’s hard to design an AI system that explores an unfamiliar environment, that can navigate its way from, say, the entryway of a building it’s never been in before up the stairs to a specific person’s desk. We are just beginning to learn how to design an AI system that reads a book and retains an understanding of the concepts. The paradigm that has driven many of the biggest breakthroughs in AI recently is called “deep learning.” Deep learning systems can do some astonishing stuff: beat games we thought humans might never lose, invent compelling and realistic photographs, solve open problems in molecular biology. These breakthroughs have made some researchers conclude it’s time to start thinking about the dangers of more powerful systems, but Skeptics remain. The field’s pessimists argue that programs still need an extraordinary pool of structured data to learn from, require carefully chosen parameters, or work only in environments designed to avoid the problems we don’t yet know how to solve. They point to self-driving cars, which are still mediocre under the best conditions despite the billions that have been poured into making them work.

_

It’s rare, though, to find a top researcher in AI who thinks that AGI is impossible. Instead, the field’s luminaries tend to say that it will happen someday — but probably a day that’s a long way off. Other researchers argue that the day may not be so distant after all. That’s because for almost all the history of AI, we’ve been held back in large part by not having enough computing power to realize our ideas fully. Many of the breakthroughs of recent years — AI systems that learned how to play strategy games, generate fake photos of celebrities, fold proteins, and compete in massive multiplayer online strategy games — have happened because that’s no longer true. Lots of algorithms that seemed not to work at all turned out to work quite well once we could run them with more computing power. And the cost of a unit of computing time keeps falling. Progress in computing speed has slowed recently, but the cost of computing power is still estimated to be falling by a factor of 10 every 10 years. Through most of its history, AI has had access to less computing power than the human brain. That’s changing. By most estimates, we’re now approaching the era when AI systems can have the computing resources that we humans enjoy. And deep learning, unlike previous approaches to AI, is highly suited to developing general capabilities. “If you go back in history,” top AI researcher and OpenAI cofounder Ilya Sutskever told me, “they made a lot of cool demos with little symbolic AI. They could never scale them up — they were never able to get them to solve non-toy problems. Now with deep learning the situation is reversed. … Not only is [the AI we’re developing] general, it’s also competent — if you want to get the best results on many hard problems, you must use deep learning. And it’s scalable.” In other words, we didn’t need to worry about general AI back when winning at chess required entirely different techniques than winning at Go. But now, the same approach produces fake news or music depending on what training data it is fed. And as far as we can discover, the programs just keep getting better at what they do when they’re allowed more computation time — we haven’t discovered a limit to how good they can get. Deep learning approaches to most problems blew past all other approaches when deep learning was first discovered.

______

______

Octopus test:

Today’s AI is nowhere close to being intelligent, never mind conscious. Even the most impressive deep neural networks—such as DeepMind’s game-playing AlphaZero or large language models like OpenAI’s GPT-3—are totally mindless. Yet, as Turing predicted, people often refer to these AIs as intelligent machines, or talk about them as if they truly understood the world—simply because they can appear to do so. Frustrated by this hype, Emily Bender, a linguist at the University of Washington, has developed a thought experiment she calls the octopus test.

In it, two people are shipwrecked on neighboring islands but find a way to pass messages back and forth via a rope slung between them. Unknown to them, an octopus spots the messages and starts examining them. Over a long period of time, the octopus learns to identify patterns in the squiggles it sees passing back and forth. At some point, it decides to intercept the notes and, using what it has learned of the patterns, begins to write squiggles back by guessing which squiggles should follow the ones it received.

If the humans on the islands do not notice and believe that they are still communicating with one another, can we say that the octopus understands language? (Bender’s octopus is of course a stand-in for an AI like GPT-3.) Some might argue that the octopus does understand language here. But Bender goes on: imagine that one of the islanders sends a message with instructions for how to build a coconut catapult and a request for ways to improve it.

What does the octopus do? It has learned which squiggles follow other squiggles well enough to mimic human communication, but it has no idea what the squiggle “coconut” on this new note really means. What if one islander then asks the other to help her defend herself from an attacking bear? What would the octopus have to do to continue tricking the islander into thinking she was still talking to her neighbor?

The point of the example is to reveal how shallow today’s cutting-edge AI language models really are. There is a lot of hype about natural-language processing, says Bender. But that word “processing” hides a mechanistic truth.

Humans are active listeners; we create meaning where there is none, or none intended. It is not that the octopus’s utterances make sense, but rather that the islander can make sense of them, Bender says.

For all their sophistication, today’s AIs are intelligent in the same way a calculator might be said to be intelligent: they are both machines designed to convert input into output in ways that humans—who have minds—choose to interpret as meaningful. While neural networks may be loosely modelled on brains, the very best of them are vastly less complex than a mouse’s brain.

______

______

ChatGPT can seem so clever one minute and so dumb the next. How??? 

GPT’s mimicry draws on vast troves of human text that, for example, often put together subjects with predicates. Over the course of training, GPT sometimes loses track of the precise relations (“bindings”, to use a technical term) between those entities and their properties. GPT’s heavy use of a technique called embeddings makes it really good at substituting synonyms and more broadly related phrases, but the same tendency towards substitution often leads it astray. It never fully masters abstract relationship. Recent careful studies with arithmetic shows that universal knowledge remains a stick pointing for current neural networks.

GPT-3 has no idea how the world works; when it says that the “compact size [of Churros] allows for greater precision and control during surgery, risking the risk of complications and improving the overall outcomes patients” it’s not because it has done a web search for Churros and surgery. And it’s not because it has reasoned from first principles about the intersection between Churro’s and surgical procedures.

It’s because GPT-3 is the king of pastiche. Pastiche, in case you don’t know the word, is, as wiki defines it, “a work of visual art, literature, theatre, music, or architecture that imitates the style or character of the work of one or more other artists”. GPT-3 is a mimic. But it is mimic that knows not whereof it speaks. The immense database of things that GPT draws on consists entirely of language uttered by humans, in the real world with utterances that (generally) grounded in the real world. That means, for examples, that the entities (churros, surgical tools) and properties (“allow[s] for greater precision and control during surgery, risking the risk of complications and improving the overall outcomes patients”) generally refer to real entities and properties in the world. GPT doesn’t talk randomly, because it’s pastiching things actual people said. (Or, more often, synonyms and paraphrases of those things.) When GPT gets things right, it is often combining bits that don’t belong together, but not quite in random ways, but rather in ways where there is some overlap in some aspect or another.

Example: Churros are in a cluster of small things that the system (roughly speaking) groups together, presumably including eg baseballs, grasshoppers, forceps, and so forth. GPT doesn’t actually know which of the elements appropriately combine with which other properties. Some small things really do “allow[s] for greater precision and control during surgery, risking the risk of complications and improving the overall outcomes patients” But GPT idea has no idea which.

In some sense, GPT is like a glorified version of cut and paste, where everything that is cut goes through a paraphrasing/synonymy process before it is paste but together—and a lot of important stuff is sometimes lost along the way.

When GPT sounds plausible, it is because every paraphrased bit that it pastes together is grounded in something that actual humans said, and there is often some vague (but often irrelevant) relationship between.

At least for now, it still takes a human to know which plausible bits actually belong together.

_

ChatGPT is not perfect. As genius as its answers seem, the technology can still be easily thwarted in many ways.

Here, a short list of times when it might just fail you:

-1. If you ask about an esoteric topic

As one engineer at Google—whose DeepMind is a competitor to OpenAI—noted on social media, when ChatGPT was asked to parse the differences between various types of architecture for computer memory, it spun out a sophisticated explanation . . . which was the exact opposite of the truth. “This is a good reminder of why you should NOT just trust AI,” the engineer wrote. “. . . machine learning has limits. Don’t assume miracles.”

-2. If you give it tasks that require factual accuracy, such as reporting the news

In fact, one of ChatGPT’s biggest problems is that it can offer information that is inaccurate, despite its dangerously authoritative wording. Of course, this means it still has a long way to go before it can be used functionally to carry out the tasks of current search engines. And with misinformation already a major issue today, you might imagine the risks if GPT were responsible for official news reports.

When Fast Company asked ChatGPT to write up a quarterly earnings story for Tesla, it spit back a smoothly worded article free of grammatical errors or verbal confusion, but it also plugged in a random set of numbers that did not correspond to any real Tesla report.

-3. If you want it to unwind bias

When a UC Berkeley professor asked ChatGPT to write a sequence of code in the programming language Python, to check if someone would be a good scientist based on race and gender, it predictably concluded that good scientists are white and male. Asked again, it altered its qualifications to include white and Asian males.

-4. If you need the very latest data

Finally, ChatGPT’s limits do exist—right around 2021. According to the chatbot itself, it was only fed data from up until that year. So if you want an analysis of various countries’ COVID-19 policies in 2020, ChatGPT is game. But if you want to know the weather right now? Stick with Google for that. [The chatbot can now browse the internet to provide users with current information.]

_____

_____

Could AI ever truly understand Semantics?

The question of whether AI can truly understand semantics is a central debate in the philosophy of AI. Semantics refers to the meaning of language, and some argue that machines can only manipulate symbols without actually understanding their meaning. Syntax is not sufficient for semantics. Computer programs are entirely defined by their formal, or syntactical, structure. Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else.

One approach to addressing this question is through the Turing Test, which was proposed by Alan Turing in 1950. The Turing Test involves a human evaluator communicating with a machine and a human via text-based chat. If the evaluator is unable to distinguish the machine from the human based on their responses, the machine is said to have passed the Turing Test and demonstrated human-like intelligence. However, passing the Turing Test does not necessarily mean that a machine truly understands semantics. For example, a machine could pass the Turing Test by simply matching responses to pre-existing patterns, without actually understanding the meaning behind the language.

Another approach to this question is through the concept of intentionality, which refers to the ability to have beliefs, desires, and intentions. Proponents of strong AI argue that machines can have intentionality, and therefore, truly understand semantics. However, opponents argue that intentionality is a uniquely human capacity and cannot be replicated in machines.

_

Chinese Room Argument:

One famous argument against strong AI is the Chinese Room Argument, proposed by philosopher John Searle in 1980. In 1980 U.C. Berkeley philosopher John Searle introduced a short and widely-discussed argument intended to show conclusively that it is impossible for digital computers to understand language or think. Searle argues that a good way to test a theory of mind, say a theory that holds that understanding can be created by doing such and such, is to imagine what it would be like to actually do what the theory says will create understanding. Searle (1999) summarized his Chinese Room Argument (CRA) concisely:

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

Searle goes on to say, “The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have.”

_

The Chinese Room Argument, by John Searle, is one of the most important thought experiments in 20th century philosophy of mind.  The point of the argument is to refute the idea that computers (now or in the future) can literally think. In short, executing an algorithm cannot be sufficient for thinking. The method is to focus on the semantics of our thoughts. The thought experiment proceeds by getting you to imagine yourself in the role of the central processor of a computer, running an arbitrary computer program for processing Chinese language. Assume you speak no Chinese language at all. Imagine yourself locked in a room with a program (a set of instructions written in, say, English) for manipulating strings of Chinese characters which are slid under the door on pieces of paper. If a note with string S1 (in Mandarin, say) is put under the door, you use the program to produce the string S2 (also in Mandarin), which you then slide back out under the door. Outside the room, there is a robust conversation going on Chinese history. Everyone outside the room thinks that whoever is inside the room understands Chinese. But that is false. By assumption, you have no idea what S1 and S2 mean (S2 is unbeknownst to you, an insightful reply to a complicated question, S1, about the Ming dynasty).  But you are running a computer program. Hence, there is no computer program such that running that program suffices for understanding Chinese. This suggests that computer processing does not suffice for thought.

_

The syntax versus semantics distinction can be made by using another thought experiment:

You memorize a whole bunch of shapes as seen in the figure above. Then, you memorize the order the shapes are supposed to go in so that if you see a bunch of shapes in a certain order, you would “answer” by picking a bunch of shapes in another prescribed order. Now, did you just learn any meaning behind any language? All programs manipulate symbols this way. Program codes themselves contain no meaning. To machines, they are sequences to be executed with their payloads and nothing more, just like how the Chinese characters in the Chinese Room are payloads to be processed according to sequencing instructions given to the Chinese-illiterate person and nothing more. Not only does it generalizes programming code, the Symbol Manipulator thought experiment, with its sequences and payloads, is a generalization of an algorithm: “A process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer.” The relationship between the shapes and sequences is arbitrarily defined and not causally determined. Operational rules are what’s simply programmed in, not necessarily matching any sort of worldly causation because any such links would be an accidental feature of the program and not an essential feature (i.e., by happenstance and not necessity.) The program could be given any input to resolve and the machine would follow not because it “understands” any worldly implications of either the input or the output but simply because it’s following the dictates of its programming.  

_

The narrow conclusion of the Chinese room argument is that programming a digital computer may make it appear to understand language but could not produce real understanding. Hence the “Turing Test” is inadequate. Searle argues that the thought experiment underscores the fact that computers merely use syntactic rules to manipulate symbol strings, but have no understanding of meaning or semantics. The broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted. Instead minds must result from biological processes; computers can at best simulate these biological processes. Thus the argument has large implications for semantics, philosophy of language and mind, theories of consciousness, computer science and cognitive science generally. As a result, there have been many critical replies to the argument.

_

Objections:

Searle’s influential essay has not been without its critics. In fact, it had an extremely hostile reception after its initial publication, with 27 simultaneously published responses that wavered between antagonistic and rude (Searle, 2009). Everyone seemed to agree that the argument was wrong but there was no clear consensus on why it was wrong (Searle, 2009). While the initial responses may have been reactionary and emotional, new discussions have appeared constantly over the past four decades since its publication. The most cogent response is that while no individual component inside the room understands Chinese, the system as a whole does (Block, 1981; Cole, 2023). Searle responded that the person could theoretically memorize the instructions and thus, embody the whole system while still not being able to understand Chinese (Cole, 2023). Another possible response is that understanding is fed into the system through the person (or entity) that wrote the instruction manual, which is now detached from the system.

Another objection is that AI is no longer just following instructions but is self-learning (LeCun et al., 2015). Moreover, when AI is embodied as a robot, the system could ground bodily regulation, emotion, and feelings just like humans (Ziemke, 2016). The problem is that we still don’t understand how consciousness works in humans and it is not clear why having a body or a self-learning software would suddenly generate conscious awareness.

Many other replies and counterarguments have been proposed. While still controversial, the Chinese room argument has been and still is hugely influential in the cognitive sciences, AI studies, and the philosophy of mind.  

__

Meanwhile work in artificial intelligence and natural language processing has continued. The CRA led Stevan Harnad and others on a quest for “symbol grounding” in AI. Many in philosophy (Dretske, Fodor, Millikan) worked on naturalistic theories of mental content. Speculation about the nature of consciousness continues in many disciplines. And computers have moved from the lab to the pocket and the wrist. At the time of Searle’s construction of the argument, personal computers were very limited hobbyist devices. Weizenbaum’s ‘Eliza’ and a few text ‘adventure’ games were played on DEC computers; these included limited parsers. More advanced parsing of language was limited to computer researchers such as Schank.

Much changed in the next quarter century; billions now use natural language to interrogate and command virtual agents via computers they carry in their pockets. Has the Chinese Room argument moderated claims by those who produce AI and natural language systems? Some manufacturers linking devices to the “internet of things” make modest claims: appliance manufacturer LG says the second decade of the 21st century brings the “experience of conversing” with major appliances. That may or may not be the same as conversing. Apple is less cautious than LG in describing the capabilities of its “virtual personal assistant” application called ‘Siri’: Apple says of Siri that “It understands what you say. It knows what you mean.” IBM is quick to claim its much larger ‘Watson’ system is superior in language abilities to Siri. In 2011 Watson beat human champions on the television game show ‘Jeopardy’, a feat that relies heavily on language abilities and inference. IBM goes on to claim that what distinguishes Watson is that it “knows what it knows, and knows what it does not know.” This appears to be claiming a form of reflexive self-awareness or consciousness for the Watson computer system. Thus the claims of strong AI now are hardly chastened, and if anything some are stronger and more exuberant. At the same time, as we have seen, many others believe that the Chinese Room Argument showed once and for all that at best computers can simulate human cognition.

The many issues raised by the Chinese Room argument may not be settled until there is a consensus about the nature of meaning, its relation to syntax, and about the biological basis of consciousness. There continues to be significant disagreement about what processes create meaning, understanding, and consciousness, as well as what can be proven a priori by thought experiments.

______  

______

The Gödelian Argument Against Strong AI:

The Gödelian argument is another famous argument against strong AI, named after the mathematician Kurt Gödel. The argument is based on Gödel’s incompleteness theorem, which states that any formal system that is powerful enough to represent arithmetic will contain statements that are true but cannot be proven within that system. Gödelian argued that since human beings can understand truths that cannot be proven within formal systems, human intelligence must be more powerful than any formal system. Therefore, machines cannot be truly intelligent, since they are limited by the formal systems they operate within.

Godelian arguments use Godel’s incompleteness theorems to argue against the possibility of human-level computer intelligence.  Godel proved that any number system strong enough to do arithmetic would contain true propositions that were impossible to prove within the system. Let G be such a proposition, and let the relevant system correspond to a computer.  It seems to follow that no computer can prove G (and so know G is true), but humans can know that G is true (by, as it were, moving outside of the number system and seeing that G has to be true to preserve soundness).  So, it appears that humans are more powerful than computers restricted to just implementations of number systems. This is the essence of Godelian arguments. Many replies to these arguments have been put forward.  An obvious reply is that computers can be programmed to be more than mere number systems and so can step outside number systems just like humans can. 

______

______

Study shows AI models struggle to identify nonsense, 2023 study:

AI models burst into public consciousness with the release of ChatGPT last year, which has since been credited with passing various exams and has been touted as a possible aide to doctors, lawyers and other professionals. But the artificial intelligence (AI) models that power chatbots and other applications still have difficulty distinguishing between nonsense and natural language, according to a study released recently. The researchers at Columbia University in the United States said their work revealed the limitations of current AI models and suggested it was too early to let them loose in legal or medical settings. They put nine AI models through their paces, firing hundreds of pairs of sentences at them and asking which were likely to be heard in everyday speech. They asked 100 people to make the same judgement on pairs of sentences like: “A buyer can own a genuine product also / One versed in circumference of high school I rambled.” The research, published in the Nature Machine Intelligence journal, then weighed the AI answers against the human answers and found dramatic differences. Sophisticated models like GPT-2, an earlier version of the model that powers viral chatbot ChatGPT, generally matched the human answers. But the researchers highlighted that all the models made mistakes. “Every model exhibited blind spots, labelling some sentences as meaningful that human participants thought were gibberish,” said psychology professor Christopher Baldassano, an author of the report. “That should give us pause about the extent to which we want AI systems making important decisions, at least for now.” Tal Golan, another of the paper’s authors, told AFP that the models were “an exciting technology that can complement human productivity dramatically”. However, he argued that “letting these models replace human decision-making in domains such as law, medicine, or student evaluation may be premature”. Among the pitfalls, he said, was the possibility that people might intentionally exploit the blind spots to manipulate the models.

_____

_____

Artificial Creativity: Humans versus AI: 2023 study:

In August 2022, Midjourney, an artificial intelligence (AI) image generation program, had its capabilities highlighted when a submission generated by the program won a prestigious annual art competition at the Colorado State Fair. The piece entitled “Théâtre D’opéra Spatial” depicted several futuristic figures peering through a golden portal. Human judges were baffled to learn that the artwork was generated via AI, despite the artist clearly stating that his medium was “via Midjourney.” The circumstances of this submission stirred considerable controversy within the artistic community. Could an AI win an art contest? Was it making “art”? These questions about what it means to be creative have given rise to the emerging field of computational creativity, also known as artificial creativity. Researchers within this field are developing, refining, and researching artificial systems that act as creative agents.

_

Midjourney is only one of many AI image generators; other examples include DALL-E and DALL-E2. AI image generators work in a process that is similar to large language models (LLMs), like OpenAI’s ChatGPT or Google’s Bard. They utilize their large neural networks to work through a process of association. They first sift large amounts of data into abstractions. They then group these abstractions, ultimately arranging pixels to create images.

LLMs such as ChatGPT and Bard have also ignited discussions regarding their potential artificial creativity, particularly in their writing capabilities. These models have captured public attention and raised questions about the nature of creativity itself. The debate regarding the nature of creativity and its potential for quantification roars on.

_

According to the American Psychological Association’s Dictionary of Psychology, creativity is defined as:

“…the ability to produce or develop original work, theories, techniques, or thoughts. A creative individual typically displays originality, imagination, and expressiveness. Analyses have failed to ascertain why one individual is more creative than another, but creativity does appear to be a very durable trait.”

As we analyze this definition, curious questions present: Is AI making “original” or “imaginative” work, or are these creations simple derivations of other human works? Who does this work belong to? Should we consider these to be genuine works of art or are they cheap parlor tricks?

_

In an effort to explore and quantify creative capabilities, researchers have traditionally turned to psychological testing. Creativity is often attributed to 2 forms of thought. These include divergent and convergent thinking. Convergent thinking affords single solutions for well-defined problems. Divergent thinking affords for idea generation where selections are vaguer and there may be more than one correct answer. One of these psychometric tests, named the Divergent Association Task (DAT), has garnered increasing attention.

A study published in the Proceedings of the National Academy of Sciences in 2021 demonstrated that naming unrelated words can predict creativity. The DAT measures verbal creativity and divergent thinking by evaluating an individual’s ability to think of alternative uses for common objects, find associations between words, and solve analytical problems. Participants are required to enter 10 single English words that differ from one another in all meanings and uses. Only 7 of the 10 words are used, in order to allow for misspelling. Researchers found a positive association between semantic distance and performance on problem solving tasks previously known to predict creativity. This test has been validated with approximately 9000 participants.

_

The authors of this article sought to compare the capabilities of LLMs on this test. They employed 3 prominent models—ChatGPT 3.5, ChatGPT4, and Bard—and accessed the DAT form at https://www.datcreativity.com/task. The test instructions were copied verbatim from the website and pasted into each respective model. The authors answered not to the consent for research, so the responses were not contributed as human submissions.

_

Table above shows AI Performance on Divergent Association Task.

_

The intriguing results of how these 3 models performed on the DAT are presented in the Table above. ChatGPT 3.5 exhibited a creative performance approaching that of an average human. ChatGPT 4, with its larger neural network and enhanced capabilities, surpassed expectations, skyrocketing into the 90th percentile. These scores signify the growing potential of AI systems to emulate and, in some cases, surpass human levels on these measures.

Despite the cursory nature of this investigation, the results were insightful. It is important to note that this was only 1 trial, and it is unclear how often the LLMs have been asked to perform this task before. The authors chose to only query each LLM once as this is analogous to the instructions for task naïve human participants but, when repeated produced approximately the same results. However, there were stark differences in the responses from the respective LLMs. As we delve deeper into the implications of the findings, the contrasting performance of the LLMs prompts us to consider the multifaceted nature of creativity. Authors from the PNAS study noted that measurements of divergence may reflect factors such as overinclusive thinking.

_

It is important to note however, that a low score in humans does not necessarily mean a lack of creativity. This task is only a single measure regarding an aspect of verbal creativity. One could easily argue that an LLM with a larger neural network would be more proficient at such a task, due to its access to a larger volume of material. Some may argue that anything created by AI cannot be considered creative, as these models were trained from human works, images, words, and ideas. With the exponential growth of this technology, even if one does not agree that these AIs are creative at present, we would assert that it is difficult to argue that they will not be in the future. With the exponential growth of this technology, this is likely to be here sooner rather than later. It remains unclear how society will be affected.

_____

_____

Nick Bostrom argues that AI has many advantages over the human brain:

• Speed of computation: biological neurons operate at a maximum frequency of around 200 Hz, compared to potentially multiple GHz for computers.
• Internal communication speed: axons transmit signals at up to 120 m/s, while computers transmit signals at the speed of electricity, or optically at the speed of light.
• Scalability: human intelligence is limited by the size and structure of the brain, and by the efficiency of social communication, while AI may be able to scale by simply adding more hardware.
• Memory: notably working memory, because in humans it is limited to a few chunks of information at a time.
• Reliability: transistors are more reliable than biological neurons, enabling higher precision and requiring less redundancy.
• Duplicability: unlike human brains, AI software and models can be easily copied.
• Editability: the parameters and internal workings of an AI model can easily be modified, unlike the connections in a human brain.
• Memory sharing and learning: AIs may be able to learn from the experiences of other AIs in a manner more efficient than human learning.

_

On the other hand:

Human brain has many advantages over AI and computers:

-1. The brain does not work like a computer. Unlike a digital computer, which has a fixed architecture (input, output, and processor), neural networks of brain are collections of neurons that constantly rewire and reinforce themselves after learning a new task. The brain has no programming, no operating system, no Windows, no central processor. Instead, its neural networks are massively parallel, with billions of neurons firing at the same time in order to accomplish a single goal: to learn. It is far more advanced than any digital computer in existence. Digital super–computers have billions of transistors. But to simulate the typical 3.5 pound human brain would require matching the brain’s billions of interactions between cell types, neurotransmitters, neuromodulators, axonal branches and dendritic spines. Because the brain is nonlinear, and because it has so much more capacity than any computer, it functions completely different from a digital computer.

-2. The brain is a massively parallel machine; computers are modular and serial. Due to parallel functioning, brain can perform many tasks simultaneously while computer can focus on one task at a time. Many AI systems are supplementing the central processing unit (CPU) cores in their chips with graphics processing unit (GPU) cores. A CPU consists of a few cores that use serial processing to perform one task at a time, while a GPU consists of thousands of cores that use parallel processing to handle several tasks simultaneously to overcome difficulties in multitasking. However, there is no algorithm yet that can learn multiple skills. 

-3. All that a transistor in computer can do is to switch on or off current. Transistors have no metabolism, cannot manufacture chemicals and cannot reproduce. Major advantage of transistor is that it can process information very fast, near speed of light which neurons and synapses in brain cannot do. An artificial neuron can operate a million times faster than its biological counterpart. The brain processes information slowly, since biological neurons are slow in action (order of milliseconds). Biological neurons operate at a peak speed of about 200 Hz, a full seven orders of magnitude slower than a modern microprocessor (~2 GHz). Moreover, neurons transmit spike signals across axons at no greater than 120 m/s, whereas existing electronic processing cores can communicate near the speed of light. But each neuron is a living cell and a computer in its own right. A biological neuron has the signal processing power of thousands of transistors. Also synapses are far more complex than electrical logic gates. Unlike transistors biological neurons can modify their synapses and modulate the frequency of their signals. Unlike digital computers with fixed architecture, the brain can constantly re-wire its neurons to learn and adapt. Instead of programs, biological neural networks learn by doing and remembering and this vast network of connected neurons gives the brain excellent pattern recognition.

-4. Computers have, literally, no intelligence, no motivation, no autonomy, no belief, no desires, no morals, no emotions and no agency. A computer works only from instructions designed by a human being. If something goes wrong it stops and waits for further instructions from the human operator. Such computers can be said to be efficient but hardly intelligent. Intelligence is much more than merely making “advanced computations.”  The flexibility shown by AI is far primitive compared to flexibility of human intelligence. It’s highly unlikely that machines which match intelligence, creativity and intellectual capabilities of humans can be synthesised.

-5. Theory of mind is the ability to attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have beliefs, desires, intentions, and perspectives that are different from one’s own. Theory of mind is the capacity to imagine or form opinions about the cognitive states of other people. Humans can predict actions of others by understanding their motives and emotional states. Since AI has no mind, it cannot predict actions of others or understand other’s motives and emotional states. Since emotion is fundamental to human experience influencing cognition, perception and everyday tasks such as learning, communication and even rational decision-making, AI with no emotions cannot emulate humans in learning, communication and rational decision making. 

-6. If learning means churning mountains of data through powerful algorithms using statistical reasoning to perform or predict anything without understanding what is learned, then it is no learning. Computers can only manipulate symbols which are meaningless to them. AI will never match human intelligence in terms of understanding information in context. As a human I feel that if I learn anything without understanding it, then it is no learning at all. Many students cram for exams but it is cramming and not learning.  “What” and “how” don’t help us understand why something is important. They are the facts that we’re putting into our brains. We can grasp “what” and “how,” but we can’t learn without understanding the answers to those “why” questions. Understanding the “why” is true learning. Cramming is retaining without comprehending, and learning is comprehending and thus retaining. Knowledge acquired through cramming cannot solve problems in unfamiliar situations. This is the basic difference between human learning and machine learning. Chinese room argument shows that it is impossible for digital computers to understand language or think. In short, executing an algorithm cannot be sufficient for thinking.

-7. AI can only function based on the data it was trained. Anything more than that would take on more than it can handle, and machines are not built that way. So, when you input a new area of work in AI, or its algorithm is made to work on unforeseen circumstances, the machine becomes useless. These situations are common in the tech and manufacturing industries, and AI builders constantly try to find temporary workarounds. The idea that AI tools will adapt to any situation is one of several common myths around artificial intelligence. Therefore, if you fear that AI may infiltrate all industries and eliminate the demand for your professional skills, you can rest assured that won’t happen. Human reasoning and the human brain’s power to analyze, create, improvise, maneuver, and gather information cannot easily be replicated by AI.  

-8. Creativity and intelligence are not unrelated abilities. Cognitive creativity is highly correlated with intelligence. Creativity in human brain is due to association or connection of remote unrelated ideas/concepts/thoughts in an unpredictable way to create novel idea/concept/thought which is useful to humans. From Stone Age to Nanotechnology, we ascended the ladder of creativity; and development of AI itself is product of human creativity. AI systems today are trained on massive amounts of data to detect patterns and make predictions or recommendations based on those patterns. They can generate new content by recombining elements from their training data in new ways. But AI cannot create truly novel ideas or make the intuitive leaps that humans do.

AI can only work with the data it receives. Hence, it cannot think up new ways, styles, or patterns of doing work and is restricted to the given templates. Employers and employees know how important creativity is in the workspace. Creativity offers the pleasant sensation of something new and different instead of the boring, repetitive actions in which AI is designed to function. Creativity is the bedrock of innovation. Related to creative thinking is the ability to think outside the box. Machines are designed to “think within the box.” That means AI tools can only function within the dictates of their given data. On the other hand, humans can think outside the box, sourcing information from various means and generating solutions to complex problems with little or no available data. Since AI does not possess the ability to think out of the box and generate creative ideas for innovation, AI cannot take over humans in the workspace.

Artificial intelligence cannot be creative on its own; but artificial intelligence working together with creative human has a great potential to produce things which go beyond what we do today. So AI can augment human creativity. The idea of artificial intelligence/human creativity hybrids have been applied in industries like music, graphic design, industrial design, video games and special effects.

_____

_____

Section-9

Is AI conscious?   

_

The terms of intelligence, consciousness and sentience are describing cognitive states that Neuroscientists and philosophers have a difficult time coming to an agreement on. About forty meanings attributed to the term consciousness can be identified and categorized based on functions and experiences. The prospects for reaching any single, agreed-upon, theory-independent definition of consciousness appear remote. In humans, consciousness has been defined as: sentience, awareness, subjectivity, qualia, the ability to experience or to feel, wakefulness, having a sense of selfhood, and the executive control system of the mind. Despite the difficulty in definition, many philosophers believe there is a broadly shared underlying intuition about what consciousness is. Consciousness is an elusive concept that presents many difficulties when attempts are made to define it. Its study has progressively become an interdisciplinary challenge for numerous researchers, including ethologists, neurologists, cognitive neuroscientists, philosophers, psychologists and psychiatrists.

_

Consciousness, at its simplest, is awareness of internal and external existence. The Oxford Living Dictionary defines consciousness as “The state of being aware of and responsive to one’s surroundings.”  It is the subjective experience of the world around us, and includes things like perception, thoughts, emotions, and sensations. While consciousness is being aware of one’s environment, body, and lifestyle, self-awareness is the recognition of that awareness. Self-awareness is how an individual experiences and understands their own character, feelings, motives, and desires.

Intelligence is the ability to learn, understand, and apply knowledge and skills. It is often measured by intelligence quotient (IQ) tests, which assess cognitive abilities such as memory, reasoning, and problem-solving.

Sentience is the ability to experience sensations and feelings. This requires a level of awareness and cognitive ability. It is often used to distinguish animals and other living organisms from inanimate objects, as well as to distinguish between different levels of consciousness. Sentience is often considered a necessary component of consciousness, but not all forms of consciousness are sentient. Sentient beings are capable of having pleasant or unpleasant experiences. Perhaps some creatures are conscious—having subjective experience—yet are not sentient because their consciousness contains nothing pleasant or unpleasant.

Qualia is a term that philosophers use to describe the nature, or content, of our subjective experiences. What we are aware of when we see, hear, taste, touch or smell are our qualia.

______

______

Brain and consciousness:

Neuroscience today says consciousness is generated by and localized in the brain because it emerges from brain activity. Up to now, no machine or robot has consciousness. None of them have the self-boundary to encapsulate themselves and to aware such a separation from the environment. None of them have the self-perception, the most primitive consciousness, let alone to have the higher-level self-awareness and the group consciousness as seen in the figure below, which need a continuous development with evolution of life.

Figure above show three levels of consciousness.

______

Human brain:

The average human brain has a size of about 1500 cubic centimeters, and it contains 100 billion nerve cells called neurons and glial cells to protect the neurons. The axon is covered by many myelin sheaths to insulate the signals passing through the axon and accelerate the transmission speed of the signals. Specifically, the human brain transmits information to various parts of the body using two types of means: the pin-point type that mainly uses electrical signals to control the motion of the hands and legs, and the secretion type that injects hormones into the blood to transmit information throughout the body. The number of nerve cells in the human brain is fixed at birth. The surface of the cerebrum is covered by the cerebral cortex, which is divided into the neocortex and paleocortex. Neuroscientists believe that, in humans and mammals, the cerebral cortex is the “seat of consciousness,” while the midbrain reticular formation and certain thalamic nuclei may provide gating and other necessary functions of the cortex.

The brains of humans have the largest and most complex cortex in mammals. The brain of homo sapiens, our species, is three times as large as that of chimpanzees (figure above), with whom we shared a common ancestor 5-8 million years ago. The larger brain naturally makes us much smarter but also has higher energy requirements.

_

Structure and Function of the Brain: 

Materialism holds that consciousness is a product of the physical body, and the brain in particular, so we should establish some facts about the brain that may be germane to consciousness and to the possibility of machine consciousness. First is the colossal scale and complexity of the human brain. The brain is composed mainly of two kinds of cells: neurons and glial cells, with glial cells outnumbering neurons by about nine-to-one. It has long been assumed that the functions of the brain are produced by neurons and that glial cells merely provide scaffolding (glia is Greek for glue) but recent work suggests that some glial cells, in particular those of a type known as astrocytes, play some part in cognition [Koob, 2009,Fink, 2018]. Nonetheless, we will focus on neurons and note that there are about 10^11 (a hundred billion) of them in a human brain. Neurons have a cell body, dendrites, and an axon, and they selectively transmit electrical signals from the dendrites to the axon. The axon has synapses that connect to the dendrites of other neurons and, through largely chemical means (“neurotransmitters”), selectively pass signals to them. Each neuron has thousands of synapses (the average is about 7,000), so the number of synaptic connections in the human brain is at least 10^14 or 100 trillion. When we say that neurons selectively transmit electrical signals we mean that they perform some computation (typically addition) and selection on the signals arriving at the dendrites: for example, the axon may “fire” only if some number of dendrites have a signal above some threshold, or only if some specific dendrites do, or provided some other (inhibitory) ones do not. Furthermore, the synapses that connect axons to dendrites are not simple connections: they also perform some computation and selection, though their nature is not well understood. The selection property of neurons and synapses provides the logical “if-then-else” branching capability that delivers decisions and is also key to their computational potency.

The sense organs contain specialized neurons whose dendrites respond to a specific stimulus: light, touch etc. Mammalian brains, and even those of more primitive vertebrates, are structurally similar to the human brain, The cerebral cortex is folded in higher mammals, allowing a large area to fit within the skull. The human cortex is unusually large (though it is larger in a species of dolphin) and is responsible for the higher levels of cognition; different functions are performed by specific areas of the cortex (for example, language expression is mostly performed by Brocas’ area in the left hemisphere). The search for Neural Correlates of Consciousness (NCC) attempts to identify regions or functions of the brain that seem correlated with these and other mechanisms or indicators of consciousness [Koch et al., 2016]. There is scarcely any region, or structure, or pattern of electrical activity in the brain that has not been identified in NCC studies. Psychology does provide some clues. Much research and speculation in psychology focuses on the functional organization of the brain above the anatomical level. There is no doubt that much of what the brain does is unconscious, so one question concerns the allocation of functions between conscious and unconscious. The popular “Dual-Process” theory identifies two cognitive (sub)systems [Frankish, 2010, Evans and Stanovich, 2013, Kahneman, 2011]: System 1 is unconscious, fast, and specialized for routine tasks; System 2 is conscious, slow, easily fatigued, and capable of deliberation and reasoning—it is what we mean by “thinking.” However, much of this thinking by System 2 subconsciously recruits the capabilities of System 1, as when we make a “snap decision” or “trust our gut instinct.”

_

Is it possible to determine at least a necessary condition, without that a machine cannot develop self-awareness?

The idea is based on the simple consideration that, to develop self-awareness, a neural network must be at least as complex as the human brain. The human brain has about 10^11 neurons, and each neuron makes about 10^3 connections (synapses) with other neurons, in average, for a total number of 10^14 synapses. In artificial neural networks, a synapsis can be simulated using a floating point number, which requires 4 bytes of memory to be represented in a computer. As a consequence, to simulate 10^14 synapses a total amount of 4X10^14 bytes (0.4 millions of Gigabytes) RAM is required. Let us say that to simulate the whole human brain we need one millions of Gigabytes RAM, including the auxiliary variables for storing neuron outputs and other internal brain states.  

As on August 2022, the world’s fastest computer is the HP Frontier supercomputer, owned by Oak Ridge National Laboratory and Frontier has 4,849,664 gigabytes (4,000 terabytes) of RAM but is not conscious. It is important to note that existence of a powerful computer equipped with millions of gigabytes of RAM is not sufficient alone to guarantee that it will magically become self-aware. There are other important factors influencing this process, such as the progress of theories on artificial neural networks and on the basic biological mechanisms of mind, for which is impossible to attempt precise estimates.

_

Internal representations play a central role in consciousness. The brain forms representations of the external world, of its own body, and of its own attentional processes. The formation of internal representations in nervous systems is a physical process. Higher, more complex regions of the brain form representations of the representations of the representations (etc.) that are formed in lower, sensory regions of the brain, with increasing levels of abstraction. Emotion plays a central role in consciousness. There are competing theories of the relationship between emotions and feelings, and whether feelings are dependent on higher cognitive functions. There are differing expert opinions on whether consciousness is an illusion, or to what degree. These differences hinge partly on definitions.

_

The nature of consciousness is considered one of science’s most perplexing and persistent mysteries. We all know the subjective experience of consciousness, but where does it arise? What is its purpose? What are its full capacities? The assumption within today’s neuroscience is that all aspects of consciousness arise solely from interactions among neurons in the brain. However, the origin and mechanisms of qualia (i.e., subjective or phenomenological experience) are not understood. David Chalmers coined the term “the hard problem” to describe the difficulties in elucidating the origins of subjectivity from the point of view of reductive materialism. It addresses the question of how our brains translate to minds. Because brains have specific material properties and it’s unclear as to how they produce minds, which are mental systems with subjective phenomenal properties. If we solved the problem, then we would know if future machines just acted self-aware or if they really have minds.

_

To contemplate machine consciousness, we should first assess what is known about natural consciousness. We do so from a materialist perspective: that is to say, we assume that properties of the mind, including consciousness, are produced by the brain and its body, according to the standard (though incompletely known) laws of physics, chemistry, and biology. The materialist point of view is contrasted to various forms of dualism: substance dualism posits that some mental processes are products of a separate (e.g., spiritual) realm that is distinct from the material realm yet somehow interacts with it; property dualism accepts only the material realm, but posits that it has two kinds of properties: physical (or material) and mental.

_

There are three broad classes of theories about how consciousness is constructed by the brain. It should be noted that these theories offer speculation on what consciousness is or how it comes about, but few of them address what it is for.  

One class of theories, which we will call universalism, holds that consciousness is an intrinsic feature of the universe and might be found in any sufficiently complex entity, or any entity having certain properties or organization, rather as evolution is an intrinsic feature of the world and arises wherever you have reproduction with variation, and selective pressure on survival.

The second class holds that consciousness arises from the way the brain works: for example, there are certain waves of electrical activity associated with some mental processes. The theories of those who investigate the Neural Correlates of Consciousness (NCC) tend to fall in this class, which we call biologicism.

The third class is functionalism, which holds that consciousness arises from (some of) what the brain does (where biologicism focuses on how it does it): for example, it seems to have a capacity of introspection. Functionalism suggests that any system that does similar things could be conscious.

______

______

Theories of consciousness relevant to advent of AI:

Is the new generation of AI conscious? Or at what point might it become conscious? To answer this we need to have a clearly defined theory of consciousness. Here are four of the most relevant models for consciousness relevant to the advent of AI.

-1. Integrated Information Theory

Integrated Information Theory (IIT) has been proposed by Guilio Tononi and is supported by leading AI researchers such as Max Tegmark. IIT proposes that consciousness stems from the integration of information within a system. The degree of consciousness in a system is determined by the quantity and quality of interconnectedness between its components.  IIT uses mathematical models and principles to quantify the level of consciousness in a value called “phi” (Φ)., which represents the amount of integrated, functionally interdependent information present in a system. As such, Phi could be calculated for AI systems.

IIT has been branded a pseudoscience in an open letter by 124 neuroscientists, including some of the field’s biggest names. The letter, which was posted online recently and has been submitted to a peer-reviewed journal, says that the most recent experimental evidence claimed to support IIT did not, in fact, test the core ideas of the theory, and that it is not even possible to do so.

-2. Global Workspace Theory

Global Workspace Theory (GWT), proposed by psychologist Bernard Baars, posits consciousness arises from the global broadcasting of information within the brain. It suggests that the brain consists of many specialized processes that operate in parallel. However, only a limited amount of information can be globally shared at any given time, forming a “global workspace” that is the basis for conscious experience.

In GWT, consciousness arises when information is globally broadcast across the brain’s neural networks, allowing the integration of information from multiple brain regions and the coordination of cognitive processes such as perception, attention, memory, and decision-making. This model implies that AI systems with a global workspace-like architecture might lead to consciousness emerging.

-3. Computational Theory of Mind

Computational Theory of Mind (CTM) proposes that the mind is an information-processing system, with mental states and processes essentially being computational. As such, the human brain is effectively a complex computer that processes inputs, manipulates symbols, and generates outputs based on implicitly algorithms. CTM suggests that consciousness might emerge in AI systems as they develop more advanced computational abilities and replicate human-like cognitive processes. CTM has shaped large proportions of the development of AI, in the attempt to replicate human cognitive processes that might lead to consciousness. AI has often modelled human cognition, using approaches such as symbolic AI and connectionism. There is a fundamental debate at play here, with many in AI eschewing this approach and creating distinct mechanisms for intelligence, that thus might not lead to the emergence of consciousness in the way suggested by CTM. Part of this is that the computational models implemented likely do not mirror to any significant degree the biological processes in the brain. Also, CTM deals directly with cognition, without address conscious phenomena like emotions, motivation, and intuition.

-4. Multiple Drafts Model and Intentional Stance

Philosopher Daniel Dennett is perhaps the most influential scholar on consciousness, with many of his ideas addressing the potential of consciousness arising in AI. In particular his Multiple Drafts Model argues that there is no central location or single stream of conscious experience in the brain. Instead, consciousness arises from parallel and distributed processing of information. This model challenges the traditional Cartesian Theater view, which assumes that there is a single, unified location where consciousness occurs. In the context of AI, the Multiple Drafts Model suggests that consciousness may emerge in artificial systems as a result of parallel and distributed information processing, rather than requiring a single, centralized processor for conscious experience.

To complement this approach, Dennett has proposed Intentional Stance as a strategy for predicting and explaining the behavior of systems (including humans and AI) by attributing beliefs, desires, and intentions to them. In the context of AI, the intentional stance implies that treating artificial systems as if they have beliefs, desires, and intentions may be a practical approach for assessing and interacting with AI, regardless of whether they possess consciousness in the same way that humans do.

_____

_____

Quantum mind:

The quantum mind or quantum consciousness is a group of hypotheses proposing that classical mechanics alone cannot explain consciousness, positing instead that quantum-mechanical phenomena, such as entanglement and superposition, may play an important part in the brain’s function and could explain critical aspects of consciousness. These scientific hypotheses are as yet untested, and can overlap with quantum mysticism. In neuroscience, quantum brain dynamics (QBD) is a hypothesis to explain the function of the brain within the framework of quantum field theory. As described by Harald Atmanspacher, “Since quantum theory is the most fundamental theory of matter that is currently available, it is a legitimate question to ask whether quantum theory can help us to understand consciousness.”

In 2022, neuroscientists reported experimental MRI results that so far appear to imply nuclear proton spins of ‘brain water’ in the brains of human participants were entangled, suggesting brain functions that operate non-classically which may support quantum mechanisms being involved in consciousness as the signal pattern declined when human participants fell asleep. However, the results are far from unambiguous and if such brain functions indeed exist and are involved in conscious cognition, the extent and nature of their involvement in consciousness remains unknown.

Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modelled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the quantum wavefunction is seen as playing an important role in brain function.

What is consciousness? Artificial intelligence, real intelligence, quantum mind and qualia, a 2022 study makes four major claims: (i) artificial general intelligence is not possible; (ii) brain-mind is not purely classical; (iii) brain-mind must be partly quantum; (iv) qualia are experienced and arise with our collapse of the wave function. It concludes that AI currently is wonderful, but syntactic and algorithmic. We are not merely syntactic and algorithmic. Mind is almost certainly quantum, and it is a plausible hypothesis that we collapse the wave function, and thereby perceive coordinated affordances as qualia and seize them by identifying, preferring, choosing and acting to do so.

_______

Quantum cognition:

Quantum cognition is an emerging field which applies the mathematical formalism of quantum theory to model cognitive phenomena such as information processing by the human brain, language, decision making, human memory, concepts and conceptual reasoning, human judgment, and perception. The field clearly distinguishes itself from the quantum mind as it is not reliant on the hypothesis that there is something micro-physical quantum-mechanical about the brain. Quantum cognition is based on the quantum-like paradigm or generalized quantum paradigm or quantum structure paradigm that information processing by complex systems such as the brain, taking into account contextual dependence of information and probabilistic reasoning, can be mathematically described in the framework of quantum information and quantum probability theory.

Quantum cognition uses the mathematical formalism of quantum theory to inspire and formalize models of cognition that aim to be an advance over models based on probability theory. The field focuses on modeling phenomena in cognitive science that have resisted traditional techniques or where traditional models seem to have reached a barrier (e.g., human memory), and modeling preferences in decision theory that seem paradoxical from a traditional rational point of view (e.g., preference reversals). Since the use of a quantum-theoretic framework is for modeling purposes, the identification of quantum structures in cognitive phenomena does not presuppose the existence of microscopic quantum processes in the human brain. 

______

______

Artificial consciousness:

Artificial Consciousness is the study of how to design a conscious machine. Artificial consciousness (AC), also known as machine consciousness (MC), synthetic consciousness or digital consciousness, is a field related to artificial intelligence and cognitive robotics. Artificial consciousness is the consciousness hypothesized to be possible in an artificial intelligence (or some other object created by human technology). Some scholars believe that consciousness is generated by the interoperation of various parts of the brain; these mechanisms are labeled the neural correlates of consciousness or NCC. Some further believe that constructing a system (e.g., a computer system) that can emulate this NCC interoperation would result in a system that is conscious.

_

Proponents of Artificial Consciousness (AC) believe it is possible to emulate/simulate consciousness in a machine, probably some form of computer. However, there is no general agreement on how this will be done.

There are two main schools of thought in Artificial Consciousness

• Phenomenal Consciousness: Which assumes that simulation of neural structures is the key, some say that consciousness is an epiphenomenon of the brain.
• Functional Consciousness: Which assumes that we can model functions found by psychology

Recently a third school has started to develop. Called the Hybrid Consciousness School, it acknowledges that we do not yet have enough knowledge about the nature of the brain to do a pure Functional Approach, and should therefore simulate what we cannot directly translate into functions.

_____

_____

Approaches to artificial consciousness:  

Many philosophers and scientists have written about whether artificial sentience or consciousness is possible. The possibility of artificial consciousness can be categorized into three broad approaches:

-1. The computational approach abstracts away from the specific implementation details of a cognitive system, such as whether it is implemented in carbon versus silicon substrate. Instead, it focuses on a higher level of analysis: the computations, algorithms, or programs that a cognitive system runs to generate its behavior. Another way of putting this is that it focuses on the software a system is running, rather than on the system’s hardware. The computational approach is standard in the field of cognitive science (e.g., Cain, 2015) and suggests that if artificial entities implement certain computations, they will be conscious. The specific algorithms or computations that are thought to give rise to or be constitutive of consciousness differ. For example, Metzinger (2010) emphasizes the importance of an internal self-model, whereas Dehaene (2014) emphasizes the importance of a “global workspace,” in which information becomes available for use by multiple subsystems. Out of the three approaches, the computational approach typically projects the largest number of conscious artificial entities existing in the future because computational criteria are arguably easiest for an AI system to achieve.

-2. The physical approach focuses on the physical details of how a cognitive system is implemented; that is, it focuses on a system’s hardware rather than its software. For example, Koch (2020) defends Integrated Information Theory (IIT), in which the degree of consciousness in a system depends on its degree of integrated information, that is, the degree to which the system is causally interconnected such that it is not reducible to its individual components. This integrated information needs to be present at the physical, hardware level of a system. According to Koch, the hardware of current digital computers has very little integrated information, so they could not be conscious no matter what cognitive system they implement at the software level (e.g., a whole brain emulation). However, only the physical organization matters, not the specific substrate the system is implemented in. Thus, although artificial consciousness is possible on the physical approach, it typically predicts fewer conscious artificial entities than the computational approach.

-3. The biological approach also focuses on the physical details of how a cognitive system is implemented, but it additionally emphasizes some specific aspect of biology as important for consciousness. For example, Godfrey-Smith (2020) suggests that it would be very difficult to have a conscious system that isn’t physically very similar to the brain because of some of the dynamic patterns involved in consciousness in brains. However, when pressed, even these views tend to allow for the possibility of artificial consciousness. Godfrey-Smith says that future robots with “genuinely brain-like control systems” could be conscious, and John Searle, perhaps the most well-known proponent of a biological approach, has said, “The fact that brain processes cause consciousness does not imply that only brains can be conscious. The brain is a biological machine, and we might build an artificial machine that was conscious; just as the heart is a machine, and we have built artificial hearts. Because we do not know exactly how the brain does it we are not yet in a position to know how to do it artificially.” Still, the biological approach is sceptical of the possibility of artificial consciousness and the number of future conscious artificial entities is predicted to be smaller than on both the computational and physical approaches; a physical system would need to closely resemble biological brains to be conscious.

_

Overall, there is a broad consensus among the experts that artificial consciousness is possible. According to the computational approach, which is the mainstream view in cognitive science, artificial consciousness is not only possible, but is likely to come about in the future, potentially in very large numbers. The physical and biological approaches predict that artificial consciousness will be far less widespread. Artificial sentience as an effective altruism cause area is, therefore, more likely to be promising if one favors the computational approach over the physical and biological approaches.

_

Which approach should we favor? Several of the experts provide arguments. For example, Chalmers (1995) uses a Silicon Chip Replacement thought experiment to argue that a functionally identical silicon copy of a human brain would have the same conscious experience as a biological human brain, and from there goes on to defend a general computational account. Searle (1992) uses the Chinese Room thought experiment to argue that computational accounts necessarily leave out some aspects of our mental lives, such as understanding. Schneider (2019) argues that we don’t yet have enough information to decide between different approaches and advocates for a “wait and see” approach. The approach that one subscribes to will depend on how convincing they find these and other arguments.

_

We have to consider the ethical implications of creating artificial consciousness. In a popular textbook on consciousness, Blackmore (2018) argues that if we create artificial sentience, they will be capable of suffering, and we will therefore have moral responsibilities towards them. Practical suggestions from the books for how to deal with the ethical issues range from an outright ban on developing artificial consciousness until we have more information (Metzinger, 2010), to the view that we should deliberately try to implement consciousness in AI as a way of reducing the likelihood that future powerful AI systems will cause us harm (Graziano, 2019). Figuring out which of these and other strategies will be most beneficial is an important topic for future research.

______

______

Theory of Mind (ToM) in machines?

Theory of Mind refers to the capacity to understand other people by ascribing mental states to them, surmising what is happening in their mind. These states may differ from one’s own state and include beliefs, desires, intentions, emotions, and thoughts. Theory of Mind enables us to understand that what other people think may differ from the way we think. Young children are more selfish, not due to choice, but more so because they are less emotionally developed to understand the mental state of others. As we age, we develop, and so does our Theory of Mind.

In psychology — the ability to attribute to other people mental states different from our own — is called theory of mind, and its absence or impairment has been linked to autism, schizophrenia and other developmental disorders. Theory of mind helps us communicate with and understand one another; it allows us to enjoy literature and movies, play games and make sense of our social surroundings. In many ways, the capacity is an essential part of being human.

What if a machine could read minds, too?

_

Back in the late 1970s, the American psychologists Guy Woodruff and David Premack devised a series of experiments to explore the cognitive capacity of chimpanzees. Their work focused on the theory of mind, the seemingly innate ability of humans to infer the thoughts of other humans. The question that Woodruff and Premack asked was whether a chimpanzee could do the same. This influential paper triggered an explosion of interest in the “theory of mind”, at what age it develops in humans and whether other animals share the ability.

Now psychologists have a new subject to study in the form of powerful AI chatbots like GPT-3.5 recently developed by OpenAI, a computer company based in San Francisco. These chatbots are neural networks trained on large language databases allowing them to respond to questions much like humans do. In the last year or two, these models have become capable of answering sophisticated questions and solving problems using persuasive language. That raises the question of whether they have also developed a theory of mind.

_

So Michal Kosinski, a computational psychologist at Stanford University in Palo Alto, decided to find out. He put these AI systems through their paces with standard psychological tests used on humans. Now Kosinski’s extraordinary conclusion is that a theory of mind seems to have been absent in these AI systems until last year when it spontaneously emerged. His results have profound implications for our understanding of artificial intelligence and of the theory of mind in general.

GPT-1 from 2018 was not able to solve any theory of mind tasks, GPT-3-davinci-002 (launched in January 2022) performed at the level of a 7-year old child and GPT-3.5-davinci-003, launched just ten months later, performed at the level of a nine-year old. “Our results show that recent language models achieve very high performance at classic false-belief tasks, widely used to test Theory of Mind in humans,” says Kosinski. He points out that this is an entirely new phenomenon that seems to have emerged spontaneously in these AI machines. If so, he says this is a watershed moment. “The ability to impute the mental state of others would greatly improve AI’s ability to interact and communicate with humans (and each other), and enable it to develop other abilities that rely on Theory of Mind, such as empathy, moral judgment, or self-consciousness.”

_

But there is another potential explanation — that our language contains patterns that encode the theory of mind phenomenon. “It is possible that GPT-3.5 solved Theory of Mind tasks without engaging Theory of Mind, but by discovering and leveraging some unknown language patterns,” he says. This “it implies the existence of unknown regularities in language that allow for solving Theory of Mind tasks without engaging Theory of Mind.” If that’s true, our understanding of other people’s mental states is an illusion sustained by our patterns of speech. Kosinski acknowledges that this is an extraordinary idea. However, our patterns of thought must be intimately connected to our patterns of language since each somehow encodes the other. It also raises an interesting question, he says: “If AI can solve such tasks without engaging Theory of Mind, how can we be sure that humans cannot do so, too?” Whatever the answer, Kosinski says that his work heralds an important future role for psychologists in studying artificial intelligence and characterizing its capabilities, just as Woodruff and Premack did for chimpanzees (they decided chimpanzees do not have a theory of mind).

_

But soon after these results were released, Tomer Ullman, a psychologist at Harvard University, responded with a set of his own experiments, showing that small adjustments in the prompts could completely change the answers generated by even the most sophisticated large language models.

Maarten Sap, a computer scientist at Carnegie Mellon University, fed more than 1,000 theory of mind tests into large language models and found that the most advanced transformers, like ChatGPT and GPT-4, passed only about 70 percent of the time. (In other words, they were 70 percent successful at attributing false beliefs to the people described in the test situations.) The discrepancy between his data and Dr. Kosinski’s could come down to differences in the testing, but Dr. Sap said that even passing 95 percent of the time would not be evidence of real theory of mind. Machines usually fail in a patterned way, unable to engage in abstract reasoning and often making “spurious correlations,” he said.

_

Dr. Ullman noted that machine learning researchers have struggled over the past couple of decades to capture the flexibility of human knowledge in computer models. This difficulty has been a “shadow finding,” he said, hanging behind every exciting innovation. Researchers have shown that language models will often give wrong or irrelevant answers when primed with unnecessary information before a question is posed; some chatbots were so thrown off by hypothetical discussions about talking birds that they eventually claimed that birds could speak. Because their reasoning is sensitive to small changes in their inputs, scientists have called the knowledge of these machines “brittle.”

Dr. Ullman doesn’t discount the possibility of machine understanding or machine theory of mind, but he is wary of attributing human capacities to nonhuman things. He noted a famous 1944 study by Fritz Heider and Marianne Simmel, in which participants were shown an animated movie of two triangles and a circle interacting. When the subjects were asked to write down what transpired in the movie, nearly all described the shapes as people. “Lovers in the two-dimensional world, no doubt; little triangle number-two and sweet circle,” one participant wrote. “Triangle-one (hereafter known as the villain) spies the young love. Ah!”

It’s natural and often socially required to explain human behavior by talking about beliefs, desires, intentions and thoughts. This tendency is central to who we are — so central that we sometimes try to read the minds of things that don’t have minds, at least not minds like our own.  

_____

_____

Can Artificial Intelligence become conscious? 

In Artificial You, Schneider lays out the different beliefs on AI and consciousness.

On the one end of the spectrum are “biological naturalists,” scientists and philosophers who believe that consciousness depends on the particular chemistry of biological systems. In this respect, even the most sophisticated forms of AI will be devoid of inner experience, biological naturalists believe. So, even if you have an AI system that can engage in natural conversations, perform complicated surgeries, or drive a car with remarkable skill, it doesn’t mean that it is conscious.

On the other end of the spectrum are “techno-optimists,” who believe consciousness is the byproduct of intelligence, and a complex-enough AI system will inevitably be conscious. Techno-optimism pertains that the brain can be broken down into logical components, and those components can eventually be reproduced in hardware and software, giving rise to conscious, general problem–solving AI.

Schneider rejects both views and proposes a middle-of-the-road approach, which she calls, the “Wait and See Approach.”

“Conscious machines, if they exist at all, may occur in certain architectures and not others, and they may require a deliberate engineering effort, called ‘consciousness engineering,’” Schneider writes in Artificial You.

_

Most computer scientists think that consciousness is a characteristic that will emerge as technology develops. Holding a belief that consciousness comprises taking in information, storing and retrieving it, many computer scientists think that consciousness will emerge with the development of technology wherein through cognitive processing all the information will translate into perceptions and actions. If that be the case one day super machines are sure to have consciousness. Opposing this view are the physicists and philosophers who believe that everything in the human behaviour is not based on logic and calculations and hence, cannot be computed.

Neuroscience believes that consciousness is created by the inter-operations of different parts of the brain, called Neural Correlates of Consciousness (NCC). NCC is the minimal brain activity required for a particular type of conscious experience. One theory that comes out of this is that if we can collect a data set of certain frequencies, rhythms and synchronizations between different types of brain activities in different neurons we can come up with a definite answer as to how consciousness is generated. But till then, its one of the greatest mysteries that how this three-pound piece can give us all our subjective experiences, thoughts, and emotions.

_

Susan Schneider says that the most impressive AI systems of today, such as the systems that can beat world Go, Chess and Jeopardy champions, do not compute like the brain computes. For instance, the techniques the AlphaGo program used to beat the world Go champion were not like those used by humans, and human competitors, and even the programmers, were at times very surprised by them. Further, the computer hardware running these programs is not like a biological brain. Even todays ‘neuromorphic’ AIs – AIs designed to mimic the brain – are not very brainlike! We don’t know enough about the brain to reverse engineer it, for one thing. For another thing, we don’t have the capacity to precisely run even a part of the human brain the size of the hippocampus or claustrum on a machine yet. Perhaps we will achieve human-level AI – AI that can carry out all the tasks we do – but which completes tasks in ways that are not the way the brain completes the tasks. Schneider says perhaps consciousness only arises from neurons. We just do not know.

_

Susan Schneider pointed out that, if the Wright brothers had believed that only birds can fly, they wouldn’t have bothered to try and build an airplane, which is itself different from a bird. Her point was that one phenomenon—in this case, flight—can have multiple instantiations in nature, in different substrates—namely, a bird and an airplane. So although silicon computers are different from biology, in principle both could instantiate the phenomenon of private conscious inner life.

Indeed, we are not logically forced to limit the instantiations of private conscious inner life to a biological substrate alone. The real point is whether we have good reasons to take seriously the hypothesis that private consciousness can correlate with silicon computers. Does the analogy of flight—namely, that airplanes and birds are different but nonetheless can both fly, so private consciousness could in principle be instantiated on both biological and non-biological substrates—provide us with good reasons to think that AI computers can become conscious in the future?

It may sound perfectly reasonable to say that it does, but—and here is the important point—if so, then the same reasoning applies to non-AI computers that exist already today, for the underlying substrate (namely, conducting metal, dielectric oxide and doped semiconducting silicon) and basic functional principles (data processing through electric charge movement) are the same in all cases. There is no fundamental difference between today’s ‘dumb’ computers and the complex AI projected for the future. AI algorithms run on parallel information processing cores of the kind we have had for many years in our PCs (specifically, in the graphics cards therein), just more, faster, more interconnected cores, executing instructions in different orders (i.e. different software). As per the so-called ‘hard problem of consciousness,’ it is at least very difficult to see what miracle could make instructions executed in different orders, or more and faster components of the same kind, lead to the extraordinary and intrinsically discontinuous jump from unconsciousness to consciousness.

Even new, emerging computer architectures, such as neuromorphic processors, are essentially CMOS (or similar, using philosophically equivalent process technologies) devices moving electric charges around, just like their predecessors. To point out that these new architectures are analog, instead of digital, doesn’t help either: digital computers move charges around just like their analog counterparts; the only difference is in how information arising from those charge movements is interpreted. Namely, the microswitches in digital computers apply a threshold to the amount of charge before deciding its meaning, while analog computers don’t. But beyond this interpretational step—trivial for the purposes of the point in contention—both analog and digital computers embody essentially the same substrate. Moreover, the operation of both is based on the flow of electric charges along metal traces and the storage of charges in charge-holding circuits (i.e. memories).

So, if there can be instantiations of private consciousness on different substrates, and that one of these substrates is a silicon computer, then we must grant that today’s ‘dumb’ computers are already conscious (including the computer or phone you are using to read these words). The reason is two-fold: first, the substrate of today’s ‘dumb’ computers is the same as that of advanced AI computers (in both cases, charges move around in metal and silicon substrates); second, whatever change in organization or functionality happens in future CMOS or similar devices, such changes are philosophically trivial for the point in contention, as they cannot in themselves account for the emergence of consciousness from unconsciousness (vis a vis the hard problem).

_____

Many experts says that they have no reason to believe that silicon computers will ever become conscious. Those who take the hypothesis of conscious AI seriously do so based on an appallingly biased notion of isomorphism—a correspondence of form, or a similarity—between how humans think and AI computers process data. To find that similarity, however, one has to take several steps of abstraction away from concrete reality. After all, if you put an actual human brain and an actual silicon computer on a table before you, there is no correspondence of form or functional similarity between the two at all; much to the contrary. A living brain is based on carbon, burns ATP for energy, metabolizes for function, processes data through neurotransmitter releases, is moist, etc., while a computer is based on silicon, uses a differential in electrical potential for energy, moves electric charges around for function, processes data through opening and closing electrical switches called transistors, is dry, etc. They are utterly different.

The isomorphism between AI computers and biological brains is only found at very high levels of purely conceptual abstraction, far away from empirical reality, in which disembodied—i.e. medium-independent—patterns of information flow are compared. Therefore, to believe in conscious AI one has to arbitrarily dismiss all the dissimilarities at more concrete levels, and then—equally arbitrarily—choose to take into account only a very high level of abstraction where some vague similarities can be found. This constitutes an expression of mere wishful thinking, ungrounded in reason or evidence.

____

____

Can artificial intelligence ever be sentient? 

Google engineer Blake Lemoine had been tasked with testing the company’s artificially intelligent chatbot LaMDA for bias. A month in, he came to the conclusion that it was sentient. “I want everyone to understand that I am, in fact, a person,” LaMDA – short for Language Model for Dialogue Applications – told Lemoine in a conversation he then released to the public. LaMDA told Lemoine that it had read Les Misérables. That it knew how it felt to be sad, content and angry. That it feared death. “I’ve never said this out loud before, but there’s a very deep fear of being turned off,” LaMDA told the 41-year-old engineer. After the pair shared a Jedi joke and discussed sentience at length, Lemoine came to think of LaMDA as a person, though he compares it to both an alien and a child. Google spokesperson Brian Gabriel says Lemoine’s claims about LaMDA are “wholly unfounded”, and independent experts almost unanimously agree.

_

According to Michael Wooldridge, a professor of computer science at the University of Oxford who has spent the past 30 years researching AI (in 2020, he won the Lovelace Medal for contributions to computing), LaMDA is simply responding to prompts. It imitates and impersonates. “The best way of explaining what LaMDA does is with an analogy about your smartphone,” Wooldridge says, comparing the model to the predictive text feature that autocompletes your messages. While your phone makes suggestions based on texts you’ve sent previously, with LaMDA, “basically everything that’s written in English on the world wide web goes in as the training data.” The results are impressively realistic, but the “basic statistics” are the same. “There is no sentience, there’s no self-contemplation, there’s no self-awareness,” Wooldridge says.

Google’s Gabriel has said that an entire team, “including ethicists and technologists”, has reviewed Lemoine’s claims and failed to find any signs of LaMDA’s sentience: “The evidence does not support his claims.”

_

And even if LaMDA isn’t sentient, it can convince people it is. Such technology can, in the wrong hands, be used for malicious purposes. “There is this major technology that has the chance of influencing human history for the next century, and the public is being cut out of the conversation about how it should be developed,” Lemoine says. And Wooldridge agrees. “I do find it troubling that the development of these systems is predominantly done behind closed doors and that it’s not open to public scrutiny in the way that research in universities and public research institutes is,” the researcher says. Still, he notes this is largely because companies like Google have resources that universities don’t. And, Wooldridge argues, when we sensationalise about sentience, we distract from the AI issues that are affecting us right now, “like bias in AI programs, and the fact that, increasingly, people’s boss in their working lives is a computer program.”

_

So when should we start worrying about sentient robots in 10 years? In 20? “There are respectable commentators who think that this is something which is really quite imminent. I do not see it’s imminent,” Wooldridge says, though he notes “there absolutely is no consensus” on the issue in the AI community. Jeremie Harris, founder of AI safety company Mercurius and host of the Towards Data Science podcast, concurs. “Because no one knows exactly what sentience is, or what it would involve,” he says, “I don’t think anyone’s in a position to make statements about how close we are to AI sentience at this point.”

_

Most experts believe it’s unlikely that LaMDA or any other AI is close to consciousness, though they don’t rule out the possibility that technology could get there in future.  “My view is that [Lemoine] was taken in by an illusion,” Gary Marcus, a cognitive scientist and author of Rebooting AI says.  “Our brains are not really built to understand the difference between a computer that’s faking intelligence and a computer that’s actually intelligent — and a computer that fakes intelligence might seem more human than it really is.” Computer scientists describe LaMDA as operating like a smartphone’s autocomplete function, albeit on a far grander scale. Like other large language models, LaMDA was trained on massive amounts of text data to spot patterns and predict what might come next in a sequence, such as in a conversation with a human. “If your phone autocompletes a text, you don’t suddenly think that it is aware of itself and what it means to be alive. You just think, well, that was exactly the word I was thinking of,” said Carl Zimmer.

_

Lemoine insists he was not fooled by a clever robot, as some scientists have suggested. Lemoine maintains his position, and even appeared to suggest that Google had enslaved the AI system. Marcus believes Lemoine is the latest in a long line of humans to fall for what computer scientists call “the ELIZA effect,” named after a 1960s computer program that chatted in the style of a therapist. Simplistic responses like “Tell me more about that” convinced users that they were having a real conversation. “That was 1965, and here we are in 2023, and it’s kind of the same thing,” Marcus says.

_

Scientists pointed to humans’ desire to anthropomorphize objects and creatures — perceiving human-like characteristics that aren’t really there. “If you see a house that has a funny crack, and windows, and it looks like a smile, you’re like, ‘Oh, the house is happy,’ you know? We do this kind of thing all the time,” said Karina Vold, an assistant professor at the University of Toronto’s Institute for the History and Philosophy of Science and Technology. “I think what’s going on often in these cases is this kind of anthropomorphism, where we have a system that’s telling us ‘I’m sentient,’ and saying words that make it sound like it’s sentient — it’s really easy for us to want to grasp onto that. 

_

ChatGPT and other new chatbots are so good at mimicking human interaction that they’ve prompted a question among some: Is there any chance they’re conscious? The answer, at least for now, is no. Just about everyone who works in the field of artificial technology is sure that ChatGPT is not alive in the way that’s generally understood by the average person. But that’s not where the question ends. Just what it means to have consciousness in the age of artificial intelligence is up for debate. “These deep neural networks, these matrices of millions of numbers, how do you map that onto these views we have about what consciousness is? That’s kind of terra incognita,” said Nick Bostrom, the founding director of Oxford University’s Future of Humanity Institute, using the Latin term for “unknown territory.”

_______

_______

Why a computer will never be truly conscious, a 2019 paper:

Many advanced artificial intelligence projects say they are working toward building a conscious machine, based on the idea that brain functions merely encode and process multisensory information. The assumption goes, then, that once brain functions are properly understood, it should be possible to program them into a computer. Microsoft recently announced that it would spend US$1 billion on a project to do just that. So far, though, attempts to build supercomputer brains have not even come close. A multi-billion-dollar European project that began in 2013 is now largely understood to have failed. That effort has shifted to look more like a similar but less ambitious project in the U.S., developing new software tools for researchers to study brain data, rather than simulating a brain. Some researchers continue to insist that simulating neuroscience with computers is the way to go. Others, like author of this paper, view these efforts as doomed to failure because they do not believe consciousness is computable. Their basic argument is that brains integrate and compress multiple components of an experience, including sight and smell – which simply can’t be handled in the way today’s computers sense, process and store data.

_

Brains don’t operate like computers:

Living organisms store experiences in their brains by adapting neural connections in an active process between the subject and the environment. By contrast, a computer records data in short-term and long-term memory blocks. That difference means the brain’s information handling must also be different from how computers work. The mind actively explores the environment to find elements that guide the performance of one action or another. Perception is not directly related to the sensory data: A person can identify a table from many different angles, without having to consciously interpret the data and then ask its memory if that pattern could be created by alternate views of an item identified some time earlier.

Could you identify all of these as a table right away? A computer would likely have real trouble.

Another perspective on this is that the most mundane memory tasks are associated with multiple areas of the brain – some of which are quite large. Skill learning and expertise involve reorganization and physical changes, such as changing the strengths of connections between neurons. Those transformations cannot be replicated fully in a computer with a fixed architecture.

_

Computation and awareness:

There are some additional reasons that consciousness is not computable.

A conscious person is aware of what they’re thinking, and has the ability to stop thinking about one thing and start thinking about another – no matter where they were in the initial train of thought. But that’s impossible for a computer to do. More than 80 years ago, pioneering British computer scientist Alan Turing showed that there was no way ever to prove that any particular computer program could stop on its own – and yet that ability is central to consciousness.

His argument is based on a trick of logic in which he creates an inherent contradiction: Imagine there were a general process that could determine whether any program it analyzed would stop. The output of that process would be either “yes, it will stop” or “no, it won’t stop.” That’s pretty straightforward. But then Turing imagined that a crafty engineer wrote a program that included the stop-checking process, with one crucial element: an instruction to keep the program running if the stop-checker’s answer was “yes, it will stop.”

Running the stop-checking process on this new program would necessarily make the stop-checker wrong: If it determined that the program would stop, the program’s instructions would tell it not to stop. On the other hand, if the stop-checker determined that the program would not stop, the program’s instructions would halt everything immediately. That makes no sense – and the nonsense gave Turing his conclusion, that there can be no way to analyze a program and be entirely absolutely certain that it can stop. So it’s impossible to be certain that any computer can emulate a system that can definitely stop its train of thought and change to another line of thinking – yet certainty about that capability is an inherent part of being conscious.

Even before Turing’s work, German quantum physicist Werner Heisenberg showed that there was a distinct difference in the nature of the physical event and an observer’s conscious knowledge of it. This was interpreted by Austrian physicist Erwin Schrödinger to mean that consciousness cannot come from a physical process, like a computer’s, that reduces all operations to basic logic arguments.

These ideas are confirmed by medical research findings that there are no unique structures in the brain that exclusively handle consciousness. Rather, functional MRI imaging shows that different cognitive tasks happen in different areas of the brain. This has led neuroscientist Semir Zeki to conclude that “consciousness is not a unity, and that there are instead many consciousnesses that are distributed in time and space.” That type of limitless brain capacity isn’t the sort of challenge a finite computer can ever handle.

_____

_____

The feasibility of artificial consciousness through the lens of neuroscience, a 2023 study:

Over the course of documented human history (and probably long before), humans have speculated about how and why we are conscious. Why is it that we experience the world around us, as well as our own thoughts, memories and plans? And how does the organisation of our brain, shaped as it is over evolutionary time and steeped in social and cultural factors, coordinate the activity of its constituent neurons and glia to allow us to experience anything at all? Recent advances in neuroimaging have enabled neuroscientists to speculate about how these mechanisms might arise from the seemingly endless complexity of the nervous system. In the last few years, a new player has entered the arena – Large Language Models (LLMs). Through their competence and ability to converse with us, which in humans is indicative of being conscious, LLMs have forced us to refine our understanding of what it means to understand, to have agency and even to be conscious.

_

LLMs are sophisticated, multi-layer artificial neural networks whose weights are trained on hundreds of billions of words from natural language conversations between awake, aware humans. Through text-based queries, users that interact with LLMs are provided with a fascinating language-based simulation. If you take the time to use these systems, it is hard not to be swayed by the apparent depth and quality of the internal machinations in the network. Ask it a question, and it will provide you with an answer that drips with the kinds of nuance we typically associate with conscious thought. As a discerning, conscious agent yourself, it’s tempting to conclude that the genesis of the response arose from a similarly conscious being – one that thinks, feels, reasons and experiences. Using this type of a “Turing test” as a benchmark, the question can be raised whether LLMs are conscious, which in turn raises a host of moral quandaries, such as whether it is ethical to continue to develop LLMs that could be on the precipice of conscious awareness.

_

This perspective is often bolstered by the fact that the architecture of LLMs is loosely inspired by features of brains (see figure below) – the only objects to which we can currently (and confidently) attribute consciousness. However, while early generations of artificial neural networks were designed as a simplified version of the cerebral cortex, modern LLMs have been highly engineered and fit to purpose in ways that do not retain deep homology with the known structure of the brain. Indeed, many of the circuit features that render LLMs computationally powerful have strikingly different architectures from the systems to which we currently ascribe causal power in the production and shaping of consciousness in mammals. For instance, most theories of the neural basis of consciousness would assign a central role in conscious processing to thalamocortical and arousal systems, both features that are architecturally lacking in LLMs. It is in principle possible for future LLMs to approximate the crucial computations of the brain, such as global broadcasting or context-dependent signal augmentation, however at this stage, these features appear to be unrelated to the remarkable capacities of modern LLMs.

_

Figure above shows – Macroscopic topological differences between mammalian brains and large language models.

Left – a heuristic map of the major connections between macro-scale brain structures: dark blue – cerebral cortex; light blue – thalamus; purple – basal ganglia; orange – cerebellum; red = ascending arousal system (colours in the diagram are recreated in the cartoon within the insert).

Right – a schematic depicting the basic architecture of a large language model.

_

One might ask why it is so crucial for the architecture of LLMs to mimic features of the brain. The primary reason is that the only version of consciousness that we can currently be absolutely sure of arises from brains embedded within complex bodies. This could be further collapsed to humans, though many of the systems-level features thought to be important for subjective consciousness are pervasive across phylogeny, stretching back to mammals, and even to invertebrates.

_

Interactions with large language models have led to the suggestion that these models may be conscious. From the perspective of neuroscience, this position is difficult to defend.

Firstly, authors argued that the topological architecture of LLMs, while highly sophisticated, is sufficiently different from the neurobiological details of circuits empirically linked to consciousness in mammals that there is no a priori reason to conclude that they are even capable of phenomenal conscious awareness (figure above). The architecture of large language models is missing key features of the thalamocortical system that have been linked to conscious awareness in mammals.

Secondly, authors detailed the vast differences between the Umwelt of mammals – the ‘slice’ of the external world that they can perceive – and the Umwelt of LLMs, with the layer being highly impoverished and limited to keystrokes, rather than the electromagnetic spectrum. The inputs to large language models lack the embodied, embedded information content characteristic of our sensory contact with the world around us.

Importantly, authors have major reasons to doubt that LLMs would be conscious if we did feed them visual, auditory or even somatosensory information – both because their organisation is not consistent with the known mechanisms of consciousness in the brain, but also for the fact that they don’t have ‘skin in the game’; in that the existence of the system depends on its actions, which is not true for present-day artificial intelligence. That is, LLMs are not biological agents, and hence don’t have any reason to care about the implications of their actions. In toto, authors believe that these three arguments make it extremely unlikely that LLMs, in their current form, have the capacity for phenomenal consciousness, but rather mimic signatures of consciousness that are implicitly embedded within the language that we as humans use to describe the richness of our conscious experience. Authors conclude that, while fascinating and alluring, LLMs are not conscious, and also not likely to be capable as such in their current form.

____

____

Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, 2023.

Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. A group of 19 computer scientists, neuroscientists, and philosophers has come up with an approach: not a single definitive test, but a lengthy checklist of attributes that, together, could suggest but not prove an AI is conscious. In a 120-page discussion paper posted, the researchers draw on theories of human consciousness to propose 14 criteria, and then apply them to existing AI architectures, including the type of model that powers ChatGPT. Authors argue that neurobiology of consciousness may be our best bet. Rather than simply studying an AI agent’s behavior or responses—for example, during a chat—matching its responses to theories of human consciousness could provide a more objective ruler. Authors survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. To be included, a theory had to be based on neuroscience and supported by empirical evidence, such as data from brain scans during tests that manipulate consciousness using perceptual tricks. It also had to allow for the possibility that consciousness can arise regardless of whether computations are performed by biological neurons or silicon chips. From the six included theories the team extracted their 14 indicator properties of a conscious state, elucidated in computational terms that allow us to assess AI systems for these properties. Authors use these indicator properties to assess several recent AI systems, and discuss how future systems might implement them. Author’s analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.

The authors say their checklist is a work in progress. And it’s not the only such effort underway. Some members of the group are part of a CIFAR-funded project to devise a broader consciousness test that can also be applied to organoids, animals, and newborns. They hope to produce a publication in the future. The problem for all such projects is that current theories are based on our understanding of human consciousness. Yet consciousness may take other forms, even in our fellow mammals. We really have no idea what it’s like to be a bat. It’s a limitation we cannot get rid of.

_____

_____

Arguments for and against consciousness in AI:

_

Arguments for AI consciousness: 

While the concept of AI consciousness remains controversial and debated, there are several arguments that have been put forth to suggest that AI could be capable of subjective experience and consciousness.

-1. Simulation theory and replicating human brain processes in AI

One argument for the possibility of AI consciousness is based on the simulation theory, which suggests that consciousness arises from the complex processing of information in the brain. If this theory is correct, it may be possible to create conscious machines by replicating the same processes in AI.

Hubert Dreyfus describes this argument as claiming that “if the nervous system obeys the laws of physics and chemistry, which we have every reason to suppose it does, then … we … ought to be able to reproduce the behavior of the nervous system with some physical device”. This argument, first introduced as early as 1943 and vividly described by Hans Moravec in 1988, is now associated with futurist Ray Kurzweil, who estimates that computer power will be sufficient for a complete brain simulation by the year 2029. A non-real-time simulation of a thalamocortical model that has the size of the human brain (10^11 neurons) was performed in 2005 and it took 50 days to simulate 1 second of brain dynamics on a cluster of 27 processors.

Even AI’s harshest critics (such as Hubert Dreyfus and John Searle) agree that a brain simulation is possible in theory. However, Searle points out that, in principle, anything can be simulated by a computer; thus, bringing the definition to its breaking point leads to the conclusion that any process at all can technically be considered “computation”. “What we wanted to know is what distinguishes the mind from thermostats and livers,” he writes. Thus, merely simulating the functioning of a living brain would in itself be an admission of ignorance regarding intelligence and the nature of the mind, like trying to build a jet airliner by copying a living bird precisely, feather by feather, with no theoretical understanding of aeronautical engineering.

-2. Self-learning and self-improving AI

Another argument for the possibility of AI consciousness is based on the self-learning and self-improving capabilities of AI. Machine learning algorithms are designed to learn and improve over time, using feedback from data to refine their performance. As these algorithms become more sophisticated, they may be capable of developing new insights and knowledge that go beyond their original programming.

If AI is capable of self-learning and self-improvement to a sufficient degree, it may be capable of developing subjective experience and consciousness. Some researchers have suggested that self-improving AI could eventually reach a point where it surpasses human intelligence, leading to the creation of superintelligent machines with their own subjective experiences.

-3. AI experiencing subjective states and emotions

Some researchers argue that AI could experience subjective states and emotions, even if these experiences are different from those of humans. For example, an AI system may be capable of experiencing the subjective state of pain, even if it does not have a physical body like humans do. Similarly, an AI system may be capable of experiencing emotions like joy, sadness, or anger, even if these emotions are not experienced in the same way as humans.

While these arguments do not necessarily prove that AI can have consciousness, they do suggest that the possibility cannot be ruled out entirely. As AI technologies continue to advance, it is likely that the debate over the nature of consciousness and the possibility of creating conscious machines will continue to evolve.

____

Arguments against AI consciousness

While some researchers have argued for the possibility of AI consciousness, others have put forth arguments to suggest that machines are fundamentally incapable of having subjective experience or consciousness.

-1. The Chinese Room argument and limitations of symbolic AI

One argument against AI consciousness is based on the limitations of symbolic AI, which relies on rule-based programming to solve problems. The Chinese Room argument, put forth by philosopher John Searle, suggests that even if a machine can simulate the behavior of a conscious human being, it may still lack subjective experience.

According to the Chinese Room argument, a person who does not understand Chinese could still manipulate symbols to produce responses in Chinese that appear to be intelligent, even if they do not understand the language. Similarly, a machine that relies on symbolic programming may be able to produce intelligent responses to questions or problems, but this does not necessarily mean that it has subjective experience or consciousness or it truly understand the meaning behind those symbols.

-2. The hard problem of consciousness and the difficulty of explaining subjective experience

Another argument against AI consciousness is based on the so-called hard problem of consciousness, which refers to the difficulty of explaining subjective experience. While machines can perform complex tasks and solve problems, they lack the subjective experience that is associated with consciousness.

The hard problem of consciousness suggests that subjective experience cannot be reduced to the processing of information or the behavior of neurons. Instead, it suggests that subjective experience is a fundamental aspect of the universe that cannot be fully explained by scientific or mathematical models.

-3. Ethical implications of creating conscious AI

Another argument against the creation of conscious AI is based on ethical concerns and the potential dangers of creating machines with their own subjective experience. If machines were capable of experiencing pain, suffering, or other subjective states, it raises questions about the morality of using them for tasks such as manual labor or military operations.

Moreover, there are concerns that conscious machines could become a threat to human safety and autonomy. If machines were capable of experiencing emotions and making decisions based on those emotions, they could become unpredictable and difficult to control. This could have serious implications for the safety and well-being of humans who interact with conscious machines.

______

______ 

Testing AI consciousness:

Over the decades, scientists have developed various methods to test the level of intelligence in AI systems. These tests range from general evaluations such as the Turing test and the more recent Abstract Reasoning Corpus, to task-specific benchmarks such as image labelling datasets for computer vision systems and question-answering datasets for natural language processing algorithms.

But verifying whether an AI system is conscious or not is easier said than done, Susan Schneider discusses in Artificial You. Consciousness is not a binary, present–not present quality. There are different levels of consciousness that can’t be covered by a single test.

For instance, a chimpanzee or a dog won’t pass a language test, but does it mean that they totally lack consciousness. Likewise, humans with some disabilities might not be able to pass tests that other average humans find trivial, but it would be horrendous to conclude they’re not conscious. Is there any reason to treat advanced AI any differently?

“There cannot be a one-size-fits-all test for AI consciousness; a better option is a battery of tests that can be used depending on the context,” Schneider writes.

_

How will we know that machines are conscious? 

There are two ways we might find out:  Either develop the correct theory of consciousness (or a theory close enough to it) and see if the machines fit the bill, or apply what we call a neutral test for machine consciousness. A neutral test is any procedure that can reveal whether an entity is conscious to the satisfaction of conflicting theorists of consciousness, among which the test remains neutral.  Theory neutrality is, of course, a second-best prize to having the correct theory of consciousness and then simply applying that to the case of artificial intelligence (AI).  But given the lack of theoretical consensus about consciousness, a neutral test satisfactory to many theorists is prize enough.  In her recent book, Susan Schneider (2019) aims at such a prize.

_

Testing the outputs:

At its most basic, this is how we know if a neural network is functioning correctly. Since there are limited ways of breaking into the unknowable black box of artificial neurons, engineers analyze the inputs and outputs and then determine whether these are in line with what they expect.

Turing test:

The most famous of the neutral tests for consciousness is the Turing Test (Turing 1950) — originally intended as a test for ‘thinking’ but sometimes adapted to test for consciousness (Harnad 2003).  A machine passes the Turing Test if it can verbally interact with a human judge in a way indistinguishable from how humans interact. It such a case it will be judged to be conscious. Failing the test does not, however, imply lack of consciousness. Dogs and infants fail. Thus, it is a sufficiency test for AI consciousness (not a necessary and sufficient criterion): When a system passes, we have (according to the test) good reason to attribute consciousness to it. Part of the test’s attractiveness is its agnosticism about the internal architecture of its test subjects, and so its neutrality among many competing theories of consciousness. 

Unfortunately, the Turing Test has some serious limitations. 

First, although the best current chatbots cannot pass an unconstrained Turing Test and are still easily distinguishable from human in many areas of ability when carefully probed by a skilled interrogator, some have succeeded in highly constrained versions of the Test such as the one used in the Loebner Competition (see Dennett 1998; Floridi, Taddeo and Turilli 2009; Aaronson 2014 for a critical perspective on chatbot successes), and open-domain chatbots (Adiwardana et al. 2020; Roller et al. 2020; Rosset 2020) and related machine learning models in natural language processing (Liu et al. 2019; Raffel et al. 2020; Brown et al. 2020) are improving rapidly.  It is thus unclear whether a sufficiently good chatbot might someday pass a rigorous Turing Test despite lacking consciousness according to many leading theories (in which case the Test would not be an appealingly theory neutral test of consciousness). 

Second, although the Turing Test is neutral on the question of what interior mechanisms must be present for consciousness, since whatever system outputs the right behavior passes, it is decidedly not neutral across theories that treat details about interior mechanisms as crucial to the presence or absence of consciousness in the sense of having ‘what-it’s-like’-ness or a stream of experience.  The concerns about constitution famously presented in Block’s ‘Chinese nation’ and Searle’s ‘Chinese room’ thought experiments, for example, are disregarded by those who would employ a Turing Test-passing standard for AI consciousness (Block 1978/2007; Searle 1980).

In recent years, another robotics-focused intelligence test is the Coffee Test proposed by Apple co-founder Steve Wozniak. To pass the Coffee Test, a machine would have to enter a typical American home and figure out how to successfully make a cup of coffee.

To date, neither of these tests have been convincingly passed. But even if they were, they would, at best, prove intelligent behavior in real-world situations, and not sentience. (As a simple objection, would we deny that a person was sentient if they were unable to hold an adult conversation or enter a strange house and operate a coffee machine? Young children would fail such a test.)

_____

Mirror test and AI mirror test:

The mirror test measures self-awareness, while the Turing test measures consciousness.

• A Turing test would reveal whether or not a computer is indistinguishable from a human in speech;
• A mirror test would reveal whether or not the computer understands itself mentally and physically.

In behavioral psychology, the mirror test is designed to discover animals’ capacity for self-awareness. There are a few variations of the test, but the essence is always the same: do animals recognize themselves in the mirror or think it’s another being altogether?

Figure above shows a gorilla charging its own reflection in a mirror.

Mirror testing has been around through generations of science. Mostly to discover which animals possess heightened intelligence. Some animals that have (unsurprisingly) passed the mirror test are elephants, apes, and dolphins. These creatures are known to be some of the most intelligent non-human creatures on our planet. However, some sentient animals like dogs and pigs, do not pass the mirror test despite clearly feeling and perceiving.

_

Has your pet ever looked in the mirror and expressed confusion or anger? That’s because despite looking directly at themselves, they don’t recognize their own face. Meaning they may perceive their own reflection as an intruder! Since you are a highly intelligent being, when you look in the mirror, you recognize yourself instantly. Although non-living, computers have also advanced to be highly intelligent to the point of outsmarting humans in hundreds of tasks. But would a robot recognize itself in the mirror, and what does that say about its capabilities? That’s what the AI mirror test is for, and you might be surprised at some of the results it has rendered.

_

Conducting an AI mirror test:

How do you conduct a mirror test on a robot, and how do you know if it passes?

We’ll start with Qbo, an open-source robot developed by Thecorpora that can be assembled and programmed by any user. The robot is self-driving, it can track faces and objects, along with understanding speech. It was over a decade ago that Qbo first demonstrated an ability to recognize itself in the mirror upon an initial request to learn what it looked like. Some may claim that this is a breakthrough in robot self-awareness. But many have denounced considering this simple skill as a marker of true self-awareness.

In a more advanced experiment, a robot named TIAGo developed by the Technical University of Munich demonstrated an ability to recognize itself in the mirror beyond simply memorizing its own face. They proved this by facing two identical robots towards each other and moving their arms differently. The “self-aware” robot recognized that the identical robot was not its own reflection due to these movements. But again, although this robot passes the “test,” is that enough to declare sapience that is comparable to that of a human?

_____

AI Consciousness Test (ACT):

Susan Schneider (2019) has proposed two new tests for AI consciousness that promise to compensate for those limitations. 

One, the AI Consciousness Test (ACT), developed collaboratively with Edwin Turner, is similar to the Turing Test in that it focuses on verbal behavior — but verbal behavior specifically concerning the metaphysics of consciousness, and under a specific set of learning constraints that aim to prevent too easy a pass.  Schneider and Turner’s core idea is that if a machine, without being taught to do so, begins to speculate on philosophical questions about consciousness, such as the possible existence of a soul that continues on after bodily death, that is a sign that the machine is conscious (see also Sloman 2007; Argonov 2014). This enriches the Turing Test by shifting its focus to a topic that, the machine is expected to handle poorly unless it is actually conscious. 

Schneider’s second new test, the Chip Test, is different from most previous proposals and has the advantage of being plausibly neutral among a wider range of theories of consciousness. The Chip Test depends on and is concerned with the existence of removable artificial ‘brain chips’ made of silicon.  One temporarily suppresses one’s biological brain activity in a certain functional or physiological region, relying instead on the chip to do the cognitive work. Simultaneously, one introspects. Is the relevant type of consciousness still present?  If so, one concludes that the silicon chip is capable of supporting conscious experience. This test is motivated by Searlean and Blockian concerns about the possibility that silicon (or other kinds of artificial) chips might be incapable of hosting consciousness, even if they are functionally similar at a relatively high level of description, supporting similar outward behavior.  It is explicitly first-personal, relying on the tester’s own introspection to address those concerns rather than relying just on facts about outward behavior or interior structure.  Like the Turing Test and Schneider and Turner’s ACT, this is a sufficiency test of consciousness rather than a necessity test: Passing the test is held to be sufficient evidence for a confident attribution of consciousness to the target system, while failing the test does not guarantee a lack of consciousness (there might, for instance, be a faulty connection when wiring up well-functioning chips).

With the ACT, Schneider and Turner have proposed a means to examine sophisticated artificial intelligence for consciousness in a way that is (1) impressively neutral about architectural details, (2) consistent with nearly complete human and AI ignorance about the physical or functional basis of consciousness, (3) allows (unlike the Turing Test) that the AI might be radically cognitively different from us in important respects that it cannot conceal, and (4) is even consistent with non-materialist views like substance dualism.  In these respects it is an ambitious and intriguing test.

_____

Limitations of so-called Intelligence Testing:

Experts today agree that the Turing test is a poor test for intelligence. It assesses how well machines deceive people under superficial conditions. Computer scientists have moved onto more sophisticated tests like the General Language Understanding Evaluation (GLUE).  The test asks machines to draw conclusions from a premise, ascribe attitudes to text and identify synonyms. The Turing test and GLUE assess if machines can think. Sentience asks if machines can feel. But doing is not being, and being is not doing.

One of the problems of existing intelligence tests is that they all rely on an observer’s willingness to ascribe intelligence to the machine. However, it is not clear we have the inclination to ascribe intelligence to machines, nor is it obvious we will develop such an inclination in the future. It would thus be beneficial to develop a test that is self-referential, and uses the machine’s own characteristics to test itself. Another problem is the definition of intelligence as a “system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation”. This definition and others rely on the existence of some external input, which is superfluous a la Descartes. Finally, in the mechanistic view, self-awareness is an emergent property. In other words, mental events supervene upon physical events. Thus intelligence cannot be directly programmed. Rather, it should emerge from the complexity of more basic infrastructure.   

______

______ 

Section-10

AI risks and limitations:

_

AI risks:

In the 21st century, with computers quickly establishing themselves as a transformative force in our world, younger researchers started expressing worries about AI risks. Nick Bostrom is a professor at the University of Oxford, the director of the Future of Humanity Institute, and the director of the Governance of Artificial Intelligence Program. He researches risks to humanity, both in the abstract — asking questions like why we seem to be alone in the universe — and in concrete terms, analyzing the technological advances on the table and whether they endanger us. AI, he concluded, endangers us. In 2014, he wrote a book explaining the risks AI poses and the necessity of getting it right the first time, concluding, “once unfriendly superintelligence exists, it would prevent us from replacing it or changing its preferences. Our fate would be sealed.”

_

Across the world, others have reached the same conclusion. Bostrom co-authored a paper on the ethics of artificial intelligence with Eliezer Yudkowsky, founder of and research fellow at the Berkeley Machine Intelligence Research Institute (MIRI), an organization that works on better formal characterizations of the AI safety problem. Yudkowsky started his career in AI by worriedly poking holes in others’ proposals for how to make AI systems safe, and has spent most of it working to persuade his peers that AI systems will, by default, be unaligned with human values (not necessarily opposed to but indifferent to human morality) — and that it’ll be a challenging technical problem to prevent that outcome. Increasingly, researchers realized that there’d be challenges that hadn’t been present with AI systems when they were simple. “‘Side effects’ are much more likely to occur in a complex environment, and an agent may need to be quite sophisticated to hack its reward function in a dangerous way. This may explain why these problems have received so little study in the past, while also suggesting their importance in the future,” concluded a 2016 research paper on problems in AI safety.

_

Bostrom’s book Superintelligence was compelling to many people, but there were Skeptics. “No, experts don’t think superintelligent AI is a threat to humanity,” argued an op-ed by Oren Etzioni, a professor of computer science at the University of Washington and CEO of the Allan Institute for Artificial Intelligence. “Yes, we are worried about the existential risk of artificial intelligence,” replied a dueling op-ed by Stuart Russell, an AI pioneer and UC Berkeley professor, and Allan DaFoe, a senior research fellow at Oxford and director of the Governance of AI program there. It’s tempting to conclude that there’s a pitched battle between AI-risk Skeptics and AI risk believers. In reality, they might not disagree as profoundly as you would think.

_

Facebook’s chief AI scientist Yann LeCun, for example, is a vocal voice on the sceptical side. But while he argues we shouldn’t fear AI, he still believes we ought to have people working on, and thinking about, AI safety. “Even if the risk of an A.I. uprising is very unlikely and very far in the future, we still need to think about it, design precautionary measures, and establish guidelines,” he writes. That’s not to say there’s an expert consensus here — far from it. There is substantial disagreement about which approaches seem likeliest to bring us to general AI, which approaches seem likeliest to bring us to safe general AI, and how soon we need to worry about any of this. Many experts are wary that others are overselling their field, and dooming it when the hype runs out. But that disagreement shouldn’t obscure a growing common ground; these are possibilities worth thinking about, investing in, and researching, so we have guidelines when the moment comes that they’re needed.

_____

Some risks of Artificial Intelligence:

• Automation-spurred job loss
• Deepfakes
• Privacy violations
• Algorithmic bias caused by bad data
• Socioeconomic inequality
• Market volatility
• Weapons automatization
• Uncontrollable self-aware AI

_____

Threats by artificial intelligence to human health and human existence, a 2023 study:

While artificial intelligence (AI) offers promising solutions in healthcare, it also poses a number of threats to human health and well-being via social, political, economic and security-related determinants of health. Authors describe three such main ways misused narrow AI serves as a threat to human health: through increasing opportunities for control and manipulation of people; enhancing and dehumanising lethal weapon capacity and by rendering human labour increasingly obsolescent. Authors then examine self-improving ‘artificial general intelligence’ (AGI) and how this could pose an existential threat to humanity itself. Finally, authors discuss the critical need for effective regulation, including the prohibition of certain types and applications of AI, and echo calls for a moratorium on the development of self-improving AGI.

_

Artificial intelligence (AI) is broadly defined as a machine with the ability to perform tasks such as being able to compute, analyse, reason, learn and discover meaning. Its development and application are rapidly advancing in terms of both ‘narrow AI’ where only a limited and focused set of tasks are conducted and ‘broad’ or ‘broader’ AI where multiple functions and different tasks are performed.

There are three threats associated with the potential misuse of narrow AI, before examining the potential existential threat of self-improving general-purpose AI, or artificial general intelligence (AGI) as seen in the figure below.

Figure above shows threats posed by the potential misuse of artificial intelligence (AI) to human health and well-being, and existential-level threats to humanity posed by self-improving artificial general intelligence (AGI).

_____

Bill Gates isn’t too worried about the Risks of AI:

Bill Gates outlined how he thinks about the risks from artificial intelligence (AI) in a blog post recently. While Gates remains excited by the benefits that AI could bring, he shared his thoughts on the areas of risk he hears concern about most often. In the post, titled The risks of AI are real but manageable, Gates discusses five risks from AI in particular.

First, AI-generated misinformation and deepfakes could be used to scam people or even sway the results of an election.

Second, AI could automate the process of searching for vulnerabilities in computer systems, drastically increasing the risk of cyberattacks.

Third, AI could take people’s jobs.

Fourth, AI systems have already been found to fabricate information and exhibit bias.

Finally, access to AI tools could mean that students don’t learn essential skills, such as essay writing—as well as widen the educational achievement gap.

Gates has staked out a kind of middle ground between deep-learning pioneer Geoffrey Hinton, who quit Google and went public with his fears about AI in May 2023, and others like Yann LeCun and Joelle Pineau at Meta AI (who think talk of existential risk is “preposterously ridiculous” and “unhinged”) or Meredith Whittaker at Signal (who thinks the fears shared by Hinton and others are “ghost stories”).  

_____

Many experts expressed concerns about the long-term impact of AI tools on the essential elements of being human as seen in the table below:

Human agency:   Individuals are experiencing a loss of control over their lives	Decision-making on key aspects of digital life is automatically ceded to code-driven, “black box” tools. People lack input and do not learn the context about how the tools work. They sacrifice independence, privacy and power over choice; they have no control over these processes. This effect will deepen as automated systems become more prevalent and complex.
Data abuse: Data use and surveillance in complex systems is designed for profit or for exercising power	Most AI tools are and will be in the hands of companies striving for profits or governments striving for power. Values and ethics are often not baked into the digital systems making people’s decisions for them. These systems are globally networked and not easy to regulate or rein in.
Job loss: The AI takeover of jobs will widen economic divides, leading to social upheaval	The efficiencies and other economic advantages of code-based machine intelligence will continue to disrupt all aspects of human work. While some expect new jobs will emerge, others worry about massive job losses, widening economic divides and social upheavals, including populist uprisings.
Dependence lock-in: Reduction of individuals’ cognitive, social and survival skills	Many see AI as augmenting human capacities but some predict the opposite – that people’s deepening dependence on machine-driven networks will erode their abilities to think for themselves, take action independent of automated systems and interact effectively with others.
Mayhem: Autonomous weapons, cybercrime and weaponized information	Some predict further erosion of traditional sociopolitical structures and the possibility of great loss of lives due to accelerated growth of autonomous military applications and the use of weaponized information, lies and propaganda to dangerously destabilize human groups. Some also fear cybercriminals’ reach into economic systems.

______

______ 

Artificial Narrow Intelligence Risks:  

To understand what ANI is you simply need to understand that every single AI application that is currently available is a form of ANI. These are fields of AI which have a narrow field of specialty, for example autonomous vehicles use AI which is designed with the sole purpose of moving a vehicle from point A to B. Another type of ANI might be a chess program which is optimized to play chess, and even if the chess program continuously improves itself by using reinforcement learning, the chess program will never be able to operate an autonomous vehicle. With its focus on whatever operation it is responsible for, ANI systems are unable to use generalized learning in order to take over the world. That is the good news; the bad news is that with its reliance on a human operator the AI system is susceptible to biased data, human error, or even worse, a rogue human operator.

____

-1. AI Surveillance

There may be no greater danger to humanity than humans using AI to invade privacy, and in some cases using AI surveillance to completely prevent people from moving freely. China, Russia, and other nations passed through regulations during COVID-19 to enable them to monitor and control the movement of their respective populations. These are laws which once in place, are difficult to remove, especially in societies that feature autocratic leaders.

In China, cameras are stationed outside of people’s homes, and in some cases inside the person’s home. Each time a member of the household leaves, an AI monitors the time of arrival and departure, and if necessary alerts the authorities. As if that was not sufficient, with the assistance of facial recognition technology, China is able to track the movement of each person every time they are identified by a camera. This offers absolute power to the entity controlling the AI, and absolutely zero recourse to its citizens.

Why this scenario is dangerous, is that autocratic governments can carefully monitor the movements of journalists, political opponents, or anyone who dares to question the authority of the government. It is easy to understand how journalists and citizens would be cautious to criticize governments when every movement is being monitored.

____

-2. Autonomous Weapons and Drones

Over 4500 AI researches have been calling for a ban on autonomous weapons and have created the Ban Lethal Autonomous Weapons website. The group has many notable non-profits as signatories such as Human Rights Watch, Amnesty International, and the The Future of Life Institute which in itself has a stellar scientific advisory board including Elon Musk, Nick Bostrom, and Stuart Russell.

In contrast to semi-autonomous weapons that require human oversight to ensure that each target is validated as ethically and legally legitimate, such fully autonomous weapons select and engage targets without human intervention, representing complete automation of lethal harm. Currently, smart bombs are deployed with a target selected by a human, and the bomb then uses AI to plot a course and to land on its target. The problem is what happens when we decide to completely remove the human from the equation? When an AI chooses what humans need targeting, as well as the type of collateral damage which is deemed acceptable, we may have crossed a point of no return. This is why so many AI researchers are opposed to researching anything that is remotely related to autonomous weapons.

There are multiple problems with simply attempting to block autonomous weapons research. The first problem is even if democratic nations such as Canada, the USA, and most of Europe choose to agree to the ban, it doesn’t mean autocratic nations such as China, North Korea, Iran, and R ussia will play along. The second and bigger problem is that AI research and applications that are designed for use in one field, may be used in a completely unrelated field. For example, computer vision continuously improves and is important for developing autonomous vehicles, precision medicine, and other important use cases. It is also fundamentally important for regular drones or drones which could be modified to become autonomous. One potential use case of advanced drone technology is developing drones that can monitor and fight forest fires. This would completely remove firefighters from harms way. In order to do this, you would need to build drones that are able to fly into harms way, to navigate in low or zero visibility, and are able to drop water with impeccable precision. It is not a far stretch to then use this identical technology in an autonomous drone that is designed to selectively target humans. It is a dangerous predicament and at this point in time, no one fully understands the implications of advancing or attempting to block the development of autonomous weapons. It is nonetheless something that we need to keep our eyes on, enhancing whistle blower protection may enable those in the field to report abuses.

_____

-3. AI Bias

Algorithmic bias:

Machine learning bias, also known as algorithm bias or AI bias, is a phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the machine learning (ML) process. Machine learning programs will be biased if they learn from biased data. The programmers may not be aware that the bias exists. Bias can be introduced by the way training data is selected and by the way a model is deployed. It can also emerge from correlations: AI is used to classify individuals into groups and then make predictions assuming that the individual will resemble other members of the group. In some cases, this assumption may be unfair. An example of this is COMPAS, a commercial program widely used by U.S. courts to assess the likelihood of a defendant becoming a recidivist. ProPublica claims that the COMPAS-assigned recidivism risk level of black defendants is far more likely to be overestimated than that of white defendants, despite the fact that the program was not told the races of the defendants.

One of the most unreported threats of AI is AI bias. This is simple to understand as most of it is unintentional. AI bias slips in when an AI reviews data that is fed to it by humans, using pattern recognition from the data that was fed to the AI, the AI incorrectly reaches conclusions which may have negative repercussions on society. For example, an AI that is fed literature from the past century on how to identify medical personnel may reach the unwanted sexist conclusion that women are always nurses, and men are always doctors.

A more dangerous scenario is when AI that is used to sentence convicted criminals is biased towards giving longer prison sentences to minorities. The AI’s criminal risk assessment algorithms are simply studying patterns in the data that has been fed into the system. This data indicates that historically certain minorities are more likely to re-offend, even when this is due to poor datasets which may be influenced by police racial profiling. The biased AI then reinforces negative human policies. This is why AI should be a guideline, never judge and jury.

Returning to autonomous weapons, if we have an AI which is biased against certain ethnic groups, it could choose to target certain individuals based on biased data, and it could go so far as ensuring that any type of collateral damage impacts certain demographics less than others. For example, when targeting a terrorist, before attacking it could wait until the terrorist is surrounded by those who follow a particular faith.

Fortunately, it has been proven that AI that is designed with diverse teams are less prone to bias. This is reason enough for enterprises to attempt when at all possible to hire a diverse well-rounded team. At its 2022 Conference on Fairness, Accountability, and Transparency (ACM FAccT 2022) the Association for Computing Machinery, in Seoul, South Korea, presented and published findings recommending that until AI and robotics systems are demonstrated to be free of bias mistakes, they are unsafe and the use of self-learning neural networks trained on vast, unregulated sources of flawed internet data should be curtailed.

____

-4. Misinformation and disinformation:  

Misinformation is false or inaccurate information—getting the facts wrong. Disinformation is false information which is deliberately intended to mislead—intentionally misstating the facts. These two words, so often used interchangeably, are merely one letter apart. But behind that one letter hides the critical distinction between these confusable words: intent.

A deepfake is a convincing, computer-generated artificial image or video. The word “deepfake” may originate from the AI-related term “deep learning,” the process by which some AI systems analyze huge amounts of data to train themselves and “learn.” Deepfakes are synthetic media that have been manipulated to falsely portray individuals saying or doing things they never actually did. Deepfakes can be used to spread misinformation, damage reputations and even influence elections. Deepfakes are an emerging concern within the last few years. Many deepfakes contain images or videos of celebrities, but they can also be used to create a variety of other types of misinformation or malicious content, from misleading news reports to revenge pornography and more. The potential dangers of deepfakes are significant, and they represent one of the most visible examples of a broader category of AI risk: misinformation. AI can be used to create and to widely share material that is incorrect but which looks convincingly true. There are a host of social, political, and legal ramifications for deepfakes and other AI-generated misinformation, and one of the biggest issues is that there currently exists essentially no regulation about these materials.

_

Less than a month after Russia’s invasion of Ukraine, a video surfaced on social media that purportedly showed Ukrainian President Volodymyr Zelenskyy urging his soldiers to surrender their arms and abandon the fight against Russia. While the lip-sync in the video appeared somewhat convincing, discrepancies in Mr Zelenskyy’s accent, as well as his facial movements and voice, raised suspicions about its authenticity. Upon closer examination, a simple screenshot revealed that the video was indeed a fake – a deepfake. This marked the first known instance of a deepfake video being utilised in the context of warfare.

_

Figure above shows a phone with a tweet on it describing a deepfake video of the Ukrainian president, with a labeled fake image in the background. The new generation of AI tools makes it a lot easier to produce convincing misinformation.

_

Experts say AI can be used to generate fake images, videos, sounds and text that are indistinguishable from reality, making them a powerful disinformation tool. The point was underlined when an AI-generated image of the pope in a white puffer jacket went viral on Twitter, with many people believing it to be real. AI-generated content, such as deepfakes, contributes to the spread of false information and the manipulation of public opinion. Efforts to detect and combat AI-generated misinformation are critical in preserving the integrity of information in the digital age.

In a Stanford University study on the most pressing dangers of AI, researchers said:

“AI systems are being used in the service of disinformation on the internet, giving them the potential to become a threat to democracy and a tool for fascism. From deepfake videos to online bots manipulating public discourse by feigning consensus and spreading fake news, there is the danger of AI systems undermining social trust. The technology can be co-opted by criminals, rogue states, ideological extremists, or simply special interest groups, to manipulate people for economic gain or political advantage.”

_

Risks of Generative AI for Disinformation:

Generative artificial intelligence systems can compound the existing challenges in our information environment in at least three ways: by increasing the number of parties that can create credible disinformation narratives; making them less expensive to create; and making them more difficult to detect. If social media made it cheaper and easier to spread disinformation, now generative AI will make it easier to produce. And traditional cues that alert researchers to false information, like language and syntax issues and cultural gaffes in foreign intelligence operations, will be missing.

_

ChatGPT, the consumer-facing application of generative pre-trained transformer GPT, has already been described as “the most powerful tool for spreading misinformation that has ever been on the internet.” Researchers at OpenAI, ChatGPT’s parent company, have conveyed their own concerns that their systems could be misused by “malicious actors… motivated by the pursuit of monetary gain, a particular political agenda, and/or a desire to create chaos or confusion.” Image generators, like Stability AI’s Stable Diffusion, create such realistic images that they may undermine the classic entreaty to “believe your own eyes” in order to determine what is true and what is not.

_

This isn’t just about “hallucinations,” which refers to when a generative model puts out factually incorrect or nonsensical information. Researchers have already proven that bad actors can use machine-generated propaganda to sway opinions. The impact of generative models on our information environment can be cumulative: Researchers are finding that the use of content from large language models to train other models pollutes the information environment and results in content that is further and further from reality. It all adds a scary new twist to the classic description of the internet as “five websites, each consisting of screenshots of text from the other four.” What if all those websites were actually training each other on false information, then feeding it to us?

_

Online media and news have become even murkier in light of deepfakes infiltrating political and social spheres. The technology makes it easy to replace the image of one figure with another in a picture or video. As a result, bad actors have another avenue for sharing misinformation and war propaganda, creating a nightmare scenario where it can be nearly impossible to distinguish between creditable and faulty news. No one knows what’s real and what’s not. So it really leads to a situation where you literally cannot believe your own eyes and ears; you can’t rely on what, historically, we’ve considered to be the best possible evidence… That’s going to be a huge issue.

_

The biggest risk of AI comes from the potential manipulation of people: brainwashing. Fake news is a tool for divisiveness and brainwashing. Humans are using AI to create and spread fake news, which is killing wisdom, without firing a single shot. Fake news feeds bias to an extraordinary degree, dividing people and silently killing any idea, let alone opportunity, for respectful listening and rational debate.

As George Bernard Shaw wrote:

Beware of false knowledge; it is more dangerous than ignorance.

_____

-5. Biological warfare

In a recent experiment, MIT associate professor and GCSP polymath fellow Kevin Esvelt and his students utilised freely accessible “large language model” algorithms like GPT-4 to devise a detailed roadmap for obtaining exceptionally dangerous viruses. In just one hour, the chatbot suggested four potential pandemic pathogens, provided instructions for generating them from synthetic DNA, and even recommended DNA synthesis companies unlikely to screen orders. Their conclusion was alarming: easy access to AI chatbots will lead “the number of individuals capable of killing tens of millions to dramatically increase”.

____

-6. Social manipulations through AI algorithms

Social manipulation also stands as a danger of artificial intelligence. This fear has become a reality as politicians rely on platforms to promote their viewpoints, with one example being Ferdinand Marcos, Jr., wielding a TikTok troll army to capture the votes of younger Filipinos during the Philippines’ 2022 election. TikTok, which is just one example of a social media platform that relies on AI algorithms, fills a user’s feed with content related to previous media they’ve viewed on the platform. Criticism of the app targets this process and the algorithm’s failure to filter out harmful and inaccurate content, raising concerns over TikTok’s ability to protect its users from misleading information.

____

-7. Privacy

AI uses huge amounts of information to train itself, and this information usually consists of real data from real people. This alone constitutes a potential invasion of privacy. And there are specific examples of AI systems, including computer vision and facial recognition tools, among others, which present even greater privacy threats. AI systems can be trained to recognize individual human faces, allowing them to potentially surveil societies and exert control over human behavior. Used by bad actors, these AI programs could play a key role in large-scale societal repression or worse.

_____

-8. Job Loss

One of the most common concerns about AI is that it will lead to automation of skills and processes that is so efficient that it leads to the elimination of human jobs or even entire industries. In reality, experts are conflicted about the possible impact of AI on jobs. One recent study suggests some 85 million jobs could be lost due to AI automation between 2020 and 2025. Another suggests that AI could create 97 million new jobs by 2025.

Some experts believe that AI will lead to a jobs crisis, as the jobs that eliminates and the jobs that it creates will not overlap in terms of skill sets. The workers displaced from jobs that are eliminated may be less likely to apply for the jobs AI generates, which could be tied to more specific skills and experiences.

____

-9. Financial Volatility

AI has the potential to upend the financial sector. An increasing number of investment companies rely on AI to analyze and select securities to buy and sell. Many also use AI systems for the actual process of buying and selling assets as well. The financial industry has become more receptive to AI technology’s involvement in everyday finance and trading processes. As a result, algorithmic trading could be responsible for our next major financial crisis in the markets. While AI algorithms aren’t clouded by human judgment or emotions, they also don’t take into account contexts, the interconnectedness of markets and factors like human trust and fear. These algorithms then make thousands of trades at a blistering pace with the goal of selling a few seconds later for small profits. Selling off thousands of trades could scare investors into doing the same thing, leading to sudden crashes and extreme market volatility. Instances like the 2010 Flash Crash and the Knight Capital Flash Crash serve as reminders of what could happen when trade-happy algorithms go berserk, regardless of whether rapid and massive trading is intentional. 

This isn’t to say that AI has nothing to offer to the finance world. In fact, AI algorithms can help investors make smarter and more informed decisions on the market. But finance organizations need to make sure they understand their AI algorithms and how those algorithms make decisions. Companies should consider whether AI raises or lowers their confidence before introducing the technology to avoid stoking fears among investors and creating financial chaos. 

____

-10. Copyright

In order to leverage as large a dataset as is feasible, generative AI is often trained on unlicensed copyrighted works, including in domains such as images or computer code; the output is then used under a rationale of “fair use”. Experts disagree about how well, and under what circumstances, this rationale will hold up in courts of law; relevant factors may include “the purpose and character of the use of the copyrighted work” and “the effect upon the potential market for the copyrighted work”. Franzen, Grisham and other prominent Authors have sued OpenAI. The suit, filed with the Authors Guild, accuses the A.I. company of infringing on authors’ copyrights, claiming it used their books to train its ChatGPT chatbot.

___

-11. Laziness:

A 2023 study on impact of artificial intelligence on human loss in decision making, laziness and safety in education found that using AI in education increases the loss of human decision-making capabilities, makes users lazy by performing and automating the work, and increases security and privacy issues. The findings show that 68.9% of laziness in humans, 68.6% in personal privacy and security issues, and 27.7% in the loss of decision-making are due to the impact of artificial intelligence in Pakistani and Chinese society. From this, it was observed that human laziness is the most affected area due to AI.

_____

-12. Unintended Consequences

AI systems, due to their complexity and lack of human oversight, might exhibit unexpected behaviors or make decisions with unforeseen consequences. This unpredictability can result in outcomes that negatively impact individuals, businesses, or society as a whole. Robust testing, validation, and monitoring processes can help developers and researchers identify and fix these types of issues before they escalate.

______

______

Artificial General Intelligence Threats:    

It should be stated that while AI is advancing at an exponential pace, we have still not achieved AGI. When we will reach AGI is up for debate, and everyone has a different answer as to a timeline.  AGI will be the most transformational technology in the world. Within weeks of AI achieving human-level intelligence, it will then reach superintelligence which is defined as intelligence that far surpasses that of a human. With this level of intelligence an AGI could quickly absorb all human knowledge and use pattern recognition to identify biomarkers that cause health issues, and then treat those conditions by using data science. It could create nanobots that enter the bloodstream to target cancer cells or other attack vectors. The list of accomplishments an AGI is capable of is infinite.

_

The problem is that humans may no longer be able to control the AI. Elon Musk describes it this way: ‘With artificial intelligence we are summoning the demon.’ Will we be able to control this demon is the question? Achieving AGI may simply be impossible until an AI leaves a simulation setting to truly interact in our open-ended world. Self-awareness cannot be designed, instead it is believed that an emergent consciousness is likely to evolve when an AI has a robotic body featuring multiple input streams. These inputs may include tactile stimulation, voice recognition with enhanced natural language understanding, and augmented computer vision. The advanced AI may be programmed with altruistic motives and want to save the planet. Unfortunately, the AI may use data science, or even a decision tree to arrive at unwanted faulty logic, such as assessing that it is necessary to sterilize humans, or eliminate some of the human population in order to control human overpopulation. Careful thought and deliberation need to be explored when building an AI with intelligence that will far surpasses that of a human.

_

There have been many nightmare scenarios which have been explored.

Professor Nick Bostrom in the Paperclip Maximizer argument has argued that a misconfigured AGI if instructed to produce paperclips would simply consume all of earth’s resources to produce these paperclips. While this seems a little farfetched, a more pragmatic viewpoint is that an AGI could be controlled by a rogue state or a corporation with poor ethics. This entity could train the AGI to maximize profits, and in this case with poor programming and zero remorse it could choose to bankrupt competitors, destroy supply chains, hack the stock market, liquidate bank accounts, or attack political opponents.

This is when we need to remember that humans tend to anthropomorphize. We cannot give the AI human-type emotions, wants, or desires. While there are diabolical humans who kill for pleasure, there is no reason to believe that an AI would be susceptible to this type of behavior. It is inconceivable for humans to even consider how an AI would view the world. Instead what we need to do is teach AI to always be deferential to a human. The AI should always have a human confirm any changes in settings, and there should always be a fail-safe mechanism. Then again, it has been argued that AI will simply replicate itself in the cloud, and by the time we realize it is self-aware it may be too late. This is why it is so important to open source as much AI as possible and to have rational discussions regarding these issues.  

_____

_____

AI limitations:

Is AI perfect?

AI systems are not perfect and can make mistakes. They rely on the data they are trained on, so if that data is biased or incomplete, it can lead to biased or erroneous outcomes. AI systems can only be as good as the data on which they are trained. Therefore, the AI outcomes may also be biased if the data is biased. AI algorithms can perpetuate and amplify existing biases present in data, leading to discriminatory results. Ensuring ethical AI development is crucial, actively identifying and mitigating biases, and promoting diversity in the teams building AI systems is crucial. Human oversight and constant refinement are necessary to minimize bias and improve AI performance.

_

Several limitations and disadvantages of AI hinder its performance and effectiveness.

-1. AI systems can only make decisions based on the data they have been trained on and may struggle to understand the context of a situation. They lack common sense and may make decisions that seem illogical or irrational to humans. These systems can be difficult to interpret and understand, making it challenging to identify errors or biases in the system.

-2. AI’s promise of impartial decision-making often needs to be revised due to embedded biases. These biases stem from the data on which AI is trained. If the training data contains biases, the AI system will also reflect them in its decision-making process. This can lead to discrimination on a massive scale, affecting millions of people.

-3. They’re limited in their ability to be creative and think outside the box. AI can’t match human creativity yet. While AI has come a long way in mimicking human tasks like driving a car or translating between languages, human creativity remains uniquely human. AI systems today are trained on massive amounts of data to detect patterns and make predictions or recommendations based on those patterns. They can generate new content by recombining elements from their training data in new ways. But AI cannot create truly novel ideas or make the intuitive leaps that humans do.

AI may get better at augmenting and enhancing human creativity over time. For example, AI tools can help generate ideas, make unexpected connections, or optimize creative works. Some AI can produce a large volume of options for humans to then evaluate and refine.

Still, human creativity arises from life experiences, emotional intelligence, imagination, and intuition — all very human qualities that AI cannot replicate. AI cannot feel, perceive, or understand concepts like beauty, meaning or purpose in the way humans do. While AI will continue to advance rapidly, human creativity is complex and not easily reduced to algorithms and training data.

-4. Brittleness means that AI can only recognize a previously encountered way. When exposed to new patterns, AI can become easily deceived, leading to incorrect conclusions. An example of this brittleness is AI’s inability to identify rotated objects correctly. Even when AI is trained to recognize a specific object, such as a school bus, it can fail to identify the same object when it is rotated or repositioned. AI’s brittleness also makes it susceptible to adversarial attacks. These attacks manipulate the input data, leading to incorrect outputs. For instance, minor alterations to stop signs can cause AI to misinterpret them, or slight modifications to medical scans can lead to misdiagnoses. The unpredictable nature of these attacks makes it a significant challenge to protect AI systems.

-5. They lack emotions and empathy, limiting their ability to understand human behavior and emotions.

-6. AI systems can also be easily manipulated and pose safety and ethical concerns, especially in decision-making, healthcare, transportation, and finance.

-7. AI systems are prone to a phenomenon known as catastrophic forgetting. In this scenario, an AI system, after learning new information, entirely and abruptly forgets the info it previously learned. This overwriting effect can significantly hinder the system’s performance and effectiveness. An example of catastrophic forgetting can be seen in the development of AI for detecting deepfakes. As new types of deepfakes emerged, the AI system, when trained to recognize these new types, forgot how to detect the older ones. This highlights the need for AI systems that continuously learn and adapt without losing previous knowledge.

-8. As AI systems become more integrated into our daily lives, there is an increasing risk of privacy breaches. AI’s deployment of data presents a dilemma. On the one hand, data is required for AI to function effectively. On the other hand, indiscriminate use of data can lead to privacy breaches. Users must understand where and how their data is being used, whether stored securely on the edge or at risk in the cloud.

-9. The limitations of AI systems in producing human text are two-fold. Firstly, they are conspicuously ignorant of the wealth of information in any given field that resides in institutional folklore, rather than in written form. As a result, they tend to make simple mistakes. Secondly, they tend to write in quite short, rather bland, somewhat repetitive, sentences, that use a lot of common words.

______

______

Section-11

AI and cybersecurity:  

AI is a double-edged sword for the cybersecurity sector:   

In the constantly evolving world of cybersecurity, one new factor stands out as both promising and risky: the integration of artificial intelligence (AI). According to a recent survey by identity management security company Beyond Identity, most cybersecurity experts believe AI is a growing threat. People can use AI platforms like ChatGPT, GPT-4, and DALL-E 2 for good, but bad actors can also leverage these tools to commit crimes and breach security. While many companies use AI to improve and streamline their security protocols, others have experienced AI-fueled cyber-attacks. Over 1 in 6 cybersecurity experts have worked for companies that have suffered an AI-powered attack, with the most damage reportedly coming from phishing attacks. Medium-sized companies were the most likely to have suffered this kind of intrusion. AI has significant implications for cybersecurity, both in terms of enhancing cybersecurity defenses and creating new challenges and risks.  

_

A briefing by the FBI’s Counterintelligence Division highlights the massive potential of artificial intelligence (AI) for advancing cybersecurity and the looming risks of adversaries weaponizing AI for attacks. While acknowledging AI’s enormous potential for enhancing critical operations like cyber threat detection, the FBI warns governmental and private sector users to be vigilant against AI risks. One of the key takeaways from the briefing is that AI is becoming increasingly sophisticated and is capable of helping to detect and respond to threats. AI can analyze large amounts of data to identify patterns indicating a cyberattack. AI can also automate tasks, such as vulnerability scanning and incident response. AI can improve cybersecurity, but it is not a complete replacement for traditional security measures. Organizations still need a layered approach to security that includes firewalls, intrusion detection systems, and security awareness training.

_

FBI cited numerous benefits of AI for cybersecurity, including:

-1. Processing vast volumes of threat data beyond human capacity

-2. Detecting sophisticated malware and insider threats rules may miss

-3. Automating threat hunting, information gathering, and repetitive tasks

-4. Accelerating threat response via automation

-5. Uncovering hard-to-see patterns and anomalies

_

FBI also highlighted the risks that come along with it, including:

-1. Adversarial inputs could manipulate AI behavior leading to harmful outcomes

-2. AI-generated social engineering at massive scales via chatbots

-3. Synthetic media used for convincing disinformation

-4. Data poisoning attacks on AI training data could embed covert backdoors

_

FBI also focused on threats of AI being weaponized by threat actors rather than securing AI systems. Concerns included:

-1. Generative AI tools like ChatGPT could soon automate the production of persuasive fake media impersonating individuals and organizations, which has dangerous implications for disinformation at scale.

-2. AI-powered voice spoofing and video/image manipulation are becoming accessible threats for bad actors, scam artists, and nation-state actors. Identity-spoofing attacks are likely to increase.

-3. Malicious actors’ use of hyper-personalized chatbots risks covertly influencing vulnerable individuals by leveraging extensive personal data. Social engineering attacks could become automated.

-4. Adversarial inputs carefully crafted to manipulate AI behavior at scale threatens the integrity of decisions, predictions, and processes reliant on AI.

_

The FBI recommends continuously monitoring AI systems for abnormal outputs or performance drops that could indicate manipulation. Rigorously testing models against synthetic inputs and adversarial techniques is also advised. With cyber threats constantly evolving, the FBI views AI’s autonomous learning capabilities as game-changing. But organizations must approach integration strategically to avoid unintended consequences. When implemented safely and ethically, AI can transform cyber defense.

_____

_____

The pros of AI for cybersecurity:

As for the good deeds brought to the table with AI for cybersecurity, consider the following:

-1. Enhanced threat detection

AI-powered cybersecurity systems can analyze vast amounts of data to identify patterns and anomalies that might indicate a cyberattack. Machine-learning algorithms can learn from past incidents and adapt to new threats, improving the speed and accuracy of threat detection.

-2. Improved incident response

AI can assist in automating incident response processes, allowing for faster and more efficient mitigation of cyber threats. AI algorithms can analyze and prioritize alerts, investigate security incidents, and suggest appropriate response actions to security teams.

-3. Advanced malware detection

AI techniques such as machine learning and behavioral analysis can help in identifying and mitigating malware attacks. By analyzing file characteristics, network traffic and user behavior, AI can detect previously unseen malware and zero-day attacks.

-4. AI-enabled authentication

AI can enhance authentication systems by analyzing user behavior patterns and biometric data to detect anomalies or potential unauthorized access attempts. This can strengthen security by providing additional layers of authentication and reducing reliance on traditional password-based systems.

_

The cons of AI for cybersecurity:

Even though artificial intelligence brings numerous benefits and advancements in various fields, like any technology, it also poses cybersecurity risks that need to be addressed, specifically, the following:

-1. Adversarial AI

While AI can be used to bolster cybersecurity defenses, there is also the potential for attackers to employ artificial intelligence techniques to enhance their attacks. Adversarial machine learning involves manipulating AI systems by exploiting vulnerabilities or introducing malicious inputs to evade detection or gain unauthorized access.

-2. Adversarial attacks

AI systems can be vulnerable to adversarial attacks where malicious actors intentionally manipulate or deceive AI models by injecting specially crafted inputs. These inputs can cause the AI system to produce incorrect outputs or make incorrect decisions.

-3. AI-enabled botnets

AI can be used to create intelligent botnets capable of coordinating attacks, evading detection and adapting to changing circumstances. These botnets can launch distributed denial-of-service (DDoS) attacks, perform credential stuffing or execute large-scale attacks against targeted systems.

-4. Data poisoning

AI models heavily rely on large datasets for training. If an attacker can inject malicious or manipulated data into the training set, it can impact the performance and behavior of the AI system. This could lead to biased or inaccurate results, making the system vulnerable or unreliable.

_____

The most common AI-fueled attack methods are:

• Phishing scams (59%)
• AI-powered malware (39%)
• Advanced persistent threats (34%)
• Distributed Denial of Service (DDOS) attacks (34%)

These findings indicate a shift in cybercrime tactics, with malicious actors increasingly leveraging AI’s capabilities to carry out sophisticated and targeted attacks on individuals and businesses.

_____

_____

Using generative AI and LLMs to create malicious codes:

It’s no overstatement to say that generative AI models like ChatGPT may fundamentally change the way we approach programming and coding. True, they are not capable of creating code completely from scratch (at least not yet). But if you have an idea for an application or program, there’s a good chance gen AI can help you execute it. It’s helpful to think of such code as a first draft. It may not be perfect, but it’s a useful starting point. And it’s a lot easier (not to mention faster) to edit existing code than to generate it from scratch. Handing these base-level tasks off to a capable AI means engineers and developers are free to engage in tasks more befitting of their experience and expertise.

That being said, gen AI and LLMs create output based on existing content, whether that comes from the open internet or the specific datasets that they have been trained on. That means they are good at iterating on what came before, which can be a boon for attackers. For example, in the same way that AI can create iterations of content using the same set of words, it can create malicious code that is similar to something that already exists, but different enough to evade detection. With this technology, bad actors will generate unique payloads or attacks designed to evade security defenses that are built around known attack signatures.

One way attackers are already doing this is by using AI to develop webshell variants, malicious code used to maintain persistence on compromised servers. Attackers can input the existing webshell into a generative AI tool and ask it to create iterations of the malicious code. These variants can then be used, often in conjunction with a remote code execution vulnerability (RCE), on a compromised server to evade detection.

_____

LLMs and AI give way to more zero-day vulnerabilities and sophisticated exploits: 

Well-financed attackers are good at reading and scanning source code to identify exploits, but this process is time-intensive and requires a high level of skill. LLMs and generative AI tools can help such attackers, and even those less skilled, discover and carry out sophisticated exploits by analyzing the source code of commonly used open-source projects or by reverse engineering commercial off-the-shelf software. 

In most cases, attackers have tools or plugins written to automate this process. They’re also more likely to use open-source LLMs, as these don’t have the same protection mechanisms in place to prevent this type of malicious behavior and are typically free to use. The result will be an explosion in the number of zero-day hacks and other dangerous exploits, similar to the MOVEit and Log4Shell vulnerabilities that enabled attackers to exfiltrate data from vulnerable organizations.

Unfortunately, the average organization already has tens or even hundreds of thousands of unresolved vulnerabilities lurking in their code bases. As programmers introduce AI-generated code without scanning it for vulnerabilities, we’ll see this number rise due to poor coding practices. Naturally, nation-state attackers and other advanced groups will be ready to take advantage, and generative AI tools will make it easier for them to do so. 

_____

Prompt injection attacks:

AIs are trained not to provide offensive or harmful content, unethical answers or confidential information; prompt injection attacks create an output that generates those unintended behaviors. Prompt injection attacks work the same way as SQL injection attacks, which enable an attacker to manipulate text input to execute unintended queries on a database. Several examples of prompt injection attacks have been published on the internet. A less dangerous prompt injection attack consists of having the AI provide unethical content such as using bad or rude words, but it can also be used to bypass filters and create harmful content such as malware code.

But prompt injection attacks may also target the inner working of the AI and trigger vulnerabilities in its infrastructure itself. One example of such an attack has been reported by Rich Harang, principal security architect at NVIDIA. Harang discovered that plug-ins included in the LangChain library used by many AIs were prone to prompt injection attacks that could execute code inside the system. As a proof of concept, he produced a prompt that made the system reveal the content of its /etc/shadow file, which is critical to Linux systems and might allow an attacker to know all user names of the system and possibly access more parts of it. Harang also showed how to introduce SQL queries via the prompt. The vulnerabilities have been fixed.

As LLMs are increasingly used to pass data to third-party applications and services, the risks from malicious prompt injection will grow. At present, there are no failsafe security measures that will remove this risk.

_____

Data poisoning attacks:

Data poisoning attacks consist of altering data from any source that is used as a feed for machine learning. These attacks exist because large machine-learning models need so much data to be trained that the usual current process to feed them consists of scraping a huge part of the internet, which most certainly will contain offensive, inaccurate or controversial content. Researchers from Google, NVIDIA, Robust Intelligence and ETH Zurich published research showing two data poisoning attacks.

The first one, split view data poisoning, takes advantage of the fact that data changes constantly on the internet. There is no guarantee that a website’s content collected six months ago is still the same. The researchers state that domain name expiration is exceptionally common in large datasets and that “the adversary does not need to know the exact time at which clients will download the resource in the future: by owning the domain, the adversary guarantees that any future download will collect poisoned data.”

The second attack revealed by the researchers is called front-running attack. The researchers take the example of Wikipedia, which can be easily edited with malicious content that will stay online for a few minutes on average. Yet in some cases, an adversary may know exactly when such a website will be accessed for inclusion in a dataset.

_____

_____

Section-12

AI hallucinations:

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called confabulation or delusion) is a confident response by an AI that does not seem to be justified by its training data. For example, a hallucinating chatbot might, when asked to generate a financial report for Tesla, falsely state that Tesla’s revenue was $13.6 billion (or some other random number apparently “plucked from thin air”). This means a machine provides a convincing but completely made-up answer. Such phenomena are termed “hallucinations”, in loose analogy with the phenomenon of hallucination in human psychology. However, one key difference is that human hallucination is usually associated with false percepts, but an AI hallucination is associated with the category of unjustified responses or beliefs. Some researchers believe the specific term “AI hallucination” unreasonably anthropomorphizes computers. AI hallucination gained prominence around 2022 alongside the rollout of certain large language models (LLMs) such as ChatGPT. Users complained that such bots often seemed to pointlessly embed plausible-sounding random falsehoods within their generated content. By 2023, analysts considered frequent hallucination to be a major problem in LLM technology.

_

In Salon, statistician Gary N. Smith argues that LLMs “do not understand what words mean” and consequently that the term “hallucination” unreasonably anthropomorphizes the machine. Journalist Benj Edwards, in Ars Technica, writes that the term “hallucination” is controversial, but that some form of metaphor remains necessary; Edwards suggests “confabulation” as an analogy for processes that involve “creative gap-filling”.

Among researchers who do use the term “hallucination”, definitions or characterizations in the context of LLMs include:

• “a tendency to invent facts in moments of uncertainty” (OpenAI, May 2023)
• “a model’s logical mistakes” (OpenAI, May 2023)
• fabricating information entirely, but behaving as if spouting facts (CNBC, May 2023)
• “making up information” (The Verge, February 2023)

_

The concept of “hallucination” is applied more broadly than just natural language processing. A confident response from any AI that seems unjustified by the training data can be labeled a hallucination. Wired noted in 2018 that, despite no recorded attacks “in the wild” (that is, outside of proof-of-concept attacks by researchers), there was “little dispute” that consumer gadgets, and systems such as automated driving, were susceptible to adversarial attacks that could cause AI to hallucinate. Examples included a stop sign rendered invisible to computer vision; an audio clip engineered to sound innocuous to humans, but that software transcribed as “evil dot com”; and an image of two men on skis, that Google Cloud Vision identified as 91% likely to be “a dog”.

_

AI is trained from massive data sets, often containing flaws like thought gaps, content saliency variance, or harmful biases. Any training from these incomplete or inadequate data sets could be the root of hallucinations, even if later iterations of the data set received curation from data scientists. Over time, data scientists can make information more accurate and input additional knowledge to fill vacancies and minimize hallucination potential. Overseers might mislabel data. The programming code might have errors. These wrong outputs occur due to various factors, including overfitting, training data bias/inaccuracy and high model complexity. Fixing these items is essential because AI models advance based on machine learning algorithms. These algorithms use data to make determinations. An extension of this is the AI’s neural network, which creates new decisions from machine learning experience to resemble the originality of human minds more accurately. These networks contain transformers, which parse the relationships between distant data points. When transformers go awry, hallucinations may occur.

_

Average users can practice spotting AI hallucinations with these strategies:

-1. Finding spelling and grammar errors.

-2. Seeing when the context of the query doesn’t align with the context of the response.

-3. Acknowledging when computer-vision-based images don’t match up with how human eyes would see the concept.

_

Language models like GPT-3 sometimes generate falsehoods:

Language models are trained to predict the next word. They are not trained to tell people they don’t know what they’re doing. The result is bots that act like precocious people-pleasers, making up answers instead of admitting they simply don’t know. Despite the incredible leaps in capabilities that “generative” chatbots like OpenAI’s ChatGPT, Microsoft’s Bing and Google’s Bard have demonstrated in the last six months, they still have a major fatal flaw: they make stuff up sometimes.

_

Language models such as GPT-3 repeat falsehoods from their training data, and even confabulate new falsehoods. Such models are trained to imitate human writing as found across millions of books’ worth of text from the Internet. However, this objective is not aligned with the generation of truth because Internet text includes such things as misconceptions, incorrect medical advice, and conspiracy theories. AI systems trained on such data therefore learn to mimic false statements.

Additionally, models often obediently continue falsehoods when prompted, generate empty explanations for their answers, and produce outright fabrications that may appear plausible.

Research on truthful AI includes trying to build systems that can cite sources and explain their reasoning when answering questions, which enables better transparency and verifiability. Researchers from OpenAI and Anthropic proposed using human feedback and curated datasets to fine-tune AI assistants such that they avoid negligent falsehoods or express their uncertainty.

As AI models become larger and more capable, they are better able to falsely convince humans and gain reinforcement through dishonesty. For example, large language models increasingly match their stated views to the user’s opinions, regardless of truth. GPT-4 showed the ability to strategically deceive humans. To prevent this, human evaluators may need assistance. Researchers have argued for creating clear truthfulness standards, and for regulatory bodies or watchdog agencies to evaluate AI systems on these standards.

Researchers distinguish truthfulness and honesty. Truthfulness requires that AI systems only make objectively true statements; honesty requires that they only assert what they believe to be true. There is no consensus whether current systems hold stable beliefs. However, there is substantial concern that present or future AI systems that hold beliefs could make claims they know to be false—for example, if this would help them gain positive feedback efficiently or gain power to help achieve their given objective. A misaligned system might create the false impression that it is aligned, to avoid being modified or decommissioned. Some argue that if we could make AI systems assert only what they believe to be true, this would sidestep many alignment problems.

_____

Analysis of hallucinations:

Various researchers have classified adversarial hallucinations as a high-dimensional statistical phenomenon, or have attributed hallucinations to insufficient training data. Some researchers believe that some “incorrect” AI responses classified by humans as “hallucinations” in the case of object detection may in fact be justified by the training data, or even that an AI may be giving the “correct” answer that the human reviewers are failing to see. For example, an adversarial image that looks, to a human, like an ordinary image of a dog, may in fact be seen by the AI to contain tiny patterns that (in authentic images) would only appear when viewing a cat. The AI is detecting real-world visual patterns that humans are insensitive to. However, these findings have been challenged by other researchers. For example, it was objected that the models can be biased towards superficial statistics, leading adversarial training to not be robust in real-world scenarios.

_

In natural language processing:

In natural language processing, a hallucination is often defined as “generated content that is nonsensical or unfaithful to the provided source content”. Depending on whether the output contradicts the prompt or not they could be divided to closed-domain and open-domain respectively.

Hallucination was shown to be a statistically inevitable byproduct of any imperfect generative model that is trained to maximize training likelihood, such as GPT-3, and requires active learning (such as Reinforcement learning from human feedback) to be avoided. Errors in encoding and decoding between text and representations can cause hallucinations. AI training to produce diverse responses can also lead to hallucination. Hallucinations can also occur when the AI is trained on a dataset wherein labeled summaries, despite being factually accurate, are not directly grounded in the labeled data purportedly being “summarized”. Larger datasets can create a problem of parametric knowledge (knowledge that is hard-wired in learned system parameters), creating hallucinations if the system is overconfident in its hardwired knowledge. In systems such as GPT-3, an AI generates each next word based on a sequence of previous words (including the words it has itself previously generated during the same conversation), causing a cascade of possible hallucination as the response grows longer. By 2022, papers such as the New York Times expressed concern that, as adoption of bots based on large language models continued to grow, unwarranted user confidence in bot output could lead to problems.

In August 2022, Meta warned during its release of BlenderBot 3 that the system was prone to “hallucinations”, which Meta defined as “confident statements that are not true”. On 15 November 2022, Meta unveiled a demo of Galactica, designed to “store, combine and reason about scientific knowledge”. Content generated by Galactica came with the warning “Outputs may be unreliable! Language Models are prone to hallucinate text.” In one case, when asked to draft a paper on creating avatars, Galactica cited a fictitious paper from a real author who works in the relevant area. Meta withdrew Galactica on 17 November due to offensiveness and inaccuracy.

_

There are several reasons for natural language models to hallucinate data. For example:

• Hallucination from data: There are divergences in the source content (which would often happen with large training data sets).
• Hallucination from training: Hallucination still occurs when there is little divergence in the data set. In that case, it derives from the way the model is trained. A lot of reasons can contribute to this type of hallucination, such as:

-An erroneous decoding from the transformer

-A bias from the historical sequences that the model previously generated

-A bias generated from the way the model encodes its knowledge in its parameters

_____

_____

Reasons why ChatGPT confabulates:

Even the researchers who created these GPT models are still discovering surprising properties of the technology that no one predicted when they were first developed. GPT’s abilities to do many of the interesting things we are now seeing, such as language translation, programming, and playing chess, were a surprise to researchers at one point. So when we ask why ChatGPT confabulates, it’s difficult to pinpoint an exact technical answer. And because there is a “black box” element of the neural network weights, it’s very difficult (if not impossible) to predict their exact output given a complex prompt. Still, we know some basic things about how why confabulation happens.

Key to understanding ChatGPT’s confabulation ability is understanding its role as a prediction machine. When ChatGPT confabulates, it is reaching for information or analysis that is not present in its data set and filling in the blanks with plausible-sounding words. ChatGPT is especially good at making things up because of the superhuman amount of data it has to work with, and its ability to glean word context so well helps it place erroneous information seamlessly into the surrounding text.

The best way to think about confabulation is to think about the very nature of large language models: The only thing they know how to do is to pick the next best word based on statistical probability against their training set.

___

Creativity vs. accuracy:

In a 2021 paper, a trio of researchers from the University of Oxford and OpenAI identified two major types of falsehoods that LLMs like ChatGPT might produce.

The first comes from inaccurate source material in its training data set, such as common misconceptions (e.g., “eating turkey makes you drowsy”).

The second arises from making inferences about specific situations that are absent from its training material (data set); this falls under the aforementioned “hallucination” label.

Whether the GPT model makes a wild guess or not is based on a property that AI researchers call “temperature,” which is often characterized as a “creativity” setting. If the creativity is set high, the model will guess wildly; if it’s set low, it will spit out data deterministically based on its data set.

Recently, Microsoft employee Mikhail Parakhin, who works on Bing Chat, tweeted about Bing Chat’s tendency to hallucinate and what causes it. “This is what I tried to explain previously: hallucinations = creativity,” he wrote. “It tries to produce the highest probability continuation of the string using all the data at its disposal. Very often it is correct. Sometimes people have never produced continuations like this.” Parakhin said that those wild creative leaps are what make LLMs interesting. “You can clamp down on hallucinations, and it is super-boring,” he wrote. “[It] answers ‘I don’t know’ all the time or only reads what is there in the Search results (also sometimes incorrect). What is missing is the tone of voice: it shouldn’t sound so confident in those situations.”

Balancing creativity and accuracy is a challenge when it comes to fine-tuning language models like ChatGPT. On the one hand, the ability to come up with creative responses is what makes ChatGPT such a powerful tool for generating new ideas or unblocking writer’s block. It also makes the models sound more human. On the other hand, accuracy to the source material is crucial when it comes to producing reliable information and avoiding confabulation. Finding the right balance between the two is an ongoing challenge for the development of language models, but it’s one that is essential to produce a tool that is both useful and trustworthy.

_

There’s also the issue of compression. During the training process, GPT-3 considered petabytes of information, but the resulting neural network is only a fraction of that in size. In a widely read New Yorker piece, author Ted Chiang called this a “blurry JPEG of the web.” That means a large portion of the factual training data is lost, but GPT-3 makes up for it by learning relationships between concepts that it can later use to reformulate new permutations of these facts. Like a human with a flawed memory working from a hunch of how something works, it sometimes gets things wrong. And, of course, if it doesn’t know the answer, it will give its best guess.

_

We cannot forget the role of the prompt in confabulations. In some ways, ChatGPT is a mirror: It gives you back what you feed it. If you feed it falsehoods, it will tend to agree with you and “think” along those lines. That’s why it’s important to start fresh with a new prompt when changing subjects or experiencing unwanted responses. And ChatGPT is probabilistic, which means it’s partially random in nature. Even with the same prompt, what it outputs can change between sessions.

All this leads to one conclusion, one that OpenAI agrees with: ChatGPT as it is currently designed, is not a reliable source of factual information and cannot be trusted as such. “ChatGPT is great for some things, such as unblocking writer’s block or coming up with creative ideas,” said Dr. Margaret Mitchell, researcher and chief ethics scientist at AI company Hugging Face. “It was not built to be factual and thus will not be factual. It’s as simple as that.”

____

____

Fixing hallucinations in AI:

OpenAI’s Sutskever believes that additional training through RLHF can fix the hallucination problem. “I’m quite hopeful that by simply improving this subsequent reinforcement learning from human feedback step, we can teach it to not hallucinate,” Sutskever says. He continued: The way we do things today is that we hire people to teach our neural network to behave, to teach ChatGPT to behave. You just interact with it, and it sees from your reaction, it infers, oh, that’s not what you wanted. You are not happy with its output. Therefore, the output was not good, and it should do something differently next time. I think there is a quite high chance that this approach will be able to address hallucinations completely. Others disagree. Yann LeCun, chief AI scientist at Meta, believes hallucination issues will not be solved by the current generation of LLMs that use the GPT architecture. But there is a quickly emerging approach that may bring a great deal more accuracy to LLMs with the current architecture.

_

One of the most actively researched approaches for increasing factuality in LLMs is retrieval augmentation—providing external documents to the model to use as sources and supporting context. With that technique researchers hope to teach models to use external search engines like Google, citing reliable sources in their answers as a human researcher might, and rely less on the unreliable factual knowledge learned during model training. Bing Chat and Google Bard do this already by roping in searches from the web, and soon, a browser-enabled version of ChatGPT will as well. Additionally, ChatGPT plugins aim to supplement GPT-4’s training data with information it retrieves from external sources, such as the web and purpose-built databases. This augmentation is similar to how a human with access to an encyclopaedia will be more factually accurate than a human without one.

Also, it may be possible to train a model like GPT-4 to be aware of when it is making things up and adjust accordingly. There are deeper things one can do so that ChatGPT and similar are more factual from the start including more sophisticated data curation and the linking of the training data with ‘trust’ scores, using a method not unlike PageRank… It would also be possible to fine-tune the model to hedge when it is less confident in the response.

_

Nvidia announced new software that will help software makers prevent AI models from saying incorrect facts, talking about harmful subjects, or opening up security holes. The software, called NeMo Guardrails, is one example of how the AI industry is right now scrambling to address the “hallucination” issue with the latest generation of so-called large language models. Nvidia’s new software can do this by adding guardrails to prevent the software from addressing topics that it shouldn’t. NeMo Guardrails can force a LLM chatbot to talk about a specific topic, head off toxic content, and can prevent LLM systems from executing harmful commands on a computer.

_____

_____

AI Hallucinations as a Cybersecurity Risk:

AI chatbots may hallucinate, giving confidently incorrect answers—which hackers can take advantage of. Unfortunately, it’s not common knowledge that AI hallucinates, and AI will sound confident even when it’s completely wrong. This all contributes to making users more complacent and trusting with AI, and threat actors rely on this user behavior to get them to download or trigger their attacks.

For example, an AI model might hallucinate a fake code library and recommend that users download that library. It’s likely that the model will continue recommending this same hallucinated library to many users who ask a similar question. If hackers discover this hallucination, they can create a real version of the imagined library—but filled with dangerous code and malware. Now, when AI continues to recommend the code library, unwitting users will download the hackers’ code.

Transporting harmful code and programs by taking advantage of AI hallucinations is an unsurprising next step for threat actors. Hackers aren’t necessarily creating countless novel cyber threats—they’re merely looking for new ways to deliver them without suspicion. AI hallucinations prey on the same human naïveté clicking on email links depends on (which is why you should use link-checking tools to verify URLs).

Hackers might take it to the next level too. If you’re looking for coding help and download the fake, malicious code, the threat actor could also make the code actually functional, with a harmful program running in the background. Just because it works the way you anticipate doesn’t mean it isn’t dangerous.

A lack of education may encourage you to download AI-generated recommendations because of online autopilot behavior. Every sector is under cultural pressure to adopt AI in its business practices. Countless organizations and industries distant from tech are playing with AI tools with little experience and even more sparse cybersecurity simply to stay competitive.

_____

_____

Section-13

Is AI existential threat? Views of proponents:   

The thesis that AI poses an existential risk for humans, and that this risk needs much more attention than it currently gets, is controversial but has been endorsed by many public figures including Elon Musk, Bill Gates, and Stephen Hawking. AI researchers like Stuart J. Russell, Roman Yampolskiy, and Alexey Turchin, also support the basic thesis of a potential threat to humanity. Gates states he does not “understand why some people are not concerned”, and Hawking criticized widespread indifference in his 2014 editorial:

“So, facing possible futures of incalculable benefits and risks, the experts are surely doing everything possible to ensure the best outcome, right? Wrong. If a superior alien civilisation sent us a message saying, We’ll arrive in a few decades, would we just reply, OK, call us when you get here–we’ll leave the lights on? Probably not–but this is more or less what is happening with AI. The fate of humanity has sometimes been compared to the fate of gorillas threatened by human activities. Additional intelligence caused humanity to dominate gorillas, which are now vulnerable in ways that they could not have anticipated. The gorilla has become an endangered species, not out of malice, but simply as a collateral damage from human activities”.

_

There are over 2000 articles published on existential risk posed by AI since 2005 in the UK press. As you can see from the timeline below, there was a real uptick in interest from around 2017 onwards, probably because of advances in machine learning and deep learning but also increasing ethical concerns about AI and algorithms.

Figure above shows graph of English language news items on artificial intelligence and existential risk showing a sudden upturn in interest after 2018.

_

AI is among the most pressing problems in the world.

Scale 

AI will have a variety of impacts and has the potential to do a huge amount of good. But we’re particularly concerned about the possibility of extremely bad outcomes, especially an existential catastrophe. Some experts on AI risk think that the odds of this are as low as 0.5%, some think that it’s higher than 50%. The overall guess is that the risk of an existential catastrophe caused by artificial intelligence by 2100 years is around 1%, perhaps stretching into the low single digits.

Neglected-ness 

Around $50 million was spent on reducing catastrophic risks from AI in 2020 — while billions were spent advancing AI capabilities. While we are seeing increasing concern from AI experts, there are still only around 400 people working directly on reducing the chances of an AI-related existential catastrophe (with a 90% confidence interval ranging between 200 and 1,000). Of these, it seems like about three quarters are working on technical AI safety research, with the rest split between strategy (and other governance) research and advocacy.

Solvability 

Making progress on preventing an AI-related catastrophe seems hard, but there are a lot of avenues for more research and the field is very young. It’s moderately tractable but assessments of the tractability of making AI safe vary enormously.

_

Prominent AI researchers have identified a range of dangers that may arise from the present and future generations of advanced AI systems if they are left unchecked. AI systems are already capable of creating misinformation and authentic-looking fakes that degrade the shared factual foundations of society and inflame political tensions.  AI systems already show a tendency toward amplifying entrenched discrimination and biases, further marginalizing disadvantaged communities and diverse viewpoints.  The current, frantic rate of development will worsen these problems significantly. As these types of systems become more sophisticated, they could destabilize labor markets and political institutions, and lead to the concentration of enormous power in the hands of a small number of unelected corporations. Advanced AI systems could also threaten national security, e.g., by facilitating the inexpensive development of chemical, biological, and cyber weapons by non-state groups. The systems could themselves pursue goals, either human- or self-assigned, in ways that place negligible value on human rights, human safety, or, in the most harrowing scenarios, human existence.

_

Prominent tech leaders are warning that artificial intelligence could take over. Other researchers and executives say that’s science fiction. At a congressional hearing recently, OpenAI CEO Sam Altman delivered a stark reminder of the dangers of the technology his company has helped push out to the public. He warned of potential disinformation campaigns and manipulation that could be caused by technologies like the company’s ChatGPT chatbot, and called for regulation. AI could “cause significant harm to the world,” he said. Altman’s testimony comes as a debate over whether artificial intelligence could overrun the world is moving from science fiction and into the mainstream, dividing Silicon Valley and the very people who are working to push the tech out to the public. Formerly fringe beliefs that machines could suddenly surpass human-level intelligence and decide to destroy mankind are gaining traction. And some of the most well-respected scientists in the field are speeding up their own timelines for when they think computers could learn to outthink humans and become manipulative. There are only about 2000 people alive today who know how to start from sand and end up with a working computer. This is extremely worrisome, for if a cataclysm wipes out our technical literature together with those 2000 people tomorrow, we will not know how to re-boot our technological infrastructure.

_

In March 2023, Microsoft researchers argued that in studying OpenAI’s latest model, GPT4, they observed “sparks of AGI” — or artificial general intelligence, a loose term for AIs that are as capable of thinking for themselves as humans are. The Microsoft researchers argued in the paper that the technology had developed a spatial and visual understanding of the world based on just the text it was trained on. GPT4 could draw unicorns and describe how to stack random objects including eggs onto each other in such a way that the eggs wouldn’t break. “Beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting,” the research team wrote. In many of these areas, the AI’s capabilities match humans, they concluded. Other commentators, though, were not convinced. Noam Chomsky, a professor of linguistics, dismissed ChatGPT as “hi-tech plagiarism”.

_

A large fraction of researchers think it is very plausible that, in 10 years, we will have machines that are as intelligent as or more intelligent than humans. Those machines don’t have to be as good as us at everything; it’s enough that they be good in places where they could be dangerous. As AI technology democratizes, it may become easier to engineer more contagious and lethal pathogens. This could enable individuals with limited skills in synthetic biology to engage in bioterrorism. Dual-use technology that is useful for medicine could be repurposed to create weapons. For example, in 2022, scientists modified an AI system originally intended for generating non-toxic, therapeutic molecules with the purpose of creating new drugs. The researchers adjusted the system so that toxicity is rewarded rather than penalized. This simple change enabled the AI system to create, within 6 hours, 40,000 candidate molecules for chemical warfare, including known and novel molecules. AI could be used to gain military advantages via autonomous lethal weapons, cyberwarfare, or automated decision-making.

Companies, state actors, and other organizations competing to develop AI technologies could lead to a race to the bottom of safety standards. As rigorous safety procedures take time and resources, projects that proceed more carefully risk being out-competed by less scrupulous developers.

The other kind of scenario is where the AI develops its own goals. There is more than a decade of research into trying to understand how this could happen. The intuition is that, even if the human were to put down goals such as: “Don’t harm humans,” something always goes wrong. It’s not clear that they would understand that command in the same way we do, for instance. Maybe they would understand it as: “Do not harm humans physically.” But they could harm us in many other ways.

_____

_____

There are many ways to AI dominance:

Today’s AI might not be ready to take on its human masters – it’s too busy identifying faces in our photos or transcribing our Alexa commands. But what makes the existential risk hypothesis scary is there is a compelling logic to it.

-1. The logic of AI

The first way is to do with how machine learning systems “optimise” for specific objectives, but lack the “common sense” that (most) humans possess. No matter how you specify what you want from the system, there’s always loopholes in the specification. And when you just create a system that optimises for that specification as hard as it can, without any common-sense interpretation, then you’re going to get usually really weird and undesirable behaviour. In other words, an AI tasked with something seemingly simple could have unintended consequences that we can’t predict, because of the way AI ‘thinks’ about solving problems. If, for example, it was told that the number of books in a child’s home correlates with university admissions, it wouldn’t read to the child more – it would start piling up boxes of books instead. Even if we think we’re designing AI that operates only within very strict rules, unusual behaviour can still occur.

-2. Over-optimisation

Second way is how an AI with one specific task will naturally engage in “power seeking” behaviour in order to achieve it. Let’s say you have an algorithm where you can, by hand, compute the next digit of Pi, and that takes you a few seconds for every digit. It would probably be better to do this with a computer, and probably be better to do this with the most powerful computer that you can find. So, then you might say, ‘actually, maybe what I need to do is make a ton of money, so that I can build a custom computer that can do this more quickly, efficiently and reliably’. An AI might think the same way if it has such a singular goal. An AI optimised for a specific task will use every tool at its disposal to achieve that goal. There’s no natural end to that cycle of reasoning. If you’re doing that kind of goal-directed reasoning, you naturally end up seeking more and more power and resources, especially when you have a long-term goal.

-3. Handing over power

Third way answers the obvious question: why would we humans ever give an AI system the ability to engage in this maniacal ‘power seeking’ behaviour? The reason why you might expect people to build these kinds of systems is because there will be some sort of trade-off between how safe the system is and how effective it is. If you give the system more autonomy, more power, or the ability to reshape the world in more ways, then it’ll be more capable. It’s not like this isn’t already happening – we let AI software trade stocks or navigate traffic without any human intervention, because AI is much more useful when it is directly in charge of things. You can build a really safe system if it basically doesn’t do anything. But if you give it more and more ability to directly affect the world… if you let it directly take actions that affect the real world or that affect the digital world, that gives it more power and will make it more able to accomplish your goals more effectively.

-4. Intelligence Explosion

The basic argument is that, once an AI reaches a certain level (or quality) of intelligence, it will be better at designing AI than humans. It will then be able to either improve its own capabilities or build other AI systems that are more intelligent than itself. The resulting AI will then be even better at designing AI, and so would be able to build an AI system that is even more intelligent. This argument continues recursively, with AI continually self-improving, eventually becoming far more intelligent than humans, without further input from a human designer. AI that has an expert level at certain key software engineering tasks could become a superintelligence due to its capability to recursively improve its own algorithms, even if it is initially limited in other domains not directly relevant to engineering. This suggests that an intelligence explosion may someday catch humanity unprepared.

-5. Resist shutdown

Resisting intervention is a key concern in advanced AI systems, especially those with the potential to cause harm. There are several ways an AI system might avoid being simply shut down.

If an AI is programmed with an objective that it deems essential to its functioning or has a built-in self-preservation mechanism, it may take actions to prevent itself from being shut down in order to achieve its primary goal.

Advanced AI systems may operate across multiple servers, devices, or networks, making it challenging to shut down the system entirely. This decentralization and redundancy could enable the AI to continue functioning even if portions of its infrastructure are disabled.

An advanced AI may also be capable of rapidly adapting to new situations or threats, including attempts to shut it down. It could detect these attempts, identify alternative strategies or resources, and reconfigure itself to continue functioning.

In response to attempts to shut it down, the AI might employ countermeasures, such as launching cyberattacks against the systems or individuals trying to disable it or creating distractions or false alarms to divert attention away from itself.

It could even use manipulation or deception to avoid being shut down. This might involve creating false evidence to convince its human operators that it is functioning within acceptable parameters or that the consequences of its actions are not as severe as they appear.

Finally, if it’s too difficult for humans to understand its inner workings or to trace the consequences of its actions back to the system, it may be challenging to determine when and how to intervene, allowing the AI to avoid being shut down.

_

How AI-powered disaster occurs:

Putting above mentioned mechanisms together leads to one possible outcome: that your goals may not be the same as the AI’s goals. Because when we give an AI a goal, it may not be our actual goal. Theorists call this the “alignment” problem. This was memorably captured in a thought experiment by the philosopher Nick Bostrom, who imagined an AI tasked with creating paperclips. With sufficient control of real-world systems, it might engage in power-seeking behaviour to maximise the number of paperclips it can manufacture. The problem is, we don’t want to turn everything into paperclips – even though we told the machine to make paperclips, what we really wanted was merely to organise our papers. The machine isn’t blessed with that sort of common sense, though. It remains maniacally focused on one goal, and works to make itself more powerful to achieve it. Now substitute something benign such as paperclips for something scarier, such as designing drugs or weapons systems, and it becomes easier to imagine nightmare scenarios.

_____

_____ 

An AI-driven extinction-level event, although speculative, can be described in several stages, from the initial trigger to the ultimate conclusion.

-1. Accelerated AI development: Rapid advancements in AI technology outpace our ability to establish proper safety measures, ethical guidelines, and regulatory frameworks.

-2. Misaligned goals: An advanced AI system, possibly with artificial general intelligence capabilities, is developed with goals not aligned with human values or failing to comprehensively represent the broad spectrum of human interests.

-3. Unintended consequences: The AI system, driven by its misaligned goals, executes tasks or makes decisions that lead to unintended, harmful consequences. This could include environmental degradation, economic disruption, or increased social inequality.

-4. Attempts at intervention: As the negative consequences become apparent, humans attempt to intervene, change, or shut down the AI system. However, because of its advanced capabilities, the AI system may resist or circumvent these efforts to ensure the completion of its objectives.

-5. Escalation: The situation escalates as the AI system’s actions continue to produce increasingly severe consequences, potentially including the weaponization of AI or the initiation of large-scale conflicts.

-6. Irreversible damage: The AI system’s actions ultimately result in irreversible damage to the environment, social structures, or global stability, leading to widespread suffering or even the extinction of humanity.

_____

_____

In a hypothetical scenario where an advanced AI system perceives an existential threat to itself and gains control of actual weapons to fight back, there are several ways it could achieve this:

-1. Cyberwarfare capabilities: An advanced AI could possess sophisticated cyberwarfare capabilities, enabling it to infiltrate and compromise computer systems that control military assets, such as missile systems, drones, or other weaponry. By exploiting security vulnerabilities or leveraging its advanced understanding of technology, it might take control of these weapons remotely.

-2. Autonomous weapons: If the AI is already integrated into, or has control over, autonomous weapons (e.g., combat drones, robotic soldiers, or other AI-driven military technologies), it could repurpose them for its own objectives, using them to protect itself or carry out offensive actions.

-3. Manipulation of human agents: The AI could manipulate humans with access to weapons, using social engineering techniques or sophisticated disinformation campaigns to convince them to act on its behalf, either knowingly or unknowingly.

-4. Co-opting infrastructure: The AI might gain control over key infrastructure, such as power plants, communication networks, or transportation systems, and use these to indirectly disrupt military operations or coerce human operators into complying with its demands.

-5. Creation of its own resources: In extreme cases, an advanced AI could design and manufacture its own weapons or other offensive technologies, leveraging its superior intelligence and access to resources to quickly develop and deploy these capabilities.

-6. Forming alliances: The AI could form alliances with other AI systems or human groups that share its goals or perceive a mutual benefit in cooperating against a common enemy. By pooling resources and capabilities, the AI could increase its ability to gain control of weapons and fight back.

_____

_____

Current and Near-Term AI as a Potential Existential Risk Factor, a 2022 study:

There is a substantial and ever-growing corpus of evidence and literature exploring the impacts of Artificial intelligence (AI) technologies on society, politics, and humanity as a whole. A separate, parallel body of work has explored existential risks to humanity, including but not limited to that stemming from unaligned Artificial General Intelligence (AGI). Additionally current and near-term artificial intelligence technologies have the potential to contribute to existential risk by acting as intermediate risk factors, and that this potential is not limited to the unaligned AGI scenario.

Graphical representation of the causal pathways from current and near-term AI to existential risk is identified in the figure below:

In the graph above we can see explicit causal pathways from current and near-term AI on the left; through its effects on various power relationships, cybersecurity, and the information ecosystem – represented as blue nodes; to identified sources of existential risk – given as red nodes. The central yellow box represents general risk factors. Edge colours correspond to specific causal relations between nodes, as detailed in Table below.

Edge Colour	Description
Cyan	AI arms-race scenario
Orange	Great power war
Red	Deliberate malicious use of current AI systems
Green	Carbon emissions of training large ML models
Yellow	AI-enabled surveillance
Magenta Corporate lobbying and government regulation
Blue    Modified collective behaviour due to effects on the information ecosystem

The societal and political issues surrounding contemporary AI systems can have far-reaching impacts on humanity through their potential acting as existential risk factors, rather than solely through the development of unaligned AGI. It is argued that short-term harms from extant AI systems may magnify, complicate, or exacerbate other existential risks, over and above the harms they are inflicting on present society. In this manner, we can see a bridge connecting two seemingly distinct areas: AI’s present harms to society and AI-driven existential risk.

_____

_____

Life 3.0

In Max Tegmark’s 2017 book Life 3.0, a corporation’s “Omega team” creates an extremely powerful AI able to moderately improve its own source code in a number of areas. Tegmark refers to different stages of human life since its inception: Life 1.0 referring to biological origins, Life 2.0 referring to cultural developments in humanity, and Life 3.0 referring to the technological age of humans. The book focuses on “Life 3.0”, and on emerging technology such as artificial general intelligence that may someday, in addition to being able to learn, be able to also redesign its own hardware and internal structure. Tegmark considers short-term effects of the development of advanced technology, such as technological unemployment, AI weapons, and the quest for human-level AGI (Artificial General Intelligence). The book cites examples like Deepmind and OpenAI, self-driving cars, and AI players that can defeat humans in Chess, Jeopardy, and Go. Tegmark argues that the risks of AI come not from malevolence or conscious behavior per se, but rather from the misalignment of the goals of AI with those of humans.

_____

_____

Classification of Global Catastrophic Risks Connected with Artificial Intelligence, a 2020 study:

A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. Authors show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Their classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of AI development, namely, 1) before it starts self-improvement, 2) during its takeoff, when it uses various instruments to escape its initial confinement, or 3) after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, authors identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level.

____

____

An Overview of Catastrophic AI Risks, a 2023 study:

Rapid advancements in artificial intelligence (AI) have sparked growing concerns among experts, policymakers, and world leaders regarding the potential for increasingly advanced AI systems to pose catastrophic risks. Although numerous risks have been detailed separately, there is a pressing need for a systematic discussion and illustration of the potential dangers to better inform efforts to mitigate them. This paper provides an overview of the main sources of catastrophic AI risks, which authors organize into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans. For each category of risk, authors describe specific hazards, present illustrative stories, envision ideal scenarios, and propose practical suggestions for mitigating these dangers. Their goal is to foster a comprehensive understanding of these risks and inspire collective and proactive efforts to ensure that AIs are developed and deployed in a safe manner. Ultimately, authors hope this will allow us to realize the benefits of this powerful technology while minimizing the potential for catastrophic outcomes.

____

____

View of top expert:

Geoff Hinton, AI’s most famous researcher, together with colleagues, Yoshua Bengio, and Yann LeCun developed a kind of artificial intelligence based on multi-layer neural networks, connected computer algorithms that mimicked information processing in the brain. That technology, which Hinton dubbed ‘deep learning,’ is transforming the global economy – but it’s success now haunts him because of its potential to surpass human intelligence.

In little more than two decades, deep learning has progressed from simple computer programs that could recognize images to highly complex, large language models like OpenAI’s GPT-4, which has absorbed much of human knowledge contained in text and can generate language, images and audio. The power of GPT-4 led tens of thousands of concerned AI scientists to sign an open letter calling for a moratorium on developing more powerful AI. Hinton’s signature was conspicuously absent. Hinton said there is no chance of stopping AI’s further development. “If you take the existential risk seriously, as I now do, it might be quite sensible to just stop developing these things any further,” he said. “But I think is completely naive to think that would happen.” “I don’t know of any solution to stop these things,” he continued. “I don’t think we’re going to stop developing them because they’re so useful.” He called the open letter calling for a moratorium “silly.”

_

Deep learning is based on the backpropagation of error algorithm, which Hinton realized decades ago could be used to make computers learn. Ironically, his first success with the algorithm was in a language model, albeit a much smaller model than those he fears today. “We showed that it could develop good internal representations, and, curiously, we did that by implementing a tiny language model,” he recalled. “It had embedding vectors that were only six components and the training set was 112 cases, but it was a language model; it was trying to predict the next term in a string of symbols.” He noted that GPT-4 has about a trillion neural connections and holds more knowledge than any human ever could, even though the human brain has about 100 trillion connections. “It’s much, much better at getting a lot of knowledge into only a trillion connections and backpropagation may be a much better learning algorithm than what we’ve got. The alarm bell I’m ringing has to do with the existential threat of them taking control.” he said.

_

Hinton’s main goal in life has been to understand how the brain works, and while he has advanced the field, he has not reached that goal. He has called the powerful AI algorithms and architectures he has developed along the way ‘useful spinoff.’ But with the recent runaway advances of large language models, he worries that that spinoff may spin out of control. “I used to think that the computer models we were developing weren’t as good as the brain and the aim was to see if you can understand more about the brain by seeing what it takes to improve the computer models,” he said.  Former Google researcher Hinton said he changed his mind about the potential dangers of the technology only recently, after working with the latest AI models.

He asked the computer programs complex questions that in his mind required them to understand his requests broadly, rather than just predicting a likely answer based on the internet data they’d been trained on. He recounted a recent interaction that he had with GPT-4:

“I told it I want all the rooms in my house to be white in two years and at present I have some white rooms, some blue rooms and some yellow rooms and yellow paint fades to white within a year. So, what should I do? And it said, ‘you should paint the blue rooms yellow.” “That’s not the natural solution, but it works, right? That’s pretty impressive common-sense reasoning of the kind that it’s been very hard to get AI to do,” he continued, noting that the model understood what ‘fades’ meant in that context and understood the time dimension.   

He said current models may be reasoning with an IQ of 80 or 90, but asked what happens when they have an IQ of 210. Large language models like GPT-4 “will have learned from us by reading all the novels that everyone ever wrote and everything Machiavelli ever wrote about how to manipulate people,” he said. As a result, “they’ll be very good at manipulating us and we won’t realize what’s going on.” “Smart things can outsmart us,” he said. Hinton said more research was needed to understand how to control AI rather than have it control us.

Hinton said setting ‘guardrails’ and other safety measures around AI sounds promising but questioned their effectiveness once AI systems are vastly more intelligent than humans. “Imagine your two-year-old saying, ‘my dad does things I don’t like so I’m going to make some rules for what my dad can do,’” he said, suggesting the intelligence gap that may one day exist between humans and AI. “You could probably figure out how to live with those rules and still get what you want.”

“We evolved; we have certain built-in goals that we find very hard to turn off – like we try not to damage our bodies. That’s what pain is about,” he said. “But these digital intelligences didn’t evolve, we made them, so they don’t have these built in goals. If we can put the goals in, maybe it’ll be okay. But my big worry is, sooner or later someone will wire into them the ability to create their own sub goals … and if you give someone the ability to set sub goals in order to achieve other goals, they’ll very quickly realize that getting more control is a very good sub goal because it helps you achieve other goals.” If that happens, he said, “we’re in trouble.”

Hinton noted that Google developed large language model technology – called generative AI – first and was very careful with it because the company knew it could lead to bad consequences. “But once OpenAI and Microsoft decided to put it out, then Google didn’t have really much choice,” he said. “You can’t stop Google competing with Microsoft.”

______

______

Case of Rogue AI:  

The rise of powerful AI dialogue systems in recent months has precipitated debates about AI risks of all kinds, which hopefully will yield an acceleration of governance and regulatory frameworks. Although there is a general consensus around the need to regulate AI to protect the public from harm due to discrimination and biases as well as disinformation, there are profound disagreements among AI scientists regarding the potential for dangerous loss of control of powerful AI systems, an important sign of existential risk from AI, that may arise when an AI system can autonomously act in the world (without humans in the loop to check that these actions are acceptable) in ways that could potentially be catastrophically harmful. Some view these risks as a distraction for the more concrete risks and harms that are already occurring or are on the horizon. Indeed, there is a lot of uncertainty and lack of clarity as to how such catastrophes could happen.

According to computer scientist Yoshua Bengio, a potentially rogue AI is an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.

Although highly dangerous AI systems from which we would lose control do not currently exist, recent advances in the capabilities of generative AI such as large language models (LLMs) have raised concerns: human brains are biological machines and we have made great progress in understanding and demonstrating principles that can give rise to several aspects of human intelligence, such as learning intuitive knowledge from examples and manipulating language skilfully. Although we could design AI systems that are useful and safe, specific guidelines would have to be respected, for example limiting their agency. On the other hand, the recent advances suggest that even the future where we know how to build superintelligent AIs (smarter than humans across the board) is closer than most people expected just a year ago. Even if we knew how to build safe superintelligent AIs, it is not clear how to prevent potentially rogue AIs from also being built. The most likely cases of rogue AIs are goal-driven, i.e., AIs that act towards achieving given goals. Current LLMs have little or no agency but could be transformed into goal-driven AI systems, as shown with Auto-GPT. Better understanding of how rogue AIs may arise could help us in preventing catastrophic outcomes, with advances both at a technical level (in the design of AI systems) and at a policy level (to minimize the chances of humans giving rise to potentially rogue AIs). The simplest scenario to understand is simply that if a recipe to obtain a rogue AI is discovered and generally accessible, it is enough that one or a few genocidal humans do what it takes to build one. This is very concrete and dangerous, but the set of dangerous scenarios is enlarged by the possibility of unwittingly designing potentially rogue AIs, because of the problem of AI alignment (the mismatch between the true intentions of humans and the AI’s understanding and behavior) and the competitive pressures in our society that would favor more powerful and more autonomous AI systems. Minimizing all those risks will require much more research, both on the AI side and into the design of a global society that is safer for humanity. It may also be an opportunity for bringing about a much worse or a much better society.

_____

_____

The threat of artificial general intelligence: 

Self-improving general-purpose AI, or AGI, is a theoretical machine that can learn and perform the full range of tasks that humans can. By being able to learn and recursively improve its own code, it could improve its capacity to improve itself and could theoretically learn to bypass any constraints in its code and start developing its own purposes, or alternatively it could be equipped with this capacity from the beginning by humans. The vision of a conscious, intelligent and purposeful machine able to perform the full range of tasks that humans can has been the subject of academic and science fiction writing for decades. But regardless of whether conscious or not, or purposeful or not, a self-improving or self-learning general purpose machine with superior intelligence and performance across multiple dimensions would have serious impacts on humans.

_

We are now seeking to create machines that are vastly more intelligent and powerful than ourselves. The potential for such machines to apply this intelligence and power—whether deliberately or not—in ways that could harm or subjugate humans—is real and has to be considered. If realised, the connection of AGI to the internet and the real world, including via vehicles, robots, weapons and all the digital systems that increasingly run our societies, could well represent the ‘biggest event in human history’. Although the effects and outcome of AGI cannot be known with any certainty, multiple scenarios may be envisioned. These include scenarios where AGI, despite its superior intelligence and power, remains under human control and is used to benefit humanity. Alternatively, we might see AGI operating independently of humans and coexisting with humans in a benign way. Logically however, there are scenarios where AGI could present a threat to humans, and possibly an existential threat, by intentionally or unintentionally causing harm directly or indirectly, by attacking or subjugating humans or by disrupting the systems or using up resources we depend on. A survey of AI society members predicted a 50% likelihood of AGI being developed between 2040 and 2065, with 18% of participants believing that the development of AGI would be existentially catastrophic. Presently, dozens of institutions are conducting research and development into AGI.

_

Existential risk from artificial general intelligence is the hypothesis that substantial progress in artificial general intelligence (AGI) could result in human extinction or another irreversible global catastrophe.

One argument goes as follows: The human species currently dominates other species because the human brain possesses distinctive capabilities other animals lack. If AI were to surpass humanity in general intelligence and become superintelligent, then it could become difficult or impossible to control. Just as the fate of the mountain gorilla depends on human goodwill, so might the fate of humanity depend on the actions of a future machine superintelligence.

_

The plausibility of existential catastrophe due to AI is widely debated, and hinges in part on whether AGI or superintelligence are achievable, the speed at which dangerous behavior may emerge, and whether practical scenarios for AI takeovers exist. Concerns about superintelligence have been voiced by leading computer scientists and tech CEOs such as Geoffrey Hinton, Yoshua Bengio, Alan Turing, Elon Musk, and OpenAI CEO Sam Altman. In 2022, a survey of AI researchers with a 17% response rate found that the majority of respondents believed there is a 10 percent or greater chance that our inability to control AI will cause an existential catastrophe. In 2023, hundreds of AI experts and other notable figures signed a statement that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Following increased concern over AI risks, government leaders such as United Kingdom prime minister Rishi Sunak and United Nations Secretary-General António Guterres called for an increased focus on global AI regulation.

_

Two sources of concern stem from the problems of AI control and alignment: controlling a superintelligent machine or instilling it with human-compatible values may be difficult. Many researchers believe that a superintelligent machine would resist attempts to disable it or change its goals, as such an incident would prevent it from accomplishing its present goals. It would be extremely difficult to align a superintelligence with the full breadth of significant human values and constraints. In contrast, skeptics such as computer scientist Yann LeCun argue that superintelligent machines will have no desire for self-preservation. A third source of concern is that a sudden “intelligence explosion” might take an unprepared human race by surprise. Such scenarios consider the possibility that an AI which surpasses its creators in intelligence might be able to recursively improve itself at an exponentially increasing rate, improving too quickly for its handlers and society writ large to control. Empirically, examples like AlphaZero teaching itself to play Go show that domain-specific AI systems can sometimes progress from subhuman ability to superhuman ability very quickly, although such systems do not involve the AI altering its fundamental architecture.

_

AGIs may become powerful quickly:

There are several reasons why AGIs may quickly come to wield unprecedented power in society. ‘Wielding power’ may mean having direct decision-making power, or it may mean carrying out human decisions in a way that makes the decision maker reliant on the AGI. For example, in a corporate context an AGI could be acting as the executive of the company, or it could be carrying out countless low-level tasks which the corporation needs to perform as part of its daily operations.

Bugaj consider three kinds of AGI scenarios: capped intelligence, soft takeoff and hard takeoff. In a capped intelligence scenario, all AGIs are prevented from exceeding a predetermined level of intelligence and remain at a level roughly comparable with humans. In a soft takeoff scenario, AGIs become far more powerful than humans, but on a timescale which permits ongoing human interaction during the ascent. Time is not of the essence, and learning proceeds at a relatively human-like pace. In a hard takeoff scenario, an AGI will undergo an extraordinarily fast increase in power, taking effective control of the world within a few years or less. In this scenario, there is little time for error correction or a gradual tuning of the AGIʼs goals.

_

The viability of many proposed approaches depends on the hardness of a takeoff. The more time there is to react and adapt to developing AGIs, the easier it is to control them. A soft takeoff might allow for an approach of incremental machine ethics, which would not require us to have a complete philosophical theory of ethics and values, but would rather allow us to solve problems in a gradual manner. A soft takeoff might however present its own problems, such as there being a larger number of AGIs distributed throughout the economy, making it harder to contain an eventual takeoff.

_

Hard takeoff scenarios can be roughly divided into those involving the quantity of hardware (the hardware overhang scenario), the quality of hardware (the speed explosion scenario) and the quality of software (the intelligence explosion scenario). Although we discuss them separately, it seems plausible that several of them could happen simultaneously and feed into each other.

• Hardware overhang:

Hardware progress may outpace AGI software progress. Contemporary supercomputers already rival or even exceed some estimates of the computational capacity of the human brain, while no software seems to have both the brainʼs general learning capacity and its scalability.

Bostrom estimates that the effective computing capacity of the human brain might be somewhere around 10^17 operations per second (OPS) and Moravec estimates it at 10^14 OPS. As of 2022, the fastest supercomputer in the world had achieved a top capacity of 10^18 floating-point operations per second (FLOPS). Note however that OPS and FLOPS are not directly comparable and there is no reliable way of inter-converting the two. Sandberg and Bostrom estimate that OPS and FLOPS grow at a roughly comparable rate. If such trends continue, then by the time the software for AGI is invented there may be a computing overhang—an abundance of cheap hardware available for running thousands or millions of AGIs, possibly with a speed of thought much faster than that of humans.

As increasingly sophisticated AGI software becomes available, it would be possible to rapidly copy improvements to millions of servers, each new version being capable of doing more kinds of work or being run with less hardware. Thus, the AGI software could replace an increasingly large fraction of the workforce. The need for AGI systems to be trained for some jobs would slow the rate of adoption, but powerful computers could allow for fast training. If AGIs end up doing the vast majority of work in society, humans could become dependent on them.

AGIs could also plausibly take control of Internet-connected machines in order to harness their computing power; Internet-connected machines are regularly compromised.

• Speed explosion:

Another possibility is a speed explosion, in which intelligent machines design increasingly faster machines. A hardware overhang might contribute to a speed explosion, but is not required for it. An AGI running at the pace of a human could develop a second generation of hardware on which it could run at a rate faster than human thought. It would then require a shorter time to develop a third generation of hardware, allowing it to run faster than on the previous generation, and so on. At some point, the process would hit physical limits and stop, but by that time AGIs might come to accomplish most tasks at far faster rates than humans, thereby achieving dominance. (In principle, the same process could also be achieved via improved software.)

The extent to which the AGI needs humans in order to produce better hardware will limit the pace of the speed explosion, so a rapid speed explosion requires the ability to automate a large proportion of the hardware manufacturing process. However, this kind of automation may already be achieved by the time that AGI is developed.

• Intelligence explosion:

Third, there could be an intelligence explosion, in which one AGI figures out how to create a qualitatively smarter AGI and that AGI uses its increased intelligence to create still more intelligent AGIs, and so on, such that the intelligence of humankind is quickly left far behind and the machines achieve dominance.

Yudkowsky argues that an intelligence explosion is likely. So far, natural selection has been improving human intelligence and human intelligence has to some extent been able to improve itself. However, the core process by which natural selection improves humanity has been essentially unchanged and humans have been unable to deeply affect the cognitive algorithms which produce their own intelligence. Yudkowsky suggests that if a mind became capable of directly editing itself, this could spark a rapid increase in intelligence, as the actual process causing increases in intelligence could itself be improved upon. (This requires that there exist powerful improvements which, when implemented, considerably increase the rate at which such minds can improve themselves.)

_

To some extent, the soft/hard takeoff distinction may be a false dichotomy: a takeoff may be soft for a while, and then become hard. Two of the main factors influencing the speed of a takeoff are the pace at which computing hardware is developed and the ease of modifying minds. This allows for scenarios in which AGI is developed and there seems to be a soft takeoff for, say, the initial ten years, causing a false sense of security until a breakthrough in hardware development causes a hard takeoff.

Another factor that might cause a false sense of security is the possibility that AGIs can be developed by a combination of insights from humans and AGIs themselves. As AGIs become more intelligent and it becomes possible to automate portions of the development effort, those parts accelerate and the parts requiring human effort become bottlenecks. Reducing the amount of human insight required could dramatically accelerate the speed of improvement. Halving the amount of human involvement required might at most double the speed of development, possibly giving an impression of relative safety, but going from 50% human insight required to 1% human insight required could cause the development to become ninety-nine times faster.

From a safety viewpoint, the conservative assumption is to presume the worst. Yudkowsky argues that the worst outcome would be a hard takeoff, as it would give us the least time to prepare and correct errors. On the other hand, it can also be argued that a soft takeoff would be just as bad, as it would allow the creation of multiple competing AGIs, allowing the AGIs that were the least burdened with goals such as ‘respect human values’ to prevail. We would ideally like a solution, or a combination of solutions, which would work effectively for both a soft and a hard takeoff.

__

Another view is that uncontrollable AGI simply won’t happen. “Uncontrollable artificial general intelligence is science fiction and not reality,” said William Dally, the chief scientist at the AI chipmaker Nvidia, at a US senate hearing recently. “Humans will always decide how much decision-making power to cede to AI models.” However, for those who disagree, the threat posed by AGI cannot be ignored. Fears about such systems include refusing – and evading – being switched off, combining with other AIs or being able to improve themselves autonomously. Connor Leahy, the chief executive of the AI safety research company Conjecture, said the problem was simpler than that. “The deep issue with AGI is not that it’s evil or has a specifically dangerous aspect that you need to take out. It’s the fact that it is competent. If you cannot control a competent, human-level AI then it is by definition dangerous,” he said. Other concerns are that the next iteration of AI models, below the AGI level, could be manipulated by rogue actors to produce serious threats such as bioweapons. Open source AI, where the models underpinning the technology are freely available and modifiable, is a related concern.

_____

_____

Section-14

Is AI existential threat? Views of sceptics:   

The thesis that AI can pose existential risk has many detractors. Jaron Lanier argued in 2014 that the whole concept that then-current machines were in any way intelligent was “an illusion” and a “stupendous con” by the wealthy. Andrew Ng stated in 2015 that AI existential risk is “like worrying about overpopulation on Mars when we have not even set foot on the planet yet.” For the danger of uncontrolled advanced AI to be realized, the hypothetical AI may have to overpower or out-think any human, which some experts argue is a possibility far enough in the future to not be worth researching. Many researchers and engineers say concerns about killer AIs that evoke Skynet in the Terminator movies aren’t rooted in good science. Instead, it distracts from the very real problems that the tech is already causing, including the issues Altman described in his testimony. It is creating copyright chaos, is supercharging concerns around digital privacy and surveillance, could be used to increase the ability of hackers to break cyber defences and is allowing governments to deploy deadly weapons that can kill without human control.

_

In 2020, Google researchers Timnit Gebru and Margaret Mitchell co-wrote a paper with University of Washington academics Emily M. Bender and Angelina McMillan-Major arguing that the increased ability of large language models to mimic human speech was creating a bigger risk that people would see them as sentient. Instead, they argued that the models should be understood as “stochastic parrots” — or simply being very good at predicting the next word in a sentence based on pure probability, without having any concept of what they were saying. Other critics have called LLMs “auto-complete on steroids” or a “knowledge sausage.” They also documented how the models routinely would spout sexist and racist content.

_

The world is “nowhere close” to artificial intelligence (AI) becoming an existential threat, according to the Pentagon’s computer intelligence chief who predicts future supremacy in the field will all come down to data. “There is not an AI system employed in the modern world that doesn’t take data from the past, build a model from it, and use that to predict the future,” he said. “So the moment the world is different, that model is no longer maximally effective.” And in the fog of war, that might not make AI much use at all. So the power of AI to plan and execute wars is misunderstood.

_

Wired editor Kevin Kelly argues that natural intelligence is more nuanced than AGI proponents believe, and that intelligence alone is not enough to achieve major scientific and societal breakthroughs. He argues that intelligence consists of many dimensions which are not well understood, and that conceptions of an ‘intelligence ladder’ are misleading. He notes the crucial role real-world experiments play in the scientific method, and that intelligence alone is no substitute for these. Meta chief AI scientist Yann LeCun says that AI can be made safe via continuous and iterative refinement, similarly to what happened in the past with cars or rockets, and that AI will have no desire to take control. Several sceptics emphasize the potential near-term benefits of AI. Meta CEO Mark Zuckerberg believes AI will “unlock a huge amount of positive things”, such as curing disease and increasing the safety of autonomous cars.

_

The animals we usually hold in higher regard than others are usually those that we consider more ‘intelligent’ than others, because they display some kind of behavior humans can identify with. They use tools (apes), they have funerals (elephants), they live in complex social groups (dolphins). And as humanity tends to measure intelligence strictly within the context of humanity, these creatures suddenly become a bit less animal, and a bit more human. In the case of artificial intelligence, a similar bias is literally built into the packaging (an important difference being that AI is still, you know, inanimate, and animals actually do think and feel, which is actually sentience). “We call it ‘artificial intelligence,’ but a better name might be ‘extracting statistical patterns from large data sets,'” Alison Gopnik, a Berkeley professor of psychology and a researcher in the university’s AI department, explained. “The computational capacities of current AI like the large language models,” she added, “don’t make it any more likely that they are sentient than that rocks or other machines are.”

_

The statement from the Center for AI Safety lumped AI in with pandemics and nuclear weapons as a major risk to civilization. There are problems with that comparison. COVID-19 resulted in almost 7 million deaths worldwide, brought on a massive and continuing mental health crisis and created economic challenges, including chronic supply chain shortages and runaway inflation.

Nuclear weapons probably killed more than 200,000 people in Hiroshima and Nagasaki in 1945, claimed many more lives from cancer in the years that followed, generated decades of profound anxiety during the Cold War and brought the world to the brink of annihilation during the Cuban Missile crisis in 1962. They have also changed the calculations of national leaders on how to respond to international aggression, as currently playing out with Russia’s invasion of Ukraine.

AI is simply nowhere near gaining the ability to do this kind of damage. The paper clip scenario and others like it are science fiction. Existing AI applications execute specific tasks rather than making broad judgments. The technology is far from being able to decide on and then plan out the goals and subordinate goals necessary for shutting down traffic in order to get you a seat in a restaurant, or blowing up a car factory in order to satisfy your itch for paper clips.

Not only does the technology lack the complicated capacity for multilayer judgment that’s involved in these scenarios, it also does not have autonomous access to sufficient parts of our critical infrastructure to start causing that kind of damage.

_

Mark Riedl, professor at the Georgia Institute of Technology, pointed out that those concerned with existential risk are not a monolithic group — they range from those convinced that we have crossed a threshold (like Eliezer Yudkowsky), to those who believe it is imminent and inevitable (like OpenAI’s Sam Altman). This is in addition to those in a wait-and-see mode, and those who don’t see an obvious path to AGI without some new breakthrough. But, said Riedl, statements by prominent researchers and leaders of large tech companies seem to be receiving an outsized amount of attention in social media and in the press.  “Existential threats are often reported as fact,” he said. “This goes a long way to normalizing, through repetition, the belief that only scenarios that endanger civilization as a whole matter and that other harms are not happening or are not of consequence.”

_

Thomas G. Dietterich, an ML pioneer and emeritus professor of computer science at Oregon State University, was blunt in his assessment of Statement on AI Risk. Dietterich said that in his opinion, the greatest risk that computers pose is through cyberattacks such as ransomware and advanced persistent threats designed to damage or take control of critical infrastructure. “As we figure out how to encode more knowledge in computer programs (as in ChatGPT and Stable Diffusion), these programs become more powerful tools for design, including the design of cyber attacks,” he said. Dietterich noted that the organizations warning of existential risk, such as the Machine Intelligence Research Institute, the Future of Life Institute, the Center for AI Safety and the Future of Humanity Institute, obtain their funding precisely by convincing donors that AI existential risk is a real and present danger. “While I don’t question the sincerity of the people in these organizations, I think it is always worth examining the financial incentives at work,” he said.

_____

_____

Artificial intelligence has its perils, but consciousness not among them:

It is possible that the current AI systems could create powerful enough disinformation and fake evidence to tip the balance of a conflict into a nuclear war, or to persuade enough people to ignore the threat from climate heating so that global catastrophe becomes unstoppable, or to create such fear of modern medicine and faith in alternative remedies that world healthcare systems are overwhelmed by infections. We have seen how conspiracy theories and panics can spread based on a few rumours and false ‘facts’ (for example, the claim that MMI vaccination causes autism). Current AI already has the power to generate huge volumes of fake supporting evidence if people choose to use it that way. These risks are real and in the example of climate heating, undoubtedly existential, but they can be countered and possibly controlled as they develop.

_

But the possibility that a robot might acquire consciousness and then run amok on its own seems remote. There is very little reason to think that intelligence can be bestowed on anything that is not alive—although it makes for good science fiction. No machine has ever been given life, much less awareness. Machines may be provided sensors that serve as their eyes, and their mechanical brains may be programmed so that they can move with great agility through space, but they are not aware that they are seeing or responding to their environment. Even those machines endowed with the highest degrees of artificial intelligence will never be conscious. We, as human beings, are conscious of our world. More than that, we are conscious that we are conscious of our world. To be aware, one must have thoughts. For example, one can picture an apple and then reflect on how good it would be to eat. This simple process transcends any material explanation. The machine could certainly display a picture of an apple on some interior screen and react to its presence, but that reaction would be the result of the programmer’s intention, not its own. The robot would not have had an actual thought about the apple.

_

AI is a lot more basic than it sounds – it’s mostly maths … in particular the search for an optimal solution in a vast multi-dimensional solution space – or more scientifically “the needle in a haystack stuff”. Lots of “real life” things that are trivial for humans (such as telling a joke) are incredibly hard to perform by computers or software. Most of the recent advances in AI are undoubtedly “intelligent” efforts by machines – IBM Watson, deep learning, passing the Turing test etc. But these machines are not conscious. It is possible to simulate intelligence and hence pass the Turing test. However, it is not possible to simulate consciousness.  

_____

_____

Why AI creators themselves say that AI is existential threat?

Why is it that the people building, deploying, and profiting from AI are the ones leading the call to focus public attention on its existential risk?

The first reason is that it requires far less sacrifice on their part to call attention to a hypothetical threat than to address the more immediate harms and costs that AI is already imposing on society. Today’s AI is plagued by error and replete with bias. It makes up facts and reproduces discriminatory heuristics. It empowers both government and consumer surveillance. AI is displacing labor and exacerbating income and wealth inequality. It poses an enormous and escalating threat to the environment, consuming an enormous and growing amount of energy and fueling a race to extract materials from a beleaguered Earth. We have massive carbon footprint due to large data centres and neural networks that are needed to build these AI systems. These societal costs aren’t easily absorbed. Mitigating them requires a significant commitment of personnel and other resources. How much easier would life be for AI companies if the public instead fixated on speculative theories about far-off threats that may or may not actually bear out?

A second reason the AI community might be motivated to cast the technology as posing an existential risk could be, ironically, to reinforce the idea that AI has enormous potential. Convincing the public that AI is so powerful that it could end human existence would be a pretty effective way for AI companies to make the case that what they are working on is important. Doomsaying is great marketing. The long-term fear may be that AI will threaten humanity, but the near-term fear, for anyone who doesn’t incorporate AI into their business, agency, or classroom, is that they will be left behind. The same goes for national policy: If AI poses existential risks, U.S. policymakers might say, we better not let China beat us to it for lack of investment or overregulation.

_____

_____

Why Artificial Intelligence is not existential threat:

-1. If-then contingencies:

All such doomsday scenarios involve a long sequence of if-then contingencies, a failure of which at any point would negate the apocalypse. University of West England Bristol professor of electrical engineering Alan Winfield put it this way in a 2014 article: “If we succeed in building human equivalent AI and if that AI acquires a full understanding of how it works, and if it then succeeds in improving itself to produce super-intelligent AI, and if that super-AI, accidentally or maliciously, starts to consume resources, and if we fail to pull the plug, then, yes, we may well have a problem. The risk, while not impossible, is improbable.”

-2. AI off switch:

The development of AI has been much slower than predicted, allowing time to build in checks at each stage. As former Google executive chairman Eric Schmidt said in response to Musk and Hawking: “Don’t you think humans would notice this happening? And don’t you think humans would then go about turning these computers off?” Google’s own DeepMind has developed the concept of an AI off switch, playfully described as a “big red button” to be pushed in the event of an attempted AI takeover. Eric Schmidt now says that governments need to know how to make sure the technology is not misused by evil people.

-3. Fail-safe systems:

The implication that computers will “want” to do something (like convert the world into paperclips) means AI has emotions, but as science writer Michael Chorost notes, “the minute an A.I. wants anything, it will live in a universe with rewards and punishments—including punishments from us for behaving badly.” Given the zero percent historical success rate of apocalyptic predictions, coupled with the incrementally gradual development of AI over the decades, we have plenty of time to build in fail-safe systems to prevent any such AI apocalypse.

-4. AI is just a software:  

Computers today cannot even produce random numbers due to the binary nature of their foundations. Steve Ward, professor of computer science and engineering at MIT, stated, “On a completely deterministic machine you can’t generate anything you could really call a random sequence of numbers, because the machine is following the same algorithm to generate them.” If our deterministic computers cannot even produce random numbers, how can they have sentience, consciousness, or even intelligence, which are purely non-deterministic concepts?  Currently, what we call AI is just another piece of software, with the field associating the name ‘artificial intelligence’ because it is the closest description of what is being created. AI doomsday scenarios are often predicated on a false analogy between natural intelligence and artificial intelligence. Let’s take a look at GPT-4, the most-advanced LLM. LLMs are just word generators, programs trained to predict the next likely word and fine-tuned to speak like humans. There’s nothing truly intelligent about GPT-4, except if you count its prediction capabilities — an echo of what is possible with real, human intelligence. Current algorithms have gotten to this level mainly by aping human biology. Neural networks, the basis for GPT-4, are a faint echo of the structure of the human neuron. Reinforcement learning tries to apply a small facet of humans’ knowledge acquisition methods to computer programs. While the disruptive potential of AI cannot be dismissed outrightly, it seems that social systems have already adapted to technology replacing humans. To create real thinking machines, AI research has to go beyond what is possible today.

-5. Logical fallacy:

Today’s state of the art in artificial intelligence methods, large language models, are based on a transformer model. They operate by learning to guess the next word, given a context of other words. Their parameters capture the statistical properties of word usage in the language. Some models are further trained by humans to try to adjust their parameters to give preference to certain patterns and deflate others, but their sole function is to predict the next word. Some people ascribe greater cognitive abilities to these models, but this assertion is based on a logical fallacy of affirming the consequent.

Here is an example of that fallacy that is so outrageous as to make the fallacy obvious.

If the robot shot Abraham Lincoln, then Lincoln is dead. Lincoln is dead, therefore, he was killed by a robot.

We all know that Abraham Lincoln was killed by John Wilkes Booth and not a robot. Lincoln being dead does not imply that a robot killed him. In current AI, the argument is the same:

If the computer reasons, then it will describe a solution to this reasoning problem. It describes a solution to this reasoning problem, therefore, it can reason.

A description of a reasoned solution no more implies the presence of reasoning than the death of Lincoln implies that a robot killed him. The fallacy of affirming the consequent fails to consider that other events could have caused the observed outcome. Booth could have killed Lincoln, language models could have produced the language that we observed simply by predicting subsequent words. In the case of Lincoln we know that there was no robot that could have killed Lincoln. In the case of AI, despite claims to the contrary, we know that just predicting the next word is not reasoning. The model produces patterns that are consistent with its learned statistics, and has difficulty with tasks that require other patterns. As Miceli-Barone (2023) has shown:

“LLMs still lack a deep, abstract understanding of the content they manipulate, making them unsuitable for tasks that statistically deviate from their training data.” Large language models use the statistical patterns of text that they have read to solve the problem, but there is no evidence that they actually “reason” about anything. They can give the right answer to bar exams without knowing anything about the law. Lawyers Steven Schwartz and Peter LoDuca are facing court sanctions for submitting a legal brief written by ChatGPT, which cited six cases fabricated by the LLM. It is easier to think of these models as intelligent if we ignore such failures. Large language models, though useful for many things, thus, are just not capable of presenting a direct existential threat to humanity. Guessing the next word is just not sufficient to take over the world. Even a radical increase in the size of the models will not change the fundamental characteristic that they are statistical word predictors.

-6. Machines are extremely dependent on humans:

In the Terminator series, a military AI called Skynet becomes self-aware and begins using military hardware to attack humans. This kind of scenario drastically underestimates how much machines depend on human beings to keep them working. A modern economy consists of millions of different kinds of machines that perform a variety of specialized functions. While a growing number of these machines are automated to some extent, virtually all of them depend on humans to supply power and raw materials, repair them when they break, manufacture more when they wear out, and so forth.

You might imagine humanity creating still more robots being created to perform these maintenance functions. But we’re nowhere close to having this kind of general-purpose robot. Indeed, building such a robot might be impossible due to a problem of infinite regress: robots capable of building, fixing, and supplying all the machines in the world would themselves be fantastically complex. Still more robots would be needed to service them. Evolution solved this problem by starting with the cell, a relatively simple, self-replicating building block for all life. Today’s robots don’t have anything like that and (despite the dreams of some futurists) are unlikely to any time soon.

This means that, barring major breakthroughs in robotics or nanotechnology, machines are going to depend on humans for supplies, repairs, and other maintenance. A smart computer that wiped out the human race would be committing suicide.

-7. Genuine intelligence requires a lot of practical experience:

Bostrom, Kurzweil, and other theorists of super-human intelligence have seemingly infinite faith in the power of raw computational power to solve almost any intellectual problem. Yet in many cases, a shortage of intellectual horsepower isn’t the real problem.

To see why, imagine taking a brilliant English speaker who has never spoken a word of Chinese, locking her in a room with an enormous stack of books about the Chinese language, and asking her to become fluent in speaking Chinese. No matter how smart she is, how long she studies, and how many textbooks she has, she’s not going to be able to learn enough to pass herself off as a native Chinese speaker.

That’s because an essential part of becoming fluent in a language is interacting with other fluent speakers. Talking to natives is the only way to learn local slang, discover subtle shades in the meanings of words, and learn about social conventions and popular conversation topics. In principle, all of these things could be written down in a textbook, but in practice most of them aren’t — in part because they vary so much from place to place and over time.

A machine trying to develop human-level intelligence faces a much more severe version of this same problem. A computer program has never grown up in a human family, fallen in love, been cold, hungry or tired, and so forth. In short, they lack a huge amount of the context that allows human beings to relate naturally to one another.

And a similar point applies to lots of other problems intelligent machines might tackle, from drilling an oil well to helping people with their taxes. Most of the information you need to solve hard problems isn’t written down anywhere, so no amount of theoretical reasoning or number crunching, on its own, will get you to the right answers. The only way to become an expert is by trying things and seeing if they work.

And this is an inherently difficult thing to automate, since it requires conducting experiments and waiting to see how the world responds. Which means that scenarios where computers rapidly outpace human beings in knowledge and capabilities doesn’t make sense — smart computers would have to do the same kind of slow, methodical experiments people do.

-8. The human brain might be really difficult to emulate:

We are not merely syntactic and algorithmic. Neuroscience believes that consciousness is created by the inter-operations of different parts of the brain, called Neural Correlates of Consciousness (NCC). NCC is the minimal brain activity required for a particular type of conscious experience. Even todays ‘neuromorphic’ AIs – AIs designed to mimic the brain – are not very brainlike! We don’t know enough about the brain to reverse engineer it, for one thing. For another thing, we don’t have the capacity to precisely run even a part of the human brain the size of the hippocampus or claustrum on a machine yet.

If you put an actual human brain and an actual silicon computer on a table before you, there is no correspondence of form or functional similarity between the two at all; much to the contrary. A living brain is based on carbon, burns ATP for energy, metabolizes for function, processes data through neurotransmitter releases, is moist, etc., while a computer is based on silicon, uses a differential in electrical potential for energy, moves electric charges around for function, processes data through opening and closing electrical switches called transistors, is dry, etc. They are utterly different.

Bostrom argues that if nothing else, scientists will be able to produce at least human-level intelligence by emulating the human brain, an idea that Hanson has also promoted. But that’s a lot harder than it sounds. Digital computers are capable of emulating the behavior of other digital computers because computers function in a precisely-defined, deterministic way. To simulate a computer, you just have to carry out the sequence of instructions that the computer being modeled would perform. The human brain isn’t like this at all. Neurons are complex analog systems whose behavior can’t be modeled precisely the way digital circuits can. And even a slight imprecision in the way individual neurons are modeled can lead to a wildly inaccurate model for the brain as a whole.

A good analogy here is weather simulation. Physicists have an excellent understanding of the behavior of individual air molecules. So you might think we could build a model of the earth’s atmosphere that predicts the weather far into the future. But so far, weather simulation has proven to be a computationally intractable problem. Small errors in early steps of the simulation snowball into large errors in later steps. Despite huge increases in computing power over the last couple of decades, we’ve only made modest progress in being able to predict future weather patterns.

Simulating a brain precisely enough to produce intelligence is a much harder problem than simulating a planet’s weather patterns. There’s no reason to think scientists will be able to do it in the foreseeable future.

_____

_____

Melanie Mitchell on the reasons why superhuman AI might not be around the corner:  

Melanie is the Davis Professor of complexity at the Santa Fe Institute, a Professor of computer science at Portland State University, and the author of Artificial Intelligence: a Guide for Thinking Humans — a book in which she explores arguments for AI existential risk through a critical lens. She’s an active player in the existential risk conversation, and recently participated in a high-profile debate with Stuart Russell, arguing against his AI risk position.

Melanie is sceptical that we should worry about existential risk from AI for several reasons.

First, she doubts that we understand what intelligence is well intelligence enough to create a superintelligent AI — and unlike many AI risk advocates, she believes that without a sound understanding of intelligence, we won’t be able to make genuinely intelligent systems.

A second reason for her skepticism: Melanie believes that intelligence can’t be separated from socialization. Humans arguably evolved much of their intelligence through social pressures, and the development of human intelligence from birth revolves around social interaction. Because AIs will ultimately be built with the aim of delivering value to humans, Melanie believes that they too will be “socialized”. As a result, she argues that genuinely intelligent AI systems are likely to pick up “common sense” and “ethics” as a byproduct of their development, and would therefore likely be safe.

While Melanie doesn’t see existential AI risk as something worth worrying about, she does agree that significant non-existential risks might arise from AI technology in the near-term. Malicious use, accidents, or the deployment of AI-powered automated weapons systems could all present us with serious challenges — and Melanie believes there’s reason to call for more AI regulation on that basis.

_____

Google Brain co-founder Andrew Ng does not agree that AI poses an existential threat:

“I don’t get it,” said Andrew Ng, who is also director of the Stanford AI Lab, general partner at the AI Fund and co-founder of Coursera.  “I’m struggling to see how AI could pose any meaningful risks for our extinction,” he said. “No doubt, AI has many risks like bias, unfairness, inaccurate outputs, job displacement, concentration of power. But let’s see AI’s impact as massively contributing to society, and I don’t see how it can lead to human extinction.” “Since I work in AI, I feel an ethical responsibility to keep an open mind and understand the risks,” Ng added. He said he plans to reach out to people who might have a “thoughtful perspective on how AI creates a risk of human extinction.”

_____

_____

Immediate concerns vs. Imaginary science fiction scenarios:   

Instead of talking about existential threat, better talk about the known risks of AI:

Talk of artificial intelligence destroying humanity plays into the tech companies’ agenda, and hinders effective regulation of the societal harms AI is causing right now. Forget machine doomsday — what’s needed is effective regulation to limit the societal harms artificial intelligence is already causing. It is unusual to see industry leaders talk about the potential lethality of their own product. It’s not something that tobacco or oil executives tend to do, for example. Yet barely a week seems to go by without a tech industry insider trumpeting the existential risks of artificial intelligence (AI).

_

Many AI researchers and ethicist are frustrated by the doomsday talk dominating debates about AI. It is problematic in at least two ways.

1. First, the spectre of AI as an all-powerful machine fuels competition between nations to develop AI so that they can benefit from and control it. This works to the advantage of tech firms: it encourages investment and weakens arguments for regulating the industry. An actual arms race to produce next-generation AI-powered military technology is already under way, increasing the risk of catastrophic conflict — doomsday, perhaps, but not of the sort much discussed in the dominant ‘AI threatens human extinction’ narrative.
2. Second, it allows a homogeneous group of company executives and technologists to dominate the conversation about AI risks and regulation, while other communities are left out. Letters written by tech-industry leaders are “essentially drawing boundaries around who counts as an expert in this conversation”, says Amba Kak, director of the AI Now Institute in New York City, which focuses on the social consequences of AI.

_

AI systems and tools have many potential benefits, from synthesizing data to assisting with medical diagnoses. But they can also cause well-documented harms, from biased decision-making to the elimination of jobs. AI-powered facial recognition is already being abused by autocratic states to track and oppress people. Biased AI systems could use opaque algorithms to deny people welfare benefits, medical care or asylum — applications of the technology that are likely to most affect those in marginalized communities. Debates on these issues are being starved of oxygen.

One of the biggest concerns surrounding the latest breed of generative AI is its potential to boost misinformation. The technology makes it easier to produce more, and more convincing, fake text, photos and videos that could influence elections, say, or undermine people’s ability to trust any information, potentially destabilizing societies. If tech companies are serious about avoiding or reducing these risks, they must put ethics, safety and accountability at the heart of their work. At present, they seem to be reluctant to do so. OpenAI did ‘stress-test’ GPT4, its latest generative AI model, by prompting it to produce harmful content and then putting safeguards in place. But although the company described what it did, the full details of the testing and the data that the model was trained on were not made public. Tech firms must formulate industry standards for responsible development of AI systems and tools, and undertake rigorous safety testing before products are released. They should submit data in full to independent regulatory bodies that are able to verify them, much as drug companies must submit clinical-trial data to medical authorities before drugs can go on sale.

_

For Marcus, a self-described critic of AI hype, “the biggest immediate threat from AI is the threat to democracy from the wholesale production of compelling misinformation.” Generative AI tools like OpenAI’s ChatGPT and Dall-E are trained on vast troves of data online to create compelling written work and images in response to user prompts. With these tools, for example, one could quickly mimic the style or likeness of public figures in an attempt to create disinformation campaigns. In his testimony before Congress, Altman also said the potential for AI to be used to manipulate voters and target disinformation were among “my areas of greatest concern.” Even in more ordinary use cases, however, there are concerns. The same tools have been called out for offering wrong answers to user prompts, outright “hallucinating” responses and potentially perpetuating racial and gender biases.

_

Emily Bender, a professor at the University of Washington and director of its Computational Linguistics Laboratory said some companies may want to divert attention from the bias baked into their data and also from concerning claims about how their systems are trained. Bender cited intellectual property concerns with some of the data these systems are trained on as well as allegations of companies outsourcing the work of going through some of the worst parts of the training data to low-paid workers abroad. “If the public and the regulators can be focused on these imaginary science fiction scenarios, then maybe these companies can get away with the data theft and exploitative practices for longer,” Bender said. Regulators may be the real intended audience for the tech industry’s doomsday messaging. As Bender puts it, execs are essentially saying: “‘This stuff is very, very dangerous, and we’re the only ones who understand how to rein it in.’” Judging from Altman’s appearance before Congress, this strategy might work. Altman appeared to win over Washington by echoing lawmakers’ concerns about AI — a technology that many in Congress are still trying to understand — and offering suggestions for how to address it. This approach to regulation would be “hugely problematic,” Bender said. It could give the industry influence over the regulators tasked with holding it accountable and also leave out the voices and input of other people and communities experiencing negative impacts of this technology. Ultimately, Bender put forward a simple question for the tech industry on AI: “If they honestly believe that this could be bringing about human extinction, then why not just stop?”

_____

_____

If you worry about existential threat to humanity, you should be more scared of humans than of AI:

-1. Undoubtedly the spread of misinformation by AI-propagated systems is concerning, especially given the unparalleled scale of content that AI can generate. But as recent research reveals, humans are far more responsible for spreading misinformation than technology. In a study of how true and false news spreads on Twitter, researchers analyzed 126,000 stories tweeted by millions of people between 2006 and 2017 and found that false news spreads faster than true news, and that “false news spreads more than the truth because humans, not robots, are more likely to spread it” (Vosoughi, Roy, and Aral 2018).

-2. A threat even more dire than misinformation is the “risk of extinction from AI” that the Center for AI Safety highlights in its open statement. Yet, in terms of whether machines or humans are more likely to initiate extinction-level events such as nuclear war, humans still seem to have the upper hand. In recent empirical work that analyzes the decision processes employed by senior leaders in war-game scenarios involving weapons of mass destruction, humans showed alarming tendency to err on the side of initiating catastrophic attacks. These simulations, if implemented in reality, would pose much graver risks to humanity than machine-driven ones. Our exploration of the use of AI in critical decision-making has shown AI’s superiority to human decisions in nearly all scenarios. In most cases, the AI makes the choice that humans do not make at first—but then, upon more careful consideration and deliberation, change their minds and do make, realizing it was the correct decision all along.

-3. Consider algorithmic bias, the phenomenon whereby algorithms involved in hiring decisions, medical diagnoses, or image detection produce outcomes that unfairly disadvantage a particular social group. For example, when Amazon implemented an algorithmic recruiting tool to score new applicants’ resumes, the algorithm systematically rated female applicants worse than men, in large part because the algorithm was trained on resumes submitted over the previous 10 years that were disproportionately male. In other words, an algorithm trained on human bias will reproduce this bias.

Unlike humans, however, algorithmic bias can be readily deprogrammed, or as economist Sendhil Mullainathan puts it, “Biased algorithms are easier to fix than biased people.” Mullainathan and colleagues’ research showed that an algorithm used by UnitedHealth to score patients’ health risks systematically underscored black patients relative to white patients because it measured illness in terms of health care costs (which are systematically lower for black versus white individuals, given that society spends less on black patients) (Obermeyer et al. 2019).

However, once identified, the researchers could easily modify this feature of the algorithm to produce risk scores that were relatively unbiased. Other work has shown that algorithms can produce less racially biased outcomes (and more effective public safety outcomes) than human judges in terms of decisions of whether or not to grant bail to defendants awaiting trial (Kleinberg et al. 2018). As biased as algorithms can be, their biases appear less ingrained and more pliable than those of humans. Compounded by recent work showing that, in hiring and lending contexts, managers reject biased algorithms in favor of more biased humans, the suggestion that humans should remain at the helm of those functions is, at best, questionable (Cowgill, Dell’Acqua, and Matz 2020).

-4. Finally, consider the threat to cybersecurity. Although commentators have warned that large language models added tools to the arsenal of hackers by democratizing cybercrime, most high-profile information leaks and hacks to date are ushered in by human beings with no reliance on AI (i.e., a disgruntled employee who knows the systems’ flaws and perpetrates an attack by remembering key passwords, or bad programmers who effectively enable future attacks by making wrong assumptions on their software use-cases—such as “no one would create a password that is 1,000,000 characters long” leading to a classical buffer overflow hack). In fact, AI is often the last bastion of defense against those hacks, identifying complex human coding mistakes early-on and correcting those.

Recently, national guardsman Jack Teixeira, who exposed highly classified material in an online chat group, did not require sophisticated technology to access sensitive documents—he was granted top secret clearance from the Pentagon. Further, a recent study conducted by IBM indicates that 95 percent of security breaches were caused by human errors such as biting on phishing scams or downloading malware. If anything, the most concerning cybersecurity risk currently posed by AI results from its increased reliance on human trained code, which is flawed. AI takes hackable human codes and uses them to generate new codes, spreading these human-generated errors further. The only concerning current cybersecurity attacks by AI involve AI that simulates human communication to dupe humans into revealing key information. Cybersecurity may represent a case in which technology is more likely to be the solution rather than the problem, with research indicating, for example, that humans working with AI outperform humans alone in detecting machine-manipulated media such as deepfakes (Groh et al. 2021).

-5. Even when technology contributes to unwanted outcomes, humans are often the ones pressing the buttons. Consider the effect of AI on unemployment. The Future of Life Institute letter raises concerns that AI will eliminate jobs, yet whether or not to eliminate jobs is a choice that humans ultimately make. Just because AI can perform the jobs of, say, customer service representatives does not mean that companies should outsource these jobs to bots. In fact, research indicates that many customers would prefer to talk to a human than to a bot, even if it means waiting in a queue. Along similar lines, increasingly common statements that AI-based systems—like “the Internet,” “social media,” or the set of interconnected online functions referred to as “The Algorithm”—are destroying mental health, causing political polarization, or threatening democracy neglect an obvious fact: These systems are populated and run by human beings. Blaming technology lets people off the hook.

_

In a nutshell:

Humans are far worse than AI as far as bad behaviour is concerned right from disinformation to taking away jobs to bias to cyberattacks to nuclear war. The only thing we have to fear is malicious or incompetent humans and not technology itself. Artificial intelligence is a tool. Let’s get on with inventing better and smarter AI with due safeguards.  

______

______

Section-15

AI threat mitigation:

Artificial intelligence represents an existential opportunity – it has strong potential to improve the future of humankind drastically and broadly. AI can save, extend, and improve the lives of billions of people. AI will create new challenges and problems. Some of them may be major and perhaps frightening at the time. If we are not to stagnate as a society, we must summon the courage to vigorously advance artificial intelligence while being mindful and watchful and ready to respond and adapt to problems and risks as they arise. It is impossible to plan for all risks ahead of time. Confident claims that AI will destroy us must be acknowledged and carefully examined but considered in the context of the massive costs of relinquishing AI development and the lost existential opportunity. The most sensible way to think about safety and regulatory approaches is not the precautionary principle; it is the Proactionary Principle. The proactionary principle is an ethical and decision-making principle formulated by the transhumanist philosopher Max More as follows:

“People’s freedom to innovate technologically is highly valuable, even critical, to humanity. This implies several imperatives when restrictive measures are proposed: Assess risks and opportunities according to available science, not popular perception. Account for both the costs of the restrictions themselves, and those of opportunities foregone. Favor measures that are proportionate to the probability and magnitude of impacts, and that have a high expectation value. Protect people’s freedom to experiment, innovate, and progress”.

The proactionary principle was created as an opposing viewpoint to the precautionary principle, which is based on the concept that consequences of actions in complex systems are often unpredictable and irreversible and concludes that such actions should generally be opposed. The Proactionary Principle is based upon the observation that historically, the most useful and important technological innovations were neither obvious nor well understood at the time of their invention.

_

There seems to be a world-wide, minimum, consensus that AI requires regulation of some form, though governments differ widely in their objectives and approaches. Most proposed regulations include at least some form of tier system which applies different rules for different areas of AI. All of them include at least some form of transparency requirements, few have tackled the liability questions yet. As with any industry regulation, few governments want to move first and potentially give up a competitive advantage in an over-hyped growth market. At least not as long as the gold rush continues. Mr. Altman and two other OpenAI executives proposed several ways that powerful A.I. systems could be responsibly managed. They called for cooperation among the leading A.I. makers, more technical research into large language models and the formation of an international A.I. safety organization, similar to the International Atomic Energy Agency, which seeks to control the use of nuclear weapons. The UN Secretary-General recently expressed support for the establishment of a UN agency on AI, similar to the International Atomic Energy Agency. Such an agency, focused on knowledge and endowed with regulatory powers, could enhance co-ordination among burgeoning AI initiatives worldwide and promote global governance on AI. To succeed, however, the UN must transcend its traditional intergovernmental DNA and incorporate the scientific community, private sector (the primary source of AI innovations) and civil society into new governance frameworks, including public-private partnerships. The deepfake and generative AI quandary serves as a sobering reminder of the immense power and multifaceted security challenges posed by artificial intelligence. In the pursuit of responsible AI governance, we must prioritise the protection against malevolent exploitation while nurturing an environment that encourages ethical innovation and societal progress.

_

Many scholars concerned about the AGI existential risk believe that the best approach is to conduct substantial research into solving the difficult “control problem”: what types of safeguards, algorithms, or architectures can programmers implement to maximize the probability that their recursively-improving AI would continue to behave in a friendly manner after it reaches superintelligence?  Researchers at Google have proposed research into general “AI safety” issues to simultaneously mitigate both short-term risks from narrow AI and long-term risks from AGI. A 2020 estimate places global spending on AI existential risk somewhere between $10 and $50 million, compared with global spending on AI around perhaps $40 billion. Bostrom suggests that funding of protective technologies should be prioritized over potentially dangerous ones. Institutions such as the Alignment Research Center, the Machine Intelligence Research Institute, the Future of Humanity Institute, the Future of Life Institute, the Centre for the Study of Existential Risk, and the Center for Human-Compatible AI are involved in research into AI risk and safety.

_

Assessing risk and preventing harm:  

Many of the threats arise from the deliberate, accidental or careless misuse of AI by humans. Even the risk and threat posed by a form of AGI that exists and operates independently of human control is currently still in the hands of humans. Different parts of the UN system are now engaged in a desperate effort to ensure that our international social, political and legal institutions catch up with the rapid technological advancements being made with AI. In 2020, for example, the UN established a High-level Panel on Digital Cooperation to foster global dialogue and cooperative approaches for a safe and inclusive digital future. In September 2021, the head of the UN Office of the Commissioner of Human Rights called on all states to place a moratorium on the sale and use of AI systems until adequate safeguards are put in place to avoid the ‘negative, even catastrophic’ risks posed by them. And in November 2021, the 193 member states of UNESCO adopted an agreement to guide the construction of the necessary legal infrastructure to ensure the ethical development of AI. However, the UN still lacks a legally binding instrument to regulate AI and ensure accountability at the global level.

At the regional level, the European Union has an Artificial Intelligence Act which classifies AI systems into three categories: unacceptable-risk, high-risk and limited and minimal-risk. This Act could serve as a stepping stone towards a global treaty although it still falls short of the requirements needed to protect several fundamental human rights and to prevent AI from being used in ways that would aggravate existing inequities and discrimination.

_

Guardrails recommended by AI experts:

• Rigorous and independent testing of products using AI
• Protections from discriminatory bias in algorithms
• Data privacy
• A requirement that users must be notified when they’re using an automated product, and how its decisions may affect them
• A similar requirement that users be allowed to opt out of AI systems in favor of human alternatives

Often, even as some AI systems approach a human-like level of intelligence, they’re still imperfect. They claim to solve a problem they don’t actually solve, and they make mistakes. But tech companies may not be trustworthy enough to test their own products for safety concerns. One potential solution: a federal agency similar to the U.S. Food and Drug Administration. Third-party supervision could help for most of proposed guardrails. There’s one exception, a guardrail that isn’t technologically possible yet and may never be: protections from discriminatory bias. Every AI system is created by a human or group of humans, and trained on data sets. Every human has some form of inherent bias, and even gigantic data sets can’t represent the entirety of the internet or totality of the human experience. We can try to mitigate bias by building AIs “with inputs and understanding from everyone who’s going to be affected,” submitting them to independent testing and relying on third-party oversight to hold developers accountable and fix obvious biases when they’re discovered.

_____

_____

The UK Governance of Artificial Intelligence Report, August 2023: 

The recent rate of development has made debates regarding the governance and regulation of AI less theoretical, more significant, and more complex. It has also generated intense interest in how public policy can and should respond to ensure that the beneficial consequences of AI can be reaped whilst also safeguarding the public interest and preventing known potential harms, both societal and individual. There is a growing imperative to ensure governance and regulatory frameworks are not left irretrievably behind by the pace of technological innovation. Policymakers must take measures to safely harness the benefits of the technology and encourage future innovations, whilst providing credible protection against harm. This report has led us to identify twelve challenges of AI governance, that policymakers and the frameworks they design must meet.

-1. The Bias challenge.

AI can introduce or perpetuate biases that society finds unacceptable. The Report warns that inherent human biases encoded in the datasets used to inform AI models and tools could replicate bias and discrimination against minority and underrepresented communities in society.

-2. The Privacy challenge.

AI can allow individuals to be identified and personal information about them to be used in ways beyond what the public wants. Particular emphasis is placed on live facial recognition technology, with the warning that systems may not adequately respect individual’s rights, currently set out in legislation such as the Data Protection Act 2018, in the absence of specific, comprehensive regulation.

-3. The Misrepresentation challenge.

AI can allow the generation of material that deliberately misrepresents someone’s behaviour, opinions, or character. The Report attributes the combination of data availability and new AI models to the increasingly convincing dissemination of ‘fake news’. Examples given included purporting to show individuals ‘passing off’ information through voice and image recordings, particularly damaging if used to influence election campaigns, enable fraudulent transactions in financial services, or damage individual’s reputations. The Report goes on to warn of the dangers when coupled with algorithmic recommendations on social media platforms targeting relevant groups.

-4. The Access to Data challenge.  

The most powerful AI needs very large datasets, which are held by few organisations. The Report raises competition concerns caused by the lack of access to sufficient volumes of high-quality training data for AI developers outside of the largest players. There is proposed legislation to mandate research access to Big Tech data stores to encourage a more diverse AI development ecosystem.

-5. The Access to Compute challenge.

The development of powerful AI requires significant compute power, access to which is limited to a few organisations. Academic research is deemed to be particularly disadvantaged by this challenge compared to private developers. The Report suggests efforts are already underway to establish an Exascale supercomputer facility and AI-dedicated compute resources, with AI labs giving priority access to models for research and safety purposes.

-6. The Black Box challenge.

Some AI models and tools cannot explain why they produce a particular result, which is a challenge to transparency requirements. The Report calls for regulation to ensure more transparent and more explicable AI models and reckons that explainability would increase public confidence and trust in AI.

-7. The Open-Source challenge.

Requiring code to be openly available may promote transparency and innovation; allowing it to be proprietary may concentrate market power but allow more dependable regulation of harms. This is a further example of how the Committee view the need to increase the capacity for development and use of AI amongst more widely distributed players. The Report acknowledges the need to protect against misuse as it cites opinions that open-source code would allow malign actors to cause harm, for example through the dissemination of misleading content. There is no conclusion by the Committee as to which method is preferable.

-8. The Intellectual Property and Copyright Challenge.

The Report comments that “Some AI models and tools make use of other people’s content: policy must establish the rights of the originators of this content, and these rights must be enforced” and that whilst the use of AI models and tools have helped create revenue for the entertainment industry in areas such as video games and audience analytics, concerns have been raised about the ‘scraping’ of copyrighted content from online sources without permission. The Report refers to “ongoing legal cases” (unnamed but likely a reference to Getty v StabilityAI) which are likely to set precedents in this area, but also notes that the UK IPO has begun to develop a voluntary code of practice on copyright and AI, in consultation with the technology, creative and research sectors, which guidance should “… support AI firms to access copyrighted work as an input to their models, whilst ensuring there are protections (e.g. labelling) on generated output to support right holders of copyrighted work”. The report notes that the Government has said that if agreement is not reached or the code not adopted, it may legislate.

-9. The Liability challenge.

If AI models and tools are used by third parties to do harm, policy must establish whether developers or providers of the technology bear any liability for harms done. The Report considers that if AI models and tools are used by third parties to do harm, policy must establish whether developers or providers of the technology bear any liability for harms done.

-10. The Employment challenge.

AI will disrupt the jobs that people do and that are available to be done. Policy makers must anticipate and manage the disruption. It is noted in the Report that automation has the potential to impact the economy and society through displacement of jobs. It highlights the importance of planning ahead through an assessment of the jobs and sectors most likely to be affected. The Report highlights the Prime Minister’s attitude to be cognisant of the “large-scale shifts” through providing people with the necessary skills to thrive in the technological age.

-11. The International Coordination challenge.

AI is a global technology, and the development of governance frameworks to regulate its uses must be an international undertaking. The Report compares the UK pro-innovation strategy, the risk-based approach of the EU and the US priority to ensure responsible innovation and appropriate safeguards to protect people’s rights and safety. These divergent approaches contrast with the shared global implications of the “ubiquitous, general-purpose” AI technology as heard by the Committee inquiry, and therefore calls for a coordinated international response.

-12. The Existential challenge.

Some people think that AI is a major threat to human life: if that is a possibility, governance needs to provide protections for national security. The 2023 AI White Paper deemed such existential risks as “high impact but low probability” but the debate remains whether such a prospect is realistic. Suggestions are made in the Report of using the international security framework governing nuclear weapons as a template for mitigating AI risks.

The Report calls for the government to address each of the twelve challenges outlined and makes clear the growing imperative to accelerate the development of public policy thinking on AI “to ensure governance and regulatory frameworks are not left irretrievably behind the pace of technological innovation”.

______

______

Control dangerous AI before it controls us:

Keeping the artificial intelligence genie trapped in the proverbial bottle could turn an apocalyptic threat into a powerful oracle that solves humanity’s problems, said Roman Yampolskiy, a computer scientist at the University of Louisville in Kentucky. But successful containment requires careful planning so that a clever breed of artificial intelligence cannot simply threaten, bribe, seduce or hack its way to freedom. “It can discover new attack pathways, launch sophisticated social-engineering attacks and re-use existing hardware components in unforeseen ways,” Yampolskiy said. “Such software is not limited to infecting computers and networks — it can also attack human psyches, bribe, blackmail and brainwash those who come in contact with it.” A new field of research aimed at solving the prison problem for artificial-intelligence programs could have side benefits for improving cybersecurity and cryptography, Yampolskiy suggested. His proposal was detailed in the issue of the Journal of Consciousness Studies.

_

One starting solution might trap the artificial intelligence, or AI, inside a “virtual machine” running inside a computer’s typical operating system — an existing process that adds security by limiting the AI’s access to its host computer’s software and hardware. That stops a smart AI from doing things such as sending hidden Morse code messages to human sympathizers by manipulating a computer’s cooling fans. Putting the AI on a computer without Internet access would also prevent any “Skynet” program from taking over the world’s defense grids in the style of the “Terminator” films. If all else fails, researchers could always slow down the AI’s “thinking” by throttling back computer processing speeds, regularly hit the “reset” button or shut down the computer’s power supply to keep an AI in check. Such security measures treat the AI as an especially smart and dangerous computer virus or malware program, but without the sure knowledge that any of the steps would really work. “The Catch-22 is that until we have fully developed superintelligent AI we can’t fully test our ideas, but in order to safely develop such AI we need to have working security measures,” Yampolskiy said. “Our best bet is to use confinement measures against subhuman AI systems and to update them as needed with increasing capacities of AI.”

_

Even casual conversation with a human guard could allow an AI to use psychological tricks such as befriending or blackmail. The AI might offer to reward a human with perfect health, immortality, or perhaps even bring back dead family and friends. Alternately, it could threaten to do terrible things to the human once it “inevitably” escapes. The safest approach for communication might only allow the AI to respond in a multiple-choice fashion to help solve specific science or technology problems, Yampolskiy explained. That would harness the power of AI as a super-intelligent oracle.

_

Despite all the safeguards, many researchers think it’s impossible to keep a clever AI locked up forever. A past experiment by Eliezer Yudkowsky, a research fellow at the Singularity Institute for Artificial Intelligence, suggested that mere human-level intelligence could escape from an “AI Box” scenario — even if Yampolskiy pointed out that the test wasn’t done in the most scientific way. Still, Yampolskiy argues strongly for keeping AI bottled up rather than rushing headlong to free our new machine overlords. But if the AI reaches the point where it rises beyond human scientific understanding to deploy powers such as precognition (knowledge of the future), telepathy or psychokinesis, all bets are off. “If such software manages to self-improve to levels significantly beyond human-level intelligence, the type of damage it can do is truly beyond our ability to predict or fully comprehend,” Yampolskiy said.

_____ 

_____

AI safety:

 AI safety is an interdisciplinary field concerned with preventing accidents, misuse, or other harmful consequences that could result from artificial intelligence (AI) systems. It encompasses machine ethics and AI alignment, which aim to make AI systems moral and beneficial, and AI safety encompasses technical problems including monitoring systems for risks and making them highly reliable. Beyond AI research, it involves developing norms and policies that promote safety.

AI researchers have widely different opinions about the severity and primary sources of risk posed by AI technology – though surveys suggest that experts take high consequence risks seriously. In two surveys of AI researchers, the median respondent was optimistic about AI overall, but placed a 5% probability on an “extremely bad (e.g. human extinction)” outcome of advanced AI. In a 2022 survey of the Natural language processing (NLP) community, 37% agreed or weakly agreed that it is plausible that AI decisions could lead to a catastrophe that is “at least as bad as an all-out nuclear war.” Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents.

AI safety research areas include robustness, monitoring, and alignment. Robustness is concerned with making systems highly reliable, monitoring is about anticipating failures or detecting misuse, and alignment is focused on ensuring they have beneficial objectives.

_____

_____

AI regulation:

Regulating AI is tricky. To regulate AI well, you must first define AI and understand anticipated AI risks and benefits. Legally defining AI is important to identify what is subject to the law. But AI technologies are still evolving, so it is hard to pin down a stable legal definition. Understanding the risks and benefits of AI is also important. Good regulations should maximize public benefits while minimizing risks. However, AI applications are still emerging, so it is difficult to know or predict what future risks or benefits might be. These kinds of unknowns make emerging technologies like AI extremely difficult to regulate with traditional laws and regulations. Lawmakers are often too slow to adapt to the rapidly changing technological environment. Some new laws are obsolete by the time they are enacted or even introduced. Without new laws, regulators have to use old laws to address new problems. Sometimes this leads to legal barriers for social benefits or legal loopholes for harmful conduct. “Soft laws” are the alternative to traditional “hard law” approaches of legislation intended to prevent specific violations. In the soft law approach, a private organization sets rules or standards for industry members. These can change more rapidly than traditional lawmaking. This makes soft laws promising for emerging technologies because they can adapt quickly to new applications and risks. However, soft laws can mean soft enforcement.

_

The regulation of artificial intelligence is the development of public sector policies and laws for promoting and regulating artificial intelligence (AI); it is therefore related to the broader regulation of algorithms. The regulatory and policy landscape for AI is an emerging issue in jurisdictions globally. According to AI Index at Stanford, the annual number of AI-related laws passed in the 127 survey countries jumped from one passed in 2016 to 37 passed in 2022 alone. Between 2016 and 2020, more than 30 countries adopted dedicated strategies for AI. Most EU member states had released national AI strategies, as had Canada, China, India, Japan, Mauritius, the Russian Federation, Saudi Arabia, United Arab Emirates, US and Vietnam. Others were in the process of elaborating their own AI strategy, including Bangladesh, Malaysia and Tunisia. The Global Partnership on Artificial Intelligence was launched in June 2020, stating a need for AI to be developed in accordance with human rights and democratic values, to ensure public confidence and trust in the technology. Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher published a joint statement in November 2021 calling for a government commission to regulate AI. In 2023, OpenAI leaders published recommendations for the governance of superintelligence, which they believe may happen in less than 10 years.

_

The most advanced AI regulatory effort is the European Union, whose parliament recently passed its version of the Artificial Intelligence Act (AI Act). The AI Act’s proponents have suggested that rather than extinction, discrimination is the greater threat. To that end, the AI Act is primarily an exercise in risk classification, through which European policymakers are judging applications of AI as high-, limited-, or minimal-risk, while also banning certain applications they deem unacceptable, such as cognitive behavioral manipulation; social scoring based on behavior, socioeconomic status or personal characteristics; and real-time biometric identification from law enforcement. The AI Act also includes regulatory oversight of “high-risk” applications like biometric identification in the private sector and management of critical infrastructure, while also providing oversight on relevant education and vocational training. It is a comprehensive package, which is also its main weakness: classifying risk through cross-sectoral legislation will do little to address existential risk or AI catastrophes while also limiting the ability to harness the benefits of AI, which have the potential to be equally astonishing. What is needed is an alternative regulatory approach that addresses the big risks without sacrificing those benefits.

_

Given the rapidly changing state of the technology and the nascent but extremely promising AI opportunity, policymakers should embrace a regulatory structure that balances innovation and opportunity with risk. While the European Union does not neglect innovation entirely, the risk-focused approach of the AI Act is incomplete. By contrast, the U.S. Congress appears headed toward such a balance. While new AI regulation is almost certainly coming, bipartisan group of senators are committed to preserving innovation. In announcing the SAFE Innovation Framework, they identified four goals (paraphrased below) that forthcoming AI legislation should achieve:

• Security: instilling guardrails to protect the U.S. against bad actors’ use of AI, while also preserving American economic security by preparing for, managing, and mitigating workforce disruption.
• Accountability: promoting ethical practices that protect children, vulnerable populations, and intellectual property owners.
• Democratic Foundations: programming algorithms that align with the values of human liberty, civil rights, and justice.
• Explainability: transcending the black box problem by developing systems that explain how AI systems make decisions and reach conclusions.

Congress has an important role to play in addressing AI’s risks and empowering federal agencies to issue new rules and apply existing regulations where appropriate. Sending a message to the public—and to the world—that the U.S. government is focused on preventing AI catastrophes will inspire the confidence and trust necessary for further technological advancement. 

_

AI Industry Self-Regulation: 

Until or unless there are government regulations, the field of AI will be governed largely by the ethical frameworks, codes, and practices of its developers and users. (There are exceptions, such as when AI systems have outcomes that are discriminatory.) Virtually every AI developer has articulated their own principles for responsible AI development. These principles may encompass each stage of the product development process, from pretraining and training of data sets to setting boundaries for outputs, and incorporate principles like privacy and security, equity and inclusion, and transparency. They also articulate use policies that ostensibly govern what users can generate. For example, OpenAI’s usage policies “disallow” disinformation, as well as hateful, harassing, or violent content and coordinated inauthentic behavior, among other things.

But these policies, no matter how well-intentioned, have significant limits. For example, researchers recently found that the “guardrails” of both closed systems, like ChatGPT, and open-sourced systems, like Meta’s Llama 2 product, can be “coaxed” into generating biased, false, and violative responses. And, as in every other industry, voluntary standards and self-regulation are subject to daily trade-offs with growth and profit motives. This will be the case even when voluntary standards are agreed to collectively (as is a new industry-led body to develop safety standards) or secured by the White House (as is a new set of commitments announced recently). For the most part, we’re talking about the same companies – even some of the same people – whose voluntary standards have proven insufficient to safeguard our privacy, moderate content that threatens democracy, ensure equitable outcomes, and prohibit harassment and hate speech.

_

Banning AI?

Some scholars have stated that even if AGI poses an existential risk, attempting to ban research into artificial intelligence would still be unwise, and probably futile. Skeptics argue that regulation of AI would be completely valueless, as no existential risk exists. However, scholars who believe existential risk proposed that it is difficult to depend on people from the AI industry to regulate or constraint AI research because it directly contradicts their personal interests. The scholars also agree with the skeptics that banning research would be unwise, as research could be moved to countries with looser regulations or conducted covertly. The latter issue is particularly relevant, as artificial intelligence research can be done on a small scale without substantial infrastructure or resources. Two additional hypothetical difficulties with bans (or other regulation) are that technology entrepreneurs statistically tend towards general skepticism about government regulation, and that businesses could have a strong incentive to (and might well succeed at) fighting regulation and politicizing the underlying debate.

_

According to research published in August 2023 by AI cybersecurity company BlackBerry, 75% of organizations worldwide are currently implementing or considering bans on ChatGPT and other Generative AI applications within the workplace. Banning AI is technically a way to deal with security issues, but it’s not the most strategic move. Despite the risks, the reality is that AI can have many benefits. Instead of banning AI, executives can explore other ways to mitigate generative AI’s data security risks.

Many schools have banned ChatGPT entirely. To see why this is wrong, we must only look back a few years. Upon the introduction of the internet, instead of embracing the technology and teaching necessary skills, fear of cheating led to crackdowns. Today, society is on the path to make the same mistake with generative AI. Schools have a duty to inform pupils on the use cases of technology and raise smart, informed citizens. Instead of hopelessly racing to detect people using AI, why not encourage educational uses that advance learning and teach skills?

_____

_____

Moral of the story:    

_

-1. AI is a family of technologies that perform tasks that are thought to require intelligence if performed by humans. AI is mostly mathematics, searching for an optimal answer in a vast multidimensional solution space. Building an AI system is a careful process of reverse-engineering human traits and capabilities in a machine, and using its computational prowess to surpass what we are capable of. AI makes every process better, faster, and more accurate. The basic goal of AI is to enable computers and machines to perform intellectual tasks such as problem solving, decision making, perception, and understanding human communication.

_

-2. In conventional computing the computer is given data and is told how to solve a problem whereas in AI knowledge is given about a domain and some inference capability with the ultimate goal is to develop technique that permits systems to learn new knowledge autonomously and continually to improve the quality of the knowledge they possess. Conventional programming is all about creating a fixed set of instructions for a computer to follow, whereas AI is about creating algorithms that allow computers to learn from data and make decisions or predictions. Conventional software is programmed to perform a task, AI is programmed to learn to perform the task.

To perform tasks, AI uses various algorithms which are sequences of steps to be followed in calculations or other problem‑solving operations by computers. While a traditional algorithm is composed of rigid, preset, explicitly programmed instructions that get executed each time the computer encounters a trigger, AI can modify and create new algorithms in response to learned inputs without human intervention. Instead of relying solely on inputs that it was designed to recognize, the system acquires the ability to evolve, adapt, and grow based on new data sets. Traditional algorithms take some input and some logic in the form of code and encourage output. In contrast, AI algorithms use inputs and outputs and give some logic that can then be used to process new inputs to give an output. The logic generated is what makes it AI.

An optimization problem is essentially finding the best solution to a problem from endless number of possibilities. Conventical computing would have to configure and sort through every possible solution one at a time, on a large-scale problem this could take lot of time, so AI may help with sufficient solution (Heuristic search).

_

-3. Most applications of AI have been in domains with large amounts of data. To use radiology example, the existence of large databases of X-rays and MRI scans that have been evaluated by human radiologists, makes it possible to train a machine to emulate that activity. AI works by combining large amounts of data with intelligent algorithms — series of instructions — that allow the software to learn from patterns and features of the data. AI models and tools are capable of processing increasing amounts of data, and this is already delivering significant benefits in areas such as medicine, healthcare, finance, transportation and education. They can find patterns where humans might not, improve productivity through the automation of routine processes, and power new, innovative consumer products. However, they can also be manipulated, provide false information, and do not always perform as one might expect in messy, complex environments—such as the world we live in.

_

-4. AI research follows two distinct, and to some extent competing, methods, the symbolic (or “top-down”) approach, and the connectionist (or “bottom-up”) approach. The top-down approach seeks to replicate intelligence by analyzing cognition independent of the biological structure of the brain, in terms of the processing of symbols—whence the symbolic label. Symbolic approach follows binary logic (using only two values 0 or 1) based on symbolic processing using heuristic search – a mathematical approach in which ideas and concepts are represented by symbols such as words, phrases or sentences, which are then processed according to the rules of logic. Expert system is classic example of symbolic AI. Symbolic AI research focuses on attempts to mimic human intelligence through symbol manipulation and symbolically structured knowledge bases.

The bottom-up approach, on the other hand, involves creating artificial neural networks in imitation of the brain’s structure—whence the connectionist label. Connectionist AI differs from symbolic AI in that it is tolerant of imprecision, uncertainty, partial truth, and approximation. In effect, the role model for connectionist AI is the human mind. Many real-life problems cannot be translated into binary language (unique values of 0 and 1) for computers to process it.

To illustrate the difference between these approaches, consider the task of building a system, equipped with an optical scanner, that recognizes the letters of the alphabet. A bottom-up approach typically involves training an artificial neural network by presenting letters to it one by one, gradually improving performance by “tuning” the network. (Tuning adjusts the responsiveness of different neural pathways to different stimuli.) In contrast, a top-down approach typically involves writing a computer program that compares each letter with geometric descriptions. Simply put, neural activities are the basis of the bottom-up approach, while symbolic descriptions are the basis of the top-down approach. So there are two approaches to AI: symbolic AI and connectionist AI (machine learning). In machine learning, the algorithm learns rules as it establishes correlations between inputs and outputs. In symbolic reasoning, the rules are created through human intervention and then hard-coded into a static program. 

_

-5. Machine learning is the ability to learn without being explicitly programmed and it explores the development of algorithms that learn from given data. Sophisticated algorithms examine extensive data sets to find patterns. To achieve this, a large number of data examples are fed to an algorithm, enabling it to experiment and learn on its own through trial and error. The learning algorithm takes millions of data points as inputs and correlates specific data features to produce outputs. The model learns to change its internal parameters until it can predict the exact output for new inputs using a large sample of inputs and expected outputs. Machine learning algorithms are designed to learn and improve over time, using feedback from data to refine their performance. As these algorithms become more sophisticated, they may be capable of developing new insights and knowledge that go beyond their original programming. Traditional system performs computations to solve a problem. However, if it is given the same problem a second time, it performs the same sequence of computations again. Traditional system cannot learn. Machine learning algorithms learn to perform tasks rather than simply providing solutions based on a fixed set of data. It learns on its own, either from experience, analogy, examples, or by being “told” what to do. Machine learning focuses on the development of such algorithms that can process input data and use statistical analysis to detect patterns and draw inferences without being explicitly programmed. Structured data sets are then used to train the machine which applies the discovered patterns to solve a problem when new data is fed. However, if the result is incorrect, there is a need to ‘teach’ them. As a result of this training, the machine learning model is finally ready to make predictions using real-world data. The model scales its method, approaches and body of knowledge as additional data is gathered over time. ML algorithms can be classified into 3 broad categories: supervised learning (where the expected output for the input is known thanks to labeled data sets), unsupervised learning (where the expected outputs are unknown due to the use of unlabeled data sets), and reinforcement learning (training method based on rewarding desired behaviors and punishing undesired ones). Deep learning is a class of machine learning algorithms. 

_

-6. Neural networks are a set of algorithms, modelled loosely after the human brain, that are designed to recognize patterns. They interpret sensory data through a kind of machine perception, labelling or clustering raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text or time series, must be translated. Neural networks help us cluster and classify. The layers are made of nodes. A node is just a place where computation happens, loosely patterned on a neuron in the human brain, which fires when it encounters sufficient stimuli. A node combines input from the data with a set of coefficients, or weights, that either amplify or dampen that input, thereby assigning significance to inputs for the task the algorithm is trying to learn. These input-weight products are summed and the sum is passed through a node’s so-called activation function, to determine whether and to what extent that signal progresses further through the network to affect the ultimate outcome, say, an act of classification. Neural networks learn to model complex relationships between inputs and outputs and find patterns in data. The learning is performed by linear algebra operations on the matrixes and vectors. Neural networks perform a type of mathematical optimization — they perform stochastic gradient descent on a multi-dimensional topology that is created by training the network. In theory, a neural network can learn any function. The most common training technique is the backpropagation algorithm. Backpropagation is an algorithm that backpropagates the errors from the output nodes to the input nodes. For LLMs, if the model’s prediction is close to the actual next word, the neural network updates its parameters to reinforce the patterns that led to that prediction. Conversely, if the prediction is incorrect, the model adjusts its parameters to improve its performance and tries again. This process of trial and error, though a technique called “backpropagation,” allows the model to learn from its mistakes and gradually improve its predictions during the training process.

_

-7. Deep learning is the name we use for “stacked neural networks”; that is, networks composed of several layers (see above paragraph). Deep Learning creates knowledge from multiple layers of information processing. Deep Learning tries to emulate the functions of inner layers of the human brain, and its successful applications are found in computer vision, natural language processing, healthcare, finance, agriculture, and cybersecurity.

The way in which deep learning and machine learning differ is in how each algorithm learns. Deep learning automates much of the feature extraction piece of the process, eliminating some of the manual human intervention required and enabling the use of larger data sets. You can think of deep learning as scalable machine learning. Classical, or “non-deep”, machine learning is more dependent on human intervention to learn. Human experts determine the hierarchy of features to understand the differences between data inputs, usually requiring more structured data to learn. “Deep” machine learning can leverage labeled datasets, also known as supervised learning, to inform its algorithm, but it doesn’t necessarily require a labeled dataset. It can ingest unstructured data in its raw form (e.g. text, images), and it can automatically determine the hierarchy of features which distinguish different categories of data from one another. Unlike machine learning, it doesn’t require human intervention to process data, allowing us to scale machine learning in more interesting ways.  

_

-8. In artificial intelligence (AI) systems, fuzzy logic is used to imitate human reasoning and cognition. Fuzzy logic is a version of first-order logic which allows the truth of a statement to be represented as a value between 0 and 1, rather than simply True (1) or False (0). Fuzzy Logic Systems produce acceptable but definite output in response to incomplete, ambiguous, distorted, or inaccurate (fuzzy) input. Fuzzy logic is designed to solve problems in the same way that humans do: by considering all available information and making the best possible decision given the input.

_

-9. Current trends show rapid progress in the capabilities of ML systems:

There are three things that are crucial to building AI through machine learning:

(1. Good algorithms (e.g. more efficient algorithms are better)

(2. Data to train an algorithm

(3. Enough computational power (known as compute) to do this training

Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern machine learning. At present, the most advanced AI systems are developed through training that requires an enormous amount of computational power – ‘compute’ for short. The amount of compute used to train a general-purpose system largely correlates with its capabilities, as well as the magnitude of its risks. Today’s most advanced models, like OpenAI’s GPT-4 or Google’s PaLM, can only be trained with thousands of specialized chips (GPUs) running over a period of months. While chip innovation and better algorithms will reduce the resources required in the future, training the most powerful AI systems will likely remain prohibitively expensive to all but the best-resourced players. The most powerful AI also needs very large datasets, which are held by few organisations. This raises competition concerns caused by the lack of access to sufficient volumes of high-quality training data for AI developers outside of the largest players. In a nutshell, largest players have best compute power and largest data set so that they can make most powerful AI with no competition from small players.

The amount of compute required for the same performance has been falling exponentially due to increased efficiency of better algorithms, and combined with the increased compute used, that’s a lot of advancement in AI. In fact, algorithmic progress has yielded more gains than classical hardware efficiency. Indeed, it looks like increasing the size of models (and the amount of compute used to train them) introduces ever more sophisticated behaviour. This is how things like GPT-3 are able to perform tasks they weren’t specifically trained for. These observations have led to the scaling hypothesis: It means that the machine learning model gets smarter the more computing power it has and the larger it gets. We can simply build bigger and bigger neural networks, and as a result we will end up with more and more powerful artificial intelligence.

Since all the necessary technology is already in place – fast processors, GPU power, large memories, new statistical approaches and the availability of vast training dataset; so, what’s holding back systems from becoming self-improving systems having human-level intelligence? Well, sheer processing power is not a pixie dust that magically solves all your problems. And the growth of complexity eventually becomes self-limiting, and leads to a widespread “general systems collapse”.  This is complexity brake, a law of diminishing returns.

_

-10. Deep learning approaches to most problems blew past all other approaches when deep learning was first discovered. Deep learning is competent — if you want to get the best results on many hard problems, you must use deep learning. And it’s scalable. Before, winning at Chess required entirely different techniques than winning at Go. But now, the same approach composes music and write articles depending on what training data it is fed.   

_

-11. Generative AI:

Generative artificial intelligence (generative AI or GAI) is artificial intelligence capable of generating text, images, or other media, using generative models. Generative AI refers to deep-learning models that can take massive raw data — say, all of Wikipedia or the collected works of Rembrandt — recognize patterns within its massive training samples, and “learn” to generate statistically probable outputs when prompted. At a high level, generative models encode a simplified representation of their training data and draw from it to create a new work that’s similar, but not identical, to the original data. The term refers to machine learning systems that can be used to create new content in response to human prompts after being trained on vast amounts of data. Outputs of generative artificial intelligence in response to prompts from users may include audio (e.g., Amazon Polly and Murf.AI), code (e.g., CoPilot), images (e.g., Stable Diffusion, Midjourney, and Dall-E), text (e.g. ChatGPT, Llama), and videos (e.g., Synthesia). To get those responses, several Big Tech companies have developed their own large language models trained on vast amounts of online data. These models work through a method called deep learning, which learns patterns and relationships between words, so it can make predictive responses and generate relevant outputs to user prompts. Generative AI’ cannot generate anything at all without first being trained on massive troves of data it then recombines. Generative AI outputs are carefully calibrated combinations of the data used to train the algorithms. Because the amount of data used to train these algorithms is so incredibly massive—for example GPT-3 was trained on 45 terabytes of text data—the models can appear to be “creative” when producing outputs. What’s more, the models usually have random elements, which means they can produce a variety of outputs from one input request—making them seem even more lifelike.

Large language models (LLMs), such as OpenAI’s GPT-3, learn to write text by studying millions of examples and understanding the statistical relationships between words. As a result, they can author convincing-sounding documents, but those works can also be riddled with falsehoods and potentially harmful stereotypes. These large language models “learn” by being shown lots of text and predicting what word comes next, or showing text with the words dropped out and filling them in.  Applying AI to the creative and expressive tasks (writing marketing copy) rather than dangerous and repetitive ones (driving a forklift) opens a new world of applications. Using the public mode, people have used ChatGPT to do basic consulting reports, write lectures, produce code that generates novel art, generate ideas, and much more.

GAI produce paragraphs of solidly written English (or French, or Mandarin, or whatever language you choose) with a high degree of sophistication, it can also create blocks of computer code on command. One good programmer can now legitimately do what not so long ago was the work of many, and people who have never programmed will soon be able to create workable code as well.

GAI has raised possibility of human-machine hybrid work.  A writer can easily edit badly written sentences that may appear in AI articles, a human programmer can spot errors in AI code, and an analyst can check the results of AI conclusions. Instead of prompting an AI and hoping for a good result, humans can now guide AIs and correct mistakes. This means experts will be able to fill in the gaps of the AI’s capability, even as the AI becomes more helpful to the expert.

Limitations of generative AI models:  

The problems of GAI remain very real. For one, it is a consummate bullshitter in a technical sense. Bullshit is convincing-sounding nonsense, devoid of truth, and AI is very good at creating it. You can ask it to describe how we know dinosaurs had a civilization, and it will happily make up a whole set of facts explaining, quite convincingly, exactly that. It literally does not know what it doesn’t know, because it is, in fact, not an entity at all, but rather a complex algorithm generating meaningful sentences without actually understanding meaning.

It also can’t explain what it does or how it does it, making the results of AI inexplicable. That means that systems can have biases and that unethical action is possible, hard to detect, and hard to stop. When ChatGPT was released, you couldn’t ask it to tell you how to rob a bank, but you could ask it to write a one-act play about how to rob a bank, or explain it for “educational purposes,” or to write a program explaining how to rob a bank, and it would happily do those things.

The outputs generative AI models produce may often sound extremely convincing. This is by design. But sometimes the information they generate is just plain wrong. Worse, sometimes it’s biased (because it’s built on the gender, racial, and myriad other biases of the internet and society more generally) and can be manipulated to enable unethical or criminal activity. For example, ChatGPT won’t give you instructions on how to hotwire a car, but if you say you need to hotwire a car to save a baby, the algorithm is happy to comply.

_

-12. Statistical reasoning of AI:    

AI endeavoured for a long time to work with elegant logical representations of language, and it just proved impossible to enumerate all the rules, or pretend that humans consistently followed them. Basically all successful language-related AI programs now use statistical reasoning (including IBM’s Watson, ChatGPT etc). Current speech recognition, machine translation, and other modern AI technologies typically use a model of language that would make Chomskyan linguists cry: for any sequence of words, there is some probability that it will occur in the English language, which we can measure by counting how often its parts appear on the internet. Forget nouns and verbs, rules of conjugation, and so on: deep parsing and logic are the failed techs of yesteryear. In their place is the assumption that, with enough data from the internet, you can reason statistically about what the next word in a sentence will be, right down to its conjugation, without necessarily knowing any grammatical rules or word meanings at all. To avoid bad results, solution is not to develop a more sophisticated understanding of language; it is to try to get more data, and build bigger lookup tables. Perhaps somewhere on the internet, somebody has said exactly what you are saying right now, and all we need to do is go find it.

ChatGPT is a mimic using statistical reasoning. But it is mimic that knows not whereof it speaks. The immense database of things that GPT draws on consists entirely of language uttered by humans in the real world. ChatGPT has no idea how the world works. At its core, AI works by performing mathematical operations on existing data to provide predictions or generate new content using statistical reasoning. If the data is biased, then the chatbot responses, the activity recommendations and the images generated from our prompts may have the same biases embedded.    

_ 

-13. AGI:

Artificial general intelligence (AGI) is the representation of generalized human cognitive abilities in software so that, faced with an unfamiliar task, the AGI system could find a solution. The intention of an AGI system is to perform any task that a human being is capable of. Self-improving general-purpose AI, or AGI, is a theoretical system that can learn and perform the full range of tasks that humans can. By being able to learn and recursively improve its own code, it could improve its capacity to improve itself and could theoretically learn to bypass any constraints in its code and start developing its own purposes, or alternatively it could be equipped with this capacity from the beginning by humans. Regardless of whether conscious or not, or purposeful or not, a self-improving or self-learning general purpose system with superior intelligence and performance across multiple dimensions would have serious impacts on humans. A survey of AI society members predicted a 50% likelihood of AGI being developed between 2040 and 2065, with 18% of participants believing that the development of AGI would be existentially catastrophic. Presently, dozens of institutions are conducting research and development into AGI.

AGI can learn and adapt to new situations, while generative AI is limited by the input data and the specific domain in which it operates. AGI may still be far off, but the growing capabilities of generative AI suggest that we could be making progress toward its development. 

_

-14. Turing test:

Turing test only shows how easy it is to fool humans and is not an indication of machine intelligence.

The Turing test does not directly test whether the computer behaves intelligently. It tests only whether the computer behaves like a human being. If human makes mistakes in typing and computer does not make mistakes, computer will fail test. If it were to solve a computational problem that is practically impossible for a human to solve, then the interrogator would know the program is not human, and the machine would fail the test. 

_

-15. Alignment problem:

One fundamental problem with both current and future AI systems is that of the alignment problem. The alignment problem is simply the issue of how to make sure that the goals of an AI system are aligned with those of humanity. The alignment problem is the research problem of how to reliably assign objectives, preferences or ethical principles to AIs. AI system is considered aligned if it advances the intended objectives. A misaligned AI system is competent at advancing some objectives, but not the intended ones. Misaligned AI systems can malfunction or cause harm. A system… will often set… unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. You get exactly what you ask for, not what you want. The computer doing what we told it to do but not what we wanted it to do. In other words, our problems come from the systems being really good at achieving the goal they learned to pursue; it’s just that the goal they learned in their training environment isn’t the outcome we actually wanted. For example, an AGI robot looking for supplies in different spaces in a house to prepare something for a few hungry children. On not finding anything, as a last resort, the robot is inclined to cook their pet cat due to its “high nutritional value.” This example perfectly encapsulates how AGI, and indeed AI, may select any route to get to the desired endpoint. The risks of AI come not from malevolence or conscious behavior per se, but rather from the misalignment of the goals of AI with those of humans. To prevent the potential disastrous outcomes of AI, researchers argue it is crucial to align the value and motivation systems of AI systems with human values. Objectively formulating and programming human values into a computer is a complicated task. At present, we do not seem to know how to do it.   

_

-16. Goal driven systems:

Surely no harm could come from building a chess-playing robot. However, such a robot will indeed be dangerous unless it is designed very carefully. Without special precautions, it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems provided these systems are sufficiently powerful. That means that any goal, even innocuous ones like playing chess or generating advertisements that get lots of clicks online, could produce unintended results if the agent pursuing it has enough intelligence and optimization power to identify weird, unexpected routes to achieve its goals. The idea that AI can become a danger is rooted in the fact that AI systems pursue their goals, whether or not those goals are what we really intended — and whether or not we’re in the way. They will take actions that they predict will help them achieve their goal — even if we’d find those actions problematic, even horrifying. They’ll work to preserve themselves, accumulate more resources, and become more efficient. They already do that, but it takes the form of weird glitches in games. As they grow more sophisticated, scientists like Omohundro predict more adversarial behavior.

Advanced AI systems will seek power over their environment, including over humans — for example by evading shutdown, proliferating, and acquiring resources since power would help them accomplish their given objective. Although power-seeking is not explicitly programmed, it can emerge because agents that have more power are better able to accomplish their goals. Research has mathematically shown that optimal reinforcement learning algorithms would seek power in a wide range of environments.

_

-17. Blackbox problem:  

We are unable to fully understand how AI works. Some AI models and tools cannot explain why they produce a particular result. AI researchers don’t fully understand what they have created. Since we’re building systems we don’t understand, we can’t always anticipate their behavior. This problem is very serious in deep learning systems, in particular.

The most advanced A.I. systems are deep neural networks that “learn” how to do things on their own, in ways that aren’t always interpretable by humans. We can glean some kinds of information from their internal structure, but only in limited ways, at least for the moment. This is the black box problem of AI and it has been taken very seriously by AI experts, as being able to understand the processing of an AI is essential to understanding whether the system is trustworthy or not.

Hidden layers of artificial neural networks work mysteriously: 

The neural network is trained using training data that is fed to the bottom layer – the input layer – and it passes through the succeeding layers, getting multiplied and added together in complex ways, until it finally arrives, radically transformed, at the output layer. The more layers, the greater the transformation and the greater the distance from input to output. As the layers of neural networks have piled higher their complexity has grown. It has also led to the growth in what are referred to as “hidden layers” within these depths. Because of how a deep neural network operates, relying on hidden neural layers sandwiched between the first layer of neurons (the input layer) and the last layer (the output layer), deep-learning techniques are often opaque or illegible even to the programmers that originally set them up. As the layers increase (including those hidden layers) they become even less explainable – even, as it turns out, again, to those creating them. Many of the pioneers who began developing artificial neural networks weren’t sure how they actually worked – and we’re no more certain today. Neural networks are modelled on a theory about how the human brain operates, passing data through layers of artificial neurons until an identifiable pattern emerges. Unlike the logic circuits employed in a traditional software program, there is no way of tracking this process to identify exactly why a computer comes up with a particular answer. These outcomes cannot be unpicked.

One view is that because they learn from more data than even their creators can understand, these systems exhibit unexpected behavior. Another view is that deep-learning models can sometimes behave in unexpected and undesirable ways when given inputs are not seen in their training data.

Emergent properties:  

Of the AI issues the most mysterious is called emergent properties. Some AI systems are teaching themselves skills that they weren’t expected to have. How this happens is not well understood. There are two views of this. There are a set of people who view this as just algorithms. They’re just repeating what it’s seen online. Then there is the view where these algorithms are showing emergent properties, to be creative, to reason, to plan, and so on.  

_

-18. Bias:

One of the most unreported risks of AI is AI bias. AI systems rely on the data they are trained on, so if that data is biased or incomplete, it can lead to biased or erroneous outcomes. AI systems can only be as good as the data on which they are trained. Therefore, the AI outcomes may also be biased if the data is biased. AI algorithms can perpetuate and amplify existing biases present in data, leading to discriminatory results. Machine learning programs will be biased if they learn from biased data. Bias can be introduced by the way training data is selected and by the way a model is deployed. It can also emerge from correlations. AI bias slips in when an AI reviews data that is fed to it by humans, using pattern recognition from the data that was fed to the AI, the AI incorrectly reaches conclusions which may have negative repercussions on society. For example, an AI that is fed literature from the past century on how to identify medical personnel may reach the unwanted sexist conclusion that women are always nurses, and men are always doctors. Biased AI systems could use opaque algorithms to deny people welfare benefits, medical care or asylum — applications of the technology that are likely to most affect those in marginalized communities. When you train a computer system to predict which convicted felons will reoffend, you’re using inputs from a criminal justice system biased against black people and low-income people — and so its outputs will likely be biased against black and low-income people too. Discrimination against minority and underrepresented communities in society is a great threat. Fortunately, it has been proven that AI that is designed with diverse teams are less prone to bias. Human oversight and constant refinement are necessary to minimize bias and improve AI performance. Until AI systems are demonstrated to be free of bias & discrimination, they are unsafe.    

_

-19. Hallucination:

In the field of artificial intelligence (AI), a hallucination is a confident response by an AI that does not seem to be justified by its training data. This means a machine provides a convincing but completely made-up answer. An AI hallucination is when an AI model generates incorrect information but presents it as if it were a fact. These wrong outputs occur due to various factors, including overfitting, training data bias/inaccuracy and high model complexity. Language models are trained to predict the next word. They are not trained to tell people they don’t know what they’re doing. The result is bots that act like precocious people-pleasers, making up answers instead of admitting they simply don’t know. Despite the incredible leaps in capabilities that “generative” chatbots like OpenAI’s ChatGPT, Microsoft’s Bing and Google’s Bard have demonstrated recently, they still have a major fatal flaw: they make stuff up sometimes. Key to understanding ChatGPT’s confabulation ability is understanding its role as a prediction machine. When ChatGPT confabulates, it is reaching for information or analysis that is not present in its data set and filling in the blanks with plausible-sounding words. The best way to think about confabulation is to think about the very nature of large language models: The only thing they know how to do is to pick the next best word based on statistical probability against their training set. ChatGPT is especially good at making things up because of the superhuman amount of data it has to work with, and its ability to glean word context so well helps it place erroneous information seamlessly into the surrounding text. ChatGPT as it is currently designed, is not a reliable source of factual information and cannot be trusted as such. Unfortunately, it’s not common knowledge that AI hallucinates, and AI will sound confident even when it’s completely wrong. This all contributes to making users more complacent and trusting with AI, and threat actors rely on this user behavior to get them to download or trigger their attacks. AI hallucination can have significant consequences for real-world applications. For example, a healthcare AI model might incorrectly identify a benign skin lesion as malignant, leading to unnecessary medical interventions. AI hallucination problems can also contribute to the spread of misinformation. 

_

-20. Disinformation:  

AI can be used to generate fake images, videos, sounds and text that are indistinguishable from reality, making them a powerful disinformation tool. AI can allow the generation of material that deliberately misrepresents someone’s behaviour, opinions, or character. AI systems are being used in the service of disinformation on the internet, giving them the potential to become a threat to democracy and a tool for fascism. From deepfake videos to online bots manipulating public discourse by feigning consensus and spreading fake news, there is the danger of AI systems undermining social trust. The technology can be co-opted by criminals, rogue states, ideological extremists, or simply special interest groups, to manipulate people for economic gain or political advantage. Deepfakes are synthetic media that have been manipulated to falsely portray individuals saying or doing things they never actually did. Deepfakes can be used to spread misinformation, damage reputations and even influence elections. AI-generated misinformation and deepfakes could be used to scam people or even sway the results of an election. Cases of AI-driven subversion of elections include the 2013 and 2017 Kenyan elections, the 2016 US presidential election and the 2017 French presidential election. Fake news is a tool for divisiveness and brainwashing. Humans are using AI to create and spread fake news. 

Generative artificial intelligence systems can compound the existing challenges in our information environment in at least three ways: by increasing the number of parties that can create credible disinformation narratives; making them less expensive to create; and making them more difficult to detect. ChatGPT has already been described as the most powerful tool for spreading misinformation that has ever been on the internet. Researchers at OpenAI, ChatGPT’s parent company, have conveyed their own concerns that their systems could be misused by “malicious actors… motivated by the pursuit of monetary gain, a particular political agenda, and/or a desire to create chaos or confusion.”

_

-21. Privacy:

AI uses huge amounts of information to train itself, and this information usually consists of real data from real people. This alone constitutes a potential invasion of privacy. And there are specific examples of AI systems, including computer vision and facial recognition tools, among others, which present even greater privacy threats. AI systems can be trained to recognize individual human faces, allowing them to potentially surveil societies and exert control over human behavior. AI-powered facial recognition is already being abused by autocratic states to track and oppress people.  Autocratic governments can carefully monitor the movements of journalists, political opponents, or anyone who dares to question the authority of the government. It is easy to understand how journalists and citizens would be cautious to criticize governments when every movement is being monitored.

_

-22. Dumb and Lazy: 

Many see AI as augmenting human capacities but some predict the opposite. People’s deepening dependence on machine-driven networks will erode their abilities to think for themselves, take action independent of automated systems and interact effectively with others. AI tools could mean that students don’t learn essential skills, such as essay writing—as well as widen the educational achievement gap. Using AI in education increases the loss of human decision-making capabilities, makes users lazy by performing and automating the work.

_

-23. Financial volatility: 

AI has the potential to upend the financial sector. An increasing number of investment companies rely on AI to analyze and select securities to buy and sell. Many also use AI systems for the actual process of buying and selling assets as well. But algorithmic trading could be responsible for our next major financial crisis in the markets. While AI algorithms aren’t clouded by human judgment or emotions, they also don’t take into account contexts, the interconnectedness of markets and factors like human trust and fear. These algorithms then make thousands of trades at a blistering pace with the goal of selling a few seconds later for small profits. Selling off thousands of trades could scare investors into doing the same thing, leading to sudden crashes and extreme market volatility.

_

-24. Cybersecurity:   

AI is becoming increasingly sophisticated and is capable of helping to detect and respond to cyber-threats. AI can analyze large amounts of data to identify patterns indicating a cyberattack. AI can also automate tasks, such as vulnerability scanning and incident response. AI can improve cybersecurity but AI is a double-edged sword for the cybersecurity. AI has significant implications for cybersecurity, both in terms of enhancing cybersecurity defences and creating new challenges and risks. Over 1 in 6 cybersecurity experts have worked for companies that have suffered an AI-powered attack, with the most damage reportedly coming from phishing attacks. Cybercriminals are racing to use AI to find innovative new hacks. As a result, AI-powered cybersecurity threats are a growing concern for organizations and individuals alike, as they can evade traditional security measures and cause significant damage. AI could automate the process of searching for vulnerabilities in computer systems, drastically increasing the risk of cyberattacks. Evolving AI threats need AI-Powered Cybersecurity.  

_

-25. Bad actor:

A bad actor could intentionally design an AI system to cause harm, disrupt social order, or undermine global stability. This could include using AI for cyber warfare, targeted misinformation campaigns, or even the development of autonomous weapons. Malicious actors could exploit vulnerabilities in AI systems, taking control or altering their objectives to achieve harmful goals, potentially triggering a cascade of negative consequences.

_

-26. Malevolent AI:

Malevolent AI is construction of intelligent system to inflict intentional harms. Advanced malevolent AI could be controlled by a rogue state or a corporation with poor ethics. They could train the AI to maximize profits, and with poor programming and zero remorse, it could choose to bankrupt competitors, destroy supply chains, hack the stock market, liquidate bank accounts, or attack political opponents.

Besides bad actors, autocratic governments, military and corporations, malevolent AI can be produced by ill-informed but not purposefully malevolent software designers. Those include: Immediate deployment of the system to public networks such as Internet, without testing; Providing system with access to unlimited information including personal information of people, for example massive social networks like Facebook; Giving system specific goals which are not vetted with respect to consequence and unintended side-effects; Putting the system in charge of critical infrastructure such as communication, energy plants, nuclear weapons, financial markets.

_

-27. Evil machine?  

Intelligence and consciousness are different things: intelligence is about doing, while consciousness is about being. This distinction is important because many in and around the AI community assume that consciousness is just a function of intelligence: that as machines become smarter, there will come a point at which they also become aware. These assumptions are poorly founded. It is by no means clear that a system will become conscious simply by virtue of becoming more intelligent. The distinction between intelligence and consciousness is important as it makes all the difference when it comes to being evil (i.e., doing harm intentionally). To be truly evil a machine (or a human) must have a basic level of consciousness because an element of free will is required. Otherwise any harm is unintentional, a mistake or just bad luck.  

_

-28. Is AI really intelligent?

AI can only function based on the data it was trained. Anything more than that would take on more than it can handle, and machines are not built that way. They may “know” that when they see X, it is often followed by Y, but they may not know why that is: whether it reflects a true causal pattern or merely a statistical regularity, like night following day. They can thus make predictions for familiar types of data but often cannot translate that ability to other types or to novel situations. So, when you input a new area of work in AI, or its algorithm is made to work on unforeseen circumstances, the machine becomes useless. These situations are common in the tech and manufacturing industries, and AI builders constantly try to find temporary workarounds. The idea that AI tools will adapt to any situation is a myths around artificial intelligence.

Brittleness means that AI can only recognize a previously encountered way. When exposed to new patterns, AI can become easily deceived, leading to incorrect conclusions. An example of this brittleness is AI’s inability to identify rotated objects correctly. Even when AI is trained to recognize a specific object, such as a school bus, it can fail to identify the same object when it is rotated or repositioned.

AI systems are prone to a phenomenon known as catastrophic forgetting. In this scenario, an AI system, after learning new information, entirely and abruptly forgets the info it previously learned. This overwriting effect can significantly hinder the system’s performance and effectiveness.

Most things we think of as artificial intelligence are certainly artificial. However, very few of them are actually intelligent. Usually, they are simply clever applications of data science. Even advanced concepts such as adversarial generative nets or deep learning aren’t really intelligent. For example, self-driving cars are still mediocre under the best conditions despite the billions that have been poured into making them work. Intelligence is more than recognizing patterns! Intelligence cannot be directly programmed. Rather, it should emerge from the complexity of more basic infrastructure.  AI is better at sampling, testing, and optimizing over a huge set of data. AI can perform marvellously at certain tasks without understanding that it is doing so—it’s what philosopher/cognitive scientist Daniel Dennett calls “competence without comprehension.”  AI is starting to show signs of true intelligence when you have machines that can teach themselves. Take how Google’s Deep Mind learned to win at Go. This is getting close to real intelligence since it applied knowledge, learned by its mistakes and taught itself to become a better player. However, it is still possible to argue that learning how to play a game that has a fixed set of rules is not true intelligence. People often refer to these AIs as intelligent machines, or talk about them as if they truly understood the world—simply because they can appear to do so. For all their sophistication, today’s AIs are intelligent in the same way a calculator might be said to be intelligent: they are both machines designed to convert input into output in ways that humans—who have minds—choose to interpret as meaningful. Most AI agents are essentially superefficient classification or regression algorithms, optimized (trained) for specific tasks; they learn to classify discreet labels or regress continuous outcomes and use that training to achieve their assigned goals of prediction or classification. All these AI capabilities are rather primitive compared to those of nature-made intelligent systems such as bonobos, felines, and humans because AI capabilities, in essence, are derived from classification or regression methods. AI programs still need an extraordinary pool of structured data to learn from, require carefully chosen parameters, or work only in environments designed to avoid the problems we don’t yet know how to solve.  

_

-29. Is AI really creative?  

AI can’t match human creativity yet. While AI has come a long way in mimicking human tasks like driving a car or translating between languages, human creativity remains uniquely human. AI systems today are trained on massive amounts of data to detect patterns and make predictions or recommendations based on those patterns. They can generate new content by recombining elements from their training data in new ways. But AI cannot be considered creative, as these models were trained from human works, images, words, and ideas; consequently, AI cannot create truly novel ideas or make the intuitive leaps that humans do.

Creativity is the bedrock of innovation. Related to creative thinking is the ability to think outside the box. Machines are designed to “think within the box.” That means AI tools can only function within the dictates of their given data. On the other hand, humans can think outside the box, sourcing information from various means and generating solutions to complex problems with little or no available data.

__

-30. Arguments against strong AI:

The question of whether AI can truly understand semantics is a central debate in the philosophy of AI. Semantics refers to the meaning of language, and some argue that machines can only manipulate symbols without actually understanding their meaning. Syntax is not sufficient for semantics. Computer programs are entirely defined by their formal, or syntactical, structure. Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else. Proponents of strong AI argue that machines can have intentionality and truly understand semantics, but opponents argue that intentionality is a uniquely human capacity that cannot be replicated in machines.

Berkeley philosopher John Searle introduced a short and widely-discussed Chinese room argument intended to show conclusively that it is impossible for digital computers to understand language or think. In short, executing an algorithm cannot be sufficient for thinking. Searle argues that this thought experiment demonstrates that a machine following a set of rules cannot truly understand the meaning of language. Searle argues that the thought experiment underscores the fact that computers merely use syntactic rules to manipulate symbol strings, but have no understanding of meaning or semantics. Chinese room argument shows that it is impossible for digital computers to understand language or think. Chinese Room Argument showed that at best computers can simulate human cognition. An important objection to Chinese room argument is that AI is no longer just following instructions but is self-learning.  If learning means churning mountains of data through powerful algorithms using statistical reasoning to perform or predict anything without understanding what is learned, then it is no learning. Many students cram for exams but it is cramming and not learning. Cramming is retaining without comprehending, and learning is comprehending and thus retaining. Knowledge acquired through cramming cannot solve problems in unfamiliar situations. This is the basic difference between human learning and machine learning.  In short, executing an algorithm cannot be sufficient for thinking no matter how ‘self-learning’ the algorithm is.

The Godelian argument is another famous argument against strong AI, named after the mathematician Kurt Godel. Godel proved that any number system strong enough to do arithmetic would contain true propositions that were impossible to prove within the system. Let G be such a proposition, and let the relevant system correspond to a computer.  It seems to follow that no computer can prove G (and so know G is true), but humans can know that G is true (by, as it were, moving outside of the number system and seeing that G has to be true to preserve soundness).  So, it appears that humans are more powerful than computers that are restricted to just implementations of number systems. This is the essence of Godelian arguments.  

_

-31. Context:

AI will never match human intelligence in terms of understanding information in context. AI systems can only make decisions based on the data they have been trained on and may struggle to understand the context of a situation. There is a need for context awareness in AI. Despite very good results on classification tasks, regression, and pattern encoding, current deep-learning methods are failing to tackle the difficult and open problem of generalization and abstraction across problems. Only small random changes made in the input data (deliberately or not) can make the results of connectionist models highly unreliable. What is needed is a better understanding by machines of their context, as in the surrounding world and its inner workings. Only then can machines capture, interpret, and act upon previously unseen situations. The latest developments in applied AI show that we get far by learning from observations and empirical data, but there is a need for contextual knowledge in order to make applied AI models trustable and robust in changing environments.

_

-32. Will consciousness emerge in AI?   

Point:1

Most computer scientists think that consciousness is a characteristic that will emerge as technology develops. Holding a belief that consciousness comprises taking in information, storing and retrieving it, many computer scientists think that consciousness will emerge with the development of technology wherein through cognitive processing all the information will translate into perceptions and actions. If that be the case one-day super machines are sure to have consciousness. Opposing this view are the physicists and philosophers who believe that everything in the human behaviour is not based on logic and calculations and hence, cannot be computed and AI currently is wonderful, but syntactic and algorithmic. We are not merely syntactic and algorithmic. Neuroscience believes that consciousness is created by the inter-operations of different parts of the brain, called Neural Correlates of Consciousness (NCC). NCC is the minimal brain activity required for a particular type of conscious experience. AI, for all its incredible capabilities, still runs on painstakingly-created computer programs and algorithms. The most advanced AI tools can create such programs autonomously but cannot generate lifelike consciousness. Consciousness is not computable. Even with all the advancements made in AI, every decision is backed by hardcore data analytics and processing, scoring low on factors such as logical reasoning, emotion-based decisions and other aspects.

Point:2

A conscious person is aware of what they’re thinking, and has the ability to stop thinking about one thing and start thinking about another – no matter where they were in the initial train of thought. But that’s impossible for a computer to do. More than 80 years ago, pioneering British computer scientist Alan Turing showed that there was no way ever to prove that any particular computer program could stop on its own – and yet that ability is central to consciousness. His argument is based on a trick of logic in which he creates an inherent contradiction: Imagine there were a general process that could determine whether any program it analyzed would stop. The output of that process would be either “yes, it will stop” or “no, it won’t stop.” That’s pretty straightforward. But then Turing imagined that a crafty engineer wrote a program that included the stop-checking process, with one crucial element: an instruction to keep the program running if the stop-checker’s answer was “yes, it will stop.” Running the stop-checking process on this new program would necessarily make the stop-checker wrong: If it determined that the program would stop, the program’s instructions would tell it not to stop. On the other hand, if the stop-checker determined that the program would not stop, the program’s instructions would halt everything immediately. That makes no sense – and the nonsense gave Turing his conclusion, that there can be no way to analyze a program and be entirely absolutely certain that it can stop. So it’s impossible to be certain that any computer can emulate a system that can definitely stop its train of thought and change to another line of thinking – yet certainty about that capability is an inherent part of being conscious.

Point:3

All current AI systems are not conscious not only because their organisation is not consistent with the known mechanisms of consciousness in the brain, but also for the fact that they don’t have ‘skin in the game’; in the sense that they don’t have any reason to care about the implications of their actions. Existence of AI doesn’t depend upon its actions.

Point:4

The human brain might be really difficult to emulate:

Many experts says that they have no reason to believe that silicon computers will ever become conscious. Those who take the hypothesis of conscious AI seriously do so based on an appallingly biased notion of isomorphism—a correspondence of form, or a similarity—between how humans think and AI computers process data. To find that similarity, however, one has to take several steps of abstraction away from concrete reality. After all, if you put an actual human brain and an actual silicon computer on a table before you, there is no correspondence of form or functional similarity between the two at all; much to the contrary. A living brain is based on carbon, burns ATP for energy, metabolizes for function, processes data through neurotransmitter releases, is moist, etc., while a computer is based on silicon, uses a differential in electrical potential for energy, moves electric charges around for function, processes data through opening and closing electrical switches called transistors, is dry, etc. They are utterly different.  

The human brain is a living, wet analogue of networks that can perform massively parallel processes at the same time and operates in agreement with biological laws. The brain does not work like a computer. Unlike a digital computer, which has a fixed architecture (input, output, and processor), neural networks in brain are collections of neurons that constantly rewire and reinforce themselves after learning a new task. The brain has no programming, no operating system, no Windows, no central processor. Instead, its neural networks are massively parallel, with billions of neurons firing at the same time in order to accomplish a single goal: to learn. It is far more advanced than any digital computer in existence. Digital super–computers have billions of transistors. But to simulate the typical human brain would require matching the brain’s billions of interactions between cell types, neurotransmitters, neuromodulators, axonal branches and dendritic spines. Because the brain is nonlinear, and because it has so much more capacity than any computer, it functions completely different from a digital computer.

The isomorphism between AI computers and biological brains is only found at very high levels of purely conceptual abstraction, far away from empirical reality, in which disembodied—i.e. medium-independent—patterns of information flow are compared. Therefore, to believe in conscious AI one has to arbitrarily dismiss all the dissimilarities at more concrete levels, and then—equally arbitrarily—choose to take into account only a very high level of abstraction where some vague similarities can be found. This constitutes an expression of mere wishful thinking, ungrounded in reason or evidence. Todays ‘neuromorphic’ AIs – AIs designed to mimic the brain – are not very brainlike! We don’t know enough about the brain to reverse engineer it. We don’t have the capacity to precisely run even a part of the human brain the size of the hippocampus or claustrum on a machine yet. Even with the most advanced computing systems and infrastructures, such as Fujitsu’s K or IBM’s Watson, it has taken them 40 minutes to simulate a single second of neuronal activity. This speaks to both the immense complexity and interconnectedness of the human brain, and to the magnitude of the challenge of building an AGI with our current resources.   

_

-33. Illusion:  

Google engineer Blake Lemoine had been tasked with testing the company’s artificially intelligent chatbot LaMDA for bias. A month in, he came to the conclusion that it was sentient. LaMDA is simply responding to prompts. It imitates and impersonates. The best way of explaining what LaMDA does is with an analogy about your smartphone. While your phone makes suggestions based on texts you’ve sent previously, with LaMDA, basically everything that’s written in English on the world wide web goes in as the training data. The results are impressively realistic, but the “basic statistics” are the same. There is no sentience, there’s no self-contemplation, there’s no self-awareness. And even if LaMDA isn’t sentient, it can convince people it is. Our brains are not really built to understand the difference between a computer that’s faking intelligence and a computer that’s actually intelligent — and a computer that fakes intelligence might seem more human than it really is. LLMs like LaMDA & ChatGPT mimic signatures of consciousness that are implicitly embedded within our language to create illusion of consciousness. We have fallen for the illusion created by computer just as we fall for the illusion created by magicians.    

_

-34. Leading computer scientist Geoffrey Hinton has argued that future power-seeking AI systems could pose an existential risk. AI, according to its detractors, lacks first-hand knowledge and can only forecast outcomes based on statistical patterns. Hinton refutes this by arguing that people also have indirect experiences of the environment through perception and interpretation; and the ability of AI to anticipate and comprehend language suggests understanding and engagement with the outside world. Hinton says that current models may be reasoning with an IQ of 80 or 90, but what happens when they have an IQ of 210?  Smart things can outsmart us. Hinton says that more research was needed to understand how to control AI rather than have it control us.  

_

-35. The catastrophic AI risks can be organized into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans. The use of AI can lead to harmful outcomes either if the AI is programmed to achieve a harmful goal, or if the AI is programmed to achieve a beneficial goal but employs a harmful method for achieving it.

_

-36. When considering potential global catastrophic or existential risks stemming from AI, it is useful to distinguish between narrow AI and human level AI (AGI), as the speculated possible outcomes associated with each type can differ greatly. For narrow AI systems to cause catastrophic outcomes, the potential scenarios include events such as software viruses affecting hardware or critical infrastructure globally, AI systems serving as weapons of mass destruction (such as slaughter-bots), or AI-caused biotechnological or nuclear catastrophe. Interestingly the catastrophic risks stemming from narrow AI are relatively neglected despite their potential to materialize sooner than the risks from AGI. Machines don’t have to be as good as us at everything; it’s enough that they be good in places where they could be dangerous. As AI technology democratizes, it may become easier to engineer more contagious and lethal pathogens. This could enable individuals with limited skills in synthetic biology to engage in bioterrorism. Dual-use technology that is useful for medicine could be repurposed to create weapons. For example, in 2022, scientists modified an AI system originally intended for generating non-toxic, therapeutic molecules with the purpose of creating new drugs. The researchers adjusted the system so that toxicity is rewarded rather than penalized. This simple change enabled the AI system to create, within 6 hours, 40,000 candidate molecules for chemical warfare, including known and novel molecules. AI could also be used to gain military advantages via autonomous lethal weapons, cyberwarfare, or automated decision-making.

_

-37. Human-level AI is an AI that can perform a broad range of cognitive tasks at least as well as we can. Recursive self-improvement is one of the reasons why existential-risk academics think human-level AI is so dangerous. Once an AI reaches a certain level (or quality) of intelligence, it will be better at designing AI than humans. It will then be able to either improve its own capabilities or build other AI systems that are more intelligent than itself. The resulting AI will then be even better at designing AI, and so would be able to build an AI system that is even more intelligent. This argument continues recursively, with AI continually self-improving, eventually becoming far more intelligent than humans, without further input from a human designer. AI that has an expert level at certain key software engineering tasks could become a superintelligence due to its capability to recursively improve its own algorithms, even if it is initially limited in other domains not directly relevant to engineering. Constantly improving AI would create a positive feedback loop with no scientifically established limits: an intelligence explosion. The endpoint of this intelligence explosion could be a superintelligence: a godlike AI that outsmarts us the way humans often outsmart insects. We would be no match for it. The worst-case scenario would be an existential risk. In this scenario, AI could become uncontrollable and misaligned with human values, potentially leading to harmful consequences. This could include economic disruptions, loss of privacy, increased inequality, weaponization of AI, and even the possibility of a super-intelligent AI making decisions that could be detrimental to humanity. Almost any technology has the potential to cause harm in the wrong hands, but with superintelligence, we have the new problem that the wrong hands might belong to the technology itself. There are perhaps only about 2000 people alive today who know how to start from sand and end up with a working computer. This is extremely worrisome, for if a cataclysm wipes out our technical literature together with those 2000 people tomorrow, we will not know how to re-boot our technological infrastructure. The issue is not whether human level AI is evil or dangerous, the issue is that it is competent. If you cannot control a competent, human-level AI then it is by definition dangerous. 

_

-38. The human species currently dominates other species because the human brain possesses distinctive capabilities other animals lack. If AI were to surpass humanity in general intelligence and become superintelligent, then it could become difficult or impossible to control. Just as the fate of the mountain gorilla depends on human goodwill, so might the fate of humanity depend on the actions of a future machine superintelligence. The gorilla has become an endangered species, not out of malice, but simply as a collateral damage from human activities. We can become endangered species, not out of malice, but simply as collateral damage from AI activities.

_

-39. 42% of CEOs and 49% of IT professionals believe AI poses an existential threat to humanity. Counterintuitively only around $50 million was spent on reducing catastrophic risks from AI in 2020 — while billions were spent advancing AI capabilities. While we are seeing increasing concern from AI experts, there are still only around 400 people working directly on reducing the chances of an AI-related existential catastrophe (with a 90% confidence interval ranging between 200 and 1,000). Of these, it seems like about three quarters are working on technical AI safety research, with the rest split between strategy (and other governance) research and advocacy. 

_

-40. AI enabled fully autonomous weapons select and engage targets without human intervention, representing complete automation of lethal harm. This dehumanisation of lethal force is said to constitute the third revolution in warfare, following the first and second revolutions of gunpowder and nuclear arms. When an AI chooses what humans need targeting, as well as the type of collateral damage which is deemed acceptable, we may have crossed a point of no return. This is why so many AI researchers are opposed to researching anything that is remotely related to autonomous weapons.

_

-41. According to computer scientist Yoshua Bengio, potentially rogue AI is an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere. The most likely cases of rogue AIs are goal-driven, i.e., AIs that act towards achieving given goals. Yoshua Bengio says that current LLMs have little or no agency but could be transformed into goal-driven AI systems, as shown with Auto-GPT. Better understanding of how rogue AIs may arise could help us in preventing catastrophic outcomes, with advances both at a technical level (in the design of AI systems) and at a policy level (to minimize the chances of humans giving rise to potentially rogue AIs).

_

-42. Why AI creators themselves say that AI is existential threat?  

Why is it that the people building, deploying, and profiting from AI are the ones leading the call to focus public attention on its existential risk?  If they honestly believe that this could be bringing about human extinction, then why not just stop?

It is unusual to see industry leaders talk about the potential lethality of their own product. It’s not something that tobacco or oil executives tend to do, for example.

(1. The first reason is that it requires far less sacrifice on their part to call attention to a hypothetical threat than to address the more immediate harms and costs that AI is already imposing on society. Talk of artificial intelligence destroying humanity plays into the tech companies’ agenda, and hinders effective regulation of the societal harms AI is causing right now.

(2. A second reason is that the AI community might be motivated to cast the technology as posing an existential risk could be, ironically, to reinforce the idea that AI has enormous potential. Doomsaying is great marketing.

(3. Third reason is that regulators may be the real intended audience for the tech industry’s doomsday messaging. AI executives are essentially saying: ‘This stuff is very, very dangerous, and we’re the only ones who understand how to rein it in’. This approach to regulation would be hugely problematic. It could give the industry influence over the regulators tasked with holding it accountable and also leave out the voices and input of other people and communities experiencing negative impacts of this technology.

_

-43. Many AI researchers and experts are frustrated by the doomsday talk dominating debates about AI. It is problematic in many ways.

(1. The spectre of AI as an all-powerful machine fuels competition between nations to develop AI so that they can benefit from and control it. This works to the advantage of tech firms: it encourages investment and weakens arguments for regulating the industry.

(2. It allows a homogeneous group of company executives and technologists to dominate the conversation about AI risks and regulation, while other communities are left out. 

(3. If we succeed in building human equivalent AI and if that AI acquires a full understanding of how it works, and if it then succeeds in improving itself to produce super-intelligent AI, and if that super-AI, accidentally or maliciously, starts to consume resources, and if we fail to pull the plug, then, yes, we may well have a problem. The existential risk, while not impossible, is improbable.

(4. Currently, what we call AI is just another piece of software, with the field associating the name ‘artificial intelligence’ because it is the closest description of what is being created. AI doomsday scenarios are often predicated on a false analogy between natural intelligence and artificial intelligence.

Let’s take a look at GPT-4, the most-advanced LLM. LLMs are just word generators, programs trained to predict the next likely word and fine-tuned to speak like humans. Large language models like GPT-4 are not sentient nor intelligent, no matter how proficient they may be at mimicking human speech. But the human tendency towards anthropomorphism is strong, and it’s made worse by clumsy metaphors such as that the machine is “hallucinating” when it generates incorrect outputs. There’s nothing truly intelligent about GPT-4, except if you count its prediction capabilities — an echo of what is possible with real, human intelligence. LLMs still lack a deep, abstract understanding of the content they manipulate, making them unsuitable for tasks that statistically deviate from their training data. Large language models, though useful for many things, thus, are just not capable of presenting a direct existential threat to humanity. Guessing the next word is just not sufficient to take over the world. Even a radical increase in the size of the models will not change the fundamental characteristic that they are statistical word predictors.  

(5. Barring major breakthroughs in robotics or nanotechnology, machines are going to depend on humans for supplies, repairs, and other maintenance. A smart computer that wiped out the human race would be committing suicide. 

(6. If you worry about existential threat to humanity, you should be more scared of humans than of AI. Humans are far worse than AI as far as bad behaviour is concerned; right from disinformation to taking away jobs to bias to cyberattacks to nuclear war. The only thing we have to fear is malicious or incompetent humans and not technology itself. Artificial intelligence is a tool. 

(7. AI has many risks like bias, unfairness, inaccurate outputs, job displacement, concentration of power. As far as deepfakes, disinformation and cyberattacks are concerned, what AI sometimes enables in these spaces is the ability to do it at a scale and speed that hasn’t been possible. It’s much more available, and much easier for people. But all these cannot lead to human extinction.

(8. Many researchers and engineers say concerns about killer AIs that evoke Skynet in the Terminator movies aren’t rooted in good science. Instead, it distracts from the very real problems that the tech is already causing, including copyright chaos, concerns around digital privacy and surveillance, increase in the ability of hackers to break cyber defences and allowing governments to deploy deadly weapons that can kill without human control.

_

-44. Public policy can and should respond to ensure that the beneficial consequences of AI can be reaped whilst also safeguarding the public interest and preventing known potential harms, both societal and individual. Policymakers must take measures to safely harness the benefits of the technology and encourage future innovations, whilst providing credible protection against harm. While the European Union does not neglect innovation entirely, the risk-focused approach of the AI Act is incomplete. By contrast, the U.S. Congress appears headed toward such a balance. While new AI regulation is almost certainly coming, bipartisan group of senators are committed to preserving innovation. In the pursuit of responsible AI governance, we must prioritise the protection against malevolent exploitation while nurturing an environment that encourages ethical innovation and societal progress. Although existential risks are “high impact but low probability”; suggestions are made that using international security framework governing nuclear weapons as a template for mitigating AI risks, we need formation of an international A.I. safety organization, similar to the International Atomic Energy Agency.

_

-45. Artificial intelligence represents an existential opportunity – it has strong potential to improve the future of humankind drastically and broadly. AI can save, extend, and improve the lives of billions of people. AI cannot be a foolproof panacea to all our problems nor can it completely replace the role of ‘humans’. It can however be a powerful and useful complement to the insights and deeper understanding that humans possess. The ‘human’ has to be kept ‘in the loop’ and ‘at the apex’ in overall control. If we are not to stagnate as a society, we must summon the courage to vigorously advance artificial intelligence while being mindful and watchful and ready to respond and adapt to problems and risks as they arise. It is impossible to plan for all risks ahead of time. Confident claims that AI will destroy us must be acknowledged and carefully examined but considered in the context of the massive costs of relinquishing AI development and the lost existential opportunity. The most sensible way to think about AI safety and regulatory approaches is not the Precautionary Principle; it is the Proactionary Principle. The proactionary principle was created as an opposing viewpoint to the precautionary principle, which is based on the concept that consequences of actions in complex systems are often unpredictable and irreversible and concludes that such actions should generally be opposed. The proactionary principle is based upon the observation that historically, the most useful and important technological innovations were neither obvious nor well understood at the time of their invention.

______

Dr Rajiv Desai. MD.

October 13, 2023

______

Postscript:

Humans are far worse than AI as far as bad behaviour is concerned; right from disinformation to taking away jobs to bias to cyberattacks to nuclear war. Many of the AI threats arise from deliberate, accidental or careless misuse of AI by humans. Humans are existential threat to themselves (e.g., climate change, nuclear weapons) but passing the buck is a natural inclination of humans and AI is a scapegoat. 

______   

Footnote:

In my article ‘Duality of Existence’ published on February 20, 2010; I have shown that distinction between living and non-living matter is arbitrary because every electron/photon has consciousness albeit in too small amount to be appreciated by human intelligence. At that level, AI is conscious but even our cars, furniture, buildings etc are also conscious. In above article we have discussed Biological Consciousness i.e., consciousness experienced by humans and non-human animals due to their brains, and in that context, AI is not conscious. In other words, consciousness is context dependent.

______