“An invention, once invented, cannot be uninvented” — Arthur Koestler
* By Francisco Camargo
Artificial intelligence (AI) is emerging as one of the most fascinating and challenging human achievements. Its ability to learn, adapt, and perform complex tasks redefines the boundaries of what we call "intelligence." However, along with its many promises, AI also raises ethical, social, and philosophical questions. To understand these challenges, we can turn to both science fiction and past philosophical reflections, exemplified by New Zealand writer Samuel Butler's article "Darwin among the Machines" (1863). Exactly a century later, in 1963, the brilliant Frank Herbert published the first volume of the "Dune" series. Set in a galactic empire thousands of years from now, the Butlerian Jihad took place in the past, in which all thinking machines were destroyed for the danger they posed to humanity.
In 1963, there was no talk of artificial intelligence, nor neural networks, there were no microcomputers, distributed networks, internet, nothing, it was the era of mainframes, with 4 kbytes of memory.
A warning from 1863: the evolution of machines
In your article Darwin among the Machines, Samuel Butler writes with wit and insight when comparing the evolution of machines to the natural evolution described by Charles Darwin. Butler argues that machines, like living beings, can evolve and answers two questions:
The first: machines will be able to build other machines, and the answer was yes, which was not evident in 1863
The second: machines will be able to self-improve, that is, to evolve, and the answer was again yes.
He warns of the potential for machines to become independent and that this would be a terrible danger to the human race, and suggests that to protect our species we must “destroy the most dangerous machines.”
Although written in the 19th century, Butler's essay resonates deeply in the age of AI. The advancement of autonomous systems, such as robots and deep learning algorithms, revives concerns about the possibility of machines becoming independent agents, capable of making decisions beyond human control. The central question is not only whether machines can evolve, but how humans should manage this evolution to avoid dystopian scenarios.
The Butlerian Jihad: Lessons from “Dune”
In the universe of Dune, Frank Herbert explores an apocalyptic event known as the Butlerian Jihad, a war between humans and intelligent machines. The conflict ends with the prohibition of creating "a machine in the image of the human mind."
Herbert uses Butlerian Jihad as a warning metaphor about the dangers of relying on radical innovations, where we cannot yet see the full consequences, or at least the most important ones.
Butlerian Jihad also raises moral and philosophical questions about the role of technology in our lives. Herbert suggests that by relying too heavily on machines, humans risk losing their essence, creativity, and autonomy. This leads us to reflect on the extent to which we should allow AI to perform roles that traditionally require human judgment and morality.
The challenges of artificial intelligence
In Butler and Herbert's time, personal computers, email, smartphones, the internet, and social media didn't exist, and they, visionaries, already envisioned the challenges of AI in the modern world, making them clearer for future generations.
Among the main dilemmas are:
1. Ethics and Control: How can we ensure that AI systems act ethically? Or rather, what ethics should we teach future AIs? Algorithms can inherit biases from their creators or from data, resulting in discrimination or questionable decisions. Furthermore, the difficulty in predicting the behavior of complex systems can lead to unexpected consequences.
But the most important thing, and for those who have already developed systems, is how to guarantee the necessary specifications for this intelligence, as with any complex system, for a given input, the outputs may not be as expected.
2. Machine Autonomy: The advancement of autonomous AI, such as self-driving cars and military drones, raises questions about liability. Who will be at fault in the event of failure or damage? The machine, the programmer, or the user? In autonomous cars, all manufacturers have stated (or believed) that the priority is always to protect passengers, otherwise they wouldn't sell a car.
3. Technological Dependence: Just like in *Dune*, the growing reliance on machines can weaken our own cognitive capacity and critical thinking skills. And this is already happening to children and young people due to screen time. Reading isn't part of the brain's innate circuitry; it has to be developed through practice and repetition.
We must ask ourselves: are we using AI as a tool or allowing it to replace our brains?
4. The Future of Employment: With task automation, there will initially be a huge gain in productivity, but then many traditional jobs will be at risk.
The challenge is not just economic, but social and psychological, as millions of people could lose not only their source of income, but also their sense of purpose.
Regulation and the future
Both Samuel Butler and Frank Herbert offer us visions that, while rooted in theoretical or fictional contexts, remain extremely relevant. The underlying message of both is clear: technological progress must be accompanied by ethical and philosophical progress.
It's not enough to create intelligent machines; we need to ensure that their evolution is aligned with human values.
Artificial intelligence is a powerful tool, with the potential to transform the world in ways we are only beginning to understand. But, as Butler and Herbert remind us, it is also a force that demands vigilance, reflection, and, above all, responsibility.
And here we come across the Regulation of Artificial Intelligence. If the main countries of the world do not make the regulations, others who are willing to do so will be at a disadvantage, because the initiative, creativity and entrepreneurship will migrate from these countries for those that do not have regulation.
Arthur Koestler was a writer, journalist, and thinker, considered one of the most influential intellectuals of the 20th century. A Hungarian dissident, he took refuge in England during the repression of the Hungarian Communist Party in 1956.
*Francisco Camargo is Founder and CEO of CLM and Treasurer of ABES
Notice: The opinion presented in this article is the responsibility of its author and not of ABES - Brazilian Association of Software Companies
Article originally published on the IT Forum website: https://itforum.com.br/colunas/desafios-inteligencia-artificial/
EN 
PT 
