There's a Perverse Thrill in Reading a Book That Presages the Possible Extinction of Humankind

Superintelligence is a serious, intellectually disorientating treatment of ideas, imagining the inevitable future when we are able to create an artificial general intelligence.

Superintelligence: Paths, Dangers, Strategies

Publisher: Oxford University Press
Length: 352 pages
Author: Nick Bostrom
Publication date: 2014-09

The future of AI and whether it poses an existential threat to humanity is a question that has seeped into the public consciousness., the online public debating forum for prominent scientists, philosophers and other intellectuals, recently discussed this question, prompted by Elon Musk’s comments, which suggested that AI is perhaps the gravest threat that humanity faces. Musk’s remarks appear to have been motivated by reading Nick Bostrom’s recent monograph: Superintelligence: Paths, Dangers and Strategies.

Superintelligence is a serious, intellectually disorientating treatment of ideas, imagining the inevitable future when we are able to create an AGI (an artificial general intelligence). An AGI would be capable of successfully performing any task that a human can. Such a machine would thus be capable of recursive self-improvement (on a digital time scale) perhaps rapidly leading to an explosion in its own intelligence. An exponentially self-improving superintelligence, according to Bostrom, would pose a significant threat to human survival.

There's a perverse thrill in reading a book that presages the possible extinction of the human species; it's the same feeling of sublime terror and morbid fascination that accompanies discussions of nuclear war, climate change, asteroid impact and global pandemics. Appropriately, Bostrom heads up Oxford University’s Future of Humanity Institute, which looks at threats to humankind's survival.

However, whereas asteroids, nuclear war and disease all seem to have a ready purchase on our everyday experience, superintelligent AI (of the sort that Bostrom envisages) seems almost unimaginably remote and incomprehensible. Depictions of an AI threat in film have largely imagined it on a human level: Terminator, AI, Blade Runner and recently, Ex Machina. The closest thing we have to what Bostrom imagines is probably HAL from 2001: A Space Odyssey, although even HAL is relatively easily vanquished by humanity.

There are probably two reasons for the lack of representation of superintelligence in popular culture. The first is that superintelligence is not inherently dramatic: depictions of conventional AI are mostly devices to examine human nature. By contrast, superintelligence represents something fundamentally different and inhuman. The second reason, equally important, is that it takes a work like Bostrom’s to expand our imaginative horizons and, in an intellectually coherent way, to discuss what form a superintelligence might plausibly take.

Bostrom isn’t modest about the scale of the challenge to control superintelligent AI. He describes it in the preface (with Fukuyamian chutzpah) as ‘the last challenge we will face’ establishing a suitably apocalyptic entry point. This is contrasted against early protestations of epistemic humility, namely, that ‘many of the points made in [the] book are probably wrong’. Nevertheless, Bostrom’s case is fastidiously argued: full of caveats and qualifications. And, broadly speaking, it is difficult to fault the logic or architecture of his argument. If we are able to construct an AGI which is capable of recursive self-improvement and it undergoes an intelligence explosion, it seems likely that Bostrom’s prognostications are correct.

The book necessarily touches upon many different fields: a section on embryo selection (for intelligence) is thought-provoking and made me re-evaluate some of my moral assumptions. Bostrom is very clear-headed on the uncomfortable topic of eugenics, for example. If the human subject and her capacities change irrevocably, suddenly, simple moral principles become far more problematic. Whether genetically engineered humans would become accepted by society is an open question. However, as Bostrom points out, IVF treatment was once seen as a moral aberrance and is now (excepting religious objection) a triviality.

Elsewhere, Superintelligence provides a corrective to human pride. Whilst the human brain is deemed by humans to be among the most complex objects in the universe, it seems all but inevitable that computers will supersede its capacities. Bostrom suggests that superintelligent machines will stand to us, by the same relation that we stand to a ‘beetle or a worm’. It's a powerful analogy.

There are points, however, wherein it seems Bostrom is balancing improbability atop improbability: when discussing the potential future economic effects of superintelligence, for instance, or speculating about the rights of workers who might live in future simulated realities. In fact, Bostrom’s best arguments are voiced in general terms, such as his discussion of what machines should value, how they might acquire their values and if it’s possible to represent values in terms of computer code. These sorts of questions seem most amenable to Bostrom’s syncretic philosophical reasoning.

Elsewhere, his convergent objective conjecture is convincingly argued. This is the idea that there are certain general objectives that most intelligent beings would want in order to maximise their chances of successfully completing any tasks they were given. Bostrom suggests that resource acquisition, self-preservation and cognitive enhancement are amongst such universals that machines would want them, too. Without proper controls, superintelligent machines could decide that humanity was an obstacle to their resource acquisition; after all, more resources and energy would almost always benefit the efficiency with which an AI could achieve its goals. Many of Bostrom’s examples concern unintended consequences and the difficulty of predicting, changing or limiting the “behaviour” of a highly complex AI.

Bostrom’s discussion of perverse instantiation, where a machine interprets human commands according to their letter and not their spirit, provides a striking example of such unintended consequences. To illustrate perverse instantiation, he gives the example of an AI whose objective was to make humans smile. To begin with, this command might be instantiated in a fatuous way: the machine telling jokes, for example. However, as intelligence increased, the AI might find increasingly bizarre and direct routes to achieving the goal: paralysing human facial muscles into a constant beaming smile. Or, if muscle alteration were explicitly ruled out by humans, implanting electrodes into the human brain to indefinitely stimulate its pleasure centres. Furthermore, as preceding discussions in the book make clear, the machine might act in a duplicitous or otherwise secretive manner to ensure the attainment of its objectives. It might therefore conceal its plans, if it thought that humans would object to its method of achieving its goal.

What emerges from reading Superintelligence is the sheer number of different and intractable problems that should be solved or given serious consideration before the inception of a superintelligent AI. Bostrom’s achievement (demonstrating his own polymathic intelligence) is a delineation of a difficult subject into a coherent and well-ordered fashion. This subject now demands more investigation. If a superintelligent AI is developed in our lifetime, the questions concerning value and control must become our principal priorities now.


In the wake of Malcolm Young's passing, Jesse Fink, author of The Youngs: The Brothers Who Built AC/DC, offers up his top 10 AC/DC songs, each seasoned with a dash of backstory.

Editor's Note: Originally published 30 July 2014.

10. “Bedlam in Belgium”
(Flick of the Switch, 1983)

This is a massively underrated barnstormer from the boys off the much-maligned (unfairly, I think) Flick of the Switch. The album was missing Mutt Lange, but the Youngs did have his very capable engineer, Tony Platt, as co-producer in the studio at Compass Point in the Bahamas. Tony’s a real pro. I think he did a perfectly fine job on this album, which also features the slamming “Nervous Shakedown”.

But what I find most interesting about “Bedlam in Belgium” is that it’s based on a fracas that broke out on stage in Kontich, Belgium, in 1977, involving Bon Scott, the rest of the band, and the local authorities. AC/DC had violated a noise curfew and things got hairy.

Yet Brian Johnson, more than half a decade later, wrote the lyrics with such insight; almost as if he was the one getting walloped by the Belgian police: He gave me a crack in the back with his gun / Hurt me so bad I could feel the blood run. Cracking lyrics, Bon-esque. Unfortunately for Brian, he was removed from lyric-writing duties from The Razors Edge (1990) onwards. All songs up to and including 2008’s Black Ice are Young/Young compositions.

Who’ll be writing the songs on the new album AC/DC has been working on in Vancouver? AC/DC fans can’t wait to hear them. Nor can I.

9. “Spellbound”
(For Those About to Rock We Salute You, 1981)

"Spellbound" really stands as a lasting monument to the genius of Mutt Lange, a man whose finely tuned ear and attention to detail filed the rough edges of Vanda & Young–era AC/DC and turned this commercially underperforming band for Atlantic Records into one of the biggest in the world. On “Spellbound” AC/DC sounds truly majestic. Lange just amplifies their natural power an extra notch. It’s crisp sounding, laden with dynamics and just awesome when Angus launches into his solo.

“Spellbound” is the closer on For Those About to Rock We Salute You, the last album Lange did with AC/DC, so chronologically it’s a significant song; it marks the end of an important era. For Those About to Rock was an unhappy experience for a lot of people. There was a lot of blood being spilled behind the scenes. It went to number one in the US but commercially was a massive disappointment after the performance of Back in Black. Much of the blame lies at the feet of Atlantic Records, then under Doug Morris, who made the decision to exhume an album they’d shelved in 1976, Dirty Deeds Done Dirt Cheap, and release it in-between Back in Black and For Those About to Rock.

In the book Phil Carson, who signed AC/DC to Atlantic, calls it “one of the most crass decisions ever made by a record-company executive” and believes it undermined sales of For Those About to Rock.

8. “Down Payment Blues”
(Powerage, 1978)

This is one of the best songs off Powerage -- perhaps the high point of Bon Scott as a lyricist -- but also significant for its connection to “Back in Black”. There are key lines in it: Sitting in my Cadillac / Listening to my radio / Suzy baby get on in / Tell me where she wanna go / I'm living in a nightmare / She's looking like a wet dream / I got myself a Cadillac / But I can't afford the gasoline.

Bon loved writing about Cadillacs. He mentions them in “Rocker” off the Australian version of TNT and the international release of Dirty Deeds Done Dirt Cheap: Got slicked black hair / Skin tight jeans / Cadillac car and a teenage dream.

Then you get to “Back in Black”. Bon’s dead but the lyrics have this spooky connection to “Down Payment Blues”: Back in the back / Of a Cadillac / Number one with a bullet, I’m a power pack.

Why was Brian singing about riding around in Cadillacs? He’d just joined AC/DC, wasn’t earning a lot and was on his best behavior. Bon had a reason to be singing about money. He was writing all the songs and just had a breakthrough album with Highway to Hell. Which begs the question: Could Bon also have written or part written the lyrics to “Back in Black”?

Bon’s late mother Isa said in 2006: “The last time we saw him was Christmas ’79, two months before he died. [Bon] told me he was working on the Back in Black album and that that was going to be it; that he was going to be a millionaire.”

7. “You Shook Me All Night Long”
(Back in Black, 1980)

Everyone knows and loves this song; it’s played everywhere. Shania Twain and Celine Dion have covered it. It’s one of AC/DC’s standbys. But who wrote it?

Former Mötley Crüe manager Doug Thaler is convinced Bon Scott, who’d passed away before the album was recorded, being replaced by Brian Johnson, wrote the lyrics. In fact he told me, “You can bet your life that Bon Scott wrote the lyrics to ‘You Shook Me All Night Long’.” That’s a pretty strong statement from a guy who used to be AC/DC’s American booking agent and knew the band intimately. I look into this claim in some depth in the book and draw my own conclusions.

I’m convinced Bon wrote it. In my opinion only Bon would have written a line like “She told me to come but I was already there.” Brian never matched the verve or wit of Bon in his lyrics and it’s why I think so much of AC/DC’s mid-'80s output suffers even when the guitar work of the Youngs was as good as it ever was.

But what’s also really interesting about this song in light of the recent hullabaloo over Taurus and Led Zeppelin is how much the opening guitar riff sounds similar to Head East’s “Never Been Any Reason”. I didn’t know a hell of a lot about Head East before I started working on this book, but came across “Never Been Any Reason” in the process of doing my research and was blown away when I heard it for the first time. AC/DC opened for Head East in Milwaukee in 1977. So the two bands crossed paths.

6. “Rock ’N’ Roll Damnation”
(Powerage, 1978)

It’s hard to get my head around the fact Mick Wall, the British rock writer and author of AC/DC: Hell Ain’t a Bad Place to Be, called this “a two-bit piece of head-bopping guff.” Not sure what track he was listening to when he wrote that -- maybe he was having a bad day -- but for me it’s one of the last of AC/DC’s classic boogie tracks and probably the best.

Mark Evans loves it almost as much as he loves “Highway to Hell". It has everything you want in an AC/DC song plus shakers, tambourines and handclaps, a real Motown touch that George Young and Harry Vanda brought to bear on the recording. They did something similar with the John Paul Young hit “Love Is in the Air”. Percussion was an underlying feature of many early AC/DC songs. This one really grooves. I never get tired of hearing it.

“Rock ’n’ Roll Damnation” was AC/DC’s first hit in the UK charts and a lot of the credit has to go to Michael Klenfner, best known as the fat guy with the moustache who stops Jake and Elwood backstage in the final reel of The Blues Brothers and offers them a recording contract. He was senior vice-president at Atlantic at the time, and insisted the band go back and record a radio-worthy single after they delivered the first cut of Powerage to New York.

Michael was a real champion of AC/DC behind the scenes at Atlantic, and never got the recognition he was due while he was still alive (he passed away in 2009). He ended up having a falling out with Atlantic president Jerry Greenberg over the choice of producer for Highway to Hell and got fired. But it was Klenfner who arguably did more for the band than anyone else while they were at Atlantic. His story deserves to be known by the fans.

Next Page

Pauline Black may be called the Queen of Ska by some, but she insists she's not the only one, as Two-Tone legends the Selecter celebrate another stellar album in a career full of them.

Being commonly hailed as the "Queen" of a genre of music is no mean feat, but for Pauline Black, singer/songwriter of Two-Tone legends the Selecter and universally recognised "Queen of Ska", it is something she seems to take in her stride. "People can call you whatever they like," she tells PopMatters, "so I suppose it's better that they call you something really good!"

Keep reading... Show less

Morrison's prose is so engaging and welcoming that it's easy to miss the irreconcilable ambiguities that are set forth in her prose as ineluctable convictions.

It's a common enough gambit in science fiction. Humans come across a race of aliens that appear to be entirely alike and yet one group of said aliens subordinates the other, visiting violence upon their persons, denigrating them openly and without social or legal consequence, humiliating them at every turn. The humans inquire why certain of the aliens are subjected to such degradation when there are no discernible differences among the entire race of aliens, at least from the human point of view. The aliens then explain that the subordinated group all share some minor trait (say the left nostril is oh-so-slightly larger than the right while the "superior" group all have slightly enlarged right nostrils)—something thatm from the human vantage pointm is utterly ridiculous. This minor difference not only explains but, for the alien understanding, justifies the inequitable treatment, even the enslavement of the subordinate group. And there you have the quandary of Otherness in a nutshell.

Keep reading... Show less

Acid house legends 808 State bring a psychedelic vibe to Berlin producer NHOAH's stunning track "Abstellgleis".

Berlin producer NHOAH's "Abstellgleis" is a lean and slinky song from his album West-Berlin in which he reduced his working instruments down to a modular synthesizer system with a few controllers and a computer. "Abstellgleis" works primarily with circular patterns that establish a trancey mood and gently grow and expand as the piece proceeds. It creates a great deal of movement and energy.

Keep reading... Show less

Beechwood offers up a breezy slice of sweet pop in "Heroin Honey" from the upcoming album Songs From the Land of Nod.

At just under two minutes, Beechwood's "Heroin Honey" is a breezy slice of sweet pop that recalls the best moments of the Zombies and Beach Boys, adding elements of garage and light tinges of the psychedelic. The song is one of 10 (11 if you count a bonus CD cut) tracks on the group's upcoming album Songs From the Land of Nod out 26 January via Alive Natural Sound Records.

Keep reading... Show less
Pop Ten
Mixed Media
PM Picks

© 1999-2017 All rights reserved.
Popmatters is wholly independently owned and operated.