Big Data Is Your Future, and 'Big Data' Is Your Field Guide

Big data will change many things, but the road will not be linear, nor the miracles without misuse. Viktor Mayer-Schönberger and Kenneth Cukier define, demystify and bring big data into perspective.

Big Data: A Revolution That Will Transform How We Live, Work, and Think

Publisher: Houghton Mifflin Harcourt
Length: 242 pages
Author: Viktor Mayer-Schönberger, Kenneth Niel Cukier
Price: $27.00
Format: Hardcover
Publication date: 2013-03

If you work anywhere remotely close to an information technology supplier or in an information technology business unit, you have heard the term "big data", even if you don’t know what it is or what it means. With Big Data: A Revolution that will Transform How We Live, Work and Think Viktor Mayer-Schönberger and Kenneth Cukier attempt to answer both questions, and raise even more.

As one of the first big books on big data, Big Data could be THAT BOOK. You know, the one your manager hands out at a meeting and asks everyone to read. Houghton Mifflin Harcourt certainty hopes that will happenm as they gave their marketing team permission to create a subtitle that needs neon to do it true justice. Mayer-Schönberger and Kenneth Cukier produce a text that is much more contemplative and humble that the subtitle suggests.

Mayer-Schönberger and Cukier attempt to balance the hype with risk and reality. Don’t get me wrong, this book is not skeptical of big data’s transformative power, but unlike some hype books written as self-serving marketing tools for an idea about to explode, this book recognizes the long road ahead and the bumps along the way.

Before I go further, I should say I’m a big data skeptic, not as an idea, but in its implementation. I wrote a post for Fast Company titled “Why Big Data Won’t Make You Smart Rich or Pretty” in response to Dirk Helbing of the Swiss Federal Institute of Technology in Zurich’s purposed €1-billion project, the topic of the December 2011 Scientific American cover story. Helbing seeks to do nothing less than foretell the future. Mayer-Schönberger and Cukier don’t mention Helbing, which places them a notch closer to reality even at first glance.

For those of you not familiar with big data, think about everything Amazon knows about what you have purchased, or Google knows about you from your searches, your Google+ and YouTube interactions, along your connections to other Google properties. Anything and everything you do, or that takes place in the digital world requires a digital record. A few thousands of those records is a database, several million or billion starts to be big data. Consider Twitter alone, which produces 12 terabytes of tweets each day. That is really big data.

Big Data starts with an ample historical context in chapters called “Now” and “More”. "More" concludes with a major change in the use of data: a movement from statistical sampling to being able to digest and analyze everything that has been captured about a particular topic. That observation leads into chapter titled “Messy” that begins by stating, “Using all available data is feasible in an increasing number of contexts. But it comes at a cost. Increasing the volume opens the door to inexactitude.”

To the authors' credit, a healthy skepticism runs throughout the book. They point out the profound implications of running science through the big data engine in the cloud not just with random samples of people not related to you, but with all data available about you and everyone else. Amazon does not need to guess what the American public is buying or is likely to buy, it knows exactly what people have purchased and for the majority of products, it knows exactly what people will likely buy tomorrow. Amazon's practice includes social science, not just marketing.

The revolution takes place across retail and hard science. Perhaps one of Mayer-Schönberger and Cukier’s more dramatic assertions comes from the elimination of the hypothesis in science. No longer do we need to ask hypothetical questions, says Wired Magazine’s Chris Anderson. If the data fits our suppositions, researchers and clinicians can simply ask the data, in near anthropomorphic fashion, what it knows and the answer will be revealed. The thoughtful pair of Big Data author’s call such an end to scientific process and theory “preposterous”.

Another point for rational thinking. They go on to point out that big data itself is a set of theories and that each attempt to interpret the vast sea of data requires conscious, human decisions about what data to use and how to ask the question. In the chapter “Dataification”, they tackle humanity’s propensity to render more-and-more of what we do, see, hear, say and otherwise experience into some digital representation. We will have every more data, about more-and-more things, so that we can eventually ask computers about almost anything.

Of course, asking data for an answer involves serious programming: select relevant data, normalizes it so it can be processed by the algorithm and finally, produce results that a human or machine can act upon. In the chapter called “Correlation” the authors tell the story of the University of Ontario Institute of Technology and IBM teaming up to analyze data coming from premature babies. As they collect 1,260 data points a second, the system can detect the onset of an infection a full 24 hours before the baby presents symptoms. Data from multiple instruments, collected, correlated and acted upon.

Mayer-Schönberger and Cukier, as authors do with this sort of book produced at the peak of a technology's hype cycle, discuss the value of big data, attempting to make the business case that more big data is better than less big data because the more we can process the more we can understand the world. They frankly say that data isn't really worth much by itself, it is the option value, how the data might be used in the future, repurposed and reapplied to solve problems or creates insight that provides the value, not the space it occupies on a hard disk.

If you haven’t thought of George Orwell or 1984, let me put him into a big data context. At the beginning of “Risks” Mayer-Schönberger and Cukier remind us that in 2007 the British Media reported that 30 surveillance cameras were deployed within 200 yards of the London apartment where Orwell wrote 1984. Big brother was indeed watching, Orwell had left the building long ago, however. If you are worried about big data, despite its miraculous correlations, you should be, and the authors are right there with you.

The “Risks” chapter focuses on privacy and free will. In it the authors ask the very disturbing question (to paraphrase): Does predicting what may happen actually increase the likelihood it will happen when not knowing might lead to a different outcome? Unfortunately, in today’s world, it’s hard to know, because most very public things we do, from checking a book out of a public library to attending a concert to buying ingredients for dinner, sit under the scrutiny of an algorithm designed to help us fulfill our inclination to do what we are thinking about doing. And in the obverse to that, if we think someone is highly likely to commit a crime, shouldn’t we prevent them, even punish them, ala Minority Report, before they commit that crime?

The fact that a trade technology book digs into such deep morale territory is a credit to the authors and the publisher. I don’t have the data, because there is no data about the future, but I wonder if people reading this book at Amazon, at IBM, at Accenture, will head the warnings and cautions and yellow flags thrown out by Mayer-Schönberger and Cukier or if they will blissfully analyze the detail out of the world oblivious to all but the question at hand?

Perhaps the most common sense attack against big data and the value of its outcomes can be derived from every person who has experienced an inaccurate credit report, missing inventory in a manufacturing plant, a lost delivery or some other glitch in his or her life at the hands of inaccurate data. Because we can cognitively describe a better approach to racial profiling, as the author do, it does not mean that we have accurate data in an operational system available to inform us that the person with the Arabic name is not the terrorist being tracked, but a third-generation school teacher from Cincinnati who coaches his son’s soccer team and contributes to NPR.

Just this week (early April 2013), as I write these words, Atlanta educators continue to surrender themselves to authorities for changing test answers to improve scores in order to meet test score standards. Instead of “test score” read “data.” These educators were changing data to affect other data that was being used to measure their performance and to determine program funding. The Atlanta educators had very clear guidance on what good was and very obvious instruments for affecting the goodness of the data. If, as Mayer-Schönberger and Cukier suggest, big data drives behavior as much as it anticipates it, should we not ask: if we measure everything, then how will we know what really matters?

At the most abstract level, algorithms are data, so we must also ask who or what is watching the algorithm to see if is behaving badly or not. The cover of Wired 17.03 reads: The Secret Formula that Destroyed Wall Street, a cover story that discusses the risk analysis algorithm used by Wall Street whose keepers didn’t understand when it’s underlying assumptions could no longer be assumed. The authors remind the reader often of the vigilance necessary to obtain positive value from data.

Big Data presents a well-rounded discussion of big data, including its risks and implications. Despite its hyperbolic subtitle, it does so with reason and reflection. What it doesn’t do is address the risks in a way that businesses and policy makers can remediate them. Most of the recommendations in the “Control” chapter read as Orwellian prescriptions where beneficent overseers make sure the algorithmists don’t do anything too wrong. I'm not sure that translates into actionable data.

Finally in the “Next” chapter, the authors explore the future and make, what I think is one of their more disturbing industrial age observations: Because correlations can be found far faster and cheaper than causation, they’re often preferable… for many everyday needs, knowing what not why is good enough.” This implies that because big data machinery pumps out a low-cost product we should be satisfied with it, and should, except in highly sensitive cases, like airline parts, not care why something is, just accept that the what can be applied in a way that results in a positive outcome. That too is a risk—a risk that we become so reliant on the quick and the ready that we cease to explore the underlying principles that govern our existence.

My guess is that Big Data will indeed become THAT BOOK because its wide-ranging examples, strong story telling moments and exhausting references (accounting for 27 pages of the books 242 pages) steep it with credibility and relevance. As reviewer Sally Adee pointed out in New Scientist, the authors seem to dialog in the book. We find likely caution from Mayer-Schönberger, author of Delete: The virtue of forgetting in the digital age and likely enthusiasm from Economist data editor Cukier. The tension between the authors ultimately delivers a kind of intellectual Fueng Shui for the purveyors big data.

People visit PopMatters for cultural insight. Services from Google to Alexa know much about traffic running through our site, and about those who traffic in popular culture. You are consuming data as you read this, and you left a series of digital bread crumbs on your way here that can be used to reconstruct and interpret your motivations and your inclinations. When I’m the you, that doesn’t bother me much, because for the most part, organizations, from big companies to governments, are generally well-meaning and very often borderline dysfunctional. In other words, they don’t intend to harm us, and if they did, they would probably botch it up in some way. I find that comforting.

If Google continuously improves the accuracy of the ads it displays on sites that run Google ads, however, that should invoke a little fear. If healthcare practitioners can improve the odds of surviving a hospital visit and do it at a lower cost, so much the better for society. We should worry less about the state of an incremental erosion privacy and convenience, and more about a Black Swan event in which a power rises intent on employing big data in a deliberately malevolent way.

Mayer-Schönberger and Cukier say little about the analog world. The analog world of politics and relationships taht complements, tempers and creates context for the digitization of everything. The analog world is the reason that digitization matters at all. Through big data we attempt to build better models of that world so that we can understand it, perhaps make it more meaningful. The analog sources the questions. If maliciousness arises, will we see it coming as we ask our computers not where evil hides, but who will win this season of American Idol?

The authors have the final word: “Big data is a resource and a tool. It is meant to inform, rather than explain; it points us toward understanding, but it can still lead to misunderstanding, depending on how well or poorly it is wielded. And however dazzling we find the power of big data to be, we much never let its seductive glimmer blind us to its inherent imperfections.”


Charlie Brown, Snoopy, and Woodstock each did their stint as a lonely Mexican cowboy, it seems. These and other things you didn't know about A Charlie Brown Christmas.

How Would You Like to Be the Director of Our Christmas Play?

It's really a beautiful little movie and has affected my life in numerous ways. For years, especially when we were poor, we always tried to find the littlest saddest Christmas tree possible. In fact, my son Eli has a Christmas tree set up right now that is just one single branch propped up in a juice bottle. And just a couple weeks ago we were at a wedding, everyone was dancing, and me and my wife Amy and my friend Garth started dancing like the Peanuts characters do in the Christmas special.

-- Comic artist James Kochalka.

Bill Melendez answers questions with the sort of vigor that men a third his age invest thousands in herbal supplements to achieve. He punctuates his speech with belly chuckles and comic strip taglines like "Oh, boy!" and "I tell 'ya!" With the reckless abandon that Melendez tosses out words like pleasure, it's clear that 41 years after its premiere, A Charlie Brown Christmas remains one of his favorite topics of conversation. "It changed my life," he states simply, "being involved with this silly little project."

Melendez celebrated his 90th birthday in November. "When I think of my last 40 or 50 years, I can't believe it," he says, capping off his comment with that inevitable one-man laugh track. The curly-mustachioed animator was born José Cuauhtemoc Melendez in Hermosillo, Mexico, in 1916. "I was literally a cowboy," he says. "From there, I crossed the border and started growing up. Just recently I went back, and when I got there I realized where my home was: across the border. When I was a little kid, I would have killed myself had I known such a thing was going to happen. I'm one of you. Whether you like it or not, I'm one of you."

Melendez recalls his blind leap into the world of animation as though the story's end still managed to catch him by surprise. "I was working in a lumberyard, and one of my mates said, 'Hey, I read in the paper that some guy up on Hyperion Avenue is hiring young guys like you who can draw.' So I went to this stranger and said, 'Hey, I understand someone here is hiring young artists.' The man asked me for my samples and I said I'd show them to him tomorrow. I went home that night and made the samples. I brought them in the next day, and he asked me what art school I went to. I'd never been to an art school. He said, 'Well, you have talent,' and he hired me to work in a place called Walt Disney."

Four years later, after lending his hand to Disney canon fodder like Bambi and Fantasia, and after fighting for his new country in World War II, he spent the next decade or so hunched over the drawing board, producing animated commercials and industrials by the thousands, including a number of spots featuring syndicated comic strip characters. Among them were the Peanuts characters.

Of all the Charlie Browns in the World, You're the Charlie Browniest.

I was around 10 when it first premiered, and seeing A Charlie Brown Christmas for the first time was enough to prove even to a young child that a well-written thing is superior to most of what is out there. I'll probably be watching it again tonight or tomorrow, because I have a copy of it and a six-year-old daughter. She loves it.

-- Comic artist Gilbert Hernandez.

By 1959 Charles "Sparky" Schulz's Peanuts kids found themselves at the center of their first print advertising campaign, pushing Ford's new Falcon make of cars. As the story goes, the idea of using Schulz's characters came from a daughter of one of Ford's advertising people.

"I think Sparky was flattered when they wanted to use his characters, says Schulz's widow, Jean, who now is one of the driving forces behind the Charles Schulz Museum in Santa Rosa, California. "It was a new way of extending his creativity. From the get-go Sparky always said a comic strip is a commercial venture. Newspapers put the comic strip in to sell newspapers. He would then bitterly say, 'No one considered comic strips art in the first place, so why would you get on your high horse about that?'"

When the time came for the characters to make their animated debut in a Ford commercial, Melendez was brought into the fold, and he brought along a cast of unknown child actors to voice the parts. The team reunited five years later, when Lee Mendelson, a filmmaker from San Francisco, requested two minutes worth of animation for a film he was shooting based on a Peanuts story line.

"I had done a Willie Mays documentary in 1963, A Man Named Mays, which had done really well," Mendelson says. "Then I was reading a Charlie Brown baseball strip, and the idea came to me: I've just done the world's greatest baseball player; now I'll do the world's worst." It's an old joke -- the same he used to open his 2000 coffee-table retrospective, A Charlie Brown Christmas: The Making of a Tradition -- but it's one for the ages. "Two years later Coca-Cola called, and I thought they were calling to do the documentary," Mendelson explains, "but they said, 'Have you guys ever thought about doing A Charlie Brown Christmas?' I said absolutely. And that's how I got in the animation business." His executive producer role on that film was the birth of a career now well into its fourth decade.

"So Lee called Sparky and said, 'Well, I just sold our Christmas show," explains Jean. "Sparky asked, 'What Christmas show?' and Lee said, 'The one we're going to write tomorrow." Sparky said, 'If we're going to do it, we need to have Bill.'"

Melendez was brought into direct, and as with the Ford commercial, he gave the parts of the Peanuts kids entirely to children, many of whom had never acted. Getting them to learn their roles was a trying task, given that Schulz's script had his characters regularly waxing philosophical and tossing off words like ailurophobia (a fear of felines, for the record). Melendez had to teach the young actors long portions of the script phonetically. "Sometimes they didn't understand a word," he remembers. "They'd say, 'Just tell me how you want it said.' Then they'd say it, and I'd turn to the engineer and ask if he recorded it. The kids were all startled when they got screen credit and happily startled when they started getting royalty checks."

Melendez's also tried to coach a voice actor for the part of Snoopy, whose lines were limited to a handful of non-words. "I recited Snoopy's lines for the actor, and the actor turned to the engineer and said, 'Did you record that? Just use what Bill has done. I don't want to repeat your words.' " This happy accident left Melendez playing the role of Snoopy and, later, his yellow bird companion Woodstock for the next 40 years.

For the film's soundtrack, Mendelson and Melendez embraced Schulz's love of jazz. "Driving back from Sparky's over the Golden Gate Bridge I heard a song called 'Cast Your Fate to the Wind,'" Mendelson writes in The Making of a Tradition. The song was written by Vince Guaraldi, a jazz pianist from the beatnik-dense San Francisco neighborhood of North Beach. It had won the musician a Grammy Award for best original jazz composition in 1962. Guaraldi enjoyed Schulz's script and happily accepted his invitation into the Charlie Brown Christmas fold.

This Doesn't Seem to Fit the Modern Spirit.

The one thing that has always bothered me about the Charlie Brown Christmas special is that the other kids never admit to Charlie Brown that he was right about the little tree. They ultimately accept the tree, but no one ever says, 'Well, Charlie Brown, I guess you were right all along. We were idiots.' However, it's still cool to see a mainstream children's program show that is so stridently nonsecular, which could never be done in this day and age. Linus gets some good face time with all that shepherd talk.

-- Pop culture critic Chuck Klosterman.

Beyond the inclusion of Schulz's cast of wildly popular characters, 1965's A Charlie Brown Christmas seemed a production earmarked for failure. The special's small crew was given a mere six months between the film's conception and its maiden broadcast. At his own insistence, Schulz signed up to pen the script, his first attempt at a screenplay. "He said that if he was going to get screen credit for something, he wanted to be doing something," says Melendez. "He was very proud and curious and didn't want credit where he didn't deserve it."

Despite the Ford commercials that gave birth to the collaboration, and Coca-Cola's strong sponsorship presence in the special, Schulz's script centered around a pensive Charlie Brown attempting to find the true meaning of Christmas. "The 1960s were when Christmas first began to start the day after Thanksgiving," says Mendelson. "There was an irony to this, given the commercialization of the comics. That wasn't really his doing. He said, 'If people want to buy stuff, that's up to them. I'm not in the business of making stuff and selling it. I'm in the business of making a comic strip, and if people want products, then so be it.' "

"We're all a little schizophrenic in that way," adds Jean Schulz. "You live in this world, and you despair. If you think at all, you're always wrestling with this. I think that's exactly what Sparky was expressing." The special opens with a characteristically distraught Charlie Brown, speaking to the perpetually blanket-wielding Linus on a snow-covered version of the brick wall, the bald third-grader's preferred location for vocalizing his ever-present inner despair. "I think there must be something wrong with me, Linus," he begins. "Christmas is coming, but I don't feel happy."

In case that wasn't enough to threatren the film's commercial potential, the producers added one final nail to the prime time coffin: Schulz's script called for Linus to deliver a subdued monologue at the film's climax, a word-for-word recitation of Jesus's birth, taken from the Gospel of Luke. "Bill said, 'You can't have the Bible on television!' Sparky said, 'If we don't do it, who will?' By the time that Coca-Cola and CBS saw it, they had no choice but to play it. They had nothing else to put in there."

What the roomful of executives saw upon the first screening was a shock -- a slow and quiet semireligious, jazz-filled 25 minutes, voiced by a cast of inexperienced children, and, perhaps most unforgivably, without a laugh track. "They said, 'We'll play it once and that will be all. Good try,' " remembers Mendelson. "Bill and I thought we had ruined Charlie Brown forever when it was done. We kind of agreed with the network. One of the animators stood up in the back of the room -- he had had a couple of drinks -- and he said, 'It's going to run for a hundred years,' and then fell down. We all thought he was crazy, but he was more right than we were."

I Never Thought It Was Such a Bad Little Tree

That show is probably the closest I've ever come to having any interest in religion. That part where Linus quotes from the bible is extremely touching and very deftly handled. I was raised in a nonreligious household, and that was a moment that actually had some religious significance to it just because Schulz expressed it so well.

-- Comic artist Seth.

Upon its airing, the special received a 50 share. The network immediately ordered four more films from the team. "We watch it every year to make sure that it actually happened. We thought it would be on one time and be gone," Mendelson says. "The message is simple. Schulz wanted to do a show on the true meaning of Christmas. Any good writer like Schulz deals in truisms and things that are timeless. There are themes about unrequited love and bullies. They work as well now as they did in the 1960s, and they'll probably work for another 50 or 100 years as well."

"I think it touches something in the viewer. We didn't do it on purpose, but there's something ethnic about it," Melendez adds. Schulz expressed his own surprise that the film found its way into the canon of holiday classics. "He would say things like, 'I never thought it would be around 25 years later,'" Jean remembers. "One of the reasons that Christmas is so great is that back in 1965 there were no VCRs or DVDs, so you saw that show once, and you had to wait a whole year to see it again. And when it came on, it still held up. It was still charming."

Forty-one years after its premiere, A Charlie Brown Christmas remains a towering if unassuming presence in holiday TV. It's an oasis of sincerity, managing never to be drowned out by its overzealous neighbors' rush to cross-promote themselves. It's a quiet testament to what children's programming could be: introspective, unpretentious and, above all, respectful of the intelligence of its target audience. "Children's programs were held in low regard by everybody -- including me," says Melendez. "But I realized that it wasn't just for kids. I was dealing with adults. They were giving me suggestions and criticism."

For a film with an anticommercial message, A Charlie Brown Christmas produced its own market bonanza. But it still suggests the spirit of its writer, who sensed the real magic of Christmas was not in the spectacle of lights, commerce and big aluminum Christmas trees, but in those fleeting moments of silence, which seem to become rarer with each passing day. "They weren't afraid to have quiet," Jean says. "Most of the time when the kids are walking, it's very quiet. We came out of a new animated movie one day, and Sparky said, 'I missed the quiet places.'"

Over the years, the Schulz-Mendelson-Melendez team created more than 75 half-hour television specials and four feature films, and five Peanuts films have been made since Schulz's death, in 2000, at the age of 77. Outside of the films, Peanuts continues to be an incredibly lucrative license for its owners, United Features Syndicate. "If Sparky had the volume of stuff crossing through the office that we have today, it would have driven him nuts," laughs Jean. "He probably would have walked through the office and said, 'We're cutting all of the licensing off. I don't want to do it anymore.'"

From genre-busting electronic music to new highs in the ever-evolving R&B scene, from hip-hop and Americana to rock and pop, 2017's music scenes bestowed an embarrassment of riches upon us.

60. White Hills - Stop Mute Defeat (Thrill Jockey)

White Hills epic '80s callback Stop Mute Defeat is a determined march against encroaching imperial darkness; their eyes boring into the shadows for danger but they're aware that blinding lights can kill and distort truth. From "Overlord's" dark stomp casting nets for totalitarian warnings to "Attack Mode", which roars in with the tribal certainty that we can survive the madness if we keep our wits, the record is a true and timely win for Dave W. and Ego Sensation. Martin Bisi and the poster band's mysterious but relevant cool make a great team and deliver one of their least psych yet most mind destroying records to date. Much like the first time you heard Joy Division or early Pigface, for example, you'll experience being startled at first before becoming addicted to the band's unique microcosm of dystopia that is simultaneously corrupting and seducing your ears. - Morgan Y. Evans

Keep reading... Show less

The Best Country Music of 2017

still from Midland "Drinkin' Problem" video

There are many fine country musicians making music that is relevant and affecting in these troubled times. Here are ten of our favorites.

Year to year, country music as a genre sometimes seems to roll on without paying that much attention to what's going on in the world (with the exception of bro-country singers trying to adopt the latest hip-hop slang). That can feel like a problem in a year when 58 people are killed and 546 are injured by gun violence at a country-music concert – a public-relations issue for a genre that sees many of its stars outright celebrating the NRA. Then again, these days mainstream country stars don't seem to do all that well when they try to pivot quickly to comment on current events – take Keith Urban's muddled-at-best 2017 single "Female", as but one easy example.

Keep reading... Show less

It's ironic that by injecting a shot of cynicism into this glorified soap opera, Johnson provides the most satisfying explanation yet for the significance of The Force.

Despite J.J. Abrams successfully resuscitating the Star Wars franchise with 2015's Star Wars: The Force Awakens, many fans were still left yearning for something new. It was comforting to see old familiar faces from a galaxy far, far away, but casual fans were unlikely to tolerate another greatest hits collection from a franchise already plagued by compositional overlap (to put it kindly).

Keep reading... Show less

Yeah Yeah Yeahs played a few US shows to support the expanded reissue of their debut Fever to Tell.

Although they played a gig last year for an after-party for a Mick Rock doc, the Yeah Yeah Yeahs hadn't played a proper NYC show in four years before their Kings Theatre gig on November 7th, 2017. It was the last of only a handful of gigs, and the only one on the East coast.

Keep reading... Show less
Pop Ten
Mixed Media
PM Picks

© 1999-2017 Popmatters.com. All rights reserved.
Popmatters is wholly independently owned and operated.