Scenario: What is true? Data, Graphics and Truths

Christina B. Class, Andreas Hütig & Elske M. Schönhals

Andrea, Alex, and Sascha come from the same small town and have been close friends since elementary school. After high school, they moved to far-flung parts of the country. All the more reason for them to relish their annual get-togethers on December 22, when they would sit around talking for hours on end. For the past few years, nothing ever stood in the way of their annual get-together: neither their various study abroad trips, nor their jobs, nor family. December 22 was reserved for old friends and Christmas Eve for parents: they wouldn’t miss it for a thing, not even the Corona virus. They’re seated at a safe distance from one another, having a beer in Sascha’s parents’ living room: Sascha is at the dining room table near the kitchen, Alex is on the sofa across the room, and Andrea has made herself comfortable in the armchair beside the fireplace.

Andrea has just started showing the others her latest project. She and a fellow student, Maren, are working on a data visualization app. They’ve gone to great lengths to develop as their unique selling point (USP) a user interface that will appeal to users who prefer not to deal with data or data visualization. No prior knowledge of programming or statistics is required; every trace of code remains hidden from view. The user selects their preferred filters, and the app allows them to present the data in various visual formats.

The user can get creative with charts, colors, ratios—even 3D graphics—to display data. The way the data is represented is easily changed or adapted to suit user needs. The goal is to allow the user to create graphics quickly and simply so they can either be sent to a computer or shared directly in social media networks using share buttons. After all, the ability to back up specific themes and theses with suitable statistics and infographics is becoming ever more imperative.

Sascha, who works for a consulting firm, tests the app on Andrea’s tablet and is thrilled: “Man-o-man, Andi! Why didn’t you have this thing ready a couple of weeks ago?! We had to put together an interim report for one of our clients and needed to compile all the data from our market analysis to support our strategic recommendations. Man, that was a lot of work! And Tommy, the project director, was impossible to please. The graphics never illustrated what he was trying to communicate quite how he wanted. It was such a pain fiddling with all those options and parameters.”

“Yeah, Sasch’,” Andrea answered with a grin, “we don’t make quite as much as you! Otherwise, we could hire a few more people and knock this stuff out more quickly.” Sascha hands Alex the tablet so he can look at it, too. Alex is as fascinated as Sascha. While Andrea and Sascha discuss the app’s market potential, Alex is thoroughly engrossed in testing its many functions. But the look on his face gradually turns sour. He furrows his brow like they’ve often joked about whenever his thoughts wander off the deep end.

Suddenly, Sascha turns to him and asks: “Hey, what’s up? Is something wrong?” Alex looks up, stares straight at Andrea, and says: “I don’t know. I have a bad feeling about this app, Andi. It runs like a charm and simplifies everything. The graphics look super professional and persuasive. But isn’t it almost too good? I can play around with all these options and snippets long enough to use the same data to create graphics that lead to opposite conclusions. That can’t be good.”

“Why not? That’s precisely the point,” Sascha says. “You have no idea how much effort goes into configuring graphics to illustrate precisely what you want them to. That’s what’s so brilliant about it: the user interface is so streamlined that it no longer requires specific skills to generate the graphics you need. You have to know what the graphics are supposed to show—the app does the rest for you.”

“Yeah, but that means that the graphics might end up showing something that’s not true; it might even mean that the data can be manipulated to extract insights that aren’t necessarily true.”

“Nonsense!” Andrea continues, “We don’t delete or change any of the data. And besides,” she adds, “the whole purpose is to make certain facts stand out. You know what they say: a picture’s worth a thousand words.”

“Yeah, right!” Alex barks back derisively, “Typical consultant! Just what I thought you’d say!”

There’s a minute of dead silence before Andrea asks, “Hey Alex, what’s the deal? What’s that about?! Sasch’ hasn’t done anything wrong.”

Alex takes a deep breath, “Yeah, I know. I’m sorry, Sascha, I didn’t mean it that way. It’s just that I’m genuinely ticked about this. Remember two years ago when Micha and I opened an escape room and an adventure pub? It was going well. Until. Well, you know. Until Covid came along. We tried to stay afloat with online offerings. Even that was getting off to a good start; then some folks managed to steal our ideas… what ya gonna do?…we had to devise a new plan.”

Then we got an offer for a new VR space. It sounded great. We installed a test version and spent three weeks testing for performance and quality using various subscription-free experiences. It was all very promising. So we signed the contract. But these *$@!’s had presented the data in a way that glossed over all the problems. They were either tucked away in corners with tiny graphics or smoothed over on a compensation curve. We didn’t pay much attention to it during the presentation. The system runs like crap, and we’re likely to lose our shirts over it. We’ve already seen an attorney. But since they didn’t falsify the data, they aren’t guilty of fraud. If jerks like this get their hands on an app that can do all this, it’s game over.”

Andrea and Sascha stand there staring at each other in silence.

Finally, Sascha says, “Dude, I’m so sorry to hear that! Unfortunately, bad apples are a dime a dozen—even in our field. But there’s nothing Andrea’s great app can do about that, is there? Ultimately, it’s your job to crunch the numbers and do the math, no matter how good the graphics look. Next time, why don’t you let me look at them before you sign anything?”

Andrea agrees, “Sure, if you feed the right data into our app, you can get it to show you things that are ultimately misleading, or that look differently taken out of context. But anything can be used for nefarious purposes, right? You can’t put that back on our app.“

But Alex wonders, “Aren’t you taking the easy way out here? Remember that not everyone has had as much statistics background in their education. All these numbers and fancy graphics make everything look so much more convincing, yet what they represent is only a fraction of the bigger picture. And ultimately, no one can make sense of it anymore—not even the app users! Where’s the accountability?”

Questions:

1. Sascha thinks a picture is worth a thousand words. At the same time, though, essential details often get lost in translation. Have we all grown accustomed to taking in everything with just one look? Why do we prefer to see graphics and images over numbers and data? Are we still willing to engage with the details behind the numbers?
2. The app promises to simplify data visualization. What practical applications might this have beyond pretty pictures (and marketing campaigns)?
3. Andrea and Maren’s app also allows users to export graphics to social networks, which is precisely where myriad half-truths, fake news, and falsified numbers circulate. Most dangerous are false and/or distorted statements based on accurate but incorrectly interpreted data. Would an infographic app like this tend to accelerate this trend or counteract it? What changes to the app could Andrea and Maren make to help support substantive content instead of simply rendering rote speculation more plausible?
4. In 2014, Lisa Zilinski and Megan Nelson published a “Data Credibility Checklist [1]. What might the minimal prerequisites for using data to construct graphics entail?
5. What criteria must a graphic meet for you to trust it? Where should the data come from? What should be taken into account? What tracking or verification options would you like to have?
6. What are the implications of these checklist items for data graphics creators? Who is responsible for ensuring that graphics are interpreted correctly?
7. On its face, accountability is informed mainly by a sense of agency. Someone is accountable to someone else for something adjudicated by a particular authority according to an agreed-upon norm. But what about this instance, where the programmers cannot know what the users may do with the app they created? Can you be called to account for something you do not know might happen? Or should they be required to at least minimize the likelihood of misuse or make it more difficult? If so, how might Andrea and Maren go about achieving that end?
8. If accountability can no longer be traced to any given “agents,” would one solution be implementing regulation at the system design level? Or are those types of interventions ineffective and fundamentally overreaching?

References:

Börner K, Bueckle A, Ginda M (2019) Data visualization literacy: definitions, conceptual frameworks, exercises, and assessments. Proc Natl Acad Sci USA 116(6):1857–1864. https://doi.org/10.1073/pnas.1807180116.

Zilinski LD, Nelson MS (2014) Thinking critically about data consumption: creating the data credibility checklist. Proc Am Soc Inf Sci Technol 51(1):1–4.

Published in Informatik Spektrum 44 (1), 2021, S. 62–64, doi: https://doi.org/10.1007/s00287-021-01337-z

— Translated from German by Lillian M. Banks