Scenario: Manipulations

Christina Class & Debora Weber-Wulff

AlgoConsult is a company that develops highly specialized computer processes for a wide range of applications. For marketing purposes, customers call these processes “algorithms,” and increasingly, this is the term used in-house, even though it’s not what they are.

With their „CompanyRate“ project, the firm is generating a rating system for investors working in the German banking sector. The ratings index is intended to facilitate future investment decisions. A pilot version was presented to a few selected beta customers last month, and the initial feedback has been glowing.

Various machine-learning approaches are combined to generate the ratings. The number of influencing factors is enormous: in addition to stock market prices and current market information, data on known advertising budgets, media presence, trade fair participation, market shares, etc., are used. AlgoConsult keeps its algorithms and influencing factors strictly confidential, especially to prevent manipulation of the „CompanyRate“ index: it’s a feature AlgoConsult touts on its project website.

Therefore, all project participants are carefully chosen from employees who’ve been with AlgoConsult for at least a year and who must sign project-specific NDAs. They are prohibited from investing privately in indexed companies or in funds in which these companies have a significant share in their portfolios. In exchange, the software analysts are paid handsomely.

Achim is a proud „CompanyRate“ core team member responsible for the index. One day, as he’s leaving for lunch, he stops by to pick up his colleague Martin. They always have lunch together at the Japanese or Mexican restaurant on the first floor of the high-rise building, but Martin says he’s brought something from home today. At the elevator, which can only be opened with a pass card, Achim realizes that he left his wallet in his raincoat, so he returns to the office.

When he reaches the office door, he can hear Martin—who is usually very quiet—speaking loudly on the phone. Since he’s alone in the hallway, Achim eavesdrops furtively on the conversation. He thinks he can make out that Martin is on the phone to someone at PFC, “People’s Fruit Company.” Despite its American name, PFC is a German company listed in the index. Achim clears his throat and enters the office. Martin quickly drops the call.

“My mother, she has to call me every day about something,” Martin chuckles nervously. Achim grabs his wallet and meets his colleagues from another department for lunch as usual, but he has trouble concentrating on the conversation.

Even though the project is at a lull because the index is being tested with beta customers, Martin is unusually focused and busy all afternoon. He doesn’t even seem to have time to break for coffee. He is still working when Achim drives home.

The following day, Achim sees that Martin entered a bunch of code the night before, and it successfully ran through the test suite overnight.

“Geez, Martin’s been busy,“ thinks Achim while looking at the logs. The boss will be happy about all sorts of documentation in various corners of the system. Achim decides to display a few of the program changes—all of which contain comments, as usual. But he wants to be sure the changes made to the documentation were, in fact, correct.

The third change throws Achim for a loop. The formulas themselves have also been changed. Now, bizarrely, a value is read from a file from the cloud instead of being calculated inside the program as before. On closer inspection, he realizes that the cloud value is only applied in cases that are an almost exact match with PFC.

Now, Achim isn’t sure what he should do. Just last week, Anne was terminated without notice, even though she is an excellent programmer. In the past few weeks, she’d seemed somewhat out of sorts and had scheduled a meeting with her boss. Afterward, security immediately escorted her to her desk to collect her belongings and showed her to the door. That was pretty shocking for the team. But to avoid being dragged into anything, everyone acted as if nothing had happened.

That evening, Achim had tried to contact Anne several times, but she’d rebuffed him. That weekend, he stopped by her place and waited outside the apartment until she left to go shopping. She snapped at him when she saw him: “Please just leave. I can’t talk!”

Adam is shaken. He’s known Anne since college. She is extremely talented and honest. He can’t imagine she’d violate the terms of the NDA or do anything that could harm AlgoConsult. Was it possible that someone else had manipulated the rating system, and she reported it? Should he take the risk of telling his boss about his observations? Was it worth putting his top-notch salary on the line? As he climbs into his car, there’s a re-broadcast of the local radio station’s economic program airing. The press secretary for an investor group is being interviewed, and he says they are developing new AI-based software designed to help small investors find good investment opportunities and help them make investment decisions.

What should Achim do?

Questions:

1. Is there any reason to justify Achim’s eavesdropping on Martin’s conversation?

2. How can Adam be sure that Martin was talking on the phone to PFC? Does that even matter in this scenario?

3. Is it okay for Achim to use changes to the program to examine Martin’s work more closely? How far should this kind of “double-checking” go?

4. What are we to make of the fact that Martin generated so many changes and additions to the internal program documentation to “conceal” the change he made to the calculation? Is it possible that Martin used the copy-paste function for comments he made to the changes? How reliable are these kinds of comments? How important is it to document programming changes precisely?

5. Is it safe for Achim to assume this change was made deliberately to give PFC an advantage?

6. Is it possible that the boss knows about Marin’s changes or that he instructed him to make them?

7. Should it matter to Achim that Anne has been fired and no longer wants to talk about it?

8. As a general rule of thumb, is it advisable to work on software systems subject to non-disclosure agreements?

9. Rating and matching algorithms can constitute a company’s core business and central asset. These algorithms and influencing factors are, therefore, often kept secret. This is understandable from an economic point of view, but how dependent do users become on the algorithms? What opportunities for manipulation arise? What dangers can arise not only for the individual user but also for economic systems or societies?

Published in Informatik-Spektrum, 39 (3) 2016, pp. 247–248

— Translated from German by Lillian M. Banks