When a machine has the final word

What if we use the World Cup (and its controversies) to talk about this?

Let's go to the VAR

I imagine many of us were watching the World Cup matches. On this side of the world they landed at a decent hour, although they often overlapped with work. Nothing a nearby screen couldn’t solve, because there’s no problem at all in enjoying football while you work (just don’t go having more than one beer while you do it).

And yes, I’ll be calling it football, because this sport is played with our feet. It doesn’t matter what others (cof cof, NFL, cof) have to say about it.

This World Cup had more controversies than I expected, many of them because of the VAR (Video Assistant Referee). All of it got me thinking about how technology can help or ruin something as normal as a football match, and how the same thing happens in our daily lives, sometimes without us noticing.

But let’s start with the World Cup…

The machine that decides everything

Portugal vs. Croatia, round of 32 of the World Cup, minute 103. Croatia is losing 2-1 and is just seconds away from going home. Then a cross comes into the box, the ball travels between five hundred and fifty-three players, there’s a rebound, and Josko Gvardiol throws himself at it to tie the match. The Croatians celebrate like they had just won the cup (and rightly so, it was almost, almost a miracle).

And then the assistant raises the flag and the VAR reviews the play… a few minutes later the referee disallows the goal. It turns out the ball “brushed” another player, which created an offside position. Now, when I say brushed, I don’t mean it touched his foot or his head… it was his hair (WTF)…

You read that right, it was the hair. The ball didn’t change direction, no camera caught it, and there weren’t even any complaints. I would dare to say that no human noticed that touch. Only the sensor inside the ball registered it, and just like that, looking at a sensor’s graph, the referee disallowed the goal and a national team was out of the World Cup because a sensor showed something nobody saw (or because someone decided to grow their hair long).
Bye bye, Croatia.

The ball just brushed his hair… really?

Nine days later, quarterfinals, England against Norway. Haaland’s team is winning 1-0 and we’re in first-half stoppage time. The Norwegian goalkeeper sends the ball long and, mid-flight, it changes direction in a strange way, like it had hit something while moving through the air.

The replay shows the change of trajectory again, and it just so happens that this occurs right where the cables holding the spider-cam are (that camera suspended in the air that gives us those incredible aerial shots)…

Well, it turns out that after that change of direction the ball lands directly at the feet of an English player, and after a couple of good passes, Bellingham makes it 1-1.

Faced with the Norwegians’ complaints, FIFA said that the blessed ball sensor didn’t register anything, so the change of direction was the work of the Holy Spirit.

What can we do? According to the sensor, nothing happened.

Two plays in the same World Cup, with the same ball and the same %&# sensor. In one, the machine saw something no human could see, and that was that. We did what the machine told us. In the other, millions of people saw something, but since the machine didn’t notice it, it never happened.

Could it be that we’re giving too much power to “the machine”?

We’re giving too much power to “the machine”

So far, nobody really knows if the ball brushed the Croatian player’s hair or not. No camera caught it clearly, and even Matanovic himself says he doesn’t know if the touch actually happened. In Norway’s case, there are images showing the ball changing course, but the referee never bothered to look at them because the sensor wasn’t triggered.

“That’s football,” I’ve heard many people say.
But I would rather say: “that’s football now”.

What I don’t like about these cases is that we blindly believe the %$&# sensor.
To put it in other words, if the machine doesn’t register it, then it “never happened”, and if the machine registered something, there’s no room for the slightest doubt or interpretation.

Now, imagine for a moment that this is not a football match. Instead, you’re applying for a mortgage, or a doctor is reviewing your CT scan… Are we removing human judgment from all of these things?

OK, OK, I won't do it again…

I’m not saying that trusting technology is wrong, what’s wrong is trusting technology blindly.

Germán… what does this have to do with AI?

Great question! That thing about trusting a device completely and blindly, accepting what it tells us with no way to appeal, and having technology so deeply woven into the rules of the game doesn’t happen only in football. It’s happening with artificial intelligence.

At what point did you find out how many decisions the VAR handles? If you’re like me, you probably found out while watching the matches (like that goal disallowed for Egypt, where they had to send the video back to the other side of the pitch to review the play).

Well, the same thing that happened to us during the World Cup can happen at your bank, at your job, at your clinic, at your university… and don’t expect them to send you a statement explaining all the data they use and how they make decisions that directly affect you with it.

Who is in charge of verifying that the tools they’re using actually work? And if something fails, who is responsible? And who do you complain to when “the system” says no?

It seems like all of this needs rules, right?

And these rules have a name: governance.

Gover…what?

…nance! Governance.

These are the rules, processes and roles inside an organization that make sure technology is used according to its goals, values and obligations.

And there are four things to keep in mind:

1. Who decides?
   Someone is in charge of approving which systems are used, who owns the data, who approves using one tool or another. They’re also the one who answers if something goes wrong.

2. What are the rules?
   The rules can be internal policies of the organization, industry standards, and even external regulation (like personal data protection laws). These rules must be hyper-clear and defined from the start.

3. How is it controlled?
   You have to continuously check that these rules are being followed and that the tools are working as they should. For this there are audits, data analysis, risk management, and a whole bunch of committees that review any case.

4. How is accountability handled?
   When something fails, there must be transparency and explanations, and above all, a way to complain.

And one more thing: if this isn’t defined, the default governance can end up being believing the machine. How many times have we heard some version of:
“Sorry, that’s what the system says”?

😢

And this is just governance for standard technology (like the blessed chips in the ball). When we talk about AI, we need to add a few things…

When AI steps onto the pitch

Sorry, I couldn’t resist the football metaphor. OK, let’s continue.

Let me start by telling you that generative AI is a completely different technology from what we’re used to. Not even the people who develop this technology can explain 100% why it responds the way it does in a specific case.

Also, it can easily give different answers to the same question (it’s happened to me). And as if that weren’t enough, there are still biases in its training data. Or is someone going to tell me they reviewed those trillions of texts by hand? And let’s not even talk about its hallucinations.

And here’s where the problem starts, because the governance questions become much harder to answer. What rules do we give a tool that we can’t know for certain how it works? The chip in the ball can be audited. We can know if it was really working properly and close the case. With AI, we can’t do the same.

To get started, here are 5 questions we can ask ourselves:

1. Can someone explain this?
   AI can make decisions, and as we’ve seen, there may be cases where we can’t explain 100% the decisions it makes. It’s a good idea to keep a record of how the AI arrived at a decision, and to have at least one human with the authority to review and contradict the system if necessary.

2. Does it work the same for everyone?
   As I was telling you, AI has been trained with millions and millions of texts, and the reality is that we don’t have the capacity to review them one by one. Even though companies do their best, it’s certain that many biases have made their way from the training texts into the model, like that time Amazon trained a recruitment model that penalized you for being a woman.

   So it’s good practice to run tests with different groups of people and repeat the test every so often (don’t go using another AI for these tests, use flesh-and-blood people).

3. Does what worked yesterday still work today?
   You know AI doesn’t always give exactly the same answer or behave in exactly the same way, so there’s no such thing as “I tested it once and it works”. These systems need to be monitored all the time.

4. Do we really know what information we’re giving them (and to whom)?
   What happens with the company’s sensitive data that someone copies and pastes into an assistant to finish a report faster? Now imagine it’s not the assistant the company gave them, but their personal assistant. Companies need an AI usage policy, but something that can be communicated quickly. One page, not a 49-page document that nobody will ever finish reading.

5. If AI does something it shouldn’t… who is responsible?
   This technology is no longer just in the chat. Today we have agents that send emails, reply to messages on Slack, approve requests or execute tasks. The ideal is to define someone responsible for that system (it’s also a good idea to know the risks of using it). That way you’ll know who to blame if “by pure coincidence” your assistant sends a message to your ex in the middle of the night.

Artificial intelligence is neither good nor bad (except for Skynet, Agent Smith, Ultron and about 55 more). The idea is to not use it with your eyes closed.

Sorry, that’s what the system says

Imagine you apply for a loan to buy an apartment. You have a good job (or at least you think you do), you pay all your bills on time and you eat all your vegetables. A few days later the answer arrives: denied.

You go to the bank to ask what happened, and the person helping you smiles kindly, looks at their screen, pauses, looks back at you and says…

Clearly you’d be furious getting this answer, but the reality is that the poor bank employee probably has no idea why your loan was rejected.

😱

Today, more and more AI systems evaluate loans, filter CVs, handle claims, analyze medical diagnoses and who knows how many other things. Now think about what that picture will look like in 5 years.

If we don’t take this seriously, we could end up losing control over many things, so I have a recommendation for you looking at that future:

Learn how this technology works

And that’s it, but before I go…

One last thing

I want to use the end of this post to publicly congratulate a national team that had one of the most remarkable performances I’ve seen in many years. Of course I’m talking about the great team of Cabo Verde. A special shout-out to their goalkeeper, Vozinha, who saved everything from everyone (with or without VAR).

Well done, Vozinha!

I confess that at some point I started looking for Lima-Praia flights, but I got discouraged when I saw that the cheapest ticket cost 7 thousand dollars!!! 😱

Anyway, I just wanted to add that.
What a World Cup we had, I’m already missing it.

Best,
G

P.S. While we’re on the topic, let me complain about the hydration breaks. Clearly they’re just a way to make money with advertising.