And the results are in! The fallout from the ‘mutant algorithm’

I clearly remember May 2018 and the constant fear of an impending data protection Armageddon. Working long hours to ensure all suppliers had signed GDPR compliant contracts, reviewing all data stored in multiple systems, reviewing privacy policies and in general feeling like the Y2K bug all over again. I remember organising events for HouseMark members with expert lawyers and attending Boards to talk about the areas to look out for (like the potential for an increased number of SARs). One of the things from that period that I remember the most was trying to understand the more obscure items in the legislation and there was none more perplexing than Article 22 also known as the ‘right to explanation of automated individual decision-making including profiling’. In principle, the idea is fairly simple: data subjects should be able to understand how an algorithm arrived at a decision which affects them but in practice, this turned out to be a very complex debate triggering the launch of an inquiry in the House of Commons, the creation of a new Centre for Data Ethics and Innovation and numerous debates about what this means for people. I remember trying to come up with good examples to illustrate this specific point and always feeling it was just a theoretical concept or an academic argument.

Two years later, a global pandemic would provide a perfect and tragic example which will forever define what it means for an unexplainable algorithm to make a decision with a major impact on people's lives. 

Image source: Daily Mail

Everyone knows the story. The coronavirus would force schools to shut down in March 2020 and a decision was made early by the government that A-levels and GCSE exams would not be taken, and an alternative method of awarding grades would be developed by Ofqual. A few months later, when the results came out and caused a significant number of students to have their expected grades downgraded, the inquest started immediately, and the culprit was identified. It was an algorithm which had taken a number of rules, crunched them and provided a result with massive real-world consequences for a whole generation of students. The fallout has been epic, and the subsequent decision to use teacher predicted grades was swift.

Image source: Getty Images

There are significant lessons for housing organisations as the sector makes more use of data and algorithms to support decisions.

The chair of Ofqual, Roger Taylor, said to the House of Commons Education Committee that it was a "fundamental mistake" to believe that it “would ever be acceptable” to use an algorithm to award grades for this year’s A-level. This is a serious blow to public confidence in algorithmic decision making and particularly ironic coming from Taylor who also chairs the Centre for Data Ethics and Innovation which was created in those early days of GDPR to provide clarity and trust in algorithms.

The importance of data-based decisions, which has been highlighted during the pandemic, will continue to grow, but housing colleagues must learn from this cautionary tale and ensure the necessary controls are in place and data can be trusted. Organisations will have to work harder than ever to regain trust as the mutant algorithm could strike again.

Who would have thought that a health crisis would be teaching us about the transparency of algorithmic decisions? Secure your place at HouseMark’s 10 Days of Data to learn more lessons from the COVID-19 crisis and explore the value of data across your business. With something for everyone, find out more about the events on offer and book your ticket here.

By Arturo Dell

Share this Article:

Share this:

Subscribe to our newsletters

Subscribe below to get regular updates from Housemark

Latest Tweets

Latest tweet

hasRole=False

leadValue=False

showPopup=False

totalGroups=0

loginStatusModel.IsLoggedIn=False

  1. id: 12469
  2. name: And the results are in! The fallout from the ‘mutant algorithm’
  3. item type: Content
  4. template alias: NewsStory
  5. created data: 11/09/2020 10:21:49
  6. Document Type Alias: NewsStory
  7. Path: -1,1101,1099,1141,1143,6127,12469
  8. Last Render: 23/09/2020 20:12:50