In a landmark case, heard on 5 February 2020, the Court of The Hague halted the use of the System Risk Indication (SyRI) algorithm, which is used to detect social security fraud and predicts the likelihood of a citizen committing benefit or tax fraud. It is based on information from government databases and was targeted exclusively at low income and minority citizens in the Netherlands and has been used since 2014.
DISCRIMINATION IN LOW INCOME NEIGHBOURHOODS
Nani Jansen Reventlow, Director of the Digital Freedom Fund, provides comment for Human Rights Pulse on this landmark decision: “A good example of effective strategic litigation to challenge discrimination in the use of technology is the challenge to the Dutch SyRI system. The system was designed to prevent the abuse of social benefits fraud. By combining data from different government services, it determined who had an increased risk of committing fraud. The system was only used in low income neighbourhoods and municipalities that housed a large percentage of non-Western immigrants. Just a few weeks ago, the court in The Netherlands found that SyRI violated the European Convention on Human Rights and struck out the legislation underpinning it.”
The court found SyRI was in violation of Article 8 (the right to respect for private life) of the European Convention on Human Rights. The UN Special Rapporteur on extreme poverty and human rights, Professor Philip Alston, commented: “This decision sets a strong legal precedent for other courts to follow. This is one of the first times a court anywhere has stopped the use of digital technologies and abundant digital information by welfare authorities on human rights grounds.”
THE BALANCING OF ARTICLE 8
The parameters where the State may interfere with the enjoyment of a protected right are set out in paragraph 2 of Article 8, such as in the interests of national security, public safety, for the prevention of disorder or crime or for the protection of the rights and freedoms. The State’s position is that the SyRI legislation contains sufficient guarantees to protect the privacy of everyone. However, the court found that SyRI does not meet the “fair balance” test the ECHR stipulates, when assessing the justifications for violating private life.
Additionally, the risk scores for individuals, which are scored on government databases for up to two years, was a concern for the Court particularly because individuals are not aware of their score and in turn they are not able to challenge it.
The two main human rights concerns that Special Rapporteur Alston expressed in relation to SyRI are:
1. The right to social security of the individuals in the Netherlands - Alston believes this is affected because SyRI can determine whether a welfare beneficiary is flagged for further investigations that can lead to the “withdrawal, reduction or suspension of individual welfare benefits.”
2. The right to privacy - SyRI utilises data collected by various public authorities.
FUTURE IMPLICATIONS AND “DIGITAL WELFARE” WARNINGS
Special Rapporteur Alston also states: “I predict that this judgment will be a wake-up call for politicians and others who were asleep before, not just in the Netherlands, but in many other countries where Governments are experimenting with digitising government.”
This ruling is an example of how automated technology can infringe on human rights. It shows how technology that monitors citizens can be challenged in the courts and through litigation. It indicates how the court views the high responsibility the State has when it has access to the analytics of citizens.
The case also shows how those from more vulnerable communities may be the target of algorithm-based programs focusing on finding “risk.” The Special Rapporteur also warns that “as humankind moves, perhaps inexorably, towards the digital welfare future it needs to alter course significantly and rapidly to avoid stumbling zombie-like into a digital welfare dystopia.”
Hafsah is a newly-called barrister, LLM graduate and youth worker. Blogging on social mobility and access to justice: Leeds to the Bar - (https://leedstothebar.law.blog/)
Twitter: @HafsahH24