February 14, 2019 | Amy Shepherd

Patently unfair - Epson takedowns continue

Hiding behind the shield of its connecter patents, the printer manufacturing giant Epson continues to shut down small UK businesses by requiring eBay and Amazon to take down compatible ink cartridge listings.

Platform takedown notice procedures are a personal patent guard-dog

As a verified rights-owner (VeRO) on eBay UK and by using Amazon UK’s reporting notice system, Seiko Epson Corporation (“Epson”) has free rein to remove any and all third-party cartridge listings that it wishes. It simply has to inform eBay or Amazon of the offending listing, provide its patent number and assert patent infringement. Listings are always removed, and affected sellers cannot prevent, challenge or appeal removal.

This one-sided system is fundamentally unfair. If Epson genuinely believes that its patents are being infringed it should issue court proceedings to enforce its rights. Instead, eBay and Amazon’s automatic takedown notice procedures provide the multi-million-dollar corporation with a blunt tool it can brazenly use to circumvent fair judicial process.

Targeting online sellers is a low move by Epson. The primary focus for patent enforcement should be compatible cartridge manufacturers or importers. Resellers are the least important part of the chain. However, online sellers have the disadvantage of being visible, and eBay and Amazon’s automatic and inflexible takedown policies make them by far the easiest target.

This is not an abstract concern: Epson’s ruthless patent-trolling is steadily shutting down small UK businesses, forcing them to lay off employees or close entirely. Takedown notices are damaging UK entrepreneurship, competition and independent business activity.

Absolute rights-holder protection originates in the E-Commerce Directive 2000

The ability of Epson to manipulate eBay and Amazon into acting as a personal patent guard-dog is not entirely of its own making, however, It is ultimately a symptom of a much larger issue, originating in the EU E-Commerce Directive 2000.

Aiming to give consumers certainty when conducting commercial transactions online, the E-Commerce Directive made online platforms responsible for illegal content on their site once notified of illegality. Nervous of the consequences of liability, platforms responded by implementing blanket takedown notice procedures which gave rights-holders complaining of illegal postings swift and absolute protection.

Subsequent judgments of the Court of Justice of the European Union (CJEU) and EU digital strategies have increasingly sought to make online platforms responsible for their content and activity. This has served to consolidate the platforms’ hands-off approach when it comes to rights-holders asserting infringement.

The takedown notice systems of eBay and Amazon deployed by Epson give affected sellers no opportunity to counter-notice. But if cartridge sellers can’t assert their legal right to post content against the platforms then they have no recourse against listing removal - and thus no recourse against the infringement of their right to free speech. Challenging listings removal is sellers’ only hope of pushback: small-time online enterprises lack the power and ready cash to force Epson to defend its patents in an open judicial forum.

Need for a Counter-Notice System

Policy attention on online platform regulation has largely focused on increasing responsibility for illegal or unwanted content; however, in this circumstance, platforms are taking action to remove content but this is having a significant negative effect on UK enterprises, both in terms of business viability and in terms of free speech.

Many UK businesses, especially individuals and small traders, rely on internet platforms to reach customers and conduct operations. However, the process of removal decision-making at platform level is often arbitrary and unfair. In situations of online defamation, counter-notice is available as a method of challenging speech removal. It allows individuals to put up a direct defence to the removal of their post. Although an imperfect system, this style of rebuttal opportunity could apply well in commercial contexts to protect businesses interests and ensure free speech rights.

For fairness and justice, there vitally needs to be a UK legislative mechanism put in place whereby online sellers can counter-notice against takedown demands and assert their rights to continue trading. If Epson doesn’t then dare take sellers to court for fear their patent might be overruled - well, that’s their choice.

If you have been affected by takedowns relating to Epson compatible ink cartridges and patent claims, please get in touch with us by emailing .

[Read more]

February 11, 2019 | Mike Morel

A new wave of Internet censorship may be on the horizon

2018 was a pivotal year for data protection. First the Cambridge Analytica scandal put a spotlight on Facebook’s questionable privacy practices. Then the new Data Protection Act and the General Data Protection Regulation (GDPR) forced businesses to better handle personal data.

As these events continue to develop, 2019 is shaping up to be a similarly consequential year for free speech online as new forms of digital censorship assert themselves in the UK and EU.

Of chief concern in the UK are several initiatives within the Government’s grand plan to “make Britain the safest place in the world to be online”, known as the Digital Charter. Its founding document proclaims “the same rights that people have offline must be protected online.”  That sounds a lot like Open Rights Group’s mission! What’s not to like?

Well, just as surveillance programmes created in the name of national security proved detrimental to privacy rights, new Internet regulations targeting “harmful content” risk curtailing free expression.

The Digital Charter’s remit is staggeringly broad. It addresses just about every conceivable evil on the Internet from bullying and hate speech to copyright infringement, child pornography and terrorist propaganda. With so many initiatives developing simultaneously it can be easy to get lost.

To gain clarity, Open Rights Group published a report surveying the current state of digital censorship in the UK. The report is broken up into two main sections -  formal censorship practices like copyright and pornography blocking, and informal censorship practices including ISP filtering and counter terrorism activity. The report shows how authorities, while often engaging in important work, can be prone to mistakes and unaccountable takedowns that lack independent means of redress.

Over the coming weeks we’ll post a series of excerpts from the report covering the following:

Formal censorship practices

  • Copyright blocking injunctions

  • BBFC pornography blocking

  • BBFC requests to “Ancillary Service Providers”

Informal censorship practices

  • Nominet domain suspensions

  • The Counter Terrorism Internet Referral Unit (CTIRU)

  • The Internet Watch Foundation (IWF)

  • ISP content filtering

The big picture

Take a step back from the many measures encompassed within the Digital Charter and a clear pattern emerges. When it comes to web blocking, the same rules do not apply online as offline. Many powers and practices the government employs to remove online content would be deemed unacceptable and arbitrary if they were applied to offline publications.

Part II of our report is in the works and will focus on threats to free speech within yet another branch of the Digital Charter known as the Internet Safety Strategy.

[Read more]

February 06, 2019 | Jim Killock

Duty of care: an empty concept

There is every reason to believe that the government and opposition are moving to a consensus on introducing a duty of care for social media companies to reduce harm and risk to their users. This may be backed by an Internet regulator, who might decide what kind of mitigating actions are appropriate to address the risks to users on different platforms.

This idea originated from a series of papers by Will Perrin and Lorna Woods and has been mentioned most recently in a recent Science and Technology committee report and by NGOs including children’s charity 5Rights.

A duty of care has some obvious merits: it could be based on objective risks, based on evidence, and ensure that mitigations are proportionate to those risks. It could take some of the politicisation out of the current debate.

However, it also has obvious problems. For a start, it focuses on risk rather than process. It moves attention away from the fact that interventions are regulating social media users just as much as platforms. It does not by itself tell us that free expression impacts will be considered, tracked or mitigated.

Furthermore, the lack of focus that a duty of care model gives to process means that platform decisions that have nothing to do with risky content are not necessarily based on better decisions, independent appeals and so on. Rather, as has happened with German regulation, processes can remain unaffected when they are outside a duty of care.

In practice, a lot of content which is disturbing or offensive is already banned on online platforms. Much of this would not be in scope under a duty of care but it is precisely these kinds of material which users often complain about, when it is either not removed when they want it gone, or is removed incorrectly. Any model of social media regulation needs to improve these issues, but a duty of care is unlikely to touch these problems.

There are very many questions about the kinds of risk, whether to individual in general, vulnerable groups, or society at large; and the evidence required to create action. The truth is that a duty of care, if cast sensibly and narrowly, will not satisfy many of the people who are demanding action; equally, if the threshold to act is low, then it will quickly be seen to be a mechanism for wide-scale Internet censorship.

It is also a simple fact that many decisions that platforms make about legal content which is not risky are not the business of government to regulate. This includes decisions about what legal content is promoted and why. For this reason, we believe that a better approach might be to require independent self-regulation of major platforms across all of their content decisions. This requirement could be a legislative one, but the regulator would need to be independent of government and platforms.

Independent self-regulation has not been truly tried. Instead, voluntary agreements have filled its place. We should be cautious about moving straight to government regulation of social media and social media users. The government refuses to regulate the press in this way because it doesn’t wish to be seen to be controlling print media. It is pretty curious that neither the media nor the government are spelling out the risks of state regulation of the speech of millions of British citizens.

That we are in this place is of course largely the fault of the social media platforms themselves, who have failed to understand the need and value of transparent and accountable systems to ensure they are acting properly. That, however, just demonstrates the problem: politically weak platforms who have created monopoly positions based on data silos are now being sliced and diced at the policy table for their wider errors. It’s imperative that as these government proposals progress we keep focus on the simple fact that it is end users whose speech will ultimately be regulated.

[Read more]

February 04, 2019 | Javier Ruiz

ORG calls for public participation in digital trade policy after Brexit

A key aspect of Brexit is the future of trade policy. The Government  has committed to abandon the UK’s customs union with the EU to enter into myriad independent trade deals with countries across the world. We don’t want to get into a discussion about the merits of this approach or whether it is likely to succeed, but assuming it will go ahead we believe that transparency and participation are critical requirements for the development of future trade agreements after Brexit.

ORG is interested in trade because these agreements include provisions that severely affect digital rights such as privacy and access to information. Copyright and other forms of IP have been part of trade deals for over 20 years, but countries such as the US now want to expand the scope to include a whole raft of issues into trade negotiations, including algorithmic transparency and data flows.

The UK Department of International Trade is already pre-negotiating deals with the US, Australia and New Zealand and is engaging with interested parties in some sectors, such as IP, which is very positive. ORG is participating in some of these discussions.

Our concern, as we get closer to actual trade negotiations, is that there will be pressure to maintain most of the information confidential. Historically, trade deals have been shrouded in secrecy, with the executive branch of government claiming exclusive prerogative as part of their role in maintaining international relations. In the past decades, as trade issues have expanded into many socio-economic spheres - such as digital, labour or environmental regulations - generating vigorous debates, this lack of transparency has become unsustainable. Even as recently as in the ACTA and TPP negotiations, civil society has been forced to rely on leaks for information and public media interventions for engagement. This secrecy did not stop the derailing of many trade agreements and in cases has fuelled more public concerns.

We recognise that some information requires to remain confidential, but believe that very high levels of transparency and public participation are possible, and indeed necessary in these unprecedented circumstances. The blocking by the House of Lords of the Trade Bill until Government provides more information  on how international trade deals will be struck and scrutinised after Brexit points at the need for change.

The current situation in Parliament and elsewhere demonstrates the difficulties in finding a social consensus around Brexit and the kind of trade policy that should follow from it. The limited public debate on trade deals so far has quickly led to concerns about food safety, with headlines about chlorinated chicken, and the takeover of public services, particularly the NHS. In this situation we think that transparency, including access to draft texts and positions, will be critical to maintain legitimacy.

Fortunately things are changing elsewhere. The WTO has improved their external transparency over the years, particularly when compared to negotiations on bilateral agreements. Documents are available online, there are solid NGO relationships and public engagement activities. We hope that the UK will go even further than the WTO to become a world leader in enabling public participation in trade policy.

[Read more]

February 01, 2019 | Mike Morel

Response to IAB statement

By: Jim Killock (Open Rights Group) Johnny Ryan (Brave) Katarzyna Szymielewicz (Panoptykon Foundation) Michael Veale (University College London)


We have taken note of media reports regarding an update to complaints made by ad-blocking browser developer Brave and Polish activist group Panoptykon Foundation to a number of European data protection authorities.


In addition to complaints in Ireland and Poland, the IAB should be aware that complaints have also been made to the UK Information Commissioner in this matter, by Jim Killock, Executive Director of the Open Rights Group, and Michael Veale of University College London. 


As with previous submissions made by Brave et al., we believe that: (1) the complaints are fundamentally misdirected at IAB Europe or the IAB Tech Lab; and (2) they fail to demonstrate any breach of EU data protection law.

Technical standards developed by IAB Tech Lab are intended to facilitate the effective and efficient functioning of technical online advertising processes, such as real-time bidding. IAB Europe’s Transparency & Consent Framework helps companies engaged in online advertising to meet certain requirements under EU data protection and privacy law, such as informing users about how their personal data is processed. The responsibility to use technologies and do business in compliance with applicable laws lies with individual companies.


The IAB proceed on a misunderstanding of the law and the facts. The complaints have detailed widespread and significant breaches of the data protection regime, in the initial complaints as submitted by our legal team, Ravi Naik of ITN Solicitors with the assistance of a leading QC. Those initial complaints from Sept 2018 have been built on with the further material served on 28 January 2018, Data Protection Day. 

Furthermore, the IAB proceed on the basis of an overly restrictive interpretation of how a data controller is defined. Much like Google tried to avoid liability for search before the ECJ, IAB cannot seek to avoid accountability for their own system.

The facts make clear that IAB are a liable controller. IAB defines the structure of the OpenRTB system. Both the IAB and Google structures could – and should – be remedied to have due regard to the rights of data subject. Whether the structure is so remedied is within the IAB and Google’s control.

The IAB system provides for the inclusion of personal data in the bid request, some of which are very intimate indeed. Indeed, the IAB explicitly recommends the inclusion of personal data in the bid request. For example, it “strongly recommends” that ID codes that are unique to the person visiting a website should be included. [1] It even goes so far as to warn companies using its system that they will earn less money if they do not include these personal data. [2]

The IAB does this in the knowledge that it is unable to exercise any control over what happens to personal data broadcast billions of times a day by its system.  An internal IAB TechLab document from May 2018 confirms that “there is no technical way to limit the way data is used after the data is received by a vendor for decisioning/bidding” once an ad auction broadcast has been made. [3] The same document notes that “thousands of vendors” receive these data. [4]


The Content Taxonomy Mapping document cited by the complainants does not, as Brave and Panoptykon seem to contend, demonstrate that taxonomies of data types that would qualify as special categories of personal data (and are subject to stricter protections under EU data protection law) are used by individual companies; nor can it be considered to prove or demonstrate that any companies making use of those taxonomies are doing so without complying with applicable EU data protection or other law. 


The categorised content content is used by a person, the categories stick to that person, and become personal data. This helps other players profile "the human using the device", as IAB puts it. 

The example bid requests in Google’s developer documentation (Google also uses the IAB RTB standard) speak for themselves. They contain the following personal data: [5]

  • pseudonymized user IDs, that can be “matched” against for re-identification, 

  • IP address, 

  • GPS coordinates (latitude and longitude), 

  • ZIP code, 

  • machine and operating system version details, 

  • and categories (“publisher verticals”). 

The IAB’s own documentation includes an example bid request that contains the personal data of a young female, using a specific iPhone 6s, reading a loading URL, and with several IDs that allow ad auction companies to identify her. [6] The bid request also shows her GPS coordinate at this instant. (Would a woman on her own on a street at night be comfortable knowing that her GPS coordinates were being sent to random parties?)


The complaints are akin to attempting to hold road builders accountable for traffic infractions, such as speeding or illegal parking, that are committed by individual motorists driving on those roads. Using this analogy, the complainants’ purported finding that EU data protection law is being breached is comparable to someone pointing out that an automobile is technically capable of exceeding the speed limit, or parking in a restricted area, and adducing this fact as “evidence” that it actually does.  A technical standard may be misused to violate the law or used in a legally compliant way, just as a car may be driven faster than the speed limit or driven at or below that limit. The mere fact that misuse is possible cannot reasonably be used as evidence that it is  actually happening. And the whole purpose of the Transparency & Consent Framework is to ensure it does not.


The IAB has failed to protect people’s data, which are broadcast billions of times a day, using the system that it defines and encourages its members to use. It cannot claim the to be a bystander. By defining and promoting the system, it plays a role in determining the purposes and means of how that data is processed. Using IAB’s own metaphor - which presents them as road builders or car producers who cannot be held liable for traffic infractions  - it is clear that IAB is the authority that sets the traffic rules for its private roads. It has the responsibility when those rules conflict with the law.

1 AdCOM Specification v1.0, Beta Draft”, IAB TechLab, 24 July 2018 (URL:

AdCOM Specification v1.0, Beta Draft”, IAB TechLab, 24 July 2018 (URL:

Pubvendors.json v1.0: Transparency & Consent Framework”, IAB TechLab, May 2018.

4 Ibid.

5 Authorized Buyers Real-Time Bidding Proto”, Google, 23 January 2018 (URL:

6  AdCOM Specification v1.0, Beta Draft”, IAB TechLab, 24 July 2018 (URL:





[Read more]

January 28, 2019 | Ed Johnson-Williams

Public Understanding of GDPR

Today is the 13th annual Data Protection Day in Europe and the first since the General Data Protection Regulation (GDPR) came into force last May. We are publishing new research today about the public understanding of GDPR and what it means for how organisations communicate about how they use data.

Over the last couple of years, we've seen a lot of attention given to data protection. For the most part, the debate has focussed on helping businesses with legal compliance.

But data protection is more than a compliance hoop for organisations to jump through. It is also about the rights it gives to individuals to know about, understand, and control information about them.

Today, we are publishing research in a new report, ‌Public Understanding of GDPR: How companies, regulators, and civil society can support data protection rights.

We look at:

  • the ways members of the general public think about GDPR and their data protection rights;
  • how regulators, civil society organisations and others who support data protection rights can best communicate to make data protection relevant to the general public
  • what organisations should do to communicate better with individuals about data protection and their rights

The report follows several rounds of interviews, user research, and website usability testing. It is also informed by our experiences from creating a website called Data Rights Finder with Projects By IF. Data Rights Finder makes the content of privacy policies easier to understand and helps people engage their data protection rights.

You can read the report in full here. It is also available in full as a PDF.

We are grateful to the Information Commissioner’s Office (ICO) for funding this research through their Research Grants Programme.

Below is a summary of the findings in this project. 

Do people understand their rights around data protection?

Our research indicated that:

  1. The British public’s awareness of their data protection rights is low. People are surprised when they become aware of the rights they have.
  1. Awareness of consent as a basis for collecting and processing user data is relatively high, but understanding of what consent means is low. The other bases for processing data are not well-known.
  1. People do not think about their lives in terms of the rights they have. They do not think first about their data protection rights and then about what problems they have that they could solve with those rights. Instead, they realise they have a problem they want to deal with and then look for ways of dealing with their problem.

Making data protection relevant to people

Considering the way people understand data protection, these are some points to consider for regulators, civil society organisations and others when communicating in support of data protection rights:

  1. Provide information and context for data protection rights. Expect members of the public to require examples of the situations in which they might find data protection rights useful or vital to solving a problem or improving their life in some way.
  1. Offer services or tools that are problem-focussed rather than rights-focussed. Services or tools that help people use their data protection rights will likely resonate with more people if it is clear which specific problems the service helps with.
  1. Make time to undertake user-centred research to understand how your target audiences think about data protection and the problems in their life. This will help you show how data protection rights can be helpful. Test your messages and products with real people from your audiences.

How organisations can communicate well about data protection rights

From our experience of analysing organisations’ privacy policies to create Data Rights Finder, and talking to people about data protection issues, we have these recommendations for how organisations can communicate better about data protection to individuals:

  1. Provide electronic means such as an email address or contact form to contact your data protection officer. We found several well-known companies who only provided a postal address as the route through which to use a data protection right.

  2. Explain how the data protection rights interact with the particular activities or business that your organisation does. Help the individuals involved to know what their rights are, how those rights are relevant to their relationship with your organisation, and finally, how and why individuals would use those rights.
  1. Use plain English to describe how you use data. Tell people clearly what data you collect and what you will use it for. Test how easy it is to find, read, and comprehend the information you provide about how you use data.
  1. As much as possible, use a granular, rather than a bundled, approach to gaining consent to collect and process personal data. It is not always reasonable to expect people to give consent to everything in your privacy policy at the very beginning of their relationship with you. Just-in-time information and consent is one way to address this.
  1. Link the data you say you collect with the purpose you will use it for. Make it clear which data is being used for which purpose.
  1. Consider alternatives to the name ‘privacy policy’. Research in America consistently finds that people misunderstand what is meant by the name ‘privacy policy’. Phrases like “How we use data” may offer a better alternative.
  1. Contribute to and run trials of machine-readable standards about how you use data. Organisations are often presenting information about how they use data in inconsistent and unstructured ways. This makes it difficult to scrutinise and provide insight into how organisations use data. Organisations should collaborate on and test machine-readable standards to communicate how they use data.

[Read more]

December 20, 2018 | Javier Ruiz

Law Commission report asks for complete reform of online offensive communications

Earlier this year the Law Commission published a comprehensive review of the law around abusive online communications with some important recommendations that support ORG's views, particularly around the problems with the term "grossly offensive"


word cloud law commission

In 2019 ORG will be doing more work on the regulation of online content and free expression, as there are various important government initiatives in the area that could impact the rights of internet users. Earlier this year the Law Commission was asked to conduct an analysis of the criminal law in relation to offensive and abusive online communications. It makes some sensible recommendations for thorough reform of the relevant legislation. The government now has to fully respond and agree to move on with more pre-legislative work.

ORG believes that the current level of online abuse - particularly against vulnerable groups: women, ethnic minorities, migrants, transgender people or those with disabilities, among others -  is unacceptable, but we also consider that there are some areas where restrictions on online speech are not consistent and may impinge on free expression. This is a difficult area, and the thorough approach of the Law Commision is welcome. ORG staff met the Law Commissioner for extensive discussions.

Their focus is whether the criminal law provides equivalent protection online and offline. Excluded from the scope are terrorism offences, child sexual abuse and exploitation offences, online fraud and contempt of court.

The key offences considered are the “communications offences” under the Malicious Communications Act 1988 (“MCA 1988”) and s127 of the Communications Act 2003, but also looks more widely at harassment and public order. The LawCom sees as positive that these do not require evidence of “actual harms” and are therefore easier to prosecute. This actually supports our argument as to their use as “consolation prizes”.

The report recommends tightening the scope of the offences. MCA 1998 is unclear on whether it covers public internet fora or only messages directed to a specific person. s127 does not cover private networks, such as bluetooth or LANs. An interesting point raised is that s127 would cover materials stored in cloud but never actually sent. This could have huge implications if followed through by prosecutors.

The Law Commission engages in a thorough discussion of the term “grossly offensive”, mentioning this is one of the issues we raised with them. The report tracks ongoing problems with defining the term since its inception in the Post Office Protection Act 1884, including the relation of “grossly offensive” to obscenity, vulgarity, vilification; and criticises its impact on freedom of expression. The term continues to lead to controversial prosecutions despite the introduction of insufficient Crown Prosecution Service guidelines because the underlying proscribed conduct is very broadly interpreted and the concepts are malleable.

We welcome the report’s recognition that “grossly offensive communication may in fact be more broadly criminalised online than in the offline world”, as long argued by ORG, due to the pervasive record created by online communications and their much broader reach. The Law Commission diplomatically argues for the term to be removed through a further review.

The report extends those criticisms to the notions of “obscenity” and “indecency” and criticises the details of a wider array of related offences, recommending a thorough review, including the potential impact on private online communications of the Obscene Publications Act 1959.

Throughout the report, we see a pattern where the potential negative impact of various offences could be even worse if the criminal justice system decided to apply them with the simplistic principle that everything that applies offline should equally apply online.

The report makes recommendations on various other areas, such as harassment and staking, hate crime and non consensual disclosure of private information, including sexually explicit images. There is also an analysis of the online environment and challenges to policing, which  explores some of the technical difficulties in establishing the identities of perpetrators,  but not in great detail. Thankfully, there are no proposals for changes to anonymity online or demands for increased data retention.

Importantly, the report also details the harms caused by offensive communications, particularly to women,  and digital rights advocates should read it carefully to increase our self-awareness and avoid potentially insensitive arguments in our quest for due process and the protection of civil liberties.

[Read more]

September 20, 2018 | Javier Ruiz

Machine learning and the right to explanation in GDPR

One of the rights in GDPR is the right to explanation. Here we take a look at some of the debates about the right and how it can be implemented.

This blogpost is a small section of a much larger research report Debates, awareness, and projects about GDPR and data protection. The report complements the launch of the Digital Rights Finder tool delivered by Projects by IF and Open Rights Group. We highlight some of the most interesting and important debates around GDPR (General Data Protection Regulation).

There is some concern about the practical feasibility of implementing the right to explanation in GDPR in the context of complex data processing such as big data, artificial intelligence and machine learning. (See this section of the report for more on debates about the existence of the right to explanation.)

Lilian Edwards and Michael Veale argue that a right to an explanation is not the remedy to harms caused to people by algorithmic decisions. They also argue that the narrowly-defined right to explanation in GDPR of “meaningful information about the logic of processing” is not compatible with how modern machine learning technologies are being developed.

The problems to tackle here are discrimination and fairness. Machine learning systems are designed to discriminate but some forms of discrimination are socially unacceptable and the systems need to be restrained. The general obligation of fairness in data protection provides the basis for the need to have some level of insight into the functioning of algorithms, particularly in profiling.

One of Edwards and Veale’s proposals is to partially remove transparency as a necessary key step towards accountability and redress. They argue that people trying to tackle data protection issues have a desire for an action, not for an explanation. The actual value of an explanation will not be to relieve or redress the emotional or economic damage suffered, but to understand why something happened and helping ensure a mistake doesn’t happen again.

Within this more limited sense, problems remain in defining  transparency in the context of algorithmic accountability. For example, providing the source code of algorithms may not be sufficient and may create other problems in terms of privacy disclosures and the gaming of technical systems. They argue that an auditing approach could be more successful instead by looking at the external inputs and outputs of a decision process, rather than at the inner workings: “explaining black boxes without opening them”.

The authors see the right to explanation as providing some grounds for explanations about specific decisions. They present two types of algorithmic explanations that could be provide: model-centric explanations (MCEs) and subject-centric explanations (SCEs), which seem broadly aligned with explanations about either systems or decisions.

SCEs are seen as the best way to provide for some remedy, although with some severe constraints if the data is just too complex. Their proposal is to break down the full model and focus on particular issues through pedagogical explanations to a particular query, “which could be real or could be fictitious or exploratory”. These explanations will necessarily involve trade offs with accuracy to reduce complexity.

Their main concern seems to be to avoid a creating a “transparency fallacy”, where similarly to the “consent fallacy” people regimen an illusion of control that does not exist, instead of being offered practical remedies to stop harmful data practices.

There is growing interest in explanation of technical decision making systems in the field of human-computer interaction design. Practitioners in this field criticise efforts to open the black box in terms of mathematically interpretable models as removed from cognitive science and the actual needs of people. Alternative approaches would be to allow users to explore the system’s behaviour freely through interactive explanations. This is quite similar to the proposals by Edwards and Veale.

A complementary approach has been put forward by Andrew Selbst and Solon Barocas, who argue that the increasing calls for explainability of automated decision making systems rely on an intuitive approach that will not work with machine learning. ML is both inscrutable and non-intuitive. Inscrutability is the back box problem, the inability to understand the inner cogs of a model, but non-intuitiveness means being unable to grasp the rules the model follows, even if we were able to open the box. Accountability requires not only knowledge of the process, but also whether it is justified, or fair.

Selbst and Barocas argue that lawyers and scholars asking for explanations will be disappointed because intuition cannot deal with the truly novel insights produced through machine learning that associate data in patterns that completely escape human logic and imagination.

Their alternative proposal is to focus accountability on the processes around ML models, not the models themselves. Policies and documentation of intent and design choices should be made available, some by default, such as impact assessment, and others in the context of a complaint or regulatory action. This approach chimes with the main thrust of GDPR, which puts accountability at the fore.

In summary, the right to an explanation as defined in GDPR may be harder than expected to implement. This does not invalidate the basic premise that individuals have a right to know what is being done with their data, but – particularly with novel machine learning techniques – it means that we need to look beyond simple calls for transparency.

[Read more]