EU Data Protection Law May End The Unknowable Algorithm

...

Millennials know exactly what they want and expectations are high - very high. They are empowered

Slated to take effect as law across the EU in 2018, the General Data Protection Regulation could require companies to explain their algorithms to avoid unlawful discrimination.

Europe's data protection rules have established a "right to be forgotten," to the consternation of technology companies like Google that have built businesses on computational memory. The rules also outline a "right to explanation," by which people can seek clarification about algorithmic decision that affect them.

In a paper published last month, Bryce Goodman, Clarendon Scholar at the Oxford Internet Institute, and Seth Flaxman, a post-doctoral researcher in Oxford's Department of Statistics, describe the challenges this right poses to businesses and the opportunities it presents to machine learning researchers to design algorithms that are open to evaluation and scrutiny.

The rationale for requiring companies to explain their algorithms is to avoid unlawful discrimination. In his 2015 book The Black Box Society, University of Maryland law professor Frank Pasquale describes the problem with opaque programming.

"Credit raters, search engines, major banks, and the TSA take in data about us and convert it into scores, rankings, risk calculations, and watch lists with vitally important consequences," Pasquale wrote. "But the proprietary algorithms by which they do so are immune from scrutiny."

Several academic studies have already explored the potential for algorithmic discrimination.

A 2015 study by researchers at Carnegie Mellon University, for example, found that Google showed ads for high income jobs to men more frequently than to women.

That's not to say Google did so intentionally. But as other researchers have suggested, algorithmic discrimination can be an unintended consequence of reliance on inaccurate or biased data.

Google did not immediately respond to a request to discuss whether it changed its advertising algorithm in response to the research findings. 

A 2014 paper from the Data & Society Research Institute echoes the finding that inappropriate algorithmic bias tends to be inadvertent. It states, "Although most companies do not intentionally engage in discriminatory hiring practices (particularly on the basis of protected classes), their reliance on automated systems, algorithms, and existing networks systematically benefits some at the expense of others, often without employers even recognizing the biases of such mechanisms."

Between Europe's General Data Protection Rules (GDPR), which are scheduled to take effect in 2018 and existing regulations, companies would do well to pay more attention to the way they implement algorithms and machine learning.

But adhering to the rules won't necessarily be easy, according to Goodman and Flaxman. They note that excluding sensitive data having to do with race or religion, for example, doesn't necessarily mean algorithms will return non-biased results. That's because other non-sensitive data points, like geographic area of residence, may have some correlation with sensitive data.

What's more, the researchers observe that many large data sets are the product of multiple smaller data sets, making it difficult if not impossible for organizations to vouch for the integrity, accuracy, and neutrality in their data.

"The GDPR thus presents us with a dilemma with two horns: Under one interpretation the non-discrimination requirement is ineffective, under the other it is infeasible," write Goodman and Flaxman.

In a phone interview, Lokke Moerel, senior of counsel at Morrison & Foerster, said the provision on automated decision making is not new. Also under the current Directive (the data rules that apply to criminal matters), companies have to inform individuals about the underlying logic involved in their automated decisions.

[See 8 Ways To Secure Data During US-EU Privacy Fight.]

Moerel acknowledged the difficulties of the rules, noting that in an era where algorithms are dynamic and self-learning, it's very difficult to know how an algorithm made a decision at any point in time, let alone communicate this to an individual in a meaningful manner. If logic is incomprehensible to the vast majority of people, the question becomes: What is the added value of providing this information in the first place?

Moerel said she found it troubling that algorithms can end up being discriminatory through data correlation. As an example, she noted that an insurance company charging higher premiums in a certain region because of higher accident rates could end up discriminating against a specific ethnic group that happens to live in that area. She also suggested there's a risk that companies may try to hide such discriminatory correlations by performing further analytics and finding other non-sensitive correlations that they know are correlated with the sensitive data. Requiring the disclosure of algorithmic logic guards against such action, she said.

In order to avoid being questioned about algorithmic logic, Moerel suggested companies give individuals affected by their decisions more control over the implications of how data are used (e.g., by giving them control over their ad preferences, whereby they can view and adjust the indicators that triggered the relevant advertisement for the visitor).

"It will help to avoid individuals questioning your logic if you give them control of the triggers that matter to them," she said. "If people are looking at a black box, it won't be acceptable for European regulators."

Goodman and Flaxman say that work is already underway to make algorithms more easily subject to inspection. And they remain optimistic that technical code can coexist with the legal code.

"We believe that, properly applied, algorithms can not only make more accurate predictions, but offer increased transparency and fairness over their human counterparts," they conclude.

Categories
APPLICATIONS
0 Comment

Leave a Reply

Captcha image


RELATED BY

  • 5300c769af79e

    Samsung Sets Up Note 7 Trade-In Kiosks at Airports

    As in the US, Aussie passengers are no longer permitted to take a Galaxy Note 7—switched on or turned off—on a flight from Oz.For those leaving Australia for an extended period of time—or permanently—Samsung will, "where possible," exchange the Note 7 for another Samsung phone—either at the terminal before departure or at your destination.
  • 5300c769af79e

    The Show: Episode 107 - HTC 10 Review is Here

    On episode 107 of the Droid Life Show, we’ll talk through our final thoughts on the HTC 10, now that our review is out.In case you haven’t read that, just know that we are quite high on the 10 and have plenty to say.
  • 5300c769af79e

    Google Hosting Indie Games Festival in San Francisco on September 24, Open and Free to Public

    During the festival, those in attendance will have a unique opportunity to play unreleased indie games from many developers, then vote to see which games will win awards.Should a developer take home a prize, it could be a SHIELD Android TV, SHIELD Tablet K1, Razer Forge TV prize pack, or other exciting stuff.
  • 5300c769af79e

    Salesforce Releases Texting Customer Service Chatbot

    Salesforce on Tuesday released LiveMessage, a chatbot that lets enterprises users engage customers through messaging apps such as Facebook Messenger, SMS or MMS directly within its Service Cloud.Businesses can add messaging capabilities to their existing toll-free customer service phone numbers and be up and running on Facebook Messenger and SMS, or both, within a day, Salesforce said.