Article excerpts from a forthcoming book — 99 Articles on the GDPR
It’s increasingly becoming evident that digital transformation not only entails changes in business models but also the automation of tasks. So while improvements in business may continue to arise, much of this transformation requires existing data to be used for future decision-making processes. At times, personal data may be used without consent or knowledge of that person’s data. To protect this data, the GDPR grants consumers the right “not to be subject to a decision…which is based solely on automated processing and which provides legal effects (on the subject).” Experts characterize this rule as a “right to an explanation.”
For organizations that use automated decision-making, GDPR creates a “right to an explanation” through Article 22 of the GDPR. Article 22 states “The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.” However, as automation incorporates personal data, profiling and other biases will arise so the GDPR will affect decision engines but not the models used in training.
The GDPR does not precisely define the scope of decisions covered by this section. The United Kingdom’s Information Commissioner’s Office (ICO) says that the right is “very likely” to apply to credit applications, recruitment, and insurance decisions. The United States has a similar model when it comes to credit bureaus. However, other agencies, law courts or the European Data Protection Board may define the scope differently. So the question remains as to how much transparency is required when an organization is needed to explain how processes work.
The problem facing data science is to explain “how” the decision was formed. While the GDPR will force data scientists to limit the techniques they use to train predictive models, the level of transparency that the GDPR requires will not be useful without agency. Although people are worried about the future development of artificial intelligence, much of technology today uses some form of machine learning sold as “customization” and “convenience” to achieve secondary outcomes (i.e., social media “discovery”). Technology does not inherently cause a generation of ethics or transparency, but frameworks like the GDPR does force, or at least attempt to force, data scientists to explain techniques that affect their decision engines so that techniques can be used to “reverse-engineer” methods.