Is over-focusing on privacy hampering the push to take full advantage of AI? 

In 2006, British mathematician Clive Humby declared that data is the new oil -- and so could be the fuel source for a new, data-driven Industrial Revolution. 

Given that he and his wife helped Tesco make £90m from its first attempt at a Clubcard, he should know. And it looks like the “derricks” out there are actually pumping that informational black gold up to the surface: the global big data analytics market is predicted to be more than $745bn by 2030 -- and while it may not be the most dependable metric, Big Tech is throwing billions at AI at a rate described as “some of the largest infusions of cash in a specific technology in Silicon Valley history”. 

That’s what we’re being told is happening. But is it really? Are organizations genuinely and consistently monetizing all the data they have -- and so using it for competitive advantage and innovation? 

More worryingly, are the safeguards we have put in place, like GDPR and other global data privacy restrictions, being deployed in ways that make them needlessly restrictive to the full flood of AI-driven innovation? 

We were keen to understand where organizations really are with that exploitation, and if data monetization and a defined, repeatable workflow really is in place. 

Specifically, we wanted to check if organizations really are leveraging their data’s full potential and are operating with the appropriate levels of protection for that data, and if that data is accessible in a time frame useful for the business. 

Therefore, we commissioned research and spoke to 600 CIOs, CTOs, CISOs, heads of data and data managers in organizations in everything from the airline sector to retail and telecommunications. 

And it was interesting to hear firsthand what they told us about the true state of data handling out there. 

Innovation through data, yes. But enough of it? No 

What these practitioners told us is that while they absolutely see the potential for innovation and profit opportunity in their data, they are struggling to capitalize on this cost-effectively. 

For example, 56 percent of all respondents said they have already achieved significant improvements in both CX and EX (customer and employee experience); 44 percent confirmed data access has directly led to a profit advantage of 6 to 10 percent -- a very impressive 20 percent put that figure at more like 11 to 20 percent. In 2023, over half (57 percen5) of the organizations we spoke to had introduced new products and services based on insights they’d got directly from mining their data. 

But 32 percent admitted it can take 3-6 months to access the data they need to power apps to do that. 37 percent said that process could take between one to two full months, and only a tiny percentage, a mere 2 percent said that they could access data in “less than a week” or instantly. 

To clarify what we mean by “data'': we’re not talking transactional data such as EPOS data -- we mean the potentially good data, about customers and their behaviors and attitudes. 

When we say that, CISOs and Data Protection Officers will immediately slap the label “sensitive and classified data” over this -- which leads to this data being put into purdah and many doors locked around it. 

Which is as it should be -- I don’t mind others knowing what kind of sandwich I bought at the petrol station for example but I do mind if you connect that purchase to my NI number. 

If the data protection wall is being set at too high a level, does that mean a miss of millions in potential profit through data exploitation (and potentially billions for UK plc) in lost opportunity? 

Is putting all your effort into cyber potentially a big mistake? 

Yes and no. We do need to protect our data -- these laws exist because of shockingly bad behavior by some technology vendors. 

But there’s a troubling confusion about what we are really protecting here. IT has somehow become obsessed with cyber security but hasn’t yet really got its head around data security. 

Thus, we have $180 billion annually spent on cybersecurity, while data security strategies are so much less robustly understood and resourced. At the end of the day all that cybersecurity exists to protect your networks, your infrastructure and endpoint… but 99 times out of 100, the bad agents don’t care about any of that -- they are after one thing, your data. 

What is happening, then, is that CIOs are putting all their attention into cybersecurity and GDPR and other data privacy laws, thinking that solves the data problem. But if you lock down your data in ways that make it essentially ‘dark’ to the kind of AI analysis you want to run over it, then you’re sitting on your oil, not pipelining it. The good news is there is a solution that will satisfy all the stakeholders in the custody and use of customer data.  

Use the right technology to protect all that sensitive information 

The way out of this is to understand that there are tried and trusted ways to let your data scientists and machine learning platforms get into the data in ways that 100 percent preserve my NI number from that visit to the petrol station, but still allow you to do something with the transaction (or sell it to third parties). 

Step forward Privacy-Enhancing Technologies (PETs) like encryption and anonymization, which can be used to protect sensitive information in ways that seal off anything that could worry the CIO, as that “sensitive and classified” personal data is replaced with a value that prevents individual consumer data from being directly identified. 

What CIOs in this survey tell us is that most organizations are heavily invested in PET through encryption, with pseudonymisation and tokenization less utilized or understood.  

Personally, we think that’s a mistake -- as pseudonymisation in particular is a highly efficient way of making your customer and partner data safe in privacy terms but open to rapid analysis. 

Nonetheless, we were very encouraged by how our analysis showed the need to look again at data security, with almost all (96 percent) of all respondents saying they plan to invest a portion of their IT budget on the issue this year -- with 49 percent thinking that needs to be between 11-15 percent of their entire IT budget. 

Summing up, these numbers show that business owners know data is their biggest asset -- but they are also crying out for a safe way to use it so they can analyze it all for advantage in ways that don’t expose anybody in the company or their supply chain.  

Not long after that famous comparison between data and oil, Australian economist Michael Palmer pointed out that while oil in its raw state is valuable, “if unrefined it cannot really be used”. Could thinking more about data than infrastructure security be the refining step we’ve been missing? 

Image Credit: Leowolfert / Dreamstime.com

Paul Mountford is CEO at Protegrity. The full results of the research discussed in this article -- The State Of Data Security Optimization And Monetization -- can be found here