Artificial intelligence and big data, three tips to integrate them
The era of business-level artificial intelligence has arrived: companies are increasingly looking to incorporate data into their decision-making processes. This is explained by Christopher Markle, Data Scientist and Business Analyst of Intersections Inc. with an intervention on the blog of Ibm Watson.
However, the idea that a company can simply connect a “black box” of artificial intelligence, feed it with terabytes of data and instantly have an effective system, is bound to fail.
For many companies, embracing the technologies of machine learning and artificial intelligence as new sources of solutions has been a journey full of promise. But also, as many have discovered, of pitfalls.
Christopher Markle shares some points of the experience made on Identity Guard, the service of Intersections of protection against identity theft.
In Identity Guard, explains Markle, the work with Watson started from experimentation and progressed along a journey through artificial intelligence. Finally, to get the Identity Guard product to stand out from the crowd. Along this path, the Identity Guard team has learned three important lessons.
The journey of Identity Guard in artificial intelligence began with an important feedback from the customers of the service. Customers wanted to have something no one else offered. That is to say a service that would help them prevent them from becoming a victim of identity theft. They wanted, in essence, proactive and timely protection.
The three lessons on artificial intelligence
Three steps that have been undertaken to build and grow an intelligent solution.
Firstly, artificial intelligence products to the benefit of customers start with customers. To protect customers, the company has developed a news monitoring solution for all types of vulnerabilities and violations. In order to warn customers if something threatening is found. And take steps to avoid identity theft.
Although the idea seems excellent, it is fraught with obstacles in its realization. It is complicated to get relevant current articles and the Internet does not lack junk content. The solution, on the other hand, works only if you can be sure that you have captured all the relevant news. That means scour hundreds of items a day. This obstacle could be overcome simply by employing more analysts. But this approach does not scale: instead, it was entrusted to Watson.
Artificial intelligence and scalability
The second lesson is, in fact, that the artificial intelligence products that scale are based on technology designed to scale.
In order to scour the almost endless stream of unwanted news items brought daily, Identity Guard needed a technology that could determine with a high degree of precision whether an article described a real and current danger to a client.
Attempts to train a single neural network to do whatever it took, Markle explains, failed. Thus, the Identity Guard team found a solution, even simpler. Instead of training a single neural network to find a needle in a haystack, he trained many neural networks each designed for a different purpose.
In Identity Guard they used the Watson Natural Language Classifiers as “building blocks” to process the content. The strategy of using many simple classifiers has solved the problem of anomaly detection.
Do not wait, and listen to your customers
The third lesson is that successful artificial intelligence products are often constructed with simpler artificial intelligence products.
Using a system of Natural Language Classifiers related to each other, explains Markle, Identity Guard is able to identify threats to their customers as they develop. It is also able to associate specific threats with specific customers and ensure that customers receive updates only on issues that pose a real threat to them.
According to Markle, building a product of artificial intelligence is an iterative process: with many small victories, failures and many opportunities. His advice is therefore to start the journey in artificial intelligence by identifying the needs of the client and the data necessary to meet this need. Deconstructing the need in activities with a limited scope and identifying those activities that can be addressed with artificial intelligence.
Take advantage of the power of conventional computing, human decision-making and building blocks of artificial intelligence in a homogeneous intelligent system. Finally, try the solution: if it does not work for customers, it will not work. And therefore iterate, asking its customers what worked and why, and what did not work and why.
