Companies have a large amount of data, but most of it goes unused. Only data that appears valuable from afar, is readily accessible, and is easily transported into an Excel chart gets used, said Lauri Ilison, Head of Big Data and Machine Learning at Nortal, in an interview for Estonian business daily Äripäev’s news portal Finantsuudised.
According to Ilison, there is a simple and human explanation for this: we tend to choose solutions from among the things we have already tried and tested. New tasks are solved with old methods because those methods are familiar and have proven to yield results. This, however, leads to stagnation and to being stuck with using the methods over and over again.
At Nortal, Ilison leads a team of 20 people whose jobs entail processing big data and placing it into the business context. According to Ilison, it is difficult to provide a simple definition for big data, but it can be said that a company’s “big data” basically includes all of the organisation’s data that has been created as a result of the organisation’s daily business activities. Including logs, databases, emails, voice and video recordings.
It is Ilison’s and his team’s job to guide companies out of this vicious circle by helping them identify usage patterns, collating data, building models, and to help them establish a company-wide data culture, i.e. a company’s own way of doing data science.
“If you ask an entrepreneur what kind of resource reserves they have that they are not using, then this is data. Data that we are simply sitting on, and that accumulates in vast quantities every day. Identifying its value is the beginning of the next period of growth for a company,” says Ilison.
Leaders need analysts who, in addition to carrying out the analysis, can also provide context around it. So where would a company find such employees? “In my opinion, it is not enough to simply say ‘go on and hire such specialists.’ Because such people simply don’t exist. The competition in this market is extremely tough. More specialists are coming onto the market, but not fast enough and also not enough of them,” offers Ilison as an explanation for the challenges faced by the sector.
Since this is a relatively new area of business, there aren’t many people in Estonia, who are able to put data science into a business context. Those coming onto the market straight out of university may be able to do data science, but they have no business knowledge. At Nortal, we find the necessary people mainly in-house. They are often people who are already good specialists, who are able to carry out development work or analyses, but would like to retrain. The retraining of employees at Nortal is carried out with the support of colleagues as part of an in-house training programme.
Ilison suggests that other company leaders do exactly this, and find the necessary specialists from among their existing analysts: there are bound to be specialists with great potential among existing employees, who would like to move onto the next level and try something new. These people are your resource. “You will not find data scientists on the market easily.”
Since increasing data capacity also means increasing risk; a new and extremely strict data protection regulation will enter into force within the EU in May 2018, which will have a direct impact on all companies and organisations.
Until now, some countries, such as Estonia and Germany, have followed the existing European data protection directive to the letter, while southern European countries have had a more relaxed approach to it. The new regulation no longer allows such creative interpretation and requires all countries to comply with it.
However, the adoption of the new regulation is not that straightforward and requires a major reorganisation and great effort from companies. The regulation will apply to all companies, organisations and fields of activity where data is stored, but particularly those companies that collect and process vast amounts of customer data, such as banks, insurance companies, retail businesses, and telecommunications companies. Among other things, the regulation states how customer data should be stored, how it can be used, and gives customers important rights to check how a company uses the data they have about them on file.
For an ordinary person this means that they will have access to information on the data a company holds on them and the right to know in which analyses their data has been used. Since the regulation states that the data belongs to the customer, not the company, the customer may also forbid the company to use and process their data.
Making sure companies are ready to comply with the new regulation is precisely the challenge Ilison and his team are faced with this year and Nortal has developed a new solution to help companies achieve compliance.