Published: March 3, 2014 - 13:25

Data Scientists on the Front Line: The Second Digital Revolution

Anyone still writing their school or academic essays on typewriters became an unwitting witness to a revolutionary event whose impact on all aspects of life conjures memories of the 19th century industrial revolution: Analogue became digital – in less than 20 years. Whereas the digital proportion of stored information amounted to just three percent in 1993, by 2007 it had already reached an estimated 94 percent.


People are now talking about a new digital revolution that renders the previously described changes as mere foreplay. Witnesses of this paradigm shift are all those who have to battle the adversities of the digital world – infinite emails that clog up inboxes on a daily basis and the endless search for important business documents that appear to be lost in a virtual Nirvana. This affects all those people who work privately or professionally with a PC, tablet or smartphone…so basically all of us.


Data Scientist


The starting point and motivation of the “second digital revolution” is to clean up the chaos that the uninhibited digitisation of the world has brought. It’s about transforming the virtual mountains of rubbish that have amassed over the past two decades into valuable material. Under the catchy motto “from trash to treasure”. What happens is that the infinite individual pieces of data in different formats such as emails, documents, photos, videos and audio files are connected to logical units. What’s so special about this? The result of this process is much greater than the sum of the individual parts.


Every revolution has its protagonists – in the original wave of digitisation it was the IT engineers who developed methods to turn analogue information to digital. Now it’s the data scientists on the front line. Their profile exactly reflects the fundamental “from trash to treasure” process of which they are the driving force. Their strength is to combine a wide range of specialised skills such as mathematics, computer science, statistics, software development, business intelligence and business process development management to extract knowledge in a way that goes far beyond the integration of the individual disciplines. Their role within a company also fits this principle as data scientists are trained to transcend departmental borders and to see the company and its information treasure as a whole – a prerequisite for e.g. optimising internal organisation, opening new business areas, or, put simply, to increase revenue and profit.


Without suitable tools however, data scientists would simply be left fighting windmills. They need a reliable database to be able to make statements about the current strengths and weaknesses of a company or to make propositions about future possibilities. These tasks are fulfilled by enterprise search solutions that have been developed since the beginning of the first digital revolution and who can play out their full benefits with the second revolution. They are capable, using different technologies such as crawling, indexing and ranking, to transform the ever rising mountain of rubbish that exists in every company into an information treasure chest. The principle of enterprise search systems is easy to understand. The benefit for data scientists however, to be able to extract the optimum value from the tools, is priceless.


The second digital revolution has only just begun. Just a few European companies have recognised what advantages the duo of data scientist + enterprise search can generate – USA and China are already ahead. It’s time to decide which side of the revolution you want to be on: The winning side or the losing side!