Go big on data or be left at home

05 June 2015 - 10:04 By FARREN COLLINS

Behavioural economics expert Don Ariely described it best when he said big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it. While experts don't seem to have agreed on a definition for it, big data - often described as the next technological frontier – is a term used to describe massive chunks of data that can’t be processed using traditional processing applications.These chunks, called data sets, comprise information from various sources including online profiles, web searches and clicks, social media feeds and machine generated data such as phone logs, but can come from any source where information is recorded.Big companies and scientists have taken the lead in exploiting the potential of big data for marketing and research, through it's usefulness for seeing patterns and determining trends. Businesses use the information to tailor customer experiences and structure offerings to clients, while researches analyse big data for better understanding of a subject.But experts say it could have just as much value to governments and civil society. The Square Kilometre Array (SKA) radio telescope being built in the Northern Cape is set to become the largest producer of big data in the world when it is completed in 2025, and will produce enough data to fill 4.5-million standard 4.7 gigabyte DVDs a day.“Big data can mean different things to different people,” said Professor Russ Taylor, big data specialist and research chair at SKA.“But essentially, if the data is so big that you can't process it, then that's big data. [The SKA] will eventually produce data at a rate that is equivalent to all the [combined] internet traffic in the world.”Drew Kennedy, partner at South African data analysis company Data Ascend, said more and more businesses were becoming aware of the value of big data, but not enough businesses were harnessing it because they were not correctly merging the data into something useful.“Whatever you do needs to be in line with the business strategy,” said Kennedy. “If not, you will spend lots of money and effort on something inconsequential and of no value to the business.” The challenge for big data remains in understanding how to process it, but Taylor believes South Africa has an opportunity to lead the world in this regard because of SKA, and that it could also benefit other sectors.“Three factors - volume, rate, and complexity or the difficulty to interpret - determine big data problems,” said Taylor. “We are investing and building the SKA big data project into a national university collaboration ... that will help us to take lead on the SKA [from other countries involved in the project]. “We [also] want to take the solutions we develop for the SKA challenge and make them more broadly applicable.”The University of Cape Town is using big data to develop ways to help municipalities make informed decisions about services around electricity and water, by using the data to learn how people respond to changes to the way services are provided."Just being able to deal with data in the public domain is important for governments who have to develop policy around how to provide services," said Taylor.“I think as a nation, if we don't develop the expertise to be part of the big data revolution, then as a country we're going to find ourselves further behind in the world economy.”..

There’s never been a more important time to support independent media.

From World War 1 to present-day cosmopolitan South Africa and beyond, the Sunday Times has been a pillar in covering the stories that matter to you.

For just R80 you can become a premium member (digital access) and support a publication that has played an important political and social role in South Africa for over a century of Sundays. You can cancel anytime.

Already subscribed? Sign in below.



Questions or problems? Email helpdesk@timeslive.co.za or call 0860 52 52 00.