'Data cloud' to help unravel mysteries of the universe

A gargantuan “research cloud” has been built by three South African universities as the country takes the lead in unravelling the mysteries of the universe.

This became a necessity when the most advanced technology in astronomy - like the Square Kilometre Array (SKA) in the Northern Cape - meant that unfathomable amounts of data from galaxies far away in space and time would need to be stored and processed by scientists or researchers.

“We now have the technology to know how the universe was made‚” says Professor Russ Taylor‚ “but there is a catch: the answer to these questions live in a massive amount of data‚ and in the next decade‚ processing that data is the biggest challenge.”

Taylor is the director of the IDIA (Inter-University Institute for Data Intensive Astronomy) which launched the IDIA Research Cloud on Monday at the Iziko Planetarium in Cape Town - a cloud built especially to deal with all the data.

The three universities that make up IDIA are the University of the Western Cape‚ University of Cape Town‚ and University of Pretoria.

“The SKA has created a data monster‚ and now we have to solve that problem. We need researchers to really be able to play with the data‚ which contains so much information‚ but right now we do not know how to tease it out. This democratises the process‚” said Taylor.

Already‚ SKA precursor MeerKAT - which consists of 64 dishes - was launched 10 days ago and is producing data that no average computer could handle.

Dr Fernando Camilo‚ chief scientist at the South African Radio Astronomy Observatory (SARAO)‚ said: “The beautiful images we see of our universe only exist after massive amounts of data have been processed after being collected over hundreds of hours.”

For Sibusiso Mdhluli‚ a master of science student at the University of the Western Cape‚ the ever-changing world of astronomy is creating new career paths for young people.

As he sharpens his skills in the world of machine learning‚ he says: "Astronomers used  to spend days studying astronomical images to look for interesting objects, but with current amounts of data coming out of telescopes, it can take them years to analyse it using the old method. Now with machine learning algorithms, they are able to analyse this large data in seconds/ hours, which is why machine learning is one of the most important parts of IDIA."

The transformative technology will soon impact on other industries like medicine‚ he said.