SANReN plans to enhance the transfer of big data in South African research facilities

Researchers in fields such as bioinformatics; meteorology; astronomy; particle physics and Earth observation share very large datasets between local and international campuses. The quicker this data is transferred and received, the sooner it will be analysed and utilised in the creation of new knowledge.

As it stands however, most South African campus networks are not designed for large data transfer. The result is that researchers, desperate to send their data to where it is required, resort to physically shipping hard drives of information.

Testing. One, two, three…

To help address this challenge, SANReN – which together with the Tertiary Education and Research Network of South Africa are responsible for the South African National Education and Research Network, is setting up proof-of-concept Data Transfer Nodes (DTNs) at specific sites. SANReN will be using a Science Demilitarised Zone architecture better known as the Science DMZ.

The Science DMZ is a network architecture that has been enhanced for the transfer of large-scale research data. It was developed by engineers at the Energy Sciences Network and National Energy Research Scientific Computing Centre. The architecture allows for a ‘scientific data enclave’ on a campus network, separate from everyday/commodity traffic. This means that big datasets can now be allocated their own pathway on the network for optimised delivery, allowing researchers and scientists to move data more efficiently.

Getting to grip with the technicalities

“The technicalities behind the current limitations are that most campus networks in South Africa make no distinction between general-purpose traffic and specialised large research data sets moving through the network to and from other research institutions,” explains SANReN network engineer Kasandra Pillay. “The result is that big-data traffic takes a long time to arrive at its destination as it is slowed down by various devices on the network which are designed for general-purpose traffic,” she adds.

To tackle the issue of operating large data optimised networks, SANReN recently hosted two pivotal workshops on data transfer and operating innovative networks with their beneficiaries – representatives from South African universities, science councils and research institutions. The workshops were held in Pretoria and in Cape Town, where approximately 50 delegates deliberated on how the Science DMZ can improve network performance when moving large datasets in and out of campus networks.

Based on the international series (http://oinworkshop.com/), the workshops equipped network engineers, managers and researchers with the hands-on knowledge needed to build next-generation campus networks optimised for data-intensive research. SANReN also invited two international experts, Jason Zurawski (Lawrence Berkeley National Laboratory/Energy Sciences Network) and Scott Chevalier (Indiana University International Networks) who presented the workshop with support from SANReN engineers and financial assistance provided through NSF grant 1638863 (IRNC: Backbone: NEAAR: Networks for European, American, and African Research).