A list of Data Storage technologies and services available to University of Newcastle Researchers

Data storage

Data Storage services provide capacity for storing raw data. When selecting external data storage facilities special consideration should be given to the volume of data to be stored, how the data will be backed up and any privacy concerns surrounding the data.

Online data storage options can be particularly useful in situations where a distributed team of researchers are working on the same data.

What is it and how do I get it? When should I use it? What caveats or obstacles are there? *Support?

ARCS DataFabric

A web based data and document storage area accessible nationally and internationally, it is available at no cost to researchers.

Suitable for small to medium size data storage requirements - by default researchers are given 25 GB of storage (more is generally available by request).

Suitable for situations where access must be restricted to specific individuals across multiple institutions (Nationally & Internationally).

The current default space allocation on the DataFabric is 25 GB but additional space can be made available with a valid research need.

This service should not be used to hold the only copy of your data, make sure you have another copy backed up for safe keeping.

To access this service a AAF user account is required.  Please contact our University IT help desk by telephone on Ext. 17000 to organise an account.

External (NeCTAR)

Dropbox

A file storage service that stores documents on internet based servers and synchronised copies of those files across the user's personal computers (PC, Apple Mac, Apple iPhone & iPad, Android smart phone & Tablet, Linux). 

Files are available on the personal computer when you are not connected to the internet and changes are synchronised the next time you go online.

The Dropbox service is suitable for maintaining additional copies of documents and data that are not privacy sensitive.

It enables access to documents across multiple computers (e.g. at work & at home) and has the ability to restore previous versions of documents in the event of corruption or accidental deletion.

It can also be used to share documents among colleagues both internal and external to the University.

Dropbox is generally easy to set-up and use, it also integrates well with the operating systems on devices. (Windows, Apple Mac, Apple iPhone&  iPad, Android smart phones & tablets and at times Linux).

Dropbox requires users to register for an account. Free accounts are limited to 2 GB and 30 days of un-do history. You have to pay a monthly or annual fee if your requirements exceed these limits.

When sharing files or folders with other users, controls around grant writing or read-only access are limited. It is not possible to share files with some users with write access and other users with read-only (i.e. a folder is either shared to all specified users as read-only or write access).

Data is stored in the cloud and transferred over the internet, which occurs securely over encrypted channels but the location of the data may be in countries that have different privacy laws and legislation than Australia. Consideration must be put to ethics requirements so sensitive research isn't contravened.

With Linux, Dropbox integrates well with Gnome but may work less well, with other window managers.

External (Dropbox)

External Hard Drives

(e.g. USB)

External Hard Drives are typically connected to a desktop PC via a USB cable. 

These drivers typically range in size from 0.5-3 TB.

Suitable for backing up copies of an individual's documents and data files that they are currently working on.

Can also be used to transfer files between off-line systems or ship larger data sets by courier (e.g courier the disk containing large data sets to a colleague).

Disk or file encryption should always be used when transporting privacy sensitive data.

These drives are not suitable for holding the only copy you possess of any document file/s (there should always be another copy of the file/s on a PC or Apple Mac device.

In the case of critical research data or long term storage of research data which supports published works;  drivers should not hold the only back up copy, another copy should be maintained on a suitable storage media (e.g. Archive Tape, the UoN RDS, RDSI Node).

USB external hard drives are highly mechanical devises that are prone to failure.The typical average life expectancy is between 3-5 years but up to 10% of these drives will fail sometime within the first 3 years (most of them in the first 6 months) of use.

Care should be taken to implement anti-virus precautions when connecting to unknown systems or PC's that are not networked (e.g. instruments).

Non-networked PC's can't run up-to-date Anti Virus software (i.e. they cannot download the latest virus signatures) but they can be infected with viruses through other user's already infected USB memory sticks.

Storing privacy sensitive data on these drives should be avoided due to the mobile nature of these devices & the subsequent risk of loss or theft. If the user requires the use of a External Hard Drive, than encryption mechanisms should be used (e.g. Bit Locker for Windows).

End User

Memory Sticks

(USB, SDRam)

External hard drives typically connected to a personal computer via a USB cable. 

These drives typically range in size from 2- 64 GB.

 

Suitable for backing up copies of an individual's documents and data files that they are currently working on.

Can also be used to transfer data sets between off-line systems (e.g. transferring data from a non-networked instrument).

These drives are not suitable for holding the only copy of any documents and file/s (i.e. there should be another copy of the file/s located on a PC or Apple Mac Detc.

In the case of research critical data or long term storage of research data which supports published works; these drives should not be the only backup copy.  Another copy should be maintained on a suitable storage media (e.g. Archive Tape, The UoN RDS, RDSI Node).

Storing privacy sensitive data on these drives should be avoided due to the mobile nature of these devices & the subsequent risk of loss or theft. If the user requires the use of a External Hard Drive, than  encryption mechanisms should be used (e.g. Bit Locker for Windows).

Care should be taken to implement anti-virus precautions when connecting to unknown systems or PCs that are not networked (e.g. Instruments). Non networked PC's cannot run up-to-data Anti Virus software (i.e. it cannot download the latest virus signatures) but they can be infected with viruses through other user's USB memory sticks already containing a virus.

End User

Network Attached Storage - NAS

An appliance accessible over the network for storing data and files.

Typically between 2 and 20 TB.

 

For the sharing and storage of data associated with a small group (5 to 20 people). 

These devises typically offer redundancy such that a single disk failure does not lose data.

Offers a lower cost per terabyte than data centre based Storage Area Network (SAN) but it has limitations with back up and data transfer speeds.

These devices do not generally come with backup facilities. 

The ability to continue to access data after a disk failure should not be considered a replacement for backups. If the data is corrupted or an individual inadvertently deletes the data, then it may not be recoverable.

The UoN* will provide limited support if the equipment was design-built and managed by the ARCS IT team, data integrity cannot be guaranteed due to the back up limitations of these devices.

*UoN Supported

Storage Area Networks - SAN

Corporate quality storage typically located in purpose built computer rooms / data centres.

Research Group purchased and ARCS Team design, build and managed.

For the sharing and storage of large quantities of data (10 Terabytes to 10 Petabytes+) to fast compute servers and / or large numbers of users (100's to 1000's).

A SAN is a higher cost per Terabyte solution than NAS but provides significant improvements in capacity and data transfer speeds along with Tape Back up for long term archiving.

The UoN* will provide limited support if the equipment was design-built and managed by the ARCS IT team, data integrity cannot be guaranteed due to the back up limitations of these devices.

*UoN Supported

Research Data Store Infrastructure - RDSI

A government funded initiative to provide large quantities of storage (100's of Petabytes nationally) for Research data.

 As of June 2012 this infrastructure was still under construction.

For the storage of Research data sets that are of national significance.

Further information can be found at the RDSI website.

Infrastructure is still in the design phase.

External(Organisation)

Research Data Store

The Research Data Store is the University of Newcastle provided storage facility for the long term archiving and curation of research data sets which support published works by UoN academics and RhD and PHD students.

To store primary research data in a centralised facility that is backed-up, archived and catalogued. The intention is to enhance data re-use (with the permission of the creating Academic) and compliance with the Research Data Policy in the Australian Code for the Responsible Conduct of Research.

Space is assigned for specific projects and applications for access are made through the Research Division using the application form located at the University's website, Data Storage web page.

UoN Supplied

UoN - U:drive

A individual U: drive is allocated to each member of staff and all students.

 

The U:drive is for documents and small data files for individual staff and student use.

Files on the U: drive are those files that you may be currently working on or are important enough that you do not want to lose them.

Deleted or corrupted files in the U: drive can be restored from up to 7 days in the past.

The default space allocation for a U: drive is 500 MB.

So the drive is not a suitable location for files that are to be shared, as access cannot be granted between staff and between students.

UoN Supplied

University of Newcastle Divisional Drives (S: drive & R: drive)

A S: drive is a shared storage space that is typically allocated to the whole of Faculty, School or Division. If your department has been allocated a shared space, it will appear as an S: drive in your computer network.

To request access to your Faculty / School / Division S: drive you need to submit a signed request for access  Application form to UoN's IT Services.

The S: drive is for documents and data files that are being collaboratively worked on or being shared amongst a University organisational group (e.g. School, Division, PRC).

These divisional drives are backed up daily.

The space allocation for S: drives vary but typical sizes are between 5 and 200 Gigabytes.

The S: drive is not suitable for large data sets as allocations more than 500 Gigabytes are unlikely.

UoN Supplied


* Support Source of Support for Service
UoN Supplied Deployed, managed and supported by the University and provided to Researchers at no additional cost.
UoN Supported Deployed, managed and supported by the University with some additional costs to the Researcher, Centre or Faculty.
End User The item is end user supported.The University is not in a position to provide technical support or assurances that the product will work within the University environment.
External (Organisation) Deployed, managed and supported by an organisation external to the University. Additional costs may apply.