Lead Researchers: A/Prof Marc Adam (Computer Science) and Dr Megan Rollo (Health)

Globally, one in five deaths are related to poor quality diets. Sub-optimal dietary intake has impacts across the lifespan including the growth and development of children through to development of chronic diseases including cardiovascular disease, type 2 diabetes and some cancers in adults. Researchers at the University of Newcastle are collaborating to capture the large sensor datasets needed to better understand and address the problem of objectively measuring eating behaviours, specifically those relating to hand-to-mouth movements. This sensor data can be used to complement self-report food intake data captured by other methods and used to identify targets for nutrition interventions across a variety of settings.

Automatic detection of intake gestures is a key element of automatic dietary monitoring. Several types of sensors, including inertial measurement units (IMU) and video cameras, have been used for this purpose. The common machine learning approaches make use of the labelled sensor data to automatically learn how to make detections. One characteristic, especially for deep learning models, is the need for large datasets. To meet this need, an interdisciplinary team of experts in human-centred computing (A/Prof Marc Adam) and nutrition and dietetics (Dr Megan Rollo) collected the Objectively Recognizing Eating Behaviour and Associated Intake (OREBA) dataset.


The OREBA dataset aims to provide a comprehensive multi-sensor recording of communal intake occasions for researchers interested in automatic detection of intake gestures and other behaviours associated with intake. The investigated scenarios include both communal eating from discrete dishes as well as communal eating from shared dishes. The collected sensor data consists of synchronised frontal video and IMU with accelerometer and gyroscope for both hands. Based on this data, the researchers apply neural networks to develop algorithms that automatically detect food intake gestures from video recordings and movement sensors. One specific application of these algorithms is the Voice-Image-Sensor technologies for Individual Dietary Assessment (VISIDA) project led by Dr Megan Rollo, which focuses on the development of voice-image-sensor technologies for individual dietary assessment in low and lower-middle income countries.

Video: Using machine learning to detect eating behaviours

The University of Newcastle acknowledges the traditional custodians of the lands within our footprint areas: Awabakal, Darkinjung, Biripai, Worimi, Wonnarua, and Eora Nations. We also pay respect to the wisdom of our Elders past and present.