Researcher Highlights
Forecasting and the power of big data

Dr Ali Eshragh is finding new ways to effectively analyse big data for forecasting. In doing so, he is setting the stage to markedly advance crucial industry practices.

Dr Ali Eshragh predicts the future, not as a fortune-teller but as a statistician who uses past data to forecast future values.

"For instance, I analyse the daily peak electricity power demand of the city of Newcastle over the past 10 years to predict the demands for the next 30 days,” explains Ali. “Knowing these forecasts in advance is crucial to the planning of generators and distributors.”

Ali’s current area of research is in statistical modelling, in conjunction with advanced machine learning techniques, to analyse big data problems.

Though several classic forecasting models still exist today, the availability of large quantities of data, referred to as “big data”, means a completely different approach to analysis is required.

But working with big data presents two main challenges: data storage limits and computational time limits. Put simply, the amount of storage needed for big data and the inordinate amount of time needed to analyse them.

“Our main goal in our projects is to develop state-of the-art machine learning techniques to overcome these two main challenges in dealing with big time series data.

“Our high-level goals are to develop some fully/semi-automated forecasting systems to analyse big time series data and generate accurate predictions for the future values.”

Forecasting and Big Data

It was Ali’s research mentor Professor Michael Mahoney, a leading scholar in the field of big data and machine learning at the University of  California, Berkeley, who inspired and supported Ali to shift his research focus towards the challenges of big data.

“His mentoring role to move my research direction towards big data problems has been significant and I always appreciate it.”

Since 2014, Ali has led several demand forecasting projects in consultation with Australian food and beverage supply chains. Industry, he believes, is good for inciting research.

“Dealing with challenges involved in those practical projects has been the main motivation for my academic research projects.”

In a proactive step to leverage collective expertise to address these problems, Ali has formed a research group called ForBiD (Forecasting and Big Data).

“Data science is a subject of ongoing intense study. Although our research team is very young, we have already developed novel results for analysing big time series data.”

In a short space of time, the team has rapidly grown and already has active members from the University of Newcastle, other universities in Australia such as the University of Queensland and Monash, and universities in the US such as the University of California Berkeley, and Yale University. In the near future, Ali hopes to establish the ForBiD Cooperative Research Centre at the University of Newcastle.

Advancing operations and performance

Lack of access to real-world big time series data has been an ongoing impediment to Ali’s research efforts, but an alliance with researchers at Yale University is changing that.

“During my recent visit to the US, I could establish a collaboration with the School of Management at Yale University, which led to a couple of joint research projects on a real-world big time series database. Our collaborators at Yale have agreed to share their big database for our joint projects.”

Forecasting, especially when involved with big data, has broad applications that can instigate personal, medical, scientific and engineering advancements. But there’s very little currently available to assist with accurate forecasting, which means Ali’s research could radically transform the way individuals and businesses operate.

“Due to the lack of a user-friendly automated forecasting system, people usually use their own knowledge and experience to predict future events for planning their lives. However, it has been shown that such judgmental forecasts include high errors in practice.”

On an organisational level, big retailers such as David Jones, Amazon and Woolworths could potentially optimise their supply chain and inventory planning logistics with a systematic approach to predicting demand for their products.

“Our research outcome could be immediately applied to any of these personal or business scenarios to improve their performance.”

Improving industry through research

In 2017, Ali was invited to attend a two-day "Science Meets Parliament” workshop in Canberra. He recalls it as a “brilliant event” through which he learnt that among 32 developed countries in the world, Australia ranked last in connections between industries and universities. It’s a position the Australian Government is motivated to improve by fervently supporting initiatives that leverage university-based research to improve industrial practice—initiatives like Ali’s.

“I am so proud that our research outcome can be directly implemented in practice and solve some industry problems. Hopefully, such research projects will pave the road towards improving the ranking between industries and universities.”