Technical Report: Big Data - Concepts, Infrastructure, Analytics, Challenges and Solutions

Abstract

Industry 5.0 is emerging while swarms of Cyber-Physical Systems (CPSs) are being integrated with other swarms of CPSs and humans to work co-actively in collaborative and sustainable environments, which leads to the development of cyber-physical-social-systems (CPSSs) by leveraging the insights filtered from the passed experiences, i.e. Big Data (BD). With Industry 5.0, the borders between humans and intelligent machines cannot be readily distinguished in the new decentralised world - what is created by AI? What is produced by humans or what is built by both? The recent advances in the CPSs and domains, cloud and edge platforms along with the advanced communication technologies are playing a crucial role in connecting the globe more than ever, which is creating large volumes of data at astonishing rates and a tsunami of computation within hyper-connectivity. Data analytic tools (e.g. ChatGPT, Gemini) are evolving rapidly to harvest these explosive increasing data volumes. In this direction, this technical report analyses the concept of BD, BD analytics, and challenges in the processing of BD and discusses practical solutions for these challenges toward Industry 5.0.