Pal2Rec LogoLamarr Institute Logo

Sensor-based activity recognition for load carriers in logistics supply chains using machine learning and artificial intelligence

Project Overview

There are more than 650 million Euro pallets in circulation in Europe, and around 100 million are produced each year. Despite their industrial significance, data on the location and condition of pallets and the goods they carry is usually only collected sporadically using sensors. Continuously tracking logistical activities such as 'Driving', 'Lifting' or 'Handling' along their life cycle is hardly possible, even though the technical requirements can already be met by smart load carriers. Due to the lack of analysis regarding the process steps, optimization potential by the gain in transparency remains unexploited.

The Pal2Rec research project conducted a feasibility study which showed that logistical activities can be recognized through the use of sensor technology and AI. The first data sets were recorded under laboratory conditions and published as the Sensor-based Pallet Activity Recognition in Logistics (SPARL) data set. In addition, an initial taxonomy of activities was developed.

While Pal2Rec provided a taxonomy and a first dataset to prove feasibility, the subsequent Lamarr Project Pal2Sim addresses the key challenge of model robustness. The high variability of logistical influences — such as vehicle types, loading scenarios, and process flows — result in a need to have a very large and diverse set of training data. To deal with this and the needed effort of collecting the data, Pal2Sim pursues the approach of generating synthetic sensor data using physical simulation.

The aim is to examine the extent to which this artificially generated data can close the gap in real data sets and thus enable the comprehensive detection of logistics activities. Real data was recorded in an industrial partner's warehouse and identical process steps were mapped in the simulation. The data sets created expand the series of SPARL data sets. Pal2Sim's outcomes aim to enable new approaches to data-driven logistics and process optimization in supply chains.

From Real Data to Synthetic Simulation

Bridging the gap between real sensor measurements and simulated environments

Real pallet sensor dataReal Data

Actual sensor measurements from pallets in logistics environments

Simulated pallet environmentSynthetic Data

Computer-generated simulation environments for model training

Example of time series and corresponding activity recognition

Example of a time series and corresponding activity recognition

Developed sensor box for data acquisition

Developed sensor box for data acquisition

Research Challenges

Interested in our project? Join the challenge with our dataset!

Challenge 1: Can you outperform our classifier?

Currently a random forest has been used so far to classify the data sets. It is expected that better results can be achieved by adjusting the model or, for example, by using deep learning approaches.

Challenge 2: Can you bridge the Sim2Real gap?

A comparison of the oscillations of the activity classes of the real and synthetic data in the frequency domain and using various metrics has shown a clear distinction. The objective is to approximate the two data sets.

Challenge 3: Can you develop an automatic annotation tool with computer vision?

Even though synthetic data may bring a major simplification in the future, real data is still important as ground truth. A computer vision annotation tool is an enabler for using real data as training data more quickly.

Research Datasets

All SPARL datasets include Video data, annotated sensor data, and the used classifiers. For follow-up work, please refer to the specified DOI and paper.

SPARL1 Dataset

The SPARL1 dataset is the smallest dataset specifically developed for the first analysis of the IMU data. The dataset was especially created to compare the influences of different test subjects and sensors and depicts a simple picking process.

SPARL2 Dataset

SPARL2 represents the first comprehensive data set and includes three scenarios from logistics with various sub-steps. The scenarios 'storage and retrieval in the warehouse', 'picking of various goods', and the process chain of 'goods dispatch' were recorded under reproducible laboratory conditions with various sensors and test subjects.

SPARL3 Dataset

SPARL3 is the first dataset to contain both real data and synthetic data. In an industrial warehouse, the regular process of goods receipt, storage, retrieval, load securing, and goods dispatch was accompanied in fast motion and carried out with various test subjects.

Coming Soon

Stay Updated

Get notified when new datasets are released or when updates to the SPARL dataset become available.

Subscribe to Updates

Getting Started

Coming soon

We are currently working on detailed implementation guides and tutorials for using the SPARL dataset.