
Sensor-based activity recognition for load carriers in logistics supply chains using machine learning and artificial intelligence
There are more than 650 million Euro pallets in circulation in Europe, and around 100 million are produced each year. Despite their industrial significance, data on the location and condition of pallets and the goods they carry is usually only collected sporadically using sensors. Continuously tracking logistical activities such as 'Driving', 'Lifting' or 'Handling' along their life cycle is hardly possible, even though the technical requirements can already be met by smart load carriers. Due to the lack of analysis regarding the process steps, optimization potential by the gain in transparency remains unexploited.
The Pal2Rec research project conducted a feasibility study which showed that logistical activities can be recognized through the use of sensor technology and AI. The first data sets were recorded under laboratory conditions and published as the Sensor-based Pallet Activity Recognition in Logistics (SPARL) data set. In addition, an initial taxonomy of activities was developed.
While Pal2Rec provided a taxonomy and a first dataset to prove feasibility, the subsequent Lamarr Project Pal2Sim addresses the key challenge of model robustness. The high variability of logistical influences — such as vehicle types, loading scenarios, and process flows — result in a need to have a very large and diverse set of training data. To deal with this and the needed effort of collecting the data, Pal2Sim pursues the approach of generating synthetic sensor data using physical simulation.
The aim is to examine the extent to which this artificially generated data can close the gap in real data sets and thus enable the comprehensive detection of logistics activities. Real data was recorded in an industrial partner's warehouse and identical process steps were mapped in the simulation. The data sets created expand the series of SPARL data sets. Pal2Sim's outcomes aim to enable new approaches to data-driven logistics and process optimization in supply chains.
Bridging the gap between real sensor measurements and simulated environments
Real DataActual sensor measurements from pallets in logistics environments
Synthetic DataComputer-generated simulation environments for model training

Example of a time series and corresponding activity recognition
Developed sensor box for data acquisition
Interested in our project? Join the challenge with our dataset!
Currently a random forest has been used so far to classify the data sets. It is expected that better results can be achieved by adjusting the model or, for example, by using deep learning approaches.
A comparison of the oscillations of the activity classes of the real and synthetic data in the frequency domain and using various metrics has shown a clear distinction. The objective is to approximate the two data sets.
Even though synthetic data may bring a major simplification in the future, real data is still important as ground truth. A computer vision annotation tool is an enabler for using real data as training data more quickly.
All SPARL datasets include Video data, annotated sensor data, and the used classifiers. For follow-up work, please refer to the specified DOI and paper.
The SPARL1 dataset is the smallest dataset specifically developed for the first analysis of the IMU data. The dataset was especially created to compare the influences of different test subjects and sensors and depicts a simple picking process.
SPARL2 represents the first comprehensive data set and includes three scenarios from logistics with various sub-steps. The scenarios 'storage and retrieval in the warehouse', 'picking of various goods', and the process chain of 'goods dispatch' were recorded under reproducible laboratory conditions with various sensors and test subjects.
SPARL3 is the first dataset to contain both real data and synthetic data. In an industrial warehouse, the regular process of goods receipt, storage, retrieval, load securing, and goods dispatch was accompanied in fast motion and carried out with various test subjects.
Coming soon
We are currently working on detailed implementation guides and tutorials for using the SPARL dataset.