You can find here several publications where our products have been used as research tools.
Terrain Type Detection for Smart Equine Gait Analysis Systems Using Inertial Sensors and Machine Learning
Jeanne I.M. Parmentier, Filipe M. Serra Bragança, Elin Hernlund and Berend Jan van der Zwaag, June 2023, 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT); https://doi.org/10.1109/DCOSS-IoT58021.2023.00029
Lameness, limping due to pain, is a significant welfare issue for horses. Veterinarians typically evaluate horses on two terrain types (hard and soft, e.g., asphalt and sand) that are known to affect the observed degree of lameness based on the origin/location of the pain. In the past years, whole-body inertial measurement units (IMU)-based gait analysis systems were developed to support diagnostics and monitor locomotion changes over time. Movement direction and gait (walk, trot) are automatically labeled, resulting in smart and easy-to-use systems. However, terrain types are not detected, leading to information loss. In this work, we explored terrain classification tasks with equine IMU data and machine and deep learning. Using the data of 111 horses equipped with IMU sensors (withers, pelvis, front, and hind limbs), we compared different features-based (FT) and time-series-based (TS) classifiers (train-test ratio: 0.7-0.3). In order to reduce the computational costs of the future system, we also evaluated the performance (F1 score) of the classifiers with different sampling frequencies (10 to 200Hz) and different IMU combinations (body and limbs). Our Convolutional Neural Network models accurately classified terrain types with only one IMU placed on the front limb. Downsampling the signals led to similar results, thus enabling real-time applications.
Read more »
Prediction of continuous and discrete kinetic parameters in horses from inertial measurement units data using recurrent artificial neural networks
J. I. M. Parmentier, S. Bosch, B. J. van der Zwaag, M. A. Weishaupt, A. I. Gmel, P. J. M. Havinga, P. R. van Weeren & F. M. Serra Braganca, 13 January 2023, Nature Scientific Reports 13, 740 (2023); https://doi.org/10.1038/s41598-023-27899-4
Vertical ground reaction force (GRFz) measurements are the best tool for assessing horses’ weight-bearing lameness. However, collection of these data is often impractical for clinical use. This study evaluates GRFz predicted using data from body-mounted IMUs and long short-term memory recurrent neural networks (LSTM-RNN). Twenty-four clinically sound horses, equipped with IMUs on the upper-body (UB) and each limb, walked and trotted on a GRFz measuring treadmill (TiF). Both systems were time-synchronised. Data from randomly selected 16, 4, and 4 horses formed training, validation, and test datasets, respectively. LSTM-RNN with different input sets (All, Limbs, UB, Sacrum, or Withers) were trained to predict GRFz curves or peak-GRFz. Our models could predict GRFz shapes at both gaits with RMSE below 0.40 N.kg−1. The best peak-GRFz values were obtained when extracted from the predicted curves by the all dataset. For both GRFz curves and peak-GRFz values, predictions made with the All or UB datasets were systematically better than with the Limbs dataset, showing the importance of including upper-body kinematic information for kinetic parameters predictions. More data should be gathered to confirm the usability of LSTM-RNN for GRFz predictions, as they highly depend on factors like speed, gait, and the presence of weight-bearing lameness.
Read more »
Stephan Bosch, 7 September 2022, University of Twente, PhD Thesis; https://doi.org/10.3990/1.9789036554350
This thesis explores how to capture and digitize human and animal motion with miniature IMU motion sensing devices and fulfill a wide range of application requirements despite resource constraints related to sensor limitations, wireless communication, processing, and energy consumption. The hypothesis is that the limitations of inertial sensing technology can be circumvented by devising intelligent algorithms that exploit application specifics. In the first part of this thesis, these limitations are circumvented by only measuring the acceleration and angular velocity magnitude, which are directionless values. This is used to track the intensity of motion over time, allowing for objective measurement of for example how active a person is, which has applications in healthcare. In the second part of this thesis, the capabilities of the sensor devices include measuring orientation and displacement with acceptable quality, yet only over short intervals of up to a minute, depending on the intensity of the motion. This is used to measure body angles and gait symmetry for human and equine applications. The time limitation for the displacement result is circumvented by exploiting the periodicity of locomotion, so that it is not needed to calculate the absolute displacement. Body angles are only measured relative to the vertical axis, so that an accurate attitude measurement is sufficient. In the third part of this thesis, an indoor positioning system is designed and analyzed that aims to meet the requirements for use by firefighters using foot-mounted motion sensors and pedestrian dead-reckoning (PDR). This is achieved by tracking the full path that a motion sensor takes trough a building while it is attached to the firefighter’s boot. Performance is impeded by drift errors on the orientation yaw direction and in the displacement result and the irregular and sometimes intense movements performed by firefighters are challenging. The collective experiments show that limitations of inertial sensing technology can be circumvented by devising intelligent algorithms that exploit application specifics. During the course of these experiments, sensor devices were developed that are now being used world-wide in academia and medical applications.
Using Different Combinations of Body-Mounted IMU Sensors to Estimate Speed of Horses—A Machine Learning Approach
Hamed Darbandi, Filipe Serra Bragança, Berend Jan van der Zwaag, John Voskamp, Annik Imogen Gmel, Eyrún Halla Haraldsdóttir and Paul Havinga, 26 January 2021, Sensors 2021, 21(3), 798; https://doi.org/10.3390/s21030798
Speed is an essential parameter in biomechanical analysis and general locomotion research. It is possible to estimate the speed using global positioning systems (GPS) or inertial measurement units (IMUs). However, GPS requires a consistent signal connection to satellites, and errors accumulate during IMU signals integration. In an attempt to overcome these issues, we have investigated the possibility of estimating the horse speed by developing machine learning (ML) models using the signals from seven body-mounted IMUs. Since motion patterns extracted from IMU signals are different between breeds and gaits, we trained the models based on data from 40 Icelandic and Franches-Montagnes horses during walk, trot, tölt, pace, and canter. In addition, we studied the estimation accuracy between IMU locations on the body (sacrum, withers, head, and limbs). The models were evaluated per gait and were compared between ML algorithms and IMU location. The model yielded the highest estimation accuracy of speed (RMSE = 0.25 m/s) within equine and most of human speed estimation literature. In conclusion, highly accurate horse speed estimation models, independent of IMU(s) location on-body and gait, were developed using ML.
Improving gait classification in horses by using inertial measurement unit (IMU) generated data and machine learning
F. M. Serra Bragança, S. Broomé, M. Rhodin, S. Björnsdóttir, V. Gunnarsson, J. P. Voskamp, E. Persson-Sjodin, W. Back, G. Lindgren, M. Novoa-Bravo, C. Roepstorff, B. J. van der Zwaag, P. R. Van Weeren & E. Hernlund, 20 October 2020, Nature Scientific Reports volume 10, 17785 (2020), https://doi.org/10.1038/s41598-020-73215-9
For centuries humans have been fascinated by the natural beauty of horses in motion and their different gaits. Gait classification (GC) is commonly performed through visual assessment and reliable, automated methods for real-time objective GC in horses are warranted. In this study, we used a full body network of wireless, high sampling-rate sensors combined with machine learning to fully automatically classify gait. Using data from 120 horses of four different domestic breeds, equipped with seven motion sensors, we included 7576 strides from eight different gaits. GC was trained using several machine-learning approaches, both from feature-extracted data and from raw sensor data. Our best GC model achieved 97% accuracy. Our technique facilitated accurate, GC that enables in-depth biomechanical studies and allows for highly accurate phenotyping of gait for genetic research and breeding. Our approach lends itself for potential use in other quadrupedal species without the need for developing gait/animal specific algorithms.
M. Tijssen , E. Hernlund, M. Rhodin, S. Bosch, J. P. Voskamp, M. Nielen, F. M. Serra Braganςa, June 3, 2020, https://doi.org/10.1371/journal.pone.0233266
For gait classification, hoof-on and hoof-off events are fundamental locomotion characteristics of interest. These events can be measured with inertial measurement units (IMUs) which measure the acceleration and angular velocity in three directions. The aim of this study was to present two algorithms for automatic detection of hoof-events from the acceleration and angular velocity signals measured by hoof-mounted IMUs in walk and trot on a hard surface. Seven Warmblood horses were equipped with two wireless IMUs, which were attached to the lateral wall of the right front (RF) and hind (RH) hooves. Horses were walked and trotted on a lead over a force plate for internal validation. The agreement between the algorithms for the acceleration and angular velocity signals with the force plate was evaluated by Bland Altman analysis and linear mixed model analysis. These analyses were performed for both hoof-on and hoof-off detection and for both algorithms separately. For the hoof-on detection, the angular velocity algorithm was the most accurate with an accuracy between 2.39 and 12.22 ms and a precision of around 13.80 ms, depending on gait and hoof. For hoof-off detection, the acceleration algorithm was the most accurate with an accuracy of 3.20 ms and precision of 6.39 ms, independent of gait and hoof. These algorithms look highly promising for gait classification purposes although the applicability of these algorithms should be investigated under different circumstances, such as different surfaces and different hoof trimming conditions.
Inertial Sensors as a Tool for Diagnosing Discopathy Lumbosacral Pathologic Gait: A Preliminary Research
Sebastian Glowinski, Karol Łosiński, Przemysław Kowiański, Monika Waśkow, Aleksandra Bryndal and Agnieszka Grochulska 2020, MDPI Diagnostics Journal, Special Issue Challenges and Advances in Monitoring and Diagnosis in Medical Sciences 10(6), 342; doi: https://doi.org/10.3390/diagnostics10060342
Background: the goal of the study is to ascertain the influence of discopathy in the lumbosacral (L-S) segment on the gait parameters. The inertial sensors are used to determine the pathologic parameters of gait. Methods: the study involved four patients (44, 46, 42, and 38 years). First, the goal of the survey was to analyze by a noninvasive medical test magnetic resonance imaging (MRI) of each patient. Next, by using inertial sensors, the flexion-extension of joint angles of the left and right knees were calculated. The statistical analysis was performed. The wavelet transform was applied to analyze periodic information in the acceleration data. Results: in the patients with discopathy, the amount of knee flexion attained during stance phase is significantly lower than that of normal (health side), which could indicate poor eccentric control or a pain avoidance mechanism. The biggest differences are observed in the Initial Swing phase. Bending of the lower limb in the knee joint at this stage reaches maximum values during the entire gait cycle. Conclusions: It has been difficult to quantify the knee angle during gait by visual inspection. The inertial measurement unit (IMU) system can be useful in determining the level of spine damage and its degree. In patients in the first stages of the intervertebral disc disease who may undergo conservative treatment, it may also partially delay or completely exclude the decision to perform a complicated imaging examination which is MRI, often showing a false positive result in this phase of the disease.
Ontology-based framework enabling smart Product-Service Systems: Application of sensing systems for machine health monitoring
Elaheh Maleki, Farouk Belkadi, Nikoletta Boli, Berend Jan van der Zwaag, Kosmas Alexopoulos, Spyridon Koukas, Mihai Marin-Perianu, Alain Bernard, Dimitris Mourtzis (2018), IEEE Internet of Things Journal, 5(2); doi: 10.1109/JIOT.2018.2831279
Providing an integrated Product-Service System (PSS) supported by Cyber-Physical Systems (CPS), as a smart solution, is increasingly offered by industrial machinery. In this approach, instead of providing products with after-sale services, the value proposition is mostly based on guaranteeing a trustable operation. As a result, capturing and analyzing information from a machine’s condition during its lifespan becomes an essential part of the smart PSS. Enabling this through lifecycle monitoring, sensing devices are at the core of smart PSS. Thus, the optimal configuration of PSS components is a critical step to ensure the efficiency of the solution. PSS design challenges arise because of the differences between characteristics and specifications of the embedded sensing system and the product. To fulfill this challenge, knowledge extraction and reuse from these domains is crucial during the design process. To do so, a sensor ontology is used as the backbone of the PSS knowledge-based framework. The focus of this paper is on defining the embedded sensing systems tailored to industrial PSS and the use of these systems in providing customized services. As part of the ICP4Life platform, the paper presents a sensing system ontology as a framework to support the smart services in industrial machinery PSS. An industrial use case is conducted to validate the efficiency of the proposed ontology and to show the benefits of the ICP4Life platform to reduce time and cost of PSS development processes.
Stephan Bosch, Filipe Serra Bragança, Mihai Marin-Perianu, Raluca Marin-Perianu, Berend Jan van der Zwaag, John Voskamp, Willem Back, René van Weeren and Paul Havinga (2018), Sensors, 18(3), 850; doi: https://doi.org/10.3390/s18030850
In this paper, we describe and validate the EquiMoves system, which aims to support equine veterinarians in assessing lameness and gait performance in horses. The system works by capturing horse motion from up to eight synchronized wireless inertial measurement units. It can be used in various equine gait modes, and analyzes both upper-body and limb movements. The validation against an optical motion capture system is based on a Bland–Altman analysis that illustrates the agreement between the two systems. The sagittal kinematic results (protraction, retraction, and sagittal range of motion) show limits of agreement of ±2.3 degrees and an absolute bias of 0.3 degrees in the worst case. The coronal kinematic results (adduction, abduction, and coronal range of motion) show limits of agreement of −8.8 and 8.1 degrees, and an absolute bias of 0.4 degrees in the worst case. The worse coronal kinematic results are most likely caused by the optical system setup (depth perception difficulty and suboptimal marker placement). The upper-body symmetry results show no significant bias in the agreement between the two systems; in most cases, the agreement is within ±5 mm. On a trial-level basis, the limits of agreement for withers and sacrum are within ±2 mm, meaning that the system can properly quantify motion asymmetry. Overall, the bias for all symmetry-related results is less than 1 mm, which is important for reproducibility and further comparison to other systems.
Validation of distal limb mounted inertial-measurement-unit sensors for stride detection in Warmblood horses at walk and trot
Bragança F.M., Bosch S., Voskamp J., Marin-Perianu M., Van der Zwaag B.J., Vernooij J. C. M., Van Weeren P.R. and Back W. (2016), Equine Veterinary Journal ISSN 2042-3306
Inertial-measurement-unit (IMU)-sensor-based techniques are becoming more popular in horses as a tool for objective locomotor assessment.
Objectives: To describe, evaluate and validate a method of stride detection and quantification at walk and trot using distal limb mounted IMU-sensors.
Study design: Prospective validation study comparing IMU-sensors and motion capture with force plate data.
Methods: Seven Warmblood horses equipped with metacarpal/metatarsal IMU-sensors and reflective markers for motion capture were hand walked and trotted over a force plate. Using four custom-built algorithms hoof-on/off timing over the force plate were calculated for each trial from the IMU data. Accuracy of the computed parameters was calculated as the mean difference in milliseconds between the IMU or motion capture generated data and the data from the force plate, precision as the s.d. of these differences and percentage of error with accuracy of the calculated parameter as a percentage of the force plate stance duration.
Results: Accuracy, precision and percentage of error of the best performing IMU algorithm for stance duration at walk were 28.5 ms, 31.6 ms and 3.7% for the forelimbs and -5.5 ms, 20.1 ms and -0.8% for the hindlimbs respectively. At trot the best performing algorithm achieved accuracy, precision and percentage of error of -27.6 ms/8.8 ms/-8.4% for the forelimbs and 6.3 ms/33.5 ms/9.1% for the hind limbs.
Main limitations: The described algorithms have not been assessed on different surfaces.
Conclusions: IMU technology can be used to determine temporal kinematic stride variables at walk and trot justifying its use in gait and performance analysis. However, precision of the method may not be sufficient to detect all possible lameness-related changes. These data seem promising enough to warrant further research to evaluate whether this approach will be useful for appraising the majority of clinically relevant gait changes encountered in practice.
Bosch, S. and Shoaib, M. and Geerlings, Stephen and Buit, Lennart and Meratnia, N. and Havinga, P.J.M. (2015) Analysis of Indoor Rowing Motion using Wearable Inertial Sensors. In: Proceedings of the 10th EAI International Conference on Body Area Networks, BODYNETS 2015, 28-30 Sep 2015, Sydney, Australia. ACM.
In this exploratory work the motion of rowers is analyzed while rowing on a rowing machine. This is performed using inertial sensors that measure the orientation at several positions on the body. Using these measurements, this work provides a preliminary analysis of the differences between experienced and novice rowers, or between a good and a bad technique. The analysis shows that the measured postural angles show no clear trend that would set apart experienced and novice rowers or a bad and a good technique. However, there are clear differences in absolute postural angle’s consistency and timing consistency of strokes between novice and experienced rowers. We also applied a machine learning technique to the data to find the similarities between different rowers and an experienced reference rower. The results can be used to compare the quality of the rowing technique with respect to a reference. In this paper, we present our initial results as well as the challenges that need to be further explored.
Acceptance and usability of technology-supported interventions for motivating patients with COPD to be physically active
Tabak, Monique and Hermens, Hermie and Marin-Perianu, Raluca and Burkow, Tatjana and Ciobanu, Ileana and Berteanu, Mihai (2013) Acceptance and usability of technology-supported interventions for motivating patients with COPD to be physically active. IADIS international journal on www/internet, 11 (3). pp. 103-115. ISSN 1645-7641
In chronic care, technology can play an important role to increase the quality and efficiency of healthcare. But to be successful, healthcare technology needs to be acceptable, usable, and easily integrated into daily life. As a consequence, end-users need to be actively involved in the design process. In the European IS-ACTIVE project, we developed technology-supported interventions that promote physical activity in patients with COPD, by using an ambulant activity coach and an interactive game. In this paper, we elaborate on the design, involving the end-users, to develop interventions that are highly usable and well accepted.
Mark Olieman, Raluca Marin-Perianu, Mihai Marin-Perianu (2012) Measurement of dynamic comfort in cycling using wireless acceleration sensors
Comfort in cycling is related to the level of vibration of the bicycle: more vibration results in less comfort for the rider. In this study, the level of vibration is measured in real time using wireless inertial acceleration sensors mounted at four places on the bike: front wheel axel, rear wheel axel, stem and seatpost. In this way, we measure both the input and output of the frame and fork, and consequently establish the transfer function of the frame and front fork. Besides the transfer of vibrations through the frame, we also investigate the input to the frame and fork. Moreover, we determine the effect of the road surface, speed, wheels and tire pressure on the vibrations induced to the frame and fork. Our analysis shows that road surface, speed and the tire pressure have a significant influence on the induced vibrations. On the contrary different wheelsets have no significant influence. Additionally, the vibrations propagate through the frame within a duration of 5 ms.
Marin-Perianu, R.S. and Marin-Perianu, M. and Havinga, P.J.M. and Taylor, S. and Begg, R. and Palaniswami, M. and Rouffet, D. (2013) A Performance Analysis of a Wireless Body-Area Network Monitoring System for Professional Cycling. Personal and Ubiquitous Computing, 17 (1). 197-209. ISSN 1617-4909
It is essential for any highly trained cyclist to optimize his pedalling movement in order to maximize the performance and minimize the risk of injuries. Current techniques rely on bicycle fitting and off-line laboratory measurements. These techniques do not allow the assessment of the kinematics of the cyclist during training and competition, when fatigue may alter the ability of the cyclist to apply forces to the pedals and thus induce maladaptive joint loading. We propose a radically different approach that focuses on determining the actual status of the cyclist’s lower limb segments in real-time and real-life conditions. Our solution is based on body area wireless motion sensor nodes that can collaboratively process the sensory information and provide the cyclists with immediate feedback about their pedalling movement. In this paper, we present a thorough study of the accuracy of our system with respect to the gold standard motion capture system. We measure the knee and ankle angles, which influence the performance as well as the risk of overuse injuries during cycling. The results obtained from a series of experiments with nine subjects show that the motion sensors are within 2.2° to 6.4° from the reference given by the motion capture system, with a correlation coefficient above 0.9. The wireless characteristics of our system, the energy expenditure, possible improvements and usability aspects are further analysed and discussed.
Bosch, S. and Marin-Perianu, R.S. and Havinga, P.J.M. and Horst, A.P. and Marin-Perianu, M. and Vasilescu, A. (2012) A study on automatic recognition of object use exploiting motion correlation of wireless sensors. Personal and Ubiquitous Computing, 16 (7). 875-895. ISSN 1617-4909
An essential component in the ubiquitous computing vision is the ability of detecting with which objects the user is interacting during his or her activities. We explore in this paper a solution to this problem based on wireless motion and orientation sensors (accelerometer and compass) worn by the user and attached to objects. We evaluate the performance in realistic conditions, characterized by limited hardware resources, measurement noise due to motion artifacts and unreliable wireless communication. We describe the complete solution, from the theoretical design, going through simulation and tuning, to the full implementation and testing on wireless sensor nodes. The implementation on sensor nodes is lightweight, with low communication bandwidth and processing needs. Compared to existing work, our approach achieves better performance (higher detection accuracy and faster response times), while being much more computationally efficient. The potential of the concept is further illustrated by means of an interactive multi-user game. We also provide a thorough discussion of the advantages, limitations and trade-offs of the proposed solution.