SOFAMEHACK : A challenge by SOFAMEA

[Alexandre Naaim]

Re: SOFAMEHACK : A challenge by SOFAMEA

2019 SOFAMEA Winner: The Deep Event (Mathieu Lempereur)

The SOFAMEHACK team is happy to finally announce the winner of the first edition of the SOFAMEHACK organised by the SOFAMEA. We would like to thank all the 8 Participants who have submitted their works and shared their idea and technic for solving the problem of gait events detection in healthy and pathological gait. We are also very pleased to see that the participants ranged from master student to full-time researcher showing that research and work for our community is done at all level.


Description of the algorithm :

The Deep Event algorithm developed by Mathieu Lempereur (research engineer at the CHRU of Brest) wins with an overall score of 3,51 which correspond to a mean error between gold standard event and the one detected by his algorithm of 0.76 frames.

The proposed algorithm (Deep Event) is based on deep learning. The development of Deep Event was firstly based on his lab gait database containing more than 10 000 gait events used. An article on proof of concepts and concurrent validity is submitted to the Journal of Biomechanics (https://doi.org/10.1016/j.jbiomech.2019.109490).

Mathieu Lempereur will be awarded a prize of 1000 euros.

Full Ranking:

1. Mathieu Lempereur (Deep Event) final score : 3,51
2. Marc Desaules and Kieran Shubert (Data Mining) final score : 1.53 e+5
3. Hugo Villi and Florent Moissenet (Joint Entropy) final score : 2.12 e+6
4. Amyn Jaafar (Recurrent neural network) 6.53 e+6
5. Team Kalf (Mean Learning) 25 e+77 (on some files no event was detected resulting in large errors)

The following team were not evaluated as they did not follow the challenge guide lines.

• Aurelia Autem and Gibran Chevalley
• Quentin Guy and Raphael
• Bigot Romain

As a reminder, as the full score is the sum of the exponential of the time differences between to the computed gait events and the gold-standard values of the gait events, even errors as low as 5 frames can increase importantly the final score.

http://sofamehack2019.sofamea.org/results

[Alexandre Naaim]

Re: SOFAMEHACK : A challenge by SOFAMEA

A reminder that the deadlign for the SOFAMEHACK is approaching quickly (30rd of June) with only 20 days left.

All information are still available at http://sofamehack2019.sofamea.org/ and if you have any question you can contact us at contact@sofamehack2019.sofamea.org

We are hoping a wide participation for its first edition and looking forward to find the best method to detect gait event in pathological gait.


The SOFAMEHACK committee.

[Alexandre Naaim]

SOFAMEHACK first edition:
Gait event detection in pathologic gait

All the following information can be found at: http://sofamehack2019.sofamea.org/
How to contact us: contact@sofamehack2019.sofamea.org

What is SOFAMEHACK?
The SOFAMEHACK is a challenge launched by the SOFAMEA (French society of Clinical Gait Analysis for Adult and Children). The goal is to develop tools that can be shared within the Clinical Gait Analysis community to improve and standardise clinical practice as well as stimulate collaborations between clinical and research groups. For each challenge, a 1000€ prize will be awarded to the best team.

Gait event detection
Gait events, i.e. foot off and foot strike, are needed to define spatiotemporal parameters and to normalise gait data in time. The clinical standard to define gait events is the use of ground reaction forces measured by forceplates (Veilleux et al. 2015; Carcreff et al. 2018). This information allows an accurate detection of the instant the foot touches or leaves the ground. However, for some patients with small stride/step length or that uses technical walking aids, the detection with forceplates might not be possible. The detection can still be done manually, but it is time consuming and relies strongly on the operator expertise. Furthermore, this process often results in data with low accuracy and low reproducibility. A solution is to develop automatic detection algorithms based on marker trajectories and/or joint kinematics, such as proposed by Zeni et al. (2009), Desailly et al. (2009) or Ghoussayni et al. (Ghoussayni et al. 2004). However, to date these methods can show a good repeatability and accuracy for patients with low impairments but not for patients with strong impairments (Bruening and Ridge 2014). Consequently, a time-consuming manual check of the events still needs to be done systematically. Thus, there is a need for algorithms that can identify gait events with good accuracy and repeatability in clinical settings.

The aim of the challenge:
To develop an algorithm based on marker trajectories and/or joint kinematics that can identify gait events for healthy and pathological gait. For this first edition, it has been chosen to use only one of the most widely used markersets in clinical settings: the Conventional Gait Model. A set of pathological gait data (cerebral palsy GMFCS I/II/III, idiopathic toe walker and foot deformities) containing marker trajectories and gold-standard gait events identified with forceplates will be made available.

Who can participate?
This challenge is open to everyone.

How to participate?

• Training data will be made available using the following link: http://sofamehack2019.sofamea.org

Results submission:
IMPORTANT! Participants should accept to make their program open source and shared with the Clinical Gait Analysis community.

For their submission, the participants should generate C3D files containing the gait events or their submission, the participants should generate C3D files containing the gait events computed by the algorithm as well as the original gold-standard events (extracted from forceplate data) that were encoded in the C3D using the prefix GS (e.g. GS_Left_Foot_Off for Gold Standards Left Foot Off). In addition, they should provide their program in either Matlab, Python or R with a list of C3D files as input and the new C3D files as output.

Description of the algorithm
In addition to their code, participants are required to upload a description of their algorithm. This document should provide sufficient details to be implemented by other groups of the CGA community.
The description should contain the following information:

• Contact details.
• Name of your team.
• Overall structure of your algorithm and a description of each step in this structure.
• Computation times (time to compute the events on the whole database).
• Score of your algorithm on the database provided for the challenge.
• If the algorithm has been tested on other datasets, you could consider including those results.

Results without a suitable description will not be considered for evaluation. Description can be submitted as a word file or pdf file.
Evaluation:
The criterion to evaluate the algorithms will be the sum of the exponential of the time differences between to the computed gait events and the gold-standard values of the gait events. The code that computes the score is available with the data. The gait event detection algorithm with the lowest score will win the challenge. The time for calculating the detection will also be taken into account. If the mean value for the detection is above 0,5s the final score will be multiplied by 10.
All algorithms will be evaluated on a second database of patients with similar impairments
Prize:
A prize of 1000€ will be awarded to the winning team. The awards will be held during a 30 minutes session at the SOFAMEA conference in January 2020. A presentation by the winning team will be done followed by a discussion to share ideas and give future direction for the improvement of the gait events detection. A visio-conference can be organised for teams that cannot attend the conference.