< Back to previous page

Project

Part-Based Syndrome Classification and Metric Spaces: Linear 3D Facial Analysis vs Geometric Deep Learning

The science of phenotyping is empowering our understanding of health, particularly in clinical genetics, where it is used to strengthen genetic testing interpretation and disease research. In next-generation phenotyping (NGP), advanced computational techniques are used to integrate phenotypic data into human health analysis, such as capturing, structuring, and interpreting complex clinical data. In this context, accurate characterization of facial shape, or facial phenotyping, is very promising for studying the role of genetic and environmental factors in determining population and individual health. Facial based NGP can be particularly useful for identifying genetic syndromes, as distinct craniofacial characteristics are manifested in approximately 30% to 40% of these syndromes, and their presence contributes to the establishment of an accurate diagnosis. Despite this, diagnosis from facial morphology remains challenging due to the wide variety, the small differences between individuals with different syndromes, and the rarity of some of these conditions. Despite the fact that 2D photographs are more readily available, 3D images provide a more accurate representation of facial shape and morphology since distortions caused by projection, positional changes, and lighting conditions are not present. This demonstrates the advantages of 3D facial shape analysis over its 2D counterpart. With an adequate dataset of 3D facial images, Artificial Intelligence techniques can objectively assist the diagnosis process by exploiting dysmorphic features associated with genetic disorders.

In this work, we propose a geometric deep learning (GDL)-based encoder, referred to as geometric encoder (GE), and first train it by a deep metric learning loss function for optimally encoding face-based properties, such as sex, age, BMI, and genetic background. We then use the same methodology to extract dysmorphic characteristics that are helpful in distinguishing genetic syndromes. A variety of methods are used to enhance the explainability of the models. More specifically, the GE architectures are modified to facilitate multi-scale and part-based learning for 3D facial shape analysis and multi-class syndrome classification in which a separate GE model is trained for each localized facial segment. By applying this part-based approach to facial photographs of controls and individuals belonging to 13 different syndrome classes, classification accuracy is improved. Through saliency maps illustrating the contribution of each facial segment to the model decision, we improve model explainability, and through ablation studies, we investigate which facial segment contains the most unique information. The proposed method encourages the use of advanced part-based GDL methods for 3D facial imaging in clinical genetics.

In addition to the GE, we further propose and construct a low dimensional metric space, referred to as clinical face phenotype space (CFPS), that models the range of facial dysmorphism present in 51 different and diverse syndromes along with general facial variations. For this purpose, we propose a geometric autoencoder and combine it with a singular value decomposition layer. The model simultaneously learns facial shape variation and reconstruction, in an unsupervised way, and group discrimination with the supervision of syndrome labels. Furthermore, to promote explainability, experiments are designed to illustrate the following properties of CFPSs: Their ability to 1) increase classification and clustering syndromes compared to a linear baseline, 2) generalize to novel syndromes, and 3) reflect the phenotypic relationship among related genetic diseases, meaning that clusters of phenotypically similar disorders reflect functional relationships between genes. By utilizing these properties, individuals without a confirmed diagnosis can be compared quickly to each other and to existing phenotypic groupings in the space, thereby establishing their position in a spectrum of known relationships between phenotype and clinical and molecular diagnosis. Thus, the space can be used to propose hypothetical clinical and molecular diagnoses. This can facilitate variant interpretation in a genome wide next-generation sequencing analysis or direct towards targeted sequencing as part of a clinical diagnostic workup.

Date:1 Sep 2018 →  27 Oct 2022
Keywords:Facial Dynamics, Computer Vision, Deep Learning, Autism Facial Dynamics, Facial Dynamics in Infants
Disciplines:Psychiatry and psychotherapy, Nursing, Other paramedical sciences, Clinical and counselling psychology, Other psychology and cognitive sciences
Project type:PhD project