M3Care: Learning with Missing Modalities in Multimodal Healthcare Data

Chaohe Zhang, Xu Chu, Liantao Ma, Yinghao Zhu, Yasha Wang, Jiangtao Wang, Junfeng Zhao

    Research output: Chapter in Book/Report/Conference proceedingConference proceedingpeer-review

    8 Citations (Scopus)


    Multimodal electronic health record (EHR) data are widely used in clinical applications. Conventional methods usually assume that each sample (patient) is associated with the unified observed modalities, and all modalities are available for each sample. However, missing modality caused by various clinical and social reasons is a common issue in real-world clinical scenarios. Existing methods mostly rely on solving a generative model that learns a mapping from the latent space to the original input space, which is an unstable ill-posed inverse problem. To relieve the underdetermined system, we propose a model solving a direct problem, dubbed learning with Missing Modalities in Multimodal healthcare data (M3Care). M3Care is an end-to-end model compensating the missing information of the patients with missing modalities to perform clinical analysis. Instead of generating raw missing data, M3Care imputes the task-related information of the missing modalities in the latent space by the auxiliary information from each patient's similar neighbors, measured by a task-guided modality-adaptive similarity metric, and thence conducts the clinical tasks. The task-guided modality-adaptive similarity metric utilizes the uncensored modalities of the patient and the other patients who also have the same uncensored modalities to find similar patients. Experiments on real-world datasets show that M3Care outperforms the state-of-the-art baselines. Moreover, the findings discovered by M3Care are consistent with experts and medical knowledge, demonstrating the capability and the potential of providing useful insights and explanations.
    Original languageEnglish
    Title of host publicationKDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    Publisher Association for Computing Machinery
    Number of pages11
    ISBN (Electronic)9781450393850
    Publication statusPublished - 14 Aug 2022
    Event28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining - Washington, United States
    Duration: 14 Aug 202218 Aug 2022
    Conference number: 28

    Publication series

    NameProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining


    Conference28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    Abbreviated titleKDD '22
    Country/TerritoryUnited States
    Internet address

    Bibliographical note

    Free access via publisher's website

    Funding Information:
    This work is supported by the National Natural Science Foundation of China (No.62172011). L. Ma is supported by the China Postdoctoral Science Foundation (2021TQ0011). J. Wang is supported by EPSRC New Investigator Award under Grant No.EP/V043544/1.

    Publisher Copyright:
    © 2022 ACM.


    • electronic health record
    • healthcare informatics
    • multimodal data

    ASJC Scopus subject areas

    • Software
    • Information Systems


    Dive into the research topics of 'M3Care: Learning with Missing Modalities in Multimodal Healthcare Data'. Together they form a unique fingerprint.

    Cite this