Multimodal Fusion of Electrocardiogram Signals and Demographic Characteristics with Cross-Attention Networks in the Classification of Cardiovascular Diseases

Creative Commons License

Çay T., Ölmez E.

Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji, cilt.1, sa.1, ss.1, 2026 (TRDizin)

Özet

Cardiovascular diseases are the leading cause of mortality worldwide. In diagnosis, low-cost and non-invasive electrocardiography (ECG) is the primary tool. Since manual examination imposes a high workload on experts, numerous deep learning approaches have been proposed in the literature. In this study, the simultaneous classification of five disease classes (NORM, MI, STTC, CD, HYP) defined by SCP codes in the PTB-XL database was targeted. The proposed architecture consists of three components. In the first component, a second-order polynomial cross-feature transformation was applied to the variables of sex, age, height, and weight. The data were projected into a dmodel-dimensional space using a neural network with a GELU activation function. In the second component, 12-channel ECG recordings were decomposed into multi-scale representations through parallel convolution (1D-CNN), and temporal dependencies were modeled using xLSTM (mLSTM) layers. In the third component, predictions were generated using class-specific context vectors obtained by fusing clinical data as the “query” and ECG data as the “key/value” through cross-attention. In training, which prevented information leakage, the BCEWithLogitsLoss loss function compatible with multilabel classification was used. The learning rate and AdamW weight decay were optimized through a 12-cell grid search, ablation studies were conducted, and the model was compared with ResNet and InceptionTime. The 0.9106 macro AUROC and 0.7151 macro F1 scores obtained within a 95% confidence interval across five independent seeds demonstrate the performance of the method. Heatmaps generated by overlaying attention weights onto the ECG signal provide temporal interpretability of the decision mechanism.