CardioTabNet: a novel hybrid transformer model for heart disease prediction using tabular medical data

The early detection and accurate prediction of cardiovascular diseases (CVDs) are critical to reduce global severe morbidity and mortality. Machine learning (ML) methods, operated by Transformers have proved its efficiency in interpreting complex data interactions. One prime example would be its notable success in Natural Language Processing (NLP), with its multi-headed self-attention mechanism to disentangle the complex interactions within high-dimensional spaces. However, the relationships between various features within biological systems remain ambiguous in these spaces, making it difficult to apply transformers in clinical datasets. We introduce CardioTabNet, a transformer-driven framework designed precisely for clinical cardiovascular data. It leverages the strength of the tab transformer architecture to effectively extract meaningful insights from clinical data. As a result, downstream classical models’ performance significantly showed outstanding results. We utilized an open-source cardiovascular dataset with 1190 instances and 11 features. These features are categorized into numerical (age, resting blood pressure, cholesterol, maximum heart rate, old peak, weight, and fasting blood sugar) and categorical (resting Electrocardiograms, exercise angina, and ST slope) variables. Tab transformer was used to extract significant features and rank them using a Random Forest (RF) feature ranking algorithm which highlighted the important clinical predictors. We used ten classical machine-learning models trained on these transformer extracted-features. An optimized ExtraTree classifier achieved an average accuracy of 94.1% and area under curve (AUC) of 95%. Furthermore, we performed nomogram analysis to draw out cardiovascular risk assessment to demonstrate clinical interpretability. Benchmarking against state-of-the-art methodologies affirmed the superior predictive capability of our CardioTabNet framework, demonstrating its potential as a robust tool for clinical decision support in cardiovascular disease prediction and early detection. In addition, SHAP (SHapley Additive exPlanations) analysis was carried out to provide insights into feature contributions and enhance model interpretability.