Intravenous immunoglobulin resistance prediction in Kawasaki disease using oversampled transformer embeddings

Abstract

Kawasaki disease (KD) is a leading cause of acquired heart disease in children under five. Although intravenous immunoglobulin (IVIG) treatment is usually effective, 10–20% of cases are resistant and at higher risk for coronary complications. Early prediction of IVIG resistance is critical but difficult due to the rarity of KD and imbalanced clinical data. To address this, we propose a novel technique called sentence transformer embeddings with synthetic minority over-sampling technique (SMOTE) oversampling (STESO), which leverages the complementary strengths of transformer-based representation learning and synthetic oversampling. Pretrained models such as paraphrase-MiniLM-L3-v2 are used to convert tabular clinical data into dense text-based embeddings, capturing deeper semantic relationships across features. By coupling these rich embeddings with SMOTE, we balance class distributions directly in the semantic space, enabling traditional machine learning (ML) models to more effectively detect minority (resistant) cases. This synergy yielded substantial improvements in sensitivity and F1-score, with random forest (RF) combined with STESO (RF-STESO) achieving the highest overall performance. Among the models evaluated, our proposed model attained best result as accuracy of 0.85, a sensitivity of 0.81, a specificity of 0.89, and an F1-score of 0.85. Our results underscore that the joint use of transformer embeddings and oversampling is more effective than either approach in isolation, offering a promising pathway for rare disease prediction tasks such as IVIG resistance prediction in KD.