A Robust Ensemble Deep Learning Framework for Detecting Deepfake Audio Using Mel-Spectrograms

This paper presents a robust deepfake audio detection framework leveraging Mel spectrogram
representations combined with ensemble deep learning models. The input audio is first converted into Mel
spectrograms, capturing essential time-frequency characteristics crucial for distinguishing synthetic speech
from genuine audio. Our approach evaluates three classification strategies: (1) training custom deep learning
architectures including CNN, RNN, and CRNN directly on Mel spectrograms; (2) applying transfer learning
using state-of-the-art computer vision models such as ResNet-18, MobileNet-V3, and (3) utilizing embeddings
extracted from advanced pre-trained audio models like YAMNet, PANNs, ECAPA-TDNN, and PyAnnote,
which are then classified by a multilayer perceptron (MLP). By ensembling the top-performing models from
these strategies, our system achieves a highly competitive Minimum Detection Cost Function (minDCF)of
0.021on the ASVspoof 2021 benchmark dataset. The experimental results demonstrate that combining Mel
spectrogram features with ensemble deep learning enhances the accuracy and robustness of deepfake audio
detection, making this framework suitable for real-world security applications.