Clustering Algorithms for Queries: A Comparative Analysis of ‎Farmer Call Center Data

Extracting insights from queries and feedback helps identify trends, enhance products and services, personalize customer interactions and craft effective marketing strategies. Data clustering, a powerful method, organizes unstructured data and refines queries by offering suggestions based on similar or related inputs, ultimately enhancing the search experience. This study compares the performance of several clustering algorithms, including Agglomerative Clustering, K-Means (KM), Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) along with various embeddings including Term Frequency-Inverse Document Frequency (TF-IDF). Sentence-Bidirectional Encoder Representations from Transformers (SBERT), Word2Vec, and GloVe. The Calinski-Harabasz Index, Davies-Bouldin Index, and Silhouette Score were used to measure the effectiveness of these algorithms. Results indicated that HDBSCAN outperformed other clustering algorithms within the farmer helpline dataset. The conclusions were derived from the average performance of the clustering algorithms. The findings showed that HDBSCAN combined with different embeddings, achieved a Silhouette Score of 0.85, Davies-Bouldin Index of 0.66, and Calinski-Harabasz Index of 4239.9.