This paper presents a novel method for singer identification from polyphonic music audio signals. It is based on the universal background model (UBM), which is a singer-independent Gaussian mixture model (GMM) trained on many songs to model the singer characteristics. For our model, singing voice separation on a polyphonic signal is used to cope with the negative influences caused by background accompaniment. Then, we construct UBM for each singer trained with the Mel-frequency Cepstral Coefficients (MFCCs) feature, using the maximum a posterior (MAP) estimation. Singer identification is realized by matching test samples to the obtained UBMs for individual singers. Another major contribution of our work is to present two new large singer identification databases with over 100 singers. The proposed system is evaluated on two public datasets and two new ones. Results indicate that UBM can build more accurate statistical models of the singer’s voice than conventional methods. The evaluation carried out on the public dataset shows that our method achieves 16% improvement in accuracy compared with the state-of-the-art singer identification system.