Malaysia is a multiethnic and multilingual nation. In order to meet this diversity, Malaysia has developed an educational system with different languages as the medium of instruction (MOI). From the Razak Report to the Rahman Talib Report and the 1961 Education Act, Malay or Bahasa Melayu (BM) has been instituted as the national language and the MOI in national schools, with the aim of promoting nation unity. However, in the case of the Chinese vernacular schools, Mandarin is the MOI. Such a difference in MOI for the two different school systems has allowed educationists and non-educationists alike to fault the Chinese vernacular schools as an obstacle to racial unity. How valid and reliable is this accusation? In contrast, Chinese educationists feel it is of utmost importance to preserve their mother tongue (Mandarin) based education. Therefore, this paper will unravel how Chinese medium schools impact on unity or the lack of it among the students via interviews and a questionnaire adapted from Fazilah, Noraziah, Zaharah, Mansor, and Azizah (2009). The findings have revealed that the students’ ethnicity and national identity are best portrayed by Mandarin because of its strong association with their Chinese roots. In addition, Mandarin is also the students’ language of choice both inside and outside the classroom, which seems to suggest the absence of a sense of unity among them.