The Hui Chinese have diverse origins. Some in the southeast coast (Guangdong, Fujian) and in major trade centers elsewhere in China are of mixed local and foreign descent. The foreign element, although greatly diluted, came from Arab (Dashi) and Persian (Bosi) traders, who brought Islam to China. These foreigners settled in China and gradually intermarried into the surrounding population while converting them to Islam, while they in turn assimilated in all aspects of Chinese culture, keeping only their distinctive religion.

A totally different explanation is available for the Hui people of Yunnan and Northwestern China, whose ethnogenesis might be a result of the convergence of large number of Mongol, Turkic, Iranian or other Central Asian settlers in these regions, who were recruited by the Mongol-founded Yuan Dynasty either as officials (the semu, who formed the second-highest stratum in the Yuan Empire's ethnic hierarchy, after the Mongols themselves, but before both northern and southern Chinese) or artisans. It was documented that a proportion of the ancestral nomad or military ethnic groups were originally Nestorian Christians many of whom later converted to Islam, while under the sinicizing pressures of the Ming and Qing states.

To these days, many Hui people and non-Hui observers say that facial features of some members of their communities make them somewhat distinct from the surrounding Han population, and reflect their Southwest Asian ancestry.

An elderly Hui man Southeastern Muslims also have a much longer tradition of synthesizing Confucian teachings with the Sharia and Qur'anic teachings, and were reported to have been contributing to the Confucian officialdom since the Tang period. Among the Northern Hui, on the other hand, there are strong influences of Central Asian Sufi schools such as Kubrawiyya, Qadiriyya, Naqshbandiyya (Khufiyya and Jahriyya) etc. mostly of the Hanafi Madhhab (whereas among the Southeastern communities the Shafi'i Madhhab is more of the norm). Before the "Ihwani" movement, a Chinese variant of the Salafi movement, Northern Hui Sufis were very fond of synthesizing Taoist teachings and martial arts practices with Sufi philosophy.

In early modern times, villages in Northern Chinese Hui areas still bore labels like "Blue-cap Huihui," "Black-cap Huihui," and "White-cap Huihui," betraying their possible Christian, Judaic and Muslim origins, even though the religious practices among North China Hui by then were by and large Islamic. Hui is also used as a catch-all grouping for Islamic Chinese who are not classified under another ethnic group.