Khan was born on 21 March 1916 into a family of traditional Muslim Musicians in Bhirung Raut Ki Gali, Dumraon, in what is now the eastern Indian state of Bihar, as the second son of Paigambar Baksh Khan and Mitthan. While named Qamruddin at birth, his grandfather Rasool Baksh Khan, also a shehnai player, exclaimed "Bismillah" upon seeing the new born. Thereafter he came to be known by this name. His father was a court musician employed by the Raja of Dumraon. His great grandfather Ustad Salar Hussain Khan and grandfather Rasool Baksh Khan were also Musicians in the Dumraon palace.