Bismillah Khan was born on March 21, 1916 at Bhirung Raut Ki Gali, in Dumraon, Bihar as the second son of Paigambar Khan and Mitthan. He was named as Qamaruddin[1] to rhyme with Shamsuddin, their first son. His grandfather, Rasool Baksh Khan uttered "Bismillah" (the basmala) after looking at the newborn, thus he was named Bismillah Khan.

His ancestors were court musicians and used to play in Naqqar khana in the princely states of Bhojpur, now in Bihar state. His father was a shehnai player in the court of Maharaja Keshav Prasad Singh of Dumraon Estate, now in Bihar. Ustad Bismillah Khan rarely accepted students, but in 1985 he had a meeting with Satguru Jagjit Singh Ji (the present master of the Namdhari Sikhs), where he saw a talented young boy called Baljit Singh Namdhari playing the tarshenai, and welcomed him as a student. Ustad Bismillah Khan & Satguru Jagjit Singh Ji had great mutual respect for each other. In 1999 he adopted two more tarshenai students, named Kirpal Singh and Gurbaksh Singh Namdhari.

On August 17, 2006, Khan was taken ill and admitted to the Heritage Hospital, Varanasi for treatment.[7] He died after four days on August 21, 2006 due to a cardiac arrest. He was ninety years old. He is survived by five sons, three daughters and a large number of grandchildren and great-grandchildren.[8] The Government of India declared one day of national mourning on his death. His body was buried at Fatemain burial ground of old Varanasi under a neem tree with 21-gun salute from Indian Army.[9]