Milne was born in Hull, Yorkshire, England. He attended Hymers College and from there he won an open scholarship in mathematics and natural science to study at Trinity College, Cambridge in 1914, gaining the largest number of marks which had ever been awarded in the examination. In 1916 he joined a group of mathematicians led by A. V. Hill for the Ministry of munitions working on the ballistics of anti-aircraft gunnery, they became known as ′Hill's Brigands′. Later Milne became an expert on sound localisation. In 1917 he became a lieutenant in the Royal Navy Volunteer Reserve. He was a fellow of Trinity College, Cambridge, 1919–1925, being assistant Director of the solar physics observatory, 1920–1924, mathematical lecturer at Trinity, 1924–1925, and university lecturer in astrophysics, 1922–1925. He was Beyer professor of applied mathematics, Victoria University of Manchester, 1924–1928, before his appointment as Rouse Ball Professor of Mathematics and to a fellowship at Wadham College, Oxford, in 1928. Milne's earlier work was in mathematical astrophysics. Much of his research in the 1930s was concerned with the theory of relativity and cosmology. His later work, concerned with the interior structure of stars, aroused controversy. Milne was President of the Royal Astronomical Society, 1943–1945. During World War II he again worked on ballistics.