Oliver started in amateur theatricals as an adolescent in Britain before pursuing a career as a jockey. However, he failed to gain an apprenticeship owing to his size. He joined the Merchant Navy at 16 and travelled the world, eventually settling in Sydney in 1956. In Australia he found work as a stockman, spending three years in this job, working his way around the country. He subsequently returned to the UK before beginning an overland trek across Asia to return to Australia. However, the journey was halted by illness and he returned to Britain where Oliver contacted Albert Finney who was then planning to produce the film Ned Kelly in Australia in 1963. Finney gave him a letter of introduction to an agency in Sydney and Oliver emigrated in 1963, later becoming an Australian citizen. Oliver became a busy theatre and television actor in Australia. He had many guest starring roles on Australian drama series, appearing frequently in the top-rated Crawford Productions police dramas Homicide, Division 4, Matlock Police and Cop Shop, and in Crawford's adventure series Hunter (1967). Oliver was then cast in serial Bellbird, playing the role of Tom Grey from 1969 to 1971, and appeared in the 1971 film Nickel Queen, directed by John McCallum. After this he returned to television guest roles for Crawfords, and other companies. He also played two roles in the Gerry Anderson series UFO – a Doctor in the episode entitled "Confetti Check A-OK" as well as a SHADO technician in the episode entitled "The Sound of Silence".