Connor was born on 11 February 1955 in Chichester or Bognor Regis, both in Sussex, England. From 1966 to 1972, he was educated at Christ's Hospital, then an all-boys independent school in Horsham, Sussex. Having been expelled from Christ's Hospital, he attended Bognor Regis School, a comprehensive school in Bognor Regis. In 1973, he matriculated into Wadham College, Oxford to study English; his tutor was Terry Eagleton. He graduated with a first class Bachelor of Arts (BA) degree in 1976. He remained at Oxford to study for a Doctor of Philosophy (DPhil) degree in English. He completed his doctorate in 1980 with a thesis titled "Prose fantasy and myth-criticism 1880–1900".