Kovner has three children and has been married twice. In 1973, at age 28, he married Artist Sarah Peter, in a Jewish ceremony in Connecticut. They divorced in 1998. In 2007, he married Suzie Fairchild, the daughter of Robert Fairchild, and great-granddaughter of Louis Fairchild who founded the company with his brother, Edmund Fairchild, that became Fairchild Fashion Media, now a division of Condé Nast Publications.