Smith was born in Baltimore, Maryland, the daughter of Anna Rosalind (née Young), an elementary school principal, and Deaver Young Smith, Jr., a coffee merchant. She has four younger siblings. Smith is an alumna of the historic Western High School. She then studied acting at Beaver College (now Arcadia University), graduating in 1971. She received her M.F.A. in Acting from the American Conservatory Theater in San Francisco, California.