Oswalt was born January 27, 1969, in Portsmouth, Virginia, the son of Carla (née Runfola) and Larry J. Oswalt, a career United States Marine Corps officer. He was named after General George S. Patton. He has one younger brother, Matt Oswalt, a comedy Writer best known for the YouTube web series Puddin'. While he was a military brat, his family lived in Ohio and Tustin, California, before settling in Sterling, Virginia. He is a 1987 graduate of Broad Run High School in Ashburn, Virginia, and subsequently graduated from The College of william & Mary, where he majored in English and was initiated into the Alpha Theta Chapter of the Phi Kappa Tau Fraternity.