Duryea was born and raised in White Plains, New York. He graduated from White Plains High School in 1924 and Cornell University in 1928. While at Cornell, Duryea was elected into the Sphinx Head Society, Cornell's oldest senior honor society. He majored in English, and in his senior year succeeded Franchot Tone as president of the college drama society.
Because his parents did not approve of his choice of an acting career, Duryea became an advertising executive. After six years, he had a heart attack that sidelined him for a year.
