Harbour was born in White Plains, New York, to Kenneth and Nancy (née Riley) Harbour, both of whom work in real estate—his mother in residential and his father in commercial. He attended Byram Hills High School in Armonk, New York, along with actors Sean Maher and Eyal Podell. He graduated from Dartmouth College in Hanover, New Hampshire in 1997, where he majored in drama and Italian and was a member of the Sigma Phi Epsilon fraternity.
As a young man in New York City, David Harbour frequented and participated in gambling at underground poker clubs, and attests that he personally knew the gangster who John Malkovich's character "Teddy KGB" was based on in the 1998 film Rounders.
