James Thomas Fallon was born in the Bay Ridge neighborhood of New York City's Brooklyn borough on September 19, 1974, the son of Gloria (née Feeley) and James Fallon. His paternal grandmother was a German immigrant from Osterholz-Scharmbeck, while his maternal grandmother's father was a Norwegian immigrant from Fredrikstad. Another set of his great-great-grandparents were an Irish couple from County Galway, with this great-great-grandmother herself being born to an Irish couple in France.
Fallon's father spent his adolescence singing in street-corner doo-wop groups then served in the Vietnam War. Shortly after his son's birth, he started working as a machine repairman for IBM in Kingston, New York. In preparation, the family moved nearby to Saugerties, New York. Fallon has described his childhood as idyllic, while his parents have been described as overprotective. He and his sister, Gloria, were unable to leave their home and had to ride their bicycles in the backyard. Fallon attended the Roman Catholic school St. Mary of the Snow. He considered becoming a priest, inspired by his experiences as an altar boy, but became more interested in comedy instead. He spent many nights listening to the radio program The Dr. Demento Show, which exposed him to both comedy and music; he often recorded it on a reel-to-reel recorder.
As a teenager, Fallon developed an obsession with the late-night comedy program Saturday Night Live. He watched it religiously, although he was only allowed to see "the clean parts" that his parents taped for him. He and Gloria would re-enact sketches such as "The Festrunk Brothers" with friends. In his teens, he impressed his parents with impersonations, including of actor James Cagney and comedian Dana Carvey. He was musically inclined and started playing guitar at age 13, going on to perform comedy and music in contests and shows. By his junior high years, he was labeled a class clown but was also described as "nice and well-mannered".
At Saugerties High School, Fallon was a performer in most stage productions and was twice a class social director. He won a young comedian's contest with an impression of Pee-wee Herman. He graduated in 1992 and then attended The College of Saint Rose in Albany, New York, where he was a computer science major before switching to communications in his senior year. He was an average student who would perform stand-up comedy on weekends. He would often board buses from his aunt's house in Fort Hamilton to perform sets at Carolines on Broadway in Times Square. He did not graduate, leaving college a semester early to pursue a comedy career.
Fourteen years later, in May 2009, Fallon returned to receive a Bachelor of Arts in communications, awarded by Saint Rose officials who granted him experiential learning credits for his television work. He joined his classmates at the Saratoga Performing Arts Center to collect his degree, where along with his BA, he was also awarded an honorary Doctor of Humane Letters.