Leahy was born in Montpelier, Vermont, the son of Alba (née Zambon) and Howard Francis Leahy. He has been legally blind in his left eye since birth. Leahy's maternal grandparents were Italian, and his father was of Irish ancestry; some of his ancestors came to Vermont in the 19th century to work at the granite quarries and manufacturing plants in Barre Town and Barre City. The Leahys ran a printing business across from the Vermont State House, and were also the publishers of the Waterbury Record newspaper. Leahy attended the parochial schools of Montpelier, and graduated from Montpelier's St. Michael's High School in 1957.
In 1961, Leahy graduated from Saint Michael's College with a Bachelor of Arts degree in government. While attending college, Leahy was a member of the National Federation of Catholic College Students. He was also involved with the Knights of Columbus, the Saint Michael's College Glee Club, and the school's pre-law society and politics club. He was active with the Reserve Officers' Training Corps rifle team, and was a member of the varsity rifle team. He was also on the staff of The Shield, the Saint Michael's College yearbook and WSSE, the school's AM radio station. In 1964, Leahy received his Juris Doctor from the Georgetown University Law Center. While in law school, Leahy was active in the Phi Delta Phi legal honor society. In addition, Leahy participated in Georgetown Law's Legal Aid Society and Legal Argument Program. He was also a representative to the school's Student Bar Association.
At graduation, Leahy was offered an E. Barrett Prettyman Fellowship, which would have enabled him to earn a Master of Laws degree from Georgetown while receiving training in courtroom advocacy. Leahy was also interviewed by the United States Attorney General, Robert F. Kennedy as one of several Georgetown Law students being considered for recruitment into the United States Department of Justice. He declined the fellowship and did not pursue a position at the Department of Justice because he intended to return to Vermont, and because he did not plan to practice criminal law.