William John Hunter was born on 27 February 1940 in Ballarat, Victoria, the son of William and Francie Hunter. He had a brother, John, and a sister, Marie Ann.
During his teens, Hunter was a champion swimmer, and briefly held a world record for the 100 yards freestyle until his record was broken by John Devitt in the very next heat 10 minutes later. Hunter qualified for the Australian swimming team for the 1956 Summer Olympics in Melbourne, Australia, before a bout of meningitis ended his Olympic hopes.