Dzundza was born in Rosenheim, Germany, to Ukrainian father Roman Dzundza and Polish mother Maria Humenecka, who respectively originated from Kalush and Lviv in what is now Ukraine. After the Nazis forced his parents into factory labor, Dzundza spent his early years living in displaced persons camps with them.
Dzundza's family moved to Amsterdam in 1949, then moved to the US in 1956, settling in the Lower East Side of Manhattan in New York City. He became a naturalized US citizen. He attended Xavier High School and St. Johns University, and studied under Stella Adler and Harold Clurman.