Pullman was born December 17, 1953, in Hornell, New York, the son of Dr. James Pullman Jr. (1914–1992), a physician, and Johanna Wilamina Pullman (née Blaas; 1911–1993), a nurse.
After graduating from Hornell High School in 1971, he attended the State University of New York at Delhi (SUNY Delhi) and the State University of New York at Oneonta (SUNY Oneonta) in the 1970s, receiving a Bachelor of Arts degree in Theater Studies from the latter in 1975. He eventually received his Master of Fine Arts degree from the University of Massachusetts Amherst in 1980. He taught theater at SUNY Delhi and was an adjunct professor at Montana State University's School of Film and Photography in Bozeman, Montana, where his students persuaded him to pursue film roles.