Elfman was born on May 29, 1953 in Los Angeles, California to a Jewish family of Russian descent. He is a son of Blossom Elfman (née Bernstein), a writer and teacher, and Milton Elfman, a teacher. He is the brother of actor, musician, and journalist Richard Elfman. Elfman was raised in Baldwin Hills, an affluent neighborhood where he spent much of his time at the local movie theater discovering classic sci-fi, fantasy and horror films and first noticed the music of such film composers as Bernard Herrmann and Franz Waxman. Elfman has admitted to fabricating stories about his past out of boredom, including a false birthplace of Amarillo, Texas, and parents who served in the United States Air Force.
In his early school days, Elfman exhibited an aptitude for science with almost no interest in music, and was even rejected from elementary school orchestra "for having no propensity for music." This would change when he switched high schools in the late 1960s and fell in with a musical crowd, who introduced him to early jazz and the work of Stravinsky and his 20th-century contemporaries. Elfman attended University High School in Los Angeles before dropping out.
Elfman left school to follow his brother Richard to France, where he performed violin with Jérôme Savary's Le Grand Magic Circus, an avant-garde musical theater group. He then embarked on a ten-month, self-guided tour through Africa, busking and collecting a range of West African percussion instruments until a series of illnesses forced him to return home. At this time, Richard was forming a new musical theater group in Los Angeles.
