Anders was born in Ashland, Kentucky, to mother Alberta "Rachel" Anders (née Steed) and father Robert "Bob" Anders. She has two sisters, one of whom, Luanna Anders, starred in her first film, Border Radio. Her paternal side has ancestry that traces back to the Southern Hatfield family and, more distantly, to George Washington's spy, Caleb Brewster, while her maternal side includes another Washington spy, Abraham Woodhull.
When Anders was 4 years old, her father abandoned the family. Anders' mother and father were divorced when she was 5. At age 12, she was gang raped by three boys at a party in Cape Canaveral, Florida, an event that influenced several of her films. After her mother moved her and her sisters to Los Angeles, Anders suffered a mental breakdown at the age of 15 and was hospitalized. When she came out of the psychiatric ward, she was placed into foster care but ran away. She hitchhiked across the country, at one point ending up in jail. After turning 17, Anders dropped out of her Los Angeles high school and moved back to Kentucky. She later moved to London with the man who fathered her first child.
In her early 20s, Anders moved back to Los Angeles with her daughter and attended a junior college, Los Angeles Valley College, while working odd jobs. Due to constant relocation as a child, Anders had not had a steady education. She said that growing up, most of her time was spent watching TV and going to movie theaters. Inspired by the films of Wim Wenders and other filmmakers, Anders applied to UCLA Film School. During her time at UCLA, Anders produced her first sound film. Wenders attended the screening. She has called Wenders' 1974 film Alice in the Cities "one of my very favorite films, and a guiding light, since I first saw it at the Nuart (Theatre) in Santa Monica in the 1970s." In 1986, Anders got her B.A. in Motion Picture-Television from UCLA.
