Cavalcanti was born in Rio de Janeiro, the son of a prominent mathematician. He was a precociously intelligent child and, by the age of 15, was studying law at university, but was expelled following an argument with a professor. His father sent him to Geneva, Switzerland, on condition that he did not study law or politics. Cavalcanti chose to study architecture instead. At 18, he moved to Paris to work for an architect, later switching to working in interior design. After a visit to Brazil, he took up a position at the Brazilian consulate in Liverpool, England.
Cavalcanti corresponded with Marcel L'Herbier, a leading light in France's avant-garde film movement, which led to a job offer from L'Herbier for Cavalcanti to work as a set designer.