Gielgud was born on 14 April 1904, in South Kensington, London, the third of the four children of Frank Henry Gielgud and his second wife, Kate Terry-Gielgud, née Terry-Lewis. Gielgud's elder brothers were Lewis, who became a senior official of the Red Cross and UNESCO, and Val, later head of BBC radio drama; his younger sister Eleanor became John's secretary for many years. On his father's side, Gielgud was of Lithuanian and Polish descent. The surname derives from Gelgaudiškis, a village in Lithuania. The Counts Gielgud had owned the Gelgaudiškis Manor on the Nemunas river, but their estates were confiscated after they took part in a failed uprising against Russian rule in 1830–31. Jan Gielgud took refuge in England with his family; one of his grandchildren was Frank Gielgud, whose maternal grandmother was the Polish actress, Aniela Aszpergerowa.
Frank married into a family with wide theatrical connections. His wife, who was on the stage until she married, was the daughter of the actress Kate Terry, and a member of the stage dynasty that included Ellen, Fred and Marion Terry, Mabel Terry-Lewis and Edith and Edward Gordon Craig. Frank had no theatrical ambitions and worked all his life as a stockbroker in the City of London.
In 1912, aged eight, Gielgud went to Hillside preparatory school in Surrey as his elder brothers had done. For a child with no interest in sport he acquitted himself reasonably well in cricket and rugby for the school. In class, he hated mathematics, was fair at classics, and excelled at English and divinity. Hillside encouraged his interest in drama, and he played several leading roles in school productions, including Mark Antony in Julius Caesar and Shylock in The Merchant of Venice.
After Hillside, Lewis and Val had won scholarships to Eton and Rugby, respectively; lacking their academic achievement, John failed to secure such a scholarship. He was sent as a day boy to Westminster School where, as he later said, he had access to the West End "in time to touch the fringe of the great century of the theatre". He saw Sarah Bernhardt act, Adeline Genée dance and Albert Chevalier, Vesta Tilley and Marie Lloyd perform in the music halls. The school choir sang in services at Westminster Abbey, which appealed to his fondness for ritual. He showed talent at sketching, and for a while thought of scenic design as a possible career.
The young Gielgud's father took him to concerts, which he liked, and galleries and museums, "which bored me rigid". Both parents were keen theatregoers, but did not encourage their children to follow an acting career. Val Gielgud recalled, "Our parents looked distinctly sideways at the Stage as a means of livelihood, and when John showed some talent for drawing his father spoke crisply of the advantages of an architect's office." On leaving Westminster in 1921, Gielgud persuaded his reluctant parents to let him take drama lessons on the understanding that if he was not self-supporting by the age of twenty-five he would seek an office post.