Wren's design for the original building was intended to house 412 veteran soldiers and their officers and was made up of a single quadrangle, known as Figure Court, surrounded on two sides by the accommodation blocks and on a third by the Great Hall and Chapel. The pavilions at each corner provided for the kitchen and infirmary and officers' accommodation.

Even before work had begun it was realised that the buildings would be insufficient and Wren added two further quadrangles to his design. In 1686 the construction of the new quadrangles on each flank was approved (now known as Light Horse and College Courts).