William Waldorf Astor built a hotel beside his aunt's house over who was "Mrs. Astor", and her son built a taller one on hers

After Charlotte Astor died in 1877, Caroline Astor, wife of William B. Astor Jr., insisted on being called simply "Mrs. Astor". Her nephew William Waldorf Astor said the title belonged to his wife, because he led the senior branch of the family. In February 1890 his father, John Jacob Astor III, died. William Waldorf inherited a fortune that made him the second richest man in America. He tore down his father's house at Fifth Avenue and 33rd Street and built the 13-story Waldorf Hotel next to his aunt's house. It opened in 1893. She lived beside the building site for about three years, then moved up Fifth Avenue. Her son, John Jacob Astor IV, pulled down her house and opened the 16-story Astoria on the site in 1897. The two hotels were joined as the Waldorf-Astoria. William Waldorf had moved to England in 1891. In 1892 his staff told American reporters he had died of pneumonia; it was false, and the press mocked him. He became a British subject in 1899, a baron in 1916 and a viscount in 1917. He died on 18 October 1919, and his ashes are in the chapel at Cliveden. The Waldorf-Astoria was sold and demolished in 1928 to make way for the Empire State Building.