Actually, Platforms 4&5 at Town Hall were planned from the start to be part of the Eastern Suburbs Line. When the City Railway was being constructed in the early 1920's, the planned route for the Eastern Suburbs Line was to run from underground platforms at Railway Square (from the Inner West) to Town Hall 4&5, and then to underground stations at Pitt & Market Streets (the present-day Pitt Street Mall), O'Connell Street, St James (unused platforms 2&3), Liverpool & College Streets (the present-day Whitlam Square) and then on to Bondi Junction via Taylor Square and Paddington. After the Central-St James and Central-Wynyard-North Sydney sections were opened in 1926 and 1932 respectively, construction of the ESR had not progressed beyond construction of the platforms at Town Hall and St James, as well as stub tunnels at each end of these platforms (those at St James being utilised for turnback facilities). When construction resumed in 1947 (which would stop in 1952, and not start again until 1967), it followed a new route, which bypassed St James, and eliminated the other proposed stations, running via Martin Place and King's Cross.
The intended route of the line south of Platforms 1&2 at Wynyard would have been to Pitt & Market Streets (at a lower level, and at an angle to, the Eastern Suburbs platform), then to a station beneath the intersection of Wentworth Avenue and Goulburn Street, and then to an underground platform at Central (beneath Chalmers Street).