I think Fireman Dave is alluding to the fact that when trackwork shuts down the main south line between Moss Vale and Glenfield, all trains are diverted via Wollongong and the Illawarra track.
In these cases of needing to divert through this route, there simply isn't enough paths for all trains (freight, pax or whatever) - so they choose to run the XPT this way, but they amex the Canberra passenger service as it cant get the paths required at the relevant times (which would theoretically mop up six paths a day, being three trains each way each day over a 24 hour period). Time wise, it may also materially delay the Canberra pass for its relatively shortish journey and quick turnarounds at each end, whereas the XPT isn't as affected as much over the bigger distances in which it theoretically could make up time (ha ha).
Overall, its available pathing that is the biggest problem. Fireman Dave wasn't saying that the XPT conventionally run this way 'all the time' at all, I think he just means during track work periods.