Less people is especially problematic for Ontario. A lot of the meagre GDP growth in Ontario of late has been population growth, not per capita growth. Take away population growth and there goes some GDP growth and the ability to pay for a ballooning deficit.
Probably the most dangerous thing for Ontario with regards to reduced immigration is reduced demand for housing. Immigration has been propping up Ontario's housing market. If population growth stalls, demand for new housing will crater and existing housing prices will stop rising and perhaps even start to drop. That's the doomsday scenario for the Ontario economy.