Good question. In the early days there seemed to be no imaginable limit on how big our population should be, except perhaps among extreme environmentalists and such who love solitude. It's hard to answer, everyone has a different feel for it. To me, there are too many people in the US, particularly east of the Mississippi. But I'm a bit of a crank Westerner that way. But I imagine many residents of teeming NYC wouldn't have it any other way.
Even if it weren't an issue of land management, unless every new taxpayer is giving more than they're taking, it's not sustainable.