The first step in the budget process is the President submits his request, with his guidelines on how much should be spent. I'm sure if the president submitted a balanced budget request, congress wouldn't add hundreds of billions or trillions to it to create a large deficit.
Also, comparing now to times past is not an aples to apples comparison. There was a gold standard during the times you mentioned, while there is no gold standard now. Also, as the budget was essentially balanced in previous years, there was not as much interest already being paid for old debt as there is now. Thirdly, the social programs were in their early stages, such as more than 10 workers per retiree not 2 or 3 workers per retiree like now. And last, America was a producing/manufacturing economy then. With Europe's and Asia's productive capacity bombed to bits, it was not hard to switch the US workers from making tanks and planes to making other goods, which the rest of the world had to buy for a time.
I know the Prez doesn't need to sign the budget, but I'm sure members of his party make sure his goals are enacted to some extent. Also, the world now is not like the post WWII world where America was the only major country not to have it's manufacturing base seriously damaged.