If you are concerned about giving out your bank acct numbers open a seperate account with a different bank and only fund it for the deposit of funds to the broker, then keep only a minimum bal in the account.
when you want money from the broker have it deposited into that account then withdraw...