Excuse me, my man, but you need a different accountant.
Service income is sourced in the location where the service is provided. Period.
If you sell any goods, that would be different.
The "state exemptions" you cite are federal law that all states must follow, which is why all 50 states have identical such exemptions.
Interstate commerce is such an exemption, as it is a federal power and not a power that the states can regulate. That's why that exemption exists.
Note also that the first exemption you cite only applies where there is a physical presence within Georgia. Even then, OP's situation would fall within the exemption. If there is no physical presence, it is interstate commerce.
Exemption means the state has no legal authority to tax it. This is not a freebie by the state. It is a matter of settled federal and state law because the US Constitution says so.
Regarding Bahamas, it was excellent advice if and only if a person wants to get fancy, which is what I said. By fancy, I mean that one has to learn what the rules/laws are. One would have to be versed on forms 926, 1120F, 5471, and possibly Treas. 90-22.1 and FATCA. If (and only if) one wants to become versed in this part of Title 26, it could be worth it. It is not for everyone. Clearly, it is not for you.
To form an S-Corporation, one would have to be familiar with forms 2553, 1120S, and Schedule K/K1. It is not much different, but the penalties are more severe if it is done wrong.
First post or not, the sourcing of income is well-established law. If you choose to pay taxes you do not owe, that is your business. If you have other factors that apply to your business that you did not mention and if those factors require you to pay taxes in multiple states, that is a different issue.
But the source of service income is where the service is provided. The location of the customer is irrelevant. As the OP described, he has no legal income tax requirements in Georgia.