OK and if you don't earn back your fortune, so what? Are you homeless? No. Happiness in life comes from acceptance of your situation and dealing with it, not avoiding pain. The fact is that you got caught up in a shitty situation is a confluence of your own bad decision making and what society has deemed appropriate for broken families.
That being said, the state these days will usually not block you from seeing your kid **even in the case of alleged or true domestic violence**. Many states are defaulting to shared parenting, joint custody. The days of high conflict divorce are now only dependent on high conflict people and the science says that if you married a high conflict woman, then guess what? You dug your own grave.
I married a high conflict woman, and divorce went pretty much as expected. 6-7 years of court, multiple agreements not adhered to (by her), etc. etc. This is the tax I paid for making a mistake many years ago. We've been divorced longer than we've been married and it's still ongoing in various ways. You see, high conflict people cannot fathom that no one wants to be with them, they believe the other party must be wrong.
I have maintained my freedom to do what I want, and yes it has cost me a lot in dollar value, but there is no situation I can fathom in which I cannot go anywhere, rent a room for $500-$1000/month and live out my life in peace, running every morning, hiking every weekend and generally just enjoying life.
No this is not ideal. Ideally, I would have no restrictions on my future success, but the fact is that compounding is difficult if you have to constantly access capital to defend against the latest accusation in court. I also like to travel.
A fortune is less important than living the live you want. What good does $10,000,000 in the bank do if you can't enjoy your life anyway? Or if your kids are unhappy?