Perhaps Ron Paul said it best: "They are over here because we are over there." There was a whole lot more validity to that argument 20-30 years ago. Todays jihadi is a whole different animal.
Think of it in terms of the BLM klan. This group has nothing in common with the civil rights groups of the 50's and 60's. Then there were actual grievances. True racist oppression and discrimination, much of it state sponsored. Todays group has to fabricate these things where none exists. It's about revenge for them. Revenge for sins of the past that they don't experience.
ISIS and other modern day terrorists aren't in it for justice. This is the holey war, end times, all infidels must die. They buy this hook, line and sinker. There is nothing to negotiate.
We are act the preverbal fork in the road. A decision must be made. We can vacate the premises entirely and watch a genocide the likes of which has never occurred in modern history, and keep our fingers crossed that they just slaughter each other and leave us out of it. Unlikely, but possible. Or, we can enter this holey war full tilt and you're right. It will cost trillions of dollars and most likely tens of thousands of American lives. Millions of Muslims will be killed. The carnage will be horrific, and then as you state, we have to stick around for decades and the whole place might still go up for grabs.
My idea at the moment. Vacate the entire region and give fair warning. One terrorist attack on American soil and we nuke the entire region. Yes, I mean nuke. Burn the whole f'n place to the ground and hit the re-set button. Or we can just be nibbled to death by a duck.
Bottom line is, we aren't going to be able to just wish this away. Decisions will be made by us, or for us. Either way it's really messy.
I believe you are right regarding the shift. Today's Jihadists seem to fit your description. But we should think long and hard about being prepared to use nuclear weapons on an entire region. If we make the kind of threat you describe, then we have to be prepared carry through with it. Is that wise? I think probably not.