Israel experienced the exact same problem, less than 3 generations after inheriting their Promised Land, although they were harassed by the neighboring Midianites, not the descendents of their former servants (Africans).
The cause was the same, and the solution every bit as difficult for us...