Unfortunately, "or you can recognize that for some reason, some 'groups' are not improving to the extent we had hoped" is too often code for "it's all their own fault" (one need only spend a brief time reviewing the posts in this forum on this subject to confirm this).
Well if you repeatedly vote for the same party, and that party does nothing for you because they take you for granted, then yes, it is partly your fault.