Probably BEFORE you need one but are due. If your work is a labor that was once fun, then it is time. Or how about, "when you want to."
Me personally, I enjoy it. It is a vacation on some level. But alas, spring is here and I will go and ski next week for the final runs of the season. Funny I feel "pressured" because the snow is going away. Yes, I know this is highly twisted. LoL
If you think you are "missing" opportunities, that is a sign. What do they say? The market will always be there. (well "always" might be a stretch) But the general comment is not to be overly needy to trade etc.
Probably the most relaxing vacation are ones where you "take a break from yourself". So for me that means not cooking nor caring about the food I eat or just caring to much about anything, within limits.
I like to fish the early season, ski the early and late season, and sometime I just stay home and do nothing I normally do. Quite liberating actually. And lastly physical exercise is key to a strong mind.