Sure sounds like dirty contacts. A hundred years ago, when I was young, we used to fix dirty radio contacts by moving the knob back and forth, rapidly, while pushing on the knob, until we "sanded" the dirt off.
I don't know if speed controls today still use contacts.... but ....if you can, move the control rapidly from low to high, changing the position of the control while you are doing it.
If that doesn't work, you might try spraying some contact cleaner into the area. If you are spraying contact cleaner, be sure that the power is off and you don't turn it on for an hour or so. You don't want to short anything out needlessly.
None of this will work if the speed control is all electronic. If so....I have no ideas.
richg99