It may be due to some allergy like food allergy or allergy from medicine you took. Stop using soap and try some alternate substitute or chane the brand of soap.
A possible explanation is that the rash is gone from sight but in the mind there may still be attachment to the idea of the rash which caused the itchiness in the first place.
If it was a rash due to a bug bite it is usually a mild allergic reaction to the liquid the bug injects into your blood stream. You could try anti-itch cream and see if that helps or take a semi cold shower to get rid of the itch.
