Yes, you should be concerned because unlike normal people you cant take over the counter medication. your fever, self medication or medicines used by non pregnant could lead to undesirable effects.
Dont get upset, before taking any over-the-counter medicines concert your doctor about temperature and symptoms. Take more fluid, take rest, don't cover you up with heavy blanket.