If the fever does not go away within a short time, make an appointment with the child's pediatrician. In the meantime, wipe the baby's face with a cool face cloth. Do not give the child aspirin or other medication. The aspirin could cause other problems. Possibly something worse than the child has.
While a fever in a baby can be benign, you should always contact your pediatrician for guidance. You can give Motrin, or Tylenol. Remove heavy blankets, and tepid bath, also.
Tylenol or ibuprophen are usually good ideas to help make them a little more comfortable. They could be teething or getting a cold. If they begin to have seizures, bluish or purple skin, unresponsiveness or lethargy you should contact a doctor or emergency room immediatley.
You could give the baby tylenol baby medicine but the best thing to do is make an appointment with your doctor and let him see the child to recommend treatment for the child.