It depends on whether they are fresh water or salt water. Fresh water fish generally just absorb water through their skin, but salt water fish do need to drink water.
It may sound silly and many people would say-no it would be stupid to say humans drink air. The fact is however that osmosis, the diffusion of water into the cells, is a process similar to drinking and so fish do in a sense undergo this process constantly.
Salt water fish actually do drink water, filtering out the salt. Fresh water fish do not drink water, there bodies absorb what they need. They all swim in water, though!