Those mice are looking for food. Seal it away in glass jars and metal canisters to discourage them. Also seal up the cracks and crevices that they enter through. Use caulk, and smooth it down so they can't pry it out of place. Cats discourage mice, but cover cat food tightly or mice will come after it.
Some hardware and sporting supply stores have just the thing you need. It's a much smaller version of traps used to catch varmints alive. Just bait it with a strong cheese and the mice will come. As soon as they start to nibble on bait the trap doors closes.
Introduce a cat to the area or conern. Cats are naturally intuitive, and small things that move around like mice easily catch their attention, they will be able to rid you of the mice if that is you goal. Cats generally also enjoy brining home presents to their family and you will soon find out that mice make great presents.
Steel wool will prevent mice from entering through crevices and small holes. Also, mice are irritated, but not harmed, by the smell of peppermint. You can use peppermint oils or candles to rid your home of the rodents.
Mice can be controlled by using fresh or dried mint. Do not leave food out overnight and put away all pet food. A cat can also serve to control the mice population in the home.