If your party has a theme, find something that fits the theme. For example, we had a "bug" party for our daughter and gave out bug catchers. Water guns are a good favor for a water party. It is also fun to make the favors at the party. You could decorate picture frames or frost cookies to take home.
Bubbles are always a hit, as are crayons, markers, Play-Doh, balloons and candy. Many party supply stores have favors that go along with character-themed plates and napkins. Children also enjoy making a craft that becomes their favor. For instance, you could have all the children paint a terra cotta pot and plant flower seeds for an all-in-one craft/favor.