Well if you are on a real tight budget then even the local superstore should have an art and craft section. Otherwise you could try sourcing your goods from china.
Yard sales and swapmeets are the best places to find things cheap. Also check clearance frequently at stores. Usually craft stores are extremely over priced so avoid those at all costs.