The difference between socialism and capitalism lies with means of production. It could be financial production or industrial production. In socialism, the means of production are owned by the state -- a government entity. In capitalism, the means of production are owned by private individuals or corporations. In the United States, less than 2% of production is state-owned.
In pure capitalism, taxes are low and government services are practically nonexistent. Under socialism, taxes are high, but the government provides schools, health care, housing, transportation, etc.