The federal government prints, mints, and distributes out the currency that we use on a daily basis. That money therefore belongs to Caesar -to the federal government, not the people. It is the government's money, not the people's.
Jesus was not saying that all your money belongs to Caesar - much of the economic activity of Jesus' day did not involve coins with Caesar's image. Jesus was suggesting to pay your taxes to Caesar and your tithes to God. (The coins that were used for tithes did not have Caesar's image on them, but the coins that were used for taxes did.)
Jesus is setting out a conservative and conciliatory view - the radical view was to resist. These taxes were oppressive and so it's hard to imagine Jesus approving of them considering what he said and did about similar oppression by the Temple system - but Jesus was not interested in stirring up political revolt, and he wasn't interested in getting into an argument with his opponents about it.
This passage does highlight something about conservatives - which is the belief that different institutions play different roles. For example, there is a way to help the poor, and a way not to.