Personally I would leave the nests alone. We've ruined so much wildlife habitat that if a bird has decided to use something of mine they are welcome to it.
Leave them alone if possible and steer clear of the area until the baby birds have fledged to avoid scaring away the parents. Hummingbirds are a treasure to behold.
You really shouldn't disturb them until the hatchlings have left the nest after that you can just remove them and if you remove them entirely they will not nest there when they come back next year but if you remove the nests with the babies still in them the adults will abandon them.