When their nest with babies is touched, the tendency is to abandon their babies or move the baby birds to another nest.Also, they transfer the babies when the nest is in bad condition..like wet or dirty.
Your cardinals are probably moving their nests because they feel threatened in some way. Maybe crows or blue jays are bothering the nest. It could be anything that is causing them to be nervous enough to change their location.
Backyard cardinals birds, In this birds smelling sense is poorly. In this bird was not abandon to touch our babies and come nearest our next. so babies moving to another next.