The nickname dates from a 1970s-era municipal tourist promotion in which people were given bracelets and awarded "charms" when they visited local landmarks.
Baltimore ran an ad campaign in the 1970's called "charm city". Visitors were given a charm bracelet and encouraged to purchase charms, and the moniker stuck for all time.
The nickname was deliberately concocted by a promotional type named Bill Evans in 1974, along with an abandoned scheme to give out charm bracelets. For some reason the name stuck.