Why would it be weird? As long as the two parties involved are comfortable, it isn't anybody's business to decide what is and isn't "weird." There are no requirements for height for people who are in a relationship.
I think it's much more important that you get along with each other than how big the height gap is between the two of you. It might look a bit odd, but ask yourself, would I rather not be with this person just because of that?
