It usually costs more to go to places where not a lot of people are travelling. They have to run a plane and they can't cover the cost if not many people use their service. It's simple business.
Nigeria is not a major tourist or business destination for travellers coming from outside of Africa. Low demand means low volume, which means high prices for the few seats available.