The bylaws and rules of the Homeowners Association will provide for the removal of a trustee. Details will include specifics that allow for consideration of removal as well as the manner in which the issue is brought forth and addressed. There will also be requirements for the manner in which the decision is made.
Before removing a condo trustee make sure everything has been done to assure the trustee has had every chance to improve. Finally, check your associations governing documents for guidelines.