Generally no, all you need to do is remove the panel on the inside that is covering its mount point, unbolt it, disconnect any linkage or wires. Connect the new mirror, and re-bolt. Then replace the interior panel.
No, it is quite easy. My husband and I replaced our Jeep's side mirror last year. It took more time to get the part from the dealership than to make the repair. We took off the door panel and removed several screws to remove the old one; set the replacement and screwed it down.
This question is a matter of opinion. I would say that replacing any side mirror is easy. Just pull off door plastic, then unbolt mirror, unplug any connectors and replace.