Because 70-80% of children reside with the mother or shared custody including mother. It's assumed that there had to be a bad reason why the mother didn't get custody.
It is because a mother is expected to always be the very best role model, supporter and an all together better care giver. People see women as a superior parent.
People feel that there must have been a major failing in the history of the relationship and in the mother's raising/treatment of her children to make her loose them.