Wait until after dusk when the hornets have returned to the nest. Use a smoky torch to smoke around the nest to calm them down and spray insecticide to soak the nest. The next day you can knock it down.
Make sure your not allergic you might get stung regardless of how cautious you are. Always spray early in the morning preferably on a cool day when they all home.