Plagiarism is quite an accusation to weigh at somebody, the question is whether he used information that was available to him to draw inspiration for a novel or whether he copied outright. I don't believe he did copy for this book though.
Angels and Demons is not plagiarized, Dan Brown was taken to court for plagiarizing his follow up novel to Angels and Demons, The Davinci Code, however the case was thrown out and he was cleared of plagiarizm.