I'm pretty sure it's a joke to refer to some form of mysterious, unknowable power conducted by secretive practitioners or something. However, if you're gonna get upset about it and accuse people of being bigots then you should probably be more specific about which religion you're referring to and what's wrong with their statement. There are a number of religions that get called "voodoo" like Louisiana Voodoo, Haitian Vodou, Hoodoo, and Juju. Some of these religions encourage secrecy, others don't. Some incorporate magic, others don't. Some use talismans, others feature spiritual possession, and so on.

Edit: like, I'm not saying you're necessarily right or wrong, but you gotta be more specific if you want to clear up misconceptions, and you have to make sure you have your facts straight. You can't just say, "X is wrong/bad" and expect everyone to go along with it if they can't see why it's wrong or bad. Yes, you could just tell people to shut up and get with the program, but the kind of people who are willing to just "shut up and get with the program" probably are not going to be strong supporters because they don't really understand what they're supporting.