Actually, most mushrooms are not poisonous. Out of all the thousands of species, the number of harmful species are under a hundred.
Mushrooms are generally
a good sign of healthy soil. So that's a good thing to have it there. Even if the mushroom were poisonous, which is possible, I doubt enough of it could get onto the plant to upset your stomach, much less to hang onto the plant through a washing to make it to your plate in sufficient quantities to cause illness.
It's possible that the mushroom is keeping the soil at a right moisture balance by absorbing any excess. But definitely make sure the rosemary is not overwatered. My mother had a twenty year old rosemary that she claimed could not be killed, no matter how often she uprooted it and moved it around her yard. It was a beautifully gnarled plant.