Yeah like they said, botrytis is highly mobile, I have dealt with it in greenhouses, it is a scourge I have yet to experience on my plants. (fingers crossed).
Now I *believe* the conditions have to be right for it to actually colonize and damage your other plants, but I would not take the chance myself.
@trailanimal is dealing with it right now maybe he can offer insight for the outdoor stuff.