The quality starts to suffer at the 1st signs of yellowing. Every issue, big or small will effect the yield and finished product. If you start removing the damaged leaves now, they wont have any leaves left to get them through the next few weeks. Removing the leaves with out treating the issue will just cause it to spread faster because the plant is still feeding off of the leaves since there is an issue (most likely a lockout) in the soil. If you want to chop em all now, maybe think about using them for something other than buds to smoke. Maybe freeze the pants and/or make bubble hash or something.