Cynthia, I think what you're talking about is on more of a commercial level. I don't believe one person getting a little compost for their back yard can possibly take enough for the forest floor to notice the difference. I agree too much removal would be a very bad thing. But even getting a pickup truck load would be almost too much work to be worth it. Small scale I think is fine.
USDA Zone 7a, Sunset Zone 32.
Failure is only a fact when you give up.