Do these bugs affect other trees? Should I move it away from other plants?
Yes and yes.
To verify that you do have Spider Mites get a piece of paper and hold it under the foliage and then tap or shake the tree. Look for little moving dots.
Spider Mites dislike moisture so the first thing I would do is to hose them off the tree trying to dislodge them physically.
A dormant oil or insecticidal soap can be used as well.