I wash them off with a stream of water from the garden hose, a technique that was recommended by my local Extension Service. You don't want such a powerful stream that it damages the plant, but you need enough force to wash the critters off. You will also have to do it periodically, to prevent new infestations.
You can use a soap solution, as you mentioned. I bought Dr. Bonner's liquid soap to use on my fuchsia recently, but any pure soap would work. Just be very careful not to use a detergent. Read the label before you use any product on plants. Cynthia recommended 1 tsp of soap per quart of water, and that seems to be working well. You will need to spray the plants every 7 to 10 days, to keep the bugs under control. You might have to wash off the spittlebugs before you spray. I'm not sure the soap solution will penetrate their protective wall of foam.