I usually just hold my plants under the faucet and wash aphids off, but if you prefer, you can kill them with a solution of pure soap and water. "Pure soap" would be a product such as Dr. Bronner's, not something like dishwashing liquid or laundry detergent. Mix 1 teaspoonful of soap in 1 quart of water.
Spray the plant thoroughly from all directions, until it's dripping wet. Do that 4 times, at 5 to 7 day intervals. One treatment won't kill all the aphids, so you have to repeat the treatments 5 to 7 days apart. The same is true if you just wash them off, i.e. you will have to wash off the plant every 5 to 7 days. Four "baths" should do the trick.
The aphids probably came in on the plant you brought in for the winter.
"Isn't it enough to see that a garden is beautiful without having to believe that there are fairies at the bottom of it too?" - Douglas Adams