If you get rid of the aphids, the ants will usually go away (though apparently there are some exceptions to that discussed above). And some times the ants are "farming" the aphids and will bring some back to restart their "farm."
Anyway, I would start with the aphids as they are easy. They are slow and stupid and just sit there and let you squish them. So you can just go over your plant and squish them, if your garden is manageable size. Otherwise you can spray with soapy water (just being sure it is real soap, not detergent, dishwashing liquid is usually detergent which can burn your plants).
Ladybugs are the best predator for them; you can buy them on line for release in your garden.
Here's a previous thread about aphid control:
Get rid of the aphids and see what happens. If you still think you need to get rid of the ants, come back here and type ant control into the Search the Forum Keyword box.