I usually put the bigger ones on the bird feeder.
Tiny ones, I'm too clumsy to even try picking up -- mush...
My DD7 insisted on raising a couple. One pupated and eclosed from the chrysalis after a few days. I had to let her release it.
The other one looked like it was going to pupate but instead turned into a mass of yellow lumpy pupa, and a swarm of tiny black wasps eclosed a few days later. I released THEM of course.
So I forgot to mention *those* tiny beneficial parasitic wasps. They look a bit like winged green aphids, except they're black and have long antennae. I wish I knew what they're called -- I thought they were trichogramma, but the Google images don't match the ones I saw.
Edited to add: OK They looked more like braconids but these didn't start out with the classic pupae-on-cat's-back. They came from inside the cat.