I usually sow extra dill and parsley seeds and designate out of the way plants for Black Swallowtail cats. You can also use Queen Ann's Lace or Fennel. I've also planted a wildflower called Zizia for their use. You can move the caterpillars to the designated plants to munch away.
Another FUN thing to do -- bring the caterpillars inside and raise them.
A large jar -- at least 1/2 gal. size pickle jar or canning jar might do. Line the bottom with a piece of paper towel cut to size. Have ready a piece of cheese/mesh cloth and a rubber band to use as a lid. You can also get a small plastic pet cage for less than $10 at a pet store. (We also reuse butterfly cages that were used to raise American Painted Lady butterflies)
Now pick that carrot top with the caterpillar on it and put in a water-filled narrow necked small bottle that fits in your jar. (I usually use spice jars -- stick the stems through the shaker top -- or brown tincture bottles). You don't want a wide-necked jar because the cats can fall in the water and drown. Provide carrot tops as you harvest them or as they eat them to sticks.
You'll want to clean the cage of the frass (poop) occasionally -- just fold up the corners of the paper towel to hold the frass in -- I usually put them in the compost. Replace with fresh paper towel if necessary. Don't forget to refill/replace the water in the jar.
The caterpillars grow through 5 stages called instars. They'll crawl -- usually up the side of the jar and sulk for a day, not eating, then molt out of their skins and will be a size bigger. Usually, their markings will be different too. Depending on what instar they are now, you may have opportunities to see them change. If you see tiny black caterpillars with a white band on them among your carrot tops, these are the 1st instar Black Swallowtail cats.
The cats are going to need a place to climb, though they may try to make chrysalis on a carrot stem -- usually not strong enough to support their weight. Provide a few sticks or strong grass stems. IF they do make a chrysalis on a flimsy stem, you can tie/tape the stem to a stick (or side of the jar). The chrysalis needs to be in a high enough location that the butterfly can hang under it and stretch their wings to dry without crumpling them.
If the chrysalis they make is green, they'll probably eclose (emerge) this fall. If the chrysalis is tan/dried grass brown, they'll overwinter in there -- just put the jar with the cheese cloth lid in an out of the way place (ours stayed on a dining room shelf. Didn't eclose until some time in late May.)