It really depends on the type of worm you have, because there are two basic types. 1) the ones the live in the leaf litter 2) the earthworms that bury into the ground. They both look very similar, especially the uninitiated. If you put the leaf-litter worms in your garden (usually referred to as Red Wigglers) they will die if you don't have a relatively deep layer of mulch, because they can't dig deep into the soil, they feed (and live) on what's on top of the soil, so if there's not a heavy layer of mulch they will die from sun exposure and if in the shade they will just dry out -- they also need the moisture in the mulch to stay alive.
If you're not sure of what type of worm, you just need to put it in a heavily mulchy area, that way if they are of the Red Wiggler type they will survive and if they are the earthworms, they'll also be ok, because they also eat in the mulch, but then they can dig down to where the prefer to live.
When I first started gardening I couldn't add worms to my garden (regardless of type), because the soil was just too sandy and dry, with very little organic matter, earthworms wouldn't be able to bury in that stuff, because there was nothing to eat and they would have dried out. However, after the first year of gardening with heavy mulch, my yard was then able to support both types of worms and my soil is no longer just plain sand.
Here's a picture of my neighbor's yard, which is what my yard use to look like. You will not find a single worm in his front yard, but my yard is packed with them now.
My yard use to look like that, notice the bare sandy spots, the grassy spots are not much different. This is what mulching does