It's where they belong (NOT in the house!). They are detritovores, part of the breaking down cycle. That said, I've done cool/warm (not hot) composting for years and I frequently find one or two smallish cockroaches in the pile, but I've never had tons of them. And they go away when the food is gone, so they are never in the finished compost. So it's not a problem. For various reasons, it is a good idea to have your compost pile away from the house--it can attract other critters also, mice, raccoons, etc.
If it totally creeps you out anyway, then you need to work harder to be sure your pile heats up, which will drive them away. Read in this forum more about how to run a hot compost pile. OR compost in a closed tumbler bin.