First that looks like a large pot for a small plant. Second - what type of soil do you have beneath the gravel?
does your container have good drainage?
Most problems with Cacti and Succulents are from over watering and/or not enough sun.
You need very well drained soil - preferably Cactus Mix. You need a well drained pot. No pot saucer.
Because I live in a region with a lot of rain my succulents are kept under the patio cover. They get direct morning sun and bright light the rest of the day. I generally ignore them. I give them a small amount of water ever week or two. They are Happy Campers.
My Succulents are planted in a 24" pot saucer with no drainage so over watering could be an issue. I wait until they just begin to shrivel before giving them water.
Kick off your shoes and stay awhile.