I disagree that compost has no fertilizer value. It may be in the range of 1-1-1, which seems small compared to 12-12-12 granular or 20-20-20 Miracle Gro soluble plant food, but on the other hand, you're putting it on by the shovelful. It has plenty of fertilizer value.
And don't add lime to your soil unless you have a soil test and know for a fact that your pH is too acidic. Even then you need to know the Ca to Mg ratio to know whether to use calcitic or dolomitic lime.
I think it's OK to use some fertilizer along with compost. Most people will not be able to change whole hog from whatever they were doing to a massive soil improvement campaign with compost, overnight. So don't feel guilty about using some fertilizer.
If your neighbors have great looking gardens, ask them what they're spraying on them.