I had a chance to get the base 2020 vs the limited. I chose the base model chalk metallic(if I could have found that Deep Marsala for a little less I would have gotten a limited for sure) and did what you mentioned. I added the OEM top case, but it isn't color matched. It's all black. I think I like the look better than the color matched panels. I also added the heated seat and signature light. So, the only thing I'm really missing is the self adjusting suspension and heated passenger grips.
From what I recall, you can have the compressor/pump added for about $600 or so. Plus, installation costs.
I recently discovered that I was riding around with 0 air in the air bag for the rear suspension. I added some air to it and the feel of the ride has completely changed.
With that said, yes there are many seemingly minor differences, but in my opinion; they make a pretty big difference in appearance, comfort, and convenience overall.

If your budget allows and there isn't a huge difference in price, the limited might be worth it. I found a great deal on my base model and I'm happy with the steps I did to upgrade it to what it is.