A little off topic I know, but
this ^ can be pretty serious! Have you ever had any medical investigation into
this bit of your post above?? Anyone checked you for
Carpal Tunnel Syndrome or
Unla Nerve Entrapment, or done a
Nerve Conductivity Test on the nerves from above your elbows down into your hands??

There's a bunch of other things besides those two mentioned that it could be of course, but they were the first things that most are likely to recognise that came to mind, and that Nerve Conductivity Test mentioned should be able to identify what & where the problem actually is -
IF you get it done!?!
I ask, because I
know that while early treatment can help or even resolve some of the potential problems entirely, letting this sort of thing go untreated for too long can be really bad for your long term ability to use your hands; and if you let it progress long enough that the damage goes too far, you might even lose the use of at least the three fingers further away your thumb on an effected hand entirely! And
THAT is guaranteed not to be any fun, and will mess a whole lot more with your riding enjoyment than the '
early sign of problems that WILL ONLY get worse' type numbness that tell us you're experiencing already!!