Fame
Actuaries are like puppeteers. People see the puppet—maybe even the strings going somewhere behind the curtain—but almost never the person manipulating it. People will always know Elmo, and never the man who secretly makes that terrible squeaking voice of his off screen.
Your anonymity is just fine by you, though. Imagine if you were constantly being hounded for your autograph and had paparazzi snapping photos of you from a window-washing scaffold outside your office on the forty-eighth floor. Might be, uh, sort of tough to concentrate.