Fame
Bob Broder. Norman Kurland. Ever heard of them? Didn't think so. Agents don't get to be famous, unlike their clients.
In the best-case scenario, a literary agent in Hollywood might have a press-worthy squabble with one of their very famous clients that leads to a thirty-second shout out on TMZ—you might do better if there's a long, drawn-out televised court-case involved—but this kind of publicity isn't the stuff that's good for an agent's career. It's likely to raise some questions with the clients you have left. But being in People magazine or featured on TV? Not likely, unless you marry someone more famous than you.
As a literary agent, the fame you have access to is within your profession. In fact, to be successful, one of your goals should be to occupy the hearts and minds of as many of the folks who decide which movies get made and for how much.
You can become known as the gal with the client-list of really talented writers, or the guy who always gets a phenomenal sum for his client's specs, or the friendly-dude whom everyone likes but questions whether or not has any taste yet somehow still manages to squeak out two or three major deals year.
On a smaller level, fame is also possible within your company as well, though this could be for reasons less desired—the gal who's been stuck as an assistant for ten years or the guy who was demoted to the mail room for costing his boss a three-million-dollar deal last year. Hopefully, nothing like that happens to you.