I once served under a general who said, "Lead for your best people, not your worst." That simple saying revolutionized the way I lead.
There are two very powerful ideas embedded in that piece of advice. First, decisions should be made based on what is right for your good people, not what is necessary for your bad ones.
Example: Our IT manager recently wanted to audit my team's e-mail. His goal was a good one, to eliminate wasted time. His solution, though, would have communicated lack of trust to all the good people who don't abuse the privilege. Instead, I tasked supervisors with monitoring employee activity for a period to determine if there was a problem.
The second idea is that expectations should be based on what your good people are capable of. Don't let attendance standards slip because some people have a hard time getting to work. Don't limit your organization's activity to only what your whole group can currently do. Don't manage for mediocrity, expect and reward excellence.
When you lead like everyone can do what your good folks are doing, you elevate the whole team. The good work done by your best people drags the rest of the group to a higher plane. When top-performers have free reign, it becomes harder for the bottom-feeders to camouflage the disparity in performance. Under-performers will struggle to catch up to your expectation that they'll be right in there with the rest.
Because you spend a lot of time dealing with your bad people, it's easy to think about them more than your good ones. So remind yourself daily to lead for your best people, and let everyone else try to keep up.