It's because the playermodels field of view is focused towards the crosshair. Look closely and you'll notice the playermodel actually turns to the side when an object (wall/player/model) is close up. If you lower 'thirdpersonside' so the crosshair is more central to the playermodel's view, it can help.
Personally, if i'm playing in third person mode, i manage to compensate for the "drift", it just takes a bit of practice getting used to. I've seen other games that use this kind of view, but they also had a feature where the crosshair floated off-center to compensate for close up targets... although it find it gets disorientating, and wouldn't call that example a solution.
♦ ♦ ♦ ♦ ♦