I don't think people (reasonable people at least) are singling Bennett out, are annoyed at Bennett for simply acting like this or are blaming him solely and excusing all other parties for their roles in it.
I think people are annoyed at his apparent hypocrisy and how such behaviour goes strictly against what Bennett has preached his entire career.
For mine at least, that's the difference, and I can totally understand people being upset about that. If you've been a passionate supporter of someone for 30 years or more, only to find out that perhaps they aren't everything you thought they were, it could most certainly be very deflating. I expect it would make you question everything you've ever believed about them. I guess that's why people say it's seldom good to meet your heroes, as more often than not, you'll be disappointed.
Conversely, Bennett has to do what he thinks is right for him - I understand that. I also understand he hasn't committed any heinous crimes, so he's hardly the devil. But he hasn't exactly been the moral compass lots of people think he is, a view I would imagine has been cultured over the span of time lots of fans have spent supporting him and rallying around him.