I'm with you on that 100%. All of it. For me, much of the dividing line is whether or not the artist is still alive to financially profit from my buying/listening/watching/whatever. Someone dead? I am much less conflicted, especially if they're long dead. Whether or not an actual crime has been committed also factors in, although perhaps not greatly. But, yes, it's easier (for me) to sometimes try to ignore when an actor has been a creep than when it's a singer singing the word he himself wrote.
A few weeks back, I had Aftermath playing. The very first time through the chorus of "Stupid Girl," my 16-year-old looked up, looked at the speakers, then slowly turned to me with an expression of "what the fuck, Dad?" on her face. I skipped to the next song, but ended up just turning the album off. Still like the Stones, still like the album—although the song's never been a favorite, to put it mildly—but it felt weird. I know the history, I get the point...but...I put on the Pretenders instead.
I'm 90 seconds in and so far this is fucking magnificent.