I don’t think they’re angry at you. I imagine they couldn’t care less if they tried, but I’m sure they’re confused how you seem to think that the target of an infatuation is somehow responsible for the actions of the infatuated person. This strikes me as a very small step away from “It’s her fault she got r***ed, look how she’s dressed”.
I’ve had this thought many times, glad I’m not alone. Also makes you wonder if possibly everyone’s “favorite” color is the same color, we just all call it different things because of how we individually perceive it.
This is a fun thought, but I can disprove this myself easily enough due to having had my favorite color change multiple times in my lifetime. Currently enjoying green.