Other people can hurt you and not be held accountable for their actions.
When you get hurt, it’s your fault now.
Explain to drivers that you’re “not like most cyclists” and that you “don’t like hanging out with other bikers”. Don’t ride with other people, lest someone think that you’re one of those cyclists.
These environments aren’t just not built for you, they’re constructed in a way that actively excludes you.
At the end of the day, you can always hang up your helmet and declare bike commuting “a great idea and all, but just not worth it”.