If your favorite thing - be it a movie or a TV show or a musician or whatever - is resolutely inaccessible, fine. If every single thing you love is inaccessible, maybe you like feeling different more than you actually like anything.
Challenging subjects can absolutely be accessible. And accessible is not the same thing as dumbed-down. Look at Kendrick Lamar for example. He's putting work into his music. But is it unpleasant to listen to? No. On the other hand, there's every dumb sitcom on CBS.
I've gone around and around on Oscar movies over the years. When I was younger I was like the typical American, wanting the big movies I'd seen to win (to wit, the "Titanic" year is still the most-watched show, I believe). As I got older, I got very far up my butt and decided I needed to see every movie nominated for every award. And I prided myself on finding a way into movies that were objectively and deliberately offputting (eg A Serious Man).
I've had the same shift in TV. But the fact is, unlike movies, which only require a two hour commitment, people just don't stick with TV shows full of characters they don't want to spend time with. And music, once the critics have their say, is purely personal reaction. If a song or an album just doesn't sound great after the third time through, it might never hit you the right way.
You can take this too far, as I've mentioned. You can try to make everything appealing to everyone and end up with a bland wet fart of a four-quadrant movie. No one needs that. That's how you get some of my former friends in Korea being genuinely excited for every Transformers movie, even to this day.
But if you have the ability to make your output accessible to someone who isn't just, you know, YOU, you should. If I see two pieces of output with the same basic goals but one succeeds at being openly communicative and resonant, it's going to be better to me.
You are welcome to disagree with this. But ultimately I think prizing obtuse and inaccessible output is just a way of cloaking oneself in postmodern pretension that does nothing but make people look far worse than they realize.
All of which is to say, The Leftovers is bad and you should feel bad.