As a foreword: This post is about pure rhythm games, and not so much about games with rhythm elements - which I should be writing about in the near future. "Near".
Alright, that's it. I'm finally accepting it. I am a rhythm game player.
I've held my fair share of anti-rhythm snobbiness in the past, internalizing the view that rhythm games are lesser games on account of being focused on a single gameplay feature.
When you want to make "press buttons in time with the music" more difficult, you generally just add more buttons and faster music. Contrast this with the variety of action puzzle genres (puzzle platformers and shooters for the most part), which add complexity by throwing in unexpected game elements - the added difficulty is in figuring out how to accomplish the new tasks. Rhythm games don't generally stray too far from the "higher levels, faster music!" formula. Even heavily gameplay-focused games and genres can still have unexpected or unforeseen complexities - skill platformers like Super Meat Boy and their ilk tend to have an overbearing complexity. Even though you're still essentially pressing buttons rapidly at the right times, it seems like their complexity involves more mental skill - you have to figure out how to best your obstacles, rather than simply reading off the list.
On the other hand, all the lauded first person shooters - which I do, indeed, love - are frequently just as simple. I don't generally find myself scratching my head over how to approach a room in Shooter McKillygame 5, I just let the instinctive part of me take over and manage the issue. That's how I lose time playing games. It's definitely more fun when a game has mental difficulty as well as physical or instinctive difficulty, but it's rather rare, in my experience. The only thing rhythm games lack in comparison to other more "serious" genres is a well-integrated story... But if you start arguing that the majority of "serious" games - I'm looking at you, f-three-ar, I'm looking at you, Mario, and I'm looking at you, every racing game ever made - can stand solely on the their storytelling, then you may need your head examined. I'm not saying that those games have bottom-of-the-well storytelling, but that's not what makes them fun.
But let's think about that for a moment - why don't rhythm games have as well-integrated a story as many other genres? When you get down to it, the problem is somewhat inherent in the genre - there's only so much that you can do to tell a story in an interesting, interactive way, when the gameplay is so rigidly structured. It might be interesting to see more rhythm games built around operatic works or ballads - something telling a story. Even then, though, the player never makes a choice - the music can always just continue playing if they fail. The other option, to make the musical gameplay entirely separate from the story (see career modes in every Guitar Band style game) just makes it worse. Then you lose out entirely on the integration effects that make video games a compelling medium in the first place.
In the end, I think that all this snobbery over game genres is rather silly. Games should be judged on their own merit - for all that you might argue that rhythm games have no depth, I will bet good money that I can make a shallower RPG-esque game than Dance Dance Revolution. You're free, of course, to have personal preferences - but when you start claiming that RPGs are an inherently stronger storytelling and experience-crafting medium, you've tipped over the line.
There. I gave you a picture. Happy?