Correct on the Butler game. I was very impressed with the size and speed of Richmond's skill talent (big, fast home run threats at every position), but slightly underwhelmed by their lines.
Nonetheless, I left that game thinking Richmond was a legit top 10 team. The loss to Dudley didn't really change that perception for me (I think Dudley is for real), but when you look at the rest of Richmond's schedule, there's just not much there to work with.
If you've seen them play in person like I have, then I think you have Richmond valued higher than if you just look at the credentials of the wins on their schedule. It's my opinion (purely speculative) that most of the poll voters have not seen Richmond play in person, hence the lack of poll support.
However, Richmond still carries tremendous street cred, so if they win out, they will skyrocket back up the rankings. High rankings will not be a problem for them if they continue to win.