Yeah definitely. Someone sent me a pretty good article that recommended that you use a screen reader a little bit on your own stuff, one for making sure it's accessible, but two because it teaches you a different way to look at your design process.
IDK how much in depth time I'm planning to commit to this whole thing but I do think applying a screen reader is necessary if I'm going to claim my stuff works with it. I can't really see it being all that effective just to apply the right classes and hints to the existing page and hoping it all works out without testing it...