Some lenders will do 80% investment loans.
I agree. I would look at major lenders. It is important to look around a bit for the best rate. If the rate and fees are roughly the same, go with the person that you feel more comfortable with and is knowledgeable. Ask tons of questions!
There are companies that go down to 600 credit score on FHA as well.
I am a mortgage consultant with Wells. You can call me and I can tell you who holds your loan.
Brian,If you haven't found anyone yet, I have done several IRRRL's during the refi boom. I am with a national lender and would be happy to provide you with some options to compare.
I would look at a well known well established mortgage company.