Unfortunately, a candidate dropping out like this is almost unheard of, so the downsides are pretty uncertain. Personally I think people are a little too worried about this but there certainly is some risk that what seems like a good candidate under casual examination ends up having some fatal flaw or hidden scandal.