I was told as a kid that this phrase comes from the investment industry following the 1840s railway bubble, in which people lost lots of money on overvalued railway stocks (the hot tech of the day.) After that, many chastened investors fell back onto residential property speculation instead, which was perceived as being a far safer investment.