You can have short term price shocks, like if there's a huge, unexpected flood of people into or out of an area. Demand for housing can change more-quickly than supply of housing.
However, over time run, supply catches up, and the price of housing is set by two things:
The cost of construction. What does it cost to get the land, get the materials, and get the labor to build more?
The cost of capital. How difficult is it to get ahold of capital to do construction?
In cities, it does cost more to get land, but you can also build up, and the land value becomes less and less significant as the heght increases.
In cities -- especiallyexpensive places like San Francisco or New York -- a major factor is usually that there's some form of height restrictions. That is, it would be possible for someone to obtain capital, buy land, build a building on it, and make a reasonable return, but there are zoning ordinances or the like that prevent this. Sometimes there are outright height restrictions. In London, there are restrictions on buildings that obstruct a network of views to various landmarks, which has a similar effect. Developers have an interest in building as long as they are allowed to do so. Eliminate height and similar zoning restrictions that prevent construction, and it will show up.
This is an article that I really like, was run in The Atlantic some years back, focusing on why housing is so expensive in New York City, due to a history of planning restrictions on construction:
California in particular has seen spread of the YIMBY movement (that is, a counter to NIMBY, where people want to block construction near them) aimed at removing restrictions on housing construction.