I wonder how many houses are vacant and/or second homes or vacation homes? IIRC back in 2007 I read through the census numbers and it was around 20M. So, if things were tight, we do have enough houses for 20M households to live in without building more. $140B/200K/house -> 700K houses. So, maybe Fannie/Freddie are sitting on 700,000 empty houses. But others (rich people?) are sitting on millions of more houses.
The reality is probably more like Freddie/Fannie don't actually have the shadow inventory. Reality is probably much of those were fraud and house is worthless or doesn't even exist! I wonder.