Rent control? Baby boom kids left the nest?
You take farm land and build a bunch of houses, and you are going to get a bunch of people in their 20's and early 30's. If they have children, it is a toddler. While an apartment is OK for a baby or toddler, not so much for older children, and you are concerned about the schools.
10 years later, and you have parents in their 30s, and a bunch of grade-school age children. Few of the houses are sold. There are some divorces, and perhaps the largest families with 6 or more children buy a bigger house, but there isn't much opportunity for newly formed families.
20 years later, and you have parents in their 40s, and early 50s and they've finished up having children and the oldest children are going off to college. There isn't a market for apartments.
30 years later and the grade schools are depopulating. Some of the houses are becoming available, as the original owners are retiring and moving to Florida, or moving to condos or townhouses, which may be built as developers start finding smaller areas to build on.
Between 1970 and 1980, Nassau saw massive drops in young children; some drop in late teenagers as the youngest children of the earliest settlers reached high school. There was actually a pretty big increase in persons in their 20s, as all the original owners had aged way past that (in 1970 the largest age cohort was 45-54). So in the 70s, at least some of the children of the settlers were sticking around - but not having as many children, and also having them later.
But there were big drops in the 35-54 population, as the settlers aged, and increases in the older population.
Most of the population drop was due to children not being replaced, while the distribution of the adult population was changing.