nope your wrong...not sure "when you were young" but its a very complicated answer. One of the biggest cost of a house is the "footprint" or the foundation. Any builder will tell you its cheaper to build a 2 (or 3) story home than a single level ranch because of the cost of land and foundation. Cheaper to build up than "out". Price per sq foot is a function of "quality". I had a friend who bought the nicest lot in the neighborhood and built a small 3000sq ft house. Price per square foot was over $750, where the other homes (much larger went for $200-$250). However they did have the most spectacular lot in the area. If you want quality finishes, hardwood, granite, high end fixtures, lighting, exterior finishes like stucco/tile roof etc then price per sq foot goes up. Region obviously makes a difference, as do building codes, now I think being closer to town is more desirable so people will pay more for shorter commutes.
Back "in the day" homes were just less expensive to build and to buy. Materials were cheaper and labor was cheaper. Larger homes sold for more than smaller homes, perhaps if they were 2 story then price per sq foot was less but again only a function of material costs(foundation etc).