If you’re remodeling your home, there are reasons why you should look at cost vs. value report. The data can be presented by region.

Q: Today I read your advice in the Washington Post to a question about cost (and value) of remodeling a bathroom. My husband was an Art Director for Remodeling Magazine for 10 years. It was a job and company he enjoyed greatly.

However, the “Cost vs. Value” article was always disturbing to us. We both knew that the data was developed using questionable methodology, poor rigor and did not use standard scientific methodology. The weaknesses include an extremely small sample, comparing apples to oranges and drawing conclusions based upon such spurious data.

Granted, it has been 10 years since he has been Art Director and the methodology might have changed, but it is doubtful. In his tenure, there were two primary data collection activities. First, the “researcher (often a columnist for the magazine – not a scientist) would contact a remodeling company and ask the cost of remodeling a kitchen/bathroom in the region/metropolitan area.

Once these data were obtained, the researcher would contact a few real estate agents (sometimes only one or two) and ask something akin to: “We have learned that the average kitchen remodel in your area cost $38,000. How much can a seller expect to get return on this investment?” Conclusions were drawn based upon this kind of barely anecdotal data and now one sees the “Cost vs. Value” article cited as the primary source on whether to invest in a remodeling project in a home.

The data was actually developed for use by home remodelers and not for homeowners to use to determine whether a capital investment was worthwhile. It might be a fair statement to say that since the target audience is a “qualified circulation” (one must apply for the magazine with proof that it is a remodeling company – it is not sold as a news rack magazine) that it is developed for use by such companies as a sales tool to convince homeowners of the benefit to home remodeling projects.

A: We use Remodeling Magazine’s Cost v. Value survey frequently in our work and were surprised to receive your letter. So, we reached out to Sal Alfano, editorial director of the Remodeling Group at Remodeling Magazine for comment. Here’s his response:

“I am glad that you decided to ask us for comment because your letter-writer’s information is hopelessly out of date. Our current methodology and information about our research team is published at our website: http://www.remodeling.hw.net/2013/costvsvalue/article/costvsvaluedatasource.aspx
http://www.remodeling.hw.net/2013/costvsvalue/article/researchteam.aspx

“Here is a little more information to fill in the gaps.

“I came to Hanley Wood in the spring of 2001, and one part of my new responsibilities was to take over management of the Cost vs. Value Report. The methodology had already changed from what your letter-writer described – for example, we were already getting cost data from a third-party publisher of remodeling cost estimating reference books and software, and we had a partnership in place with the National Association of Realtors to gather value-at-sale data from its members. But there was plenty of room for improvement. The Report covered just 10 projects each year, and the number of Realtors provided value data was very small.

“Fortunately, we were able to put changes in place that rapidly improved both the number of projects covered and the statistical significance of the results. One big change came in 2003 when we shifted from a call-center to a digital survey of Realtors. Using email to make initial contact, and a web-based survey enabled us to reach a much larger sample; today we get about 4,000 responses from Realtors nationwide, and in 2013 our confidence interval on national data is 95% +/-1.6. We also present the data by region, and while this necessarily means that the regional data represents fewer responses, all 9 regions in 2013 show a confidence interval of 95% with a variance ranging between 3.7 and 7.1points. This data analysis comes from The Farnsworth Group, an online market research company that has hosted the survey for a decade.

“We have expanded both the number of cities covered (from 35 to 81, and our plan is to cover 100 in 2014) as well as the number of projects (from 10 to 35). We include two versions of some projects, midrange and upscale, a change that was introduced to better serve our audience of professional contractors, who wanted to be able to provide a range of data to prospective customers. But contrary to the claim of your letter writer, this in no way means that the data in the report are tailored to place remodeling projects in the best light for the purposes of boosting sales. The data is independently gathered, and is available for anyone to see and analyze.

“I hope this answers your questions, but please do not hesitate to get back to me if you need more information.”