The Danger in Relying Too Much on Algorithms

By: Steve Radick

September 22, 2017

As published in HIVE and Builder.


Brunner experts discuss the power of data and its impact on the home building industry

Go to any marketing conference today and you’ll see topics like big data, algorithms, artificial intelligence, virtual reality, programmatic media, and machine learning dominating the event.

I sat down with David Sladack, the lead for Brunner’s Home Enrichment practice, to ask him if there’s any hope for us humans or if we should all start brushing up on our coding and statistics skills.


Google searches for the term "Big Data" over time. I guess all data before 2012 was “small data.”Google searches for the term "Big Data" over time. I guess all data before 2012 was “small data.”

Radick: From technology that can track homes that are wasting water to realtors using data to deliver targeted ads to home buyers, big data algorithms are being used all across the home building category. Why is that?

Sladack: Like most other parts of our life, technology and big data are converging. There is no denying that these algorithms—when properly applied and tested—can benefit both brands and consumers. But, they also have limitations.

Take Zillow for example. While the immediacy of seeing a home value with a few strokes of a keyboard has huge benefits, the accuracy of this data is in question. The algorithm gives you a false sense of transparency and validity.

Radick: Have you had some personal experience with big data bias?

Sladack: I have an 11-year old home situated in a residential development within a rural county. For the first five years after purchasing my home, the value, according to Zillow, increased by 13%. Then, in just three months, from August 2016 to December 2016, the house’s value suddenly dropped by almost 20%. And it continued to drop over a 12-month period to almost 22%. Over the same 12-month period, Zillow showed overall homes within my ZIP code were actually increasing by 14%.

What I found was that Zillow, in an attempt to improve accuracy, upgraded its algorithm in July 2016. Unfortunately, homes like mine—newer and in a rural area without strong inventory turn for comparable properties—were unfairly penalized. The algorithm change was met with strong homeowner backlash, and Zillow just announced a contest in May 2017 offering a $1 million award to an individual or team who could offer a solution that most improved the Zillow algorithm.

Even though Zillow saw the flaws in its technology, people still often put more stock into the Zillow algorithm than human real estate agents who have studied that community for decades.

Radick: So, I’m guessing you’re not deciding to buy or sell based on a Zestimate?

Sladack: Actually, I love the concept as Zillow offers buyers and sellers alike a level of transparency like never before. But, it illustrates my point that there can be alligators lurking in the algorithms.

Implementing an algorithmic-based form of estimating like Zillow has done can take time to correct itself and deliver better accuracy. In a recent New York Times article, Stan Humphries, Zillow’s chief economist, suggested that beginning in 2006, its Zestimates had a median error rate of 14%. That suggests half of the homes being valued 14% above their actual price while the other half are estimated 14% below.

Today, Humphries claims the median error rate is closer to 5%. Part of the improved accuracy is due to the increased volume of homes listed. Zillow has grown from 43 million homes on its site in 2006 to more than 110 million today. But 5% on a $500,000 home is still a $25,500 swing.

Radick: That can create quite a divide between a buyer and seller.

Sladack: That’s right. And, there are still pockets where Zillow’s information is limited, especially where comparable homes are not turned over frequently. Additionally, Zillow’s algorithms don’t account for homeowner improvements to a property, unless that improvement was large enough to warrant a local permit. I self-reported recent improvements on my own Zillow profile, and while my property value received an initial bump, it quickly returned to its downward trend.

Radick: So what advice do you have for brands who are using algorithms to harness big data as a way to better serve their prospects and customers?

Sladack: Algorithms are successfully applied in many ways, whether it’s Houzz serving up relevant content to a homeowner looking for a new kitchen design or a lender estimating your credit score for a home improvement loan. But, don’t do it at the expense of the human emotional factor.

My recommendation for both marketers and consumers is to embrace the technology, but use it as a guide. Let’s go back to the Zillow example. Just as the Zestimate algorithm considers several factors, so should a homeowner. Consider the Zestimate as one input and then apply other sources—including the expertise and knowledge of your local realtor and appraiser, and doing homework on your own on common sense factors like comparable properties, health of the local market, and recent improvements that might not be reflected in the Zestimate.

Following this conversation, I was conflicted. I came away believing there’s even more opportunity for human interpretation and intervention with big data than I had realized. At the same time, I began to wonder about all the home decisions that I’ve blindly abdicated to an algorithm: the contractor with the best Angie’s List rating, the temperature that my Ecobee3 says my house should be, the movies that Netflix says I’ll like. Maybe I should start asking myself where all that data is coming from and how it’s being used? Sometimes a big data algorithm is more of a distraction than a useful tool.

Now that I think about it, know of any good movies I should watch?


VP, Director of PR, Steve Radick

SVP, Director of Sales and Marketing, David Sladack

To read more about Home Enrichment, please visit our blog.