Maybe the atrocious levels of diversity among wealth managers aren’t due to failures to recruit and promote women and minority advisors, or the systemic disadvantages they face—maybe it’s all in our heads.

According to a recent study by Chicago-based Spectrem Group, a preference for older, white and male advisors might be hardwired into the brains of the affluent.

In a white paper titled “Snap Judgments: Do First Visual Impressions Impact Financial Advisor Selection?” the consulting firm showed respondents a photo of eight advisors—four women and four men of varying ages and races—and asked them to choose whom they would be most likely to hire based on appearance.

An older white male advisor was selected by a plurality of respondents across wealth and age cohorts and genders, with around one in three of Spectrem’s respondents selecting the oldest white man in the picture. The same advisor was chosen by 40 percent of ultra-high-net-worth respondents with more than $5 million in investable assets.

In a Wednesday conference call, Spectrem Group president George Walper said the results show that investor perception presents a barrier to diversifying the industry.

“This industry is trying to recruit younger, minority and female advisors,” Walper said. “If a young minority advisor thinking about this as a career saw these numbers, they would be frustrated.”

Cathy McBreen, managing director at Spectrem Group, said on Wednesday that the results are not necessarily representative of general investor sentiment.

“We do acknowledge that our sample, because we look at high-net-worth investors, tends to be older,” McBreen said. “High-net-worth investors are also skewed more towards white, Caucasian people, because that’s reflective of what the overall wealth market is.”

A middle-aged white male advisor was the second most popular overall selection, chosen by approximately 25 percent of the respondents. A young white woman and a middle-aged white woman were the third and fourth most popular selections; each was chosen by just over 10 percent of the respondents.

A younger white male, a middle-aged minority male and a younger minority female were each chosen by fewer than 5 percent of the respondents.

Spectrem said that prospects would be more likely to become clients of older, white male advisors, who are perceived to have the most experience and to be the most trustworthy.

“People are choosing these advisors because that’s what they believe an advisor looks like,” McBreen said. “That’s what we’ve grown up looking at, so the older white male has become our perception of a professional advisor.”

Women preferred an older white male advisor simply because of appearance, but not as strongly as men. While the older white male advisor was favored by 39 percent of men, he was selected by 30 percent of the women surveyed. Both men and women selected the middle-aged white male advisor as the second most popular choice.

Mass affluent respondents (those with between $100,000 and $1 million in assets) were most likely to pick the oldest white male in the group, with 33 percent naming him their first choice. The next most popular choice was a middle-aged white male, who was selected by 23 percent of the respondents.

“As we try to get more females and more ethnicities involved in this industry, we think this will change,” McBreen said. “Our goal has been to get people to gravitate towards people who are younger and different, and that’s not happening now. We’ll see if that changes as time goes by.”

The mass affluent were the most likely out of all wealth segments to select women: 13 percent selected the younger white female, 12 percent the older white female and another 12 percent selected the middle-aged white female.

Spectrem said the mass affluent are likely gravitating toward older advisors because of the perception of those advisors’ experience. A young white male advisor was selected by only 4 percent of the mass affluent group, while only 2 percent said they would choose a young minority female and a middle-aged minority male.

Mass affluent seniors aged 65 and up and baby boomers aged 55 to 54 were the most likely to choose the older white male advisor (39 percent of the seniors and 34 percent of the baby boomers chose him). The middle-aged white advisor was the second most popular option with both cohorts.

Mass affluent millennials younger than 35 years old were the least likely age group to choose the older white male advisors; the greatest number of them, 31 percent, selected the middle-aged white male. Millennials were also more likely than other age groups to choose the young white female advisor, who was picked by 21% of them.

“The prevailing theory is that younger people would not pick older investment advisors,” McBreen said. “However, even those under the age of 35 chose an older white male, but they chose a middle-aged white male at a higher percentage than other age groups. It could be that the middle-aged white male looks older to this age group.”

Affluent minority investors were also more likely to choose the older white male (selected by 29 percent), or the middle-aged white male, (selected by 21 percent). Only 18 percent chose the young white female; only 12% chose the middle-aged white female; only 5 percent chose the middle-aged minority male; and only 5 percent chose the younger minority female.

“Is it the chicken or the egg?” Walper asked. Is the industry failing to mirror the current U.S. and global demographic diversity, “or is the industry more representative of society and what investors have been trained to think about?”

Spetrem’s research was based on surveys of 3,152 households with between $100,000 and $25 million in net worth, with 1,875 male respondents and 1,277 female respondents.