It’s not surprising that Charles Scharf, the chief executive officer of Wells Fargo, would point to a pipeline problem as one reason his company has a poor record of hiring and promoting Black employees. The sentiment is all too common, even among leaders like Scharf, who are doing lots of the right things when it comes to fighting racism at work. “If only there were more qualified candidates” is also a wish commonly expressed by executives trying to promote more women.

“While it might sound like an excuse, the unfortunate reality is that there is a very limited pool of Black talent to recruit from,” Scharf said in a memo. He expressed a similar sentiment in a Zoom call with employees over the summer, according to Reuters. On Wednesday, the CEO apologized, saying the comment reflected his own unconscious bias.

Some observers might be scratching their heads wondering what’s so offensive about what Scharf said. To explain, let’s unpack his statement.

First, baked into it is an assumption that every single white employee at a company is talented, highly qualified and deserving of his position on merit. This is simply not true. At any firm, there are stars, there are B players, and there are laggards. At any firm, there are people who have their jobs because they had an in. Some of these people are also very qualified; others would not have gotten their foot in the door without the connection. To be truly fair, Wells Fargo should be hiring mediocre women and people of color at the same rate they hire mediocre white men.

What’s more, the statement is factually inaccurate — there are lots of talented Black people whom companies like Wells Fargo could be hiring. While it’s true that Black people graduate from college at lower rates than white people do, the disparity isn’t enough to explain a dismal record on diversity. Twenty-two percent of Black Americans have a bachelor’s degree or higher, compared with 37% of whites. That’s an appalling gap, and needs to be addressed. Nevertheless, there are literally millions of Black Americans with advanced degrees — which doesn’t sound like a limited talent pool to me.

Managers who say they “just can’t find any” qualified candidates of color for job openings should do a better job of looking. If you want to hire more Black people, stop searching in the exact same places — schools, professional associations, job boards — that attract predominantly white candidates.

The “so few qualified people” complaint also places the responsibility for the problem outside the company’s control. As Scharf himself admitted, it’s an excuse — and a weak one. It almost sounds like blaming Black people for not being more hirable.

Finally, the statement ignores the reasons that the talents of non-white non-men go unrecognized. For starters, white men are much more likely to be seen as innately talented. A study of 14 million student reviews of professors showed that professors in fields dominated by White men were much more likely to be called “brilliant” or “genius” — something that can’t be explained by other factors, unless we really think that professors of music or philosophy, fields dominated by white men, are innately more brilliant than faculty in more diverse fields, such as biology and psychology.

White men are also more likely to be promoted on potential, or because of their social capital, while racial minorities and women are often required to have a proven track record. This may be because of race or gender bias, or because hiring someone “different” is often seen as a risk. Whatever the reason, hiring managers often want to see more qualifications on the resumes of White women and people of color. This is one of the more robust findings in the social science literature on hiring.

And speaking of track records, it’s also much harder for women and racial minorities to get the kind of money-making clients and assignments that lead to stellar CVs. White men tend to award those plums to other white men, perhaps because white male managers see themselves in younger, white male proteges. Moreover, white men’s networks are much more remunerative. White men are much more likely to refer one another for jobs, and to benefit from job referrals. A study by PayScale found that “White women are 12% less likely, men of color are 26% less likely, and women of color are 35% less likely to receive a referral.” And when the referred candidate gets the job, it results in a much bigger raise if the new hire is a man; he gets a bump of $8,200, on average, versus $3,700 for a woman.

One result of all this is that, after a few years in the workforce, white men are more likely to find their careers advancing more quickly. Their gains compound like interest. To focus on the paucity of women and minorities in talent pipelines is to miss half the picture: an abundance of white men. But when was the last time you heard a CEO complain that the reason his company isn’t more diverse is that there are just so many outstanding white men?

Carmichael is an editor with Bloomberg Opinion. She was previously managing editor of ideas and commentary at Barron’s, and an executive editor at Harvard Business Review, where she hosted the HBR Ideacast.