The Algorithm Doesn't See You (And That's By Design)

or what happens when 40,000 job applications vanish because of one biased AI system

About a year ago, I got a request from a friend, a professional who wanted to enrich her career after immigrating to the U.S. from Liberia decades ago.

She had a master’s in social work, decades of healthcare experience, and a genuine desire to help others.

But after all of her applications either fell into the ether or got rejected, she got dejected.

She heard on TikTok that many people struggling in this cold job market were using ChatGPT, Claude, and similar tools to optimize their resumes.

She wasn’t too familiar with AI, so she called me on a random Saturday afternoon to convene with her at a local coffee shop and create a battle plan for action.

Now, everyone in my circle knows my history with employment and how, after a long drought ended in 2019, jobs fell into my lap as the proverbial apple fell into Newton’s.

Whether in sales, customer success, tech, or even retail, I have never been out of work for more than a month at a time.

I got the big guns and gave her both solid prompts that worked for me in the past to spruce up resumes without blatant lies, fairy tales, or fallacies, and a ChatGPT tutorial.

A year later, I saw that same friend at a holiday party. I inquired about whether the prompting strategies worked. The story she told me in response still leaves me flabbergasted.

She explained that after months of applying, she began to notice something strange. Not “the job market is bad”—strange. Not “maybe my resume needs another verb” is strange. The kind of strange that makes you open a spreadsheet at midnight because your intuition refuses to be gaslit.

Her legal name is beautiful, phonetic, and unmistakably ethnic. The kind of name that tells a story before she ever gets to speak. It tells you where her family is from. It tells you that a teacher probably paused before calling attendance. It tells you that recruiters have probably practiced it under their breath, or worse, decided not to.

For months, she had been applying to roles she was genuinely qualified for. Not dream jobs; she was manifesting her way into them. Realistic roles. Lateral roles. Jobs where her experience matched the posting almost line-for-line. But the rejections came back fast. Sometimes within hours. Sometimes, before the day was over. No phone screen. No recruiter note. No, “we went with another candidate.” Just the cold, little automated message: “Thank you for your interest. Unfortunately.”

At first, she blamed herself. She rewrote her bullet points. She added keywords. She ran her resume through every scanner, checklist, and career influencer template known to mankind. She used the prompts I gave her. She made the resume cleaner, sharper, and easier to read. Still, the same thing kept happening.

Then, almost as an experiment, she submitted the same resume under a shortened, more “Americanized” version of her name. Same degree. Same job titles. Same skills. Same city. Same formatting. The only meaningful difference was the name at the top.

And suddenly, the doors that had been silently closing started cracking open.

The version of her resume with the nickname got recruiter views. Then callbacks. Then actual conversations. Meanwhile, applications under her full name kept disappearing into the same polite rejection machine.

She told me this at a holiday party, holding a plastic cup of punch like she was telling me the plot twist in a thriller. I remember laughing at first because I thought she was exaggerating. Then I realized she was not laughing.

What made the story so disturbing was that it wasn’t cartoon-villain racism. There was no hiring manager twirling a mustache, saying, “Absolutely not, this name has too many syllables.” It was quieter than that. More deniable. More modern. A system trained on old preferences, old patterns, and old ideas of who “fits,” wrapped in the language of efficiency.

And that is when it clicked for me: sometimes the problem is not that people do not know how to use AI. Sometimes the problem is that AI already knows how to use the world’s existing bias—just faster, cleaner, and with a better user interface.

---

Stanford researchers published the largest independent study of AI hiring algorithms ever conducted. They didn’t just look at the theory of bias. They looked at real hiring data — real applicants, real algorithms, real rejections.

What they found:

26% of Black applicants applied to positions where the AI system discriminated against their racial group. Not theoretically. Not hypothetically. By the federal government’s own standard for discrimination.

If the algorithm had recommended Black and Asian candidates at the same rate as white applicants, 40,000 more applications would have advanced to the next stage.

Forty. Thousand.

But here’s the part that made me set my phone down: the study found something called systemic rejection. When one company’s AI rejects you, it actually predicts that the next company’s AI will reject you too—because they’re using the same vendor. The same algorithm. The same bias, compounding.

Imagine applying to ten jobs. All of them are using the same AI screener you’ve never seen, built by people who’ve never met you, trained on data that never represented you. And being told no, ten times in a row—not because of your skills, but because of your name.

The University of Washington found that AI resume-screening tools favored white-associated names 85% of the time. Male-associated names 52% of the time. And none of these preferred Black male-associated names over white male-associated names.

This isn’t a glitch. It’s architecture.

---

The “Neutral” Machine That Isn’t

We keep hearing that AI removes human bias from the hiring process. That algorithms are objective. The machine doesn’t see race.

But the machine was trained by humans. On human data. From human systems that were never fair to begin with.

When Amazon built an AI recruiting tool, it learned from ten years of predominantly male hiring data. The system taught itself to penalize resumes that included the word “women’s” — as in “women’s chess club.” It downgraded graduates of all-women’s colleges.

They scrapped it. But the same logic is at work in hundreds of tools currently used to screen millions of applications.

A Berkeley Haas study examined 133 AI hiring programs and found that 44% showed gender bias. The researcher put it plainly: “Black male names tend to get most of the discrimination, but women and people who are experiencing intersectionality oftentimes end up being on the losing end of that search.”

Hobbies get flagged. Career breaks for raising children are penalized. Softball instead of baseball. The algorithm doesn’t know why these patterns exist. It just knows they correlate. And it acts accordingly.

---

Beyond Hiring: The Bias Blueprint

This isn’t just about getting a job. It’s about how AI is encoding inequality into every system it touches.

Healthcare: An AI suicide prediction tool successfully detected 62% of suicides among white patients—but only 10% among Black patients. Dermatology AI was trained on datasets in which only 11 of 100,000+ images represented brown or Black skin tones. AI chatbots have promoted false claims about racial differences in skin thickness.

Mental health: AI models made inferior treatment recommendations for Black patients when race was mentioned. Language-based models underperformed on predicting depression severity for Black patients because they were trained on how white patients describe symptoms.

Financial services: Apple’s credit card algorithm offered women significantly lower credit limits than their husbands — even when women had higher credit scores. A student loan company just paid $2.5 million for AI that was more likely to deny loans to Black and Hispanic borrowers.

Content moderation: AI trained on crowdsourced hate speech data disproportionately flagged Black vernacular English as “offensive.” The moderation tools designed to protect us are silencing the way we speak.

The pattern is always the same: train on biased data, deploy at scale, harm the people who were already harmed.

---

So What Do We Actually Do?

I’m not here to tell you AI is hopeless. I’m not even here to tell you to stop using it. I’m here to tell you the truth: the tools are not neutral, and knowing that changes how you use them.

Here’s what I want you to walk away with:

If you’re job searching: Know that AI screeners are likely reviewing your resume before a human ever sees it. Lean into your network. Make personal connections. The old-school approach isn’t outdated — it’s a survival strategy. And if you suspect an AI system rejected you unfairly, document it. Colorado’s AI hiring audit law takes effect in 2026. The EU AI Act designates hiring algorithms as high-risk, with compliance starting August 2, 2026.

If you’re building with AI: Audit your tools. Ask what data they were trained on. Ask who was in the room when they were designed. If the answer is “we don’t know” or “it doesn’t matter”—it matters.

If you’re creating content about AI, talk about this. Share this. The UN High Commissioner for Human Rights said this year: “If the data are only collected from one part of the world, if only men are developing AI, then unconscious bias will be built in.” This isn’t a niche issue. This is the defining equity question of our generation.

If you’re raising kids who use AI: A study across 12 countries found that AI labels young women as “fragile” 56% of the time, recommends they seek external validation 6x more than young men, and redirects their career aspirations toward social sciences 75% more. The AI your daughter talks to is not her friend. It’s a mirror of every bias we haven’t fixed yet.

The algorithm doesn’t see you.

But you see it now.

And that’s where the power shifts.

---

Sources:

1. Stanford HAI — 26% of Black applicants face discrimination, 40,000 applications affected, systemic rejection
• Study: “AI Hiring Tools Can Yield Racial Bias and Systemic Rejection” — Stanford Institute for Human-Centered AI (HAI)
• Published: May 26, 2026
• Link: https://hai.stanford.edu/news/ai-hiring-tools-can-yield-racial-bias-and-systemic-rejection
• Coverage: https://fortune.com/2026/05/26/ai-hiring-algorithm-racial-disparities-pymetrics-stanford-study/

2. University of Washington — AI resume screening favored white-associated names 85% of the time
• Study: “AI tools show biases in ranking job applicants’ names” — University of Washington
• Published: October 31, 2024
• Link: https://www.washington.edu/news/2024/10/31/ai-bias-resume-screening-race-gender/
• Also: https://www.washington.edu/populationhealth/2024/12/12/uw-research-finds-racial-and-gender-bias-in-ai-tools-ranking-job-applicants-names/

3. Berkeley Haas — 44% of 133 AI hiring programs showed gender bias
• Coverage: “Study shows AI hiring tools discriminate against women, minorities”
• Published: April 2, 2026
• Link: https://turnto10.com/i-team/consumer-advocate/study-shows-ai-hiring-tools-discriminate-against-women-minorities-artificial-intelligence-names-hobbies-software-resume-april-2-2026
• Also: https://eveprogramme.com/en/ressources/articles/44-des-IA-comportent-des-biais-sexistes/ (March 20, 2026)

4. Amazon AI recruiting tool — penalized women’s resumes, scrapped
• Original reporting: Reuters, October 2018
• Reference: https://mediawell.ssrc.org/news-items/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-reuters/

Healthcare, Mental Health & Financial AI Bias

5. KFF — AI suicide prediction (62% white vs. 10% Black), AI healthcare racial disparities, race-based medicine claims
• Report: “The Growing Use of Artificial Intelligence in Health Care and Implications for Disparities” — KFF (Kaiser Family Foundation)
• Published: April 30, 2026
• Link: https://www.kff.org/racial-equity-and-health-policy/the-growing-use-of-artificial-intelligence-in-health-care-and-implications-for-disparities/

Gender Bias, Content Moderation & Policy/Regulation

9. LLYC — AI labels young women “fragile” 56% of the time, external validation 6x more, career steering
• Report: “Illusion of AI” — LLYC (12 countries, 9,600 recommendations, 5 AI models)
• Published: March 3, 2026
• Link: https://llyc.global/en/noticias/ai-amplifies-gender-bias-for-young-women-fragile-in-56-of-cases-more-dependent-and-with-a-vocation-for-the-social-sciences/