The Ghost in the Code
Your Algorithm Learned Prejudice From Us
Consider this a case file. The suspect has no face, the decisions arrive as scores, and everybody insists the process is objective because it involves mathematics.
Then someone checks who keeps getting rejected.
The old examples of algorithmic bias have not become irrelevant. But research published in 2026 gives this story a new chapter: hiring systems that repeat rejection across employers, AI evaluators that favour certain AI writers, and fairness tests whose conclusions change when the experiment changes.
The ghost has been busy. Apparently, it has updated its software.
Garbage in, imperium bias out
The old programmer’s phrase is “garbage in, garbage out.” I would give it a small imperial upgrade: garbage in, imperium bias out.
By imperium bias, I mean the moment a system’s judgment acquires authority simply because it arrives through the machinery. A score becomes a shortlist. A shortlist becomes an opportunity. Somewhere along the way, a prediction starts wearing a crown.
This is my name for that habit, not a formal scientific category. The distinction matters. We have quite enough impressive terminology wandering around without identification.
A model learns patterns that help it meet a target. But someone chooses the examples, the target, the tests and what happens to the answer. Bias can enter through any of those doors. Historical prejudice is one route; a poorly chosen definition of success is another.
The uncomfortable question is therefore larger than “What did the machine learn from us?” It is also: what authority did we give what it learned?
Different employers, the same invisible gatekeeper
Research published by Stanford researchers in 2026 examined roughly four million applications screened by algorithms from a single hiring vendor. The records came from 2018–2022: this is newly published evidence about earlier hiring decisions, rather than a snapshot of every hiring tool operating today.
Across all positions combined, the recommendations did not cross the researchers’ chosen adverse-impact threshold. Looking at individual positions exposed racial disparities that the overall picture concealed. The study also found that applicants could receive repeated rejection recommendations across employers more often than expected if those decisions were statistically independent. Stanford Report
That is a rather different problem from one company having one questionable filter. When employers share a screening supplier, an applicant may encounter similar judgments behind several apparently separate doors.
My reading? Changing the company name at the top of the application does not necessarily change the gatekeeper underneath it.
The study concerns one vendor’s systems and records. It does not establish that every hiring algorithm behaves this way. It does give us a practical reason to ask how independent our supposedly separate opportunities really are.
When AI starts marking AI’s homework
Another 2026 study, by Michael Andreas Ebling, Stefania Ionescu and Nicolò Pagan, examined a newer arrangement: applicants use language models to write cover letters, while evaluators use language models to assess them.
In their controlled experiments, evaluator models generally preferred letters produced by certain writer models. Proprietary models were favoured over locally hosted alternatives, and some evaluators also showed a preference for their own outputs. In the most adverse pairing, approximately two of twenty-five qualified candidates were filtered out because of the writer–evaluator model pairing rather than their credentials. proceedings.mlr.press
These were experimental results, not a count of people losing real jobs. Still, the possibility deserves attention: a candidate’s choice of writing assistant may influence how another assistant judges the application.
Wonderful. We taught the machines to help us write, and now we must investigate whether they like one another’s prose.
Here, the ghost is not simply an old prejudice copied from an archive. It may also be a preference created by the interaction between tools. The applicant brings the qualifications; the software brings an additional audition nobody mentioned.
Even the fairness test needs checking
Before we crown the audit as our new emperor, there is another complication.
Work listed by Microsoft Research in October 2026 reviewed thirty-six papers on gender and racial bias in generative AI résumé screening. The researchers then ran 1,134 variations of experimental designs. Changing design choices reversed the reproduced experiment’s conclusion around 22 percent of the time. Even the wording of a job description could matter. Microsoft Research
That does not make discrimination imaginary. It means the test conditions belong beside the result.
“Biased” and “fair” are substantial claims. We should know which model version was tested, on which applications, for which jobs, with which instructions and against which definition of fairness.
A clean certificate is reassuring. I would still like to know what was cleaned.
Why deleting the sensitive column is not enough
Removing race or gender from an input does not, by itself, establish that the resulting decisions are fair. The Stanford findings are a useful reminder that we must examine outcomes at the level where decisions affect people, rather than rely on a reassuring overall average. hai.stanford.edu
The newer cover-letter experiments add another question: what happens when the tools used to prepare an application interact with the tools used to judge it? That deserves testing alongside the more familiar demographic checks. proceedings.mlr.press
The sensible response is to test the actual process. Look at who advances, who disappears and whether seemingly irrelevant changes alter the result. Then investigate why.
Deleting a column is an action. Demonstrating fairness requires evidence.
Fix the model… and examine its job description
The original dilemma still stands: do we fix the model or the world it learned from?
I would first inspect the assignment.
What does “good candidate” mean here? Who decided? What evidence connects the score to doing the job well? And how much of the decision has quietly been handed to the score?
These questions matter because improving a prediction and justifying its use are separate pieces of work. A system might become more consistent while still measuring something poorly suited to the decision.
Human oversight needs a job description too. I want the reviewer to have time to inspect the recommendation, permission to disagree and a route for correcting mistakes. Otherwise, “a human checked it” can become another reassuring label pasted over an unopened box.
The ghost does not need consciousness. It needs an unquestioned recommendation and somewhere consequential to send it.
Five questions to ask when an algorithm decides something about you
You do not need a computer science degree to ask these. You need someone willing to answer.
- What is it actually predicting? Ask what the score measures and why that measure is relevant to the decision.
- Who was it tested on? Ask about results for the people and situations the system will actually encounter.
- Does the reassuring average hide uneven outcomes? Ask what happens when results are examined by position, group or context.
- Which version and conditions were audited? A test describes a particular setup; ask how closely it matches the system in use.
- Who can explain, challenge and change the decision? Find the person with authority to correct it.
A score you cannot question has been given rather a lot of power.
Your turn
The ghost in the code is an old human habit finding new accommodation: turning a judgment into a number, then treating the number as permission to stop asking questions.
This year’s research gives us more precise questions. About shared gatekeepers. About AI judging AI. About what an audit can actually establish.
So tell me in the comments: if you could audit one decision-making system in your own life—a hiring filter, a credit score or a search ranking—which would you choose, and what would you want explained?
Pour a coffee first. The emperor has brought a spreadsheet. ☕
Personalized learning wIth AI on Amazon
Website enhanced with Royal Addons & Neve theme by themeisle.
Human touch by Samburu & Fleeky
Website hosted on WA, with Agents and AI tools like niche & keyword search.
Article written with help of chatGPT and ACE
Images made with Kre8or and Picsart.
