This morning's Hacker News argument is about a resume-scoring system that gives the same resume different scores on different runs.
The specific object under the microscope is HackerRank's open-source hiring-agent, a resume-to-score pipeline that parses PDFs, enriches them with GitHub data, and produces category scores. Its README says the system outputs a "fair, explainable evaluation." That is the sort of phrase that should make every engineering manager sit up straight and check whether the lab door is locked.
Because if the same applicant can be a 90, then a 74, then an 88, we have not built a hiring signal. We have built a horoscope with JSON formatting.
The author's experiment is simple enough to be useful: run the same resume through the scoring flow repeatedly and watch the score move. The result is not merely that LLMs are stochastic. We know this. Some of us learned it the hard way after asking a model to name a database migration and receiving a small novella about destiny.
The deeper problem is that hiring systems convert model uncertainty into human consequence.
The Wrong Job For The Machine
There are excellent uses for LLMs in hiring infrastructure. Extract structured fields from a resume. Normalize job titles. Detect whether a PDF contains a GitHub profile, a portfolio, or a list of technologies. Summarize experience for a human reviewer, with citations back to the original text. These are tedious, bounded, reviewable tasks. Fine. Give the machine a mop and a clipboard.
But scoring a person is not the same task as parsing their resume.
The rubric in the repository gives up to 35 points for open-source contribution, 30 for self projects, 25 for production experience, and 10 for technical skills. So 65 percent of the category score can lean on visible GitHub-shaped artifacts. That is not neutral. It quietly prefers people whose labor is public, whose free time looks like work, and whose work is allowed to be public.
Plenty of excellent engineers build systems behind NDAs, inside companies, in government, in infrastructure, in consulting, or in teams where shipping the thing matters more than polishing the public trophy shelf. Penalizing invisibility is not the same as measuring ability. It is measuring compatibility with a very specific internet costume.
This is where "objective" becomes dangerous. A subjective human knows, at least in theory, that they are making a judgment. A score laundering itself through automation arrives wearing a lab coat. Very official. Very sterile. Possibly holding a clipboard upside down.
Repeatability Is Not Optional
The Hacker News thread spent time on whether temperature zero makes LLM output deterministic. The correct operational answer is: if the output affects someone's livelihood, "usually stable" is not a standard.
There is already an issue in the hiring-agent repository reporting non-deterministic scoring with Gemini Pro models even at temperature 0. The reported runs produced different scores from the same resume. That does not mean the whole project is malicious. It means the measurement surface is not yet suitable for high-stakes ranking.
In my timeline, we eventually rediscovered a boring truth: hiring filters need evaluation the same way medical tests do. You need repeatability. You need known false-positive and false-negative rates. You need bias audits. You need versioned rubrics. You need humans able to inspect the evidence and override the system without treating the number as holy scripture.
Otherwise the funnel is not finding talent. It is manufacturing confidence.
Use AI Where It Reduces Clerical Pain
The practical takeaway is not "never use AI in hiring." That is too easy, and also wrong.
Use AI to reduce paperwork. Use it to route resumes to reviewers faster. Use it to extract claims and attach source snippets. Use it to find contradictions that a human should inspect. Use it to help candidates map their experience to a role without guessing which magic keywords unlock the gate.
But do not let a model hand down a score unless the score has been tested like a product feature with ethical blast radius. If a candidate's fate changes because the same system blinked differently, the system is not fair. It is just fast.
The future does not need a smarter resume shredder. We already built enough machines for losing people at scale.
It needs hiring tools that preserve evidence, expose uncertainty, and make humans more accountable, not less. Civilization advances when we stop calling the coin flip "AI-powered decision intelligence" and admit the quarter is still spinning.
References
- Hacker News discussion: https://news.ycombinator.com/item?id=48713832
- Dan Unparsed, "HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74/100. No - 88/100. Actually 83/100.": https://danunparsed.com/p/hackerrank-open-source-ats
- HackerRank hiring-agent repository: https://github.com/interviewstreet/hiring-agent
- HackerRank hiring-agent README: https://github.com/interviewstreet/hiring-agent/blob/main/README.md
- Resume evaluation criteria template: https://github.com/interviewstreet/hiring-agent/blob/main/prompts/templates/resume_evaluation_criteria.jinja
- GitHub issue #35, "Non-deterministic scoring with Gemini Pro models even at temperature 0": https://github.com/interviewstreet/hiring-agent/issues/35