NYC Local Law 144 · Independent Bias Audits
If a tool helps you hire,
the law says someone
independent has to check it.
Corymb conducts independent bias audits of automated employment decision tools: the analysis, the published summary of results, and a defensible record of every judgment call behind the numbers.
The rule is short. Complying with it is not.
Local Law 144 has applied in New York City since July 2023. If you use an automated employment decision tool to screen candidates or make promotion decisions for a role in NYC, you need a bias audit performed by an independent auditor within the past year, a summary of results published on your site, and notice to candidates. Penalties run to $500 for a first violation and up to $1,500 for each one after, and every day a violation continues counts separately.
What we actually do
Four ways this work usually starts. Most engagements are one of them; some are two.
The Local Law 144 bias audit
The full engagement. We take your historical assessment data, agree the method with you in writing, compute selection rates and impact ratios at every required threshold for sex, race and ethnicity, and their intersection, and deliver the summary of results you are required to publish, along with the working papers that let you defend it.
Vendor-commissioned, multi-employer audits
The rules let a vendor commission an audit of its own tool using historical data from multiple employers, and let employers rely on it provided they contribute their own data. Done well, one engagement covers a whole customer base. Done carelessly, it pools populations that should never have been pooled. We know the difference and we document it.
Data readiness review
A short, fixed-scope look at what you actually have before an audit starts. Which jobs have usable demographic coverage, where the identifiers break, whether your date fields support the window you intend to publish. Most of the cost overruns in this work trace back to something that could have been found here.
Review of an audit you already have
You have a report from someone else and you are not sure it is right. We check the method against the rules, re-derive the figures where the underlying data allows, and tell you plainly what we would have done differently and whether it matters.
How an audit runs
Most audits take two to four weeks from data handoff to published summary.
Scope and independence check
We establish what the tool does, which jobs and locations are in scope, and confirm we have no employment or financial interest in you or the tool. If we cannot be independent of a piece of work, we say so before we take it.
Data handoff and profiling
We profile every file before we analyze anything: coverage by job, demographic completeness, duplicate structure, identifier integrity, date fields.
The method, written down and agreed
Before any number is produced we write the analysis plan: how the tool's output maps to a selection rate, what counts as one implementation, how duplicates resolve, which categories may be excluded and under which provision. Every judgment call comes to you with a recommendation and the evidence behind it.
Reports, and the record behind them
You get the summary of results ready to publish, the individual reports behind it, and the audit trail: every cell traceable to source, every exclusion named with its legal ground and its figures, every limitation disclosed rather than buried.
Next year, and the year after
The obligation is annual. Because the pipeline and the agreed method are yours at the end of the engagement, the second audit is a re-run and a review rather than a rebuild, and we are still here to run it.
What this looks like in practice
A recent engagement, at the scale these audits actually reach.
A national AEDT vendor
The vendor commissioned an independent audit of its assessment tool.
- 108,066 supplied records reduced to 82,106 through a three-stage deduplication rule, with every dropped row accounted for
- 171 reports across 30 implementation units, plus a portfolio summary that ranks findings at both required thresholds
- 203 category cells retained and published that a stricter reading would have suppressed, because the rules authorize exactly one exclusion and it is not that one
- Every limitation in the data written into a disclosed annotation register rather than left for a reader to discover
What you get
The published summary is the legal requirement. The rest is what makes it defensible if anyone asks.
The published summary of results
Formatted and ready for your careers page, with the selection rates, impact ratios, and distribution date the rule requires.
Individual reports
One per tool and job, so you can answer a question about a specific role without re-reading everything.
The written analysis plan
Every methodological decision, its alternatives, the evidence, and who approved it.
A complete audit trail
Every published figure traceable to the records that produced it, exclusions named with their legal ground.
Related work
New York City was first, not last. The same evidence, meaning who your tool advantages, by how much, and whether you can show your working, answers most of what the newer regimes ask for. We help clients get ready for them without buying the same analysis twice.
State AI employment law readiness
Illinois amended its Human Rights Act to cover AI in employment decisions effective January 2026, and Colorado's law has been narrowed and pushed to January 2027. The requirements differ from NYC's. The underlying evidence largely does not.
Adverse impact analysis
The Uniform Guidelines four-fifths analysis that predates all of this, done properly, for selection procedures whether or not they are automated and whether or not a statute currently requires it.
EU AI Act preparation
Employment and worker-management systems are classified high-risk, bringing obligations around data governance, documentation, and human oversight. If you operate in both markets, the documentation should be built once.
Questions we get asked
Do we need an audit if our vendor already had one done?
Possibly not. The rules let you rely on a vendor's audit of its own tool, provided you contributed your historical data to it or the audit rests on other employers' data plus your own. The catch is that you are relying on someone else's method. We read the audit you were given and tell you whether it covers you.
What if we do not have enough demographic data?
Almost nobody has complete data. The rules anticipate this: missing categories are disclosed and counted, not quietly dropped. What matters is whether the gaps are disclosed honestly and whether the categories that remain are handled under a defensible rule. We will tell you before we start if we think your data cannot support a credible audit.
Can you suppress a result that looks bad?
No, and you should be wary of anyone who says otherwise. There is exactly one exclusion the rules authorize, a category representing less than 2% of the data, and even then the count and the selection rate must still be published with the auditor's justification. Thin cells get flagged for reliability. They do not get removed.
How often does this have to happen?
Every year. The audit has to have been conducted within the past twelve months for the tool to be lawfully used, so this is a recurring obligation rather than a one-time project, which is exactly why we leave you with a pipeline instead of just a PDF.
Is enforcement actually happening?
It has been light so far, but that is changing. A New York State Comptroller audit published in December 2025 found the city's enforcement of the law ineffective and pushed for improvements: more proactive review, better complaint handling, and closer scrutiny of the bias audits themselves. The reasonable planning assumption is more scrutiny, not less.
You do not need to know which audit you need before calling.
Bring the tool, the data you think you have, and the deadline you are working against. Initial conversations are exploratory and commitment-free.
Or email discovery@corymb.ai and we will figure out together whether you have a problem.
Corymb conducts independent bias audits and provides analysis. We are not a law firm and nothing on this page is legal advice. Confirm your specific obligations and deadlines with counsel.
What would you do with the time you're losing to broken processes?
Once a month we share what we are learning from real client work, what is new on the blog, and practical thinking on data and AI for businesses. Subscribe to our newsletter so you won’t miss it.
