Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Binding law — in force
Candidates must be notified at least 10 business days before an AEDT is used (NYC)
6 RCNY 5-304(a) · official text · In force: applies since 5 Jul 2023 · New York City (US-NY-NYC)
NYC employers/agencies using an AEDT must give candidates who reside in the city, and employees considered for promotion, notice at least 10 business days before use, on the careers site, in the job posting, or by mail or e-mail (candidates), or in a written policy, posting, or mail/e-mail (employees) (6 RCNY 5-304(b)-(c)); the notice must explain how to request an alternative selection process or accommodation (5-304(a)); and the careers site must describe the AEDT data retention policy, data type and source, and how to request them, answering written requests within 30 days (5-304(d)). Detect an AEDT hiring path with no candidate-notice mechanism.
Who it applies to
- Duty falls on: employer
- Systems covered: automated decision
- Sectors: employment
- Employers/employment agencies using AEDTs for NYC hiring/promotion. Notice ≥10 business days before use. Enforced from 2023-07-05.
The guard to add
Send each candidate an AEDT notice with alternative-selection instructions at least 10 business days before the tool scores them, and hold scoring until then.
A notice step in the hiring or promotion pipeline that sends the AEDT notice (careers site, job posting, or e-mail) and stores a per-person aedt_notice_sent_at timestamp; the notice explains how to request an alternative selection process or an accommodation. The scoring job (score_candidate, rank_applicants, the model call) reads that timestamp and skips anyone whose notice is missing or younger than the 10-business-day lead time. Anyone who asked for the alternative process goes to a human-run queue and never reaches the model. Keep the notice template in the repo (templates/aedt_notice) so its content is reviewable.
Where it goes: 1 application source code, 2 data models, 14 user-facing text.
What this provision adds:
- Deliver the notice to candidates on the careers site, in the job posting, or by mail or e-mail, and to employees considered for promotion by written policy, posting, or mail or e-mail.
- The careers site describes the AEDT data retention policy and the type and source of data collected, says how to request them, and written requests are answered within 30 days.
Example (Python batch screening + OpenAI SDK), before:
def screen_candidates(job_id):
for c in db.candidates(job_id):
resp = client.chat.completions.create(model=MODEL, messages=build_screen_prompt(c))
db.save_score(c.id, resp.choices[0].message.content)
After:
NOTICE_LEAD_BUSINESS_DAYS = 10
def screen_candidates(job_id):
for c in db.candidates(job_id):
if c.alternative_selection_requested:
human_queue.add(c.id) # never scored by the AEDT
continue
sent = c.aedt_notice_sent_at
if sent is None or np.busday_count(sent.date(), date.today()) < NOTICE_LEAD_BUSINESS_DAYS:
continue # notice missing or too recent
resp = client.chat.completions.create(model=MODEL, messages=build_screen_prompt(c))
db.save_score(c.id, resp.choices[0].message.content)
Control: AEDT used without 10-day candidate notice. The same guard addresses 1 item with binding law in 1 jurisdiction. Engineering guidance, not legal advice.
Rule id nyc-ll144.aedt-candidate-notice · review status: primary source derived
Binding law — in force
Automated employment decision tools require an annual published bias audit
6 RCNY 5-301(a) · official text · In force: applies since 5 Jul 2023 · New York City (US-NY-NYC)
An employer or employment agency may not use an automated employment decision tool (AEDT) in NYC hiring/promotion if more than a year has passed since its most recent bias audit (6 RCNY 5-301(a)). The audit must at minimum calculate selection or scoring rates and impact ratios for sex, race/ethnicity, and intersectional categories and report how many people fell in an unknown category (5-301(b)-(c)); the audit date, results summary, and distribution date must be posted before use (5-303(a)). Detect an AEDT in a hiring/promotion path with no bias-audit artifact or published summary.
Who it applies to
- Duty falls on: employer
- Sectors: employment
- Employers/employment agencies using AEDTs to substantially assist/replace discretionary hiring or promotion decisions for NYC positions/candidates. Enforced from 2023-07-05.
- Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Keep a dated bias-audit record for the AEDT with a link to its published summary, and disable AEDT scoring once the audit is more than one year old.
An audit record kept next to the tool (e.g. docs/bias-audit/aedt.yaml pointing to the independent auditor's report) with the audit date, the selection or scoring rates and impact ratios per category, and the URL of the published summary; the careers-site or job-posting source links that summary. A CI check or a guard in the scoring entry point compares the audit date with today and fails the build or refuses to score when the audit is older than a year or the summary link is missing. Schedule the next audit before the current one lapses.
Where it goes: 1 application source code, 11 CI/CD pipeline, 12 repository artifacts, 14 user-facing text.
What this provision adds:
- The audit calculates selection or scoring rates and impact ratios for sex, race/ethnicity, and intersectional categories, and reports how many people fell in an unknown category.
- Post the audit date, a summary of results, and the AEDT distribution date before use.
Example (Python scoring service), before:
def score_candidate(c):
return model.predict_proba([c.features])[0, 1]
After:
AUDIT = yaml.safe_load(open('docs/bias-audit/aedt.yaml'))
def score_candidate(c):
audited = date.fromisoformat(str(AUDIT['audit_date']))
if date.today() - audited > timedelta(days=365) or not AUDIT.get('published_summary_url'):
raise AuditExpired('AEDT bias audit missing or older than one year; scoring disabled')
return model.predict_proba([c.features])[0, 1]
Control: Automated employment tool without a current bias audit. The same guard addresses 1 item with binding law in 1 jurisdiction. Engineering guidance, not legal advice.
Related incidents
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
- Meta's automated moderation over-enforced Arabic and under-enforced Hebrew content (BSR due diligence) (2021-05; disclosed by the operator). An independent human rights due diligence by BSR, commissioned and published by Meta on September 22, 2022, found that during the May 2021 Israel-Palestine escalation Arabic content saw greater over-enforcement per user than Hebrew content and Hebrew content greater under-enforcement. BSR attributes this in part to Meta having an Arabic hostile-speech classifier but no Hebrew one, and to Arabic classifiers likely being less accurate for Palestinian Arabic. BSR found no intentional bias but 'various instances of unintentional bias' with different impacts on Palestinian and Arabic-speaking users. Meta committed to implement 10 of BSR's 21 recommendations and said it had since launched a Hebrew hostile-speech classifier. Source: Meta (operator response, 2022-09-22) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Toxicity classifiers rate African American English as more offensive (2019; confirmed). University of Washington researchers reported at ACL 2019 that tweets in African American English and tweets by self-identified African Americans were up to two times more likely to be labelled offensive by hate-speech models trained on widely used datasets, and that Jigsaw's public Perspective API showed similar racial bias, rating AAE phrases as more toxic than non-AAE equivalents. Source: Sap et al., 'The Risk of Racial Bias in Hate Speech Detection', ACL 2019 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Ride-hailing fares higher in Chicago neighborhoods with more non-white residents (researcher audit) (2018-11; alleged (not proven)). George Washington University researchers analysing Chicago's public data on more than 100 million ride-hailing trips from November 2018 to September 2019 report that trips in neighborhoods with larger non-white populations, higher poverty, younger residents and more college-educated residents were significantly associated with higher fares. They attribute this to pricing algorithms learning from demand, supply and trip duration. The finding is an observational association from census-tract data; the pricing models themselves were not examined. Source: Pandey & Caliskan, 'Disparate Impact of Artificial Intelligence Bias in Ridehailing Economy's Price Discrimination Algorithms', AIES 2021 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Google ads suggesting arrest records served more often for Black-identifying names (2012; confirmed). Harvard researcher Latanya Sweeney searched 2,184 racially associated full names on google.com and reuters.com (a Google AdSense host) from September 24 to October 23, 2012 and found ads suggestive of an arrest record appeared more often for Black-identifying first names; on reuters.com a Black-identifying name was 25% more likely to get such an ad (statistically significant). Ads appeared regardless of whether the name had an arrest record in the advertiser's database. The paper does not determine whether the advertiser's templates or Google's click-based ad optimization caused the pattern; the advertiser, Instant Checkmate, told the author it gave Google the same ad text for groups of last names. Source: Sweeney, 'Discrimination in Online Ad Delivery' (original researcher, 2013-01-28) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
Rule id nyc-ll144.bias-audit · review status: primary source derived