Anti-discrimination filter

Purpose and deployment context

The Anti-discrimination filter reviews job advertisements on creation or editing. The aim is to alert the employer to wording that may conflict with the prohibition of discrimination and to prevent publication without an amendment or a documented genuine occupational requirement. Employers, candidates and internal moderators are affected.

Classification under the AI Act

Under Regulation (EU) 2024/1689 (the AI Act), we assess this component as outside Annex III, point 4(a), because it serves exclusively to review the wording of job advertisements and does not serve for the recruitment or selection of natural persons. According to the interpretive guidance on the AI Act, a system used exclusively to identify non-inclusive or discriminatory wording in an advertisement description does not fall within the use case of Annex III, point 4(a). We keep this classification under ongoing review and it is subject to a final legal assessment. Even outside the high-risk systems regime, the Operator voluntarily maintains technical documentation of the rules, evaluation results, a model card, post-market monitoring and an override audit; serious incidents would be reported to the competent authorities under the AI Act and Czech implementing legislation.

Inputs and outputs

The input is the Czech title and description of the position, or English phrases within the text. The text is normalised to lowercase, diacritics are folded and it is compared against the rules. The PII categories processed are limited to the content that the employer itself inserts into the advertisement; the system does not require candidates' personal data. The output is hasIssue, a list of categories, the section found and a Czech rewrite recommendation.

Models used and versions

The component is a rule-based classifier (it does not use an LLM, embeddings or fine-tuning), rule version from 2026-05-12. The categories in the code are age, gender, nationality, race_ethnicity, religion, sexual_orientation, disability, family_status, political_views and union_membership. A knowledge cutoff does not apply; the rules are versioned in the repository.

Training data

The model is not trained by gradient. The rules are based on an internal Czech evaluation set of 140 training rows for tuning the rules and 60 unseen test rows with no duplicates between the splits. No prompt is used. The legal framework is based on Act No. 198/2009 Coll., Directive 2000/78/EC and Directive 2000/43/EC.

Limitations and known risks

The rules may produce false positives for citations of the law, legitimate requirements of physical fitness or parallel gender forms. False negatives are a risk with new coded expressions, irony and very indirect discrimination. Coverage is tuned mainly for Czech and basic English.

Fairness and anti-discrimination

The filter explicitly protects the categories set out in Act No. 198/2009 Coll. and related regulations. The mitigations include Czech rewrite suggestions, the option to document a genuine occupational requirement, an override audit, an internal review and an internal fairness-audit methodology. Last audit: 2026-05-14, passed.

Item	Value
Last audit	2026-05-14, passed

A broader production audit is planned for Q1 after launch.

Human oversight

With no finding, the advertisement continues in the normal flow. On a finding, the employer must amend the text or document a BFOQ/GORD justification. The override is stored in the audit and is visible to administrators in the review process; disputed cases can be manually approved, rejected or sent back for amendment by Trust & Safety.

Performance metrics

On the unseen internal split, the filter has a macro-F1 of 0.815. Per-category F1: age 0.750, gender 0.667, nationality 0.667, race and ethnic origin 0.857, religion 0.889, sexual orientation 1.000, disability 0.857, family status 0.857, political opinions 0.750, trade-union membership 0.857. Production drift will be measured from the Sprint 5 launch window.

Security and privacy

The review runs locally in the application code and does not send the advertisement text to a third party. The processing of personal data is governed by Regulation (EU) 2016/679 (GDPR); for the validation of advertisements, the legal basis is the performance of the contract with the employer under Article 6(1)(b) GDPR and legitimate interest in the prevention of discrimination and misuse of the service under Article 6(1)(f) GDPR. The override audit records contain text sections, categories, justifications and the identity of the approving user; access is to be restricted to authorised administrators. Details are in the privacy policy.

Complaints and contact

Send complaints about a flag or a request for review to [email protected]. Direct formal personal data protection requests to the privacy contact in the privacy policy.

Card version

Card version 1.0.3, last updated 2026-07-08.