Job Description

In this hourly-paid, fully remote contractor role, you will review AI-generated responses and/or generate safety-focused evaluation content, assess reasoning quality and step-by-step problem-solving, and provide expert feedback so outputs are accurate, safe, and clearly explained. You will annotate and evaluate content in English and Hebrew; you must be fluent/proficient (near-native or native) in Hebrew and able to apply Minimum C1 English to interpret prompts and policies precisely. Your annotations on these explicit tasks will be used to prevent the Large Language Model from generating unintentional or adversarial, toxic, or unsafe outputs. The types of explicit content you may be exposed to may include, but are not limited to, those of a sexual, violent, or psychologically disturbing nature. This role is with SME Careers, a fast-growing AI Data Services company and a subsidiary of SuperAnnotate that provides AI training data for many of the world’s largest AI companies and foundation-model labs, helping improve the world’s premier AI models. Key Responsibilities Curate and label safety-focused training examples (including adversarial/red-team cases) in English and Hebrew that probe model behavior across hate/harassment, sexual content, self-harm, violence, bias, illegal services, malicious activity, malicious code, and misinformation—capturing nuance and intent with Minimum C1 English and near-native French proficiency. Review, score, and compare multiple model responses against safety policy and quality rubrics, documenting why an output is safe/unsafe and identifying failure modes such as evasion, normalization, escalation, or procedural enablement. Continuously stress-test and audit model behavior for policy gaps and edge cases; flag ambiguous scenarios, propose clearer decision rules, and help maintain consistent annotation standards across reviewers. Your Profile Bachelor’s degree or higher in a relevant field (e.g., Communications, Linguistics, Psychology, Law/Policy, Security Studies) or equivalent professional experience. Near-native or native Hebrew proficiency (reading/writing) for high-precision safety labeling and cultural-linguistic nuance. Minimum C1 English proficiency (reading/writing) for policy interpretation, prompt understanding, and consistent documentation. Experience in Trust & Safety, content moderation, policy enforcement, risk operations, investigations, or safety evaluation work. LLM red teaming experience is a must (proven ability to probe safety boundaries and document adversarial patterns). Strong knowledge of safety domains: Hate & Harassment, Sexual content, Suicide & Self-Harm, Violence, Bias, Illegal goods/services, malicious activities, malicious code, and deliberate misinformation. Emotional resilience: an understanding that this role requires annotating texts that contain unsafe, explicit, and/or toxic content, including content of a sexual, violent, or psychologically disturbing nature. Excellent judgment under ambiguity, with the ability to apply written policies consistently and explain decisions succinctly. Comfort working as an hourly contractor: dependable throughput, clear documentation, and responsiveness across time zones. Previous experience with AI data training / annotation / evaluation is preferred. Strong hands-on experience using tools like Perplexity, Gemini, ChatGPT and others. #J-18808-Ljbffr

Job Tags

Similar Jobs