# LLMOps Engineer Job ALX Nairobi, Kenya Canonical URL: https://primejobalerts.com/jobs/llmops-engineer-job-alx-nairobi-kenya Company: ALX Nairobi, Kenya Category: Information & Communication Technology Location: Kenya Work type: Full-time Work arrangement: On-site Published: 2026-08-04 Expires: 2026-09-18 ## Job Description Role Summary - Project A is ALX’s AI learning platform — a set of LLM products used by learners. Every one generates a stream of LLM data, and every one has hypotheses baked into it about what “working” means. The LLMOps Engineer owns the analyzer function: turning that stream into an honest answer about whether the products work. Take RAG as one example — documents must be stored accurately, fetched accurately, and fetched in the right mixture: three separate failure modes, each needing its own eval. Every product decomposes like that. This is a junior-to-mid role with a deliberate growth path: you start close to the technical lead’s designs and grow into full ownership of the function. - You will work in collaboration with Anthropic Engineers, a cross functional  team of AI engineers, product managers and data scientists to design world class learning experiences. ### Specific Responsibilities Evaluation Suites - Build and run eval suites per product, decomposed by failure mode, running on schedule and on every release — regression testing so nothing ships if it broke what worked. - Keep evals cost-effective as the product line grows. Reporting, Data & Collaboration - Own the reporting loop — findings from evals and platform data in front of the team and stakeholders, including surfacing unintended or problematic model behaviour before learners do. - Steward the core datasets the team depends on, including classified customer-support data. - Partner with the AI Product Manager on instrumentation — they instrument the product, you build the evals over what is captured. This is a measurement role, not infrastructure — no model hosting or serving. Skill Requirements – Essential - Python & data: solid Python and a data inclination,  comfortable shaping and analysing messy LLM-generated data. - Decomposition: the ability to look at an AI product and decompose it into success and failure metrics. - Eval landscape: familiarity with Langfuse, RAGAS, DSPy, or similar — depth in one, awareness of the rest. These tools are learnable; we hire the fundamentals underneath them. - Desirable (not required): experience keeping evals cheap at scale; dashboarding and reporting; classical statistics. Essential Traits for Success - You want to own a function, not execute tickets. - You communicate well and like collaborating,  you will support every builder on the team. - You can point to any project, even a small one, where you measured an AI system honestly. ### How to Apply Click Here to Apply ## Apply Use the canonical Prime Job Alerts page for full application instructions and source verification.