Software Engineer (AI Platform)

Are you passionate about AI? 🤖 At Solstice Analytics Analytics, we aim to change the world one algorithm at a time by bringing clarity to global brands through Data & AI. From cloud-based ecosystems for fintech to predictive models for airlines, our cutting-edge solutions cover the entire data lifecycle—from ingestion to AI applications. As a fast-growing scale-up, our team of 100+ tech specialists—including Data Engineers, Data Scientists, and more—delivers innovative analytics solutions across industries like FMCG, retail, manufacturing and FSI. Join us as we lead the data revolution in South-Eastern Europe and beyond! Together with a partnering company, we're looking for a strong Software Engineer to build the systems behind an AI agent evaluation platform — systems that test, grade, and stress AI agents at scale, and turn the results into actionable signal about how those agents behave.This is a software engineering role first — designing services, data models, and pipelines that have to be correct, tested, and maintainable — applied to an AI problem domain. You'll work with LLMs regularly, but the core of the job is engineering, not prompt-tuning. What Your Day Might Look Like: Backend services & APIs : Build and maintain the services, data models, and APIs that power the platform — designed for correctness, testability, and scale. Simulation & orchestration : Work on the systems that coordinate complex, multi-step interactions between AI agents and external systems, improving their reliability and throughput. Evaluation & scoring : Design systems that grade agent outputs, combining deterministic checks with model-assisted judgment — and make scoring reliable, explainable, and reproducible. Data pipelines : Build pipelines that generate, transform, and quality-check large volumes of structured data and benchmark content. Quality & reliability : Add the tests, instrumentation, and safeguards needed to trust outputs from systems that are inherently non-deterministic. Requirements Your Superpowers 🚀 4+ years building and shipping production software, with strong proficiency in Python. Deep software engineering fundamentals: system and API design, data modeling, concurrency/async, testing strategy, debugging, and code review. You can own a non-trivial service end-to-end. Experience designing and operating distributed or service-oriented systems (queues, workers, APIs) — not just calling them. Comfort designing schemas and working with relational databases, plus the migrations and performance concerns that come with them. Working knowledge of LLM APIs — orchestration, structured outputs, and handling non-determinism. We expect you to use LLMs effectively, but this is not a prompt-engineering role. Ability to reason about correctness of probabilistic systems: how to test, measure, and trust outputs that aren't byte-for-byte deterministic. High quality bar: you write tests, types, and docs by default, and you keep changes small and reviewable. Bonus points for: Experience building agentic or multi-agent systems, tool-use, or orchestration frameworks. Background in evaluation / benchmarking of ML or LLM systems (rubrics, golden datasets, model-as-judge, inter-rater reliability). Experience with distributed task queues and async workloads. Modern Python tooling and typed codebases (e.g. type checkers, linters, Pydantic, FastAPI). Retrieval / search experience and working with data ingest pipelines. Some comfort with the infra side (Docker, CI/CD) so you can ship what you build. Benefits Perks on Perks Competitive salary. Training budget to level up your skills from top tech partners like Microsoft, AWS, Salesforce, and Databricks – whether it’s certifications or courses, we’ve got you covered. Private insurance, top-tier tech gear, and the chance to work with a stellar crew. Ready to create some data magic with us? Hit that apply button and let’s get started. ✨ Originally posted on Himalayas