Senior Software Engineer - LLM Ops & Evals
Plaats
netherlandsNederland
Nederland
Contract
Vooltijds, Onbepaalde duurDatum gepubliceerd
2026-09-30Online tot
2026-11-29EUR 20.00 HOUR
Senior Software Engineer - LLM Ops & Evals
- DataSnipper
- Nederland netherlands
- 5 uur geleden
Which one is yours?
Time to find the right role for you at DataSnipper
The time is now DataSnipper is a place where you can dive into an idea and bring your unique touch to work. How do we do it? We don't just settle for the way things are, we listen to each other, we look out for each other, we sit down to see how we could improve together. So, we're always looking for people who think differently.
Open positions Senior Software Engineer - LLM Ops & Evals Location Amsterdam
Employment Type Full time
Location Type Hybrid
Department Engineering
Every AI call in DataSnipper goes through us. Product teams do not talk to model providers directly, they talk to our gateway. We are responsible for how inference is routed, how it fails over, what it costs, and how anyone can tell whether the output is any good.
The second half of the job is evaluation. We are building the platform teams use to measure AI quality: versioned datasets, experiment tracking, and evaluation runs they can act on. It is a hard problem and largely an open one, so you will have real influence over how we solve it.
This is a small team with a large blast radius. You will own real production systems, set the standards other teams build against, and see your work in front of hundreds of thousands of users in audit and finance.
About DataSnipper Audit and finance are still massively manual and we are changing that. DataSnipper is a $1B, bootstrapped unicorn with 600,000+ users across 180+ countries, already embedded in the daily workflows of top audit and accounting firms. Now, we are taking things further with our Excel Agent, bringing AI directly into where the work actually happens. Unlike generic AI tools, we do not sit on the sidelines. Our AI operates inside Excel, with access to real documents and audit evidence, meaning it does not just generate answers, it does the work, with full traceability. We are not just applying AI, we are redefining how audit gets done. If you want to build something category-defining at scale, this is the place.
What you will do Technical Delivery
Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers
Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code
Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run
Own the infrastructure for async and long-running AI workloads
Reliability, Security & On-Call
Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case
Take part in the on-call rotation, run incidents, and close the follow-ups
Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO
Collaboration & Impact
Define and maintain clean integration contracts between the platform and the teams that consume it
Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue
What you will bring Must-Have
5+ years in backend or platform engineering, with strong production Python
Experience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution
Experience with cloud at the infrastructure level and infrastructure as code (we run across Azure and GCP with Terraform)
Experience running a shared service in production: on-call, incidents, postmortems, SLOs
Hands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts
Comfort with privacy and compliance work: PII handling, anonymisation, retention, access control
Experience building platform or shared-service capabilities consumed by multiple internal teams
Nice-to-Have
Experience with LLM or agent evaluation
Temporal or another durable workflow engine
Self-hosted inference, capacity planning, load testing
Synthetic data or document anonymisation pipelines
Document AI: VLMs, OCR, structured extraction and the metrics that go with it
Domain experience in audit, accounting or fintech
What we expect
Ownership : You own work end-to-end, anticipate issues, and ensure high-quality delivery without close supervision
Growth Mindset : You encourage open feedback exchange and provide clear, balanced feedback that helps others grow
Collaboration : You build strong cross-functional relationships and influence peers through expertise, data, and empathy
Adaptability : You navigate ambiguity calmly, model positive behavior, and help peers adjust through clear communication
Judgment : You exercise sound judgment in ambiguous situations, balance speed and accuracy, and adjust priorities proactively
Recruitment steps
Recruiter screen
Hiring Manager interview
Peer programming session
System design interview
Final interviews with Engineering leadership
Ready to join? Blaze a new path into your career and take the next step
#J-18808-Ljbffr
The time is now DataSnipper is a place where you can dive into an idea and bring your unique touch to work. How do we do it? We don't just settle for the way things are, we listen to each other, we look out for each other, we sit down to see how we could improve together. So, we're always looking for people who think differently.
Open positions Senior Software Engineer - LLM Ops & Evals Location Amsterdam
Employment Type Full time
Location Type Hybrid
Department Engineering
Every AI call in DataSnipper goes through us. Product teams do not talk to model providers directly, they talk to our gateway. We are responsible for how inference is routed, how it fails over, what it costs, and how anyone can tell whether the output is any good.
The second half of the job is evaluation. We are building the platform teams use to measure AI quality: versioned datasets, experiment tracking, and evaluation runs they can act on. It is a hard problem and largely an open one, so you will have real influence over how we solve it.
This is a small team with a large blast radius. You will own real production systems, set the standards other teams build against, and see your work in front of hundreds of thousands of users in audit and finance.
About DataSnipper Audit and finance are still massively manual and we are changing that. DataSnipper is a $1B, bootstrapped unicorn with 600,000+ users across 180+ countries, already embedded in the daily workflows of top audit and accounting firms. Now, we are taking things further with our Excel Agent, bringing AI directly into where the work actually happens. Unlike generic AI tools, we do not sit on the sidelines. Our AI operates inside Excel, with access to real documents and audit evidence, meaning it does not just generate answers, it does the work, with full traceability. We are not just applying AI, we are redefining how audit gets done. If you want to build something category-defining at scale, this is the place.
What you will do Technical Delivery
Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers
Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code
Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run
Own the infrastructure for async and long-running AI workloads
Reliability, Security & On-Call
Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case
Take part in the on-call rotation, run incidents, and close the follow-ups
Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO
Collaboration & Impact
Define and maintain clean integration contracts between the platform and the teams that consume it
Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue
What you will bring Must-Have
5+ years in backend or platform engineering, with strong production Python
Experience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution
Experience with cloud at the infrastructure level and infrastructure as code (we run across Azure and GCP with Terraform)
Experience running a shared service in production: on-call, incidents, postmortems, SLOs
Hands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts
Comfort with privacy and compliance work: PII handling, anonymisation, retention, access control
Experience building platform or shared-service capabilities consumed by multiple internal teams
Nice-to-Have
Experience with LLM or agent evaluation
Temporal or another durable workflow engine
Self-hosted inference, capacity planning, load testing
Synthetic data or document anonymisation pipelines
Document AI: VLMs, OCR, structured extraction and the metrics that go with it
Domain experience in audit, accounting or fintech
What we expect
Ownership : You own work end-to-end, anticipate issues, and ensure high-quality delivery without close supervision
Growth Mindset : You encourage open feedback exchange and provide clear, balanced feedback that helps others grow
Collaboration : You build strong cross-functional relationships and influence peers through expertise, data, and empathy
Adaptability : You navigate ambiguity calmly, model positive behavior, and help peers adjust through clear communication
Judgment : You exercise sound judgment in ambiguous situations, balance speed and accuracy, and adjust priorities proactively
Recruitment steps
Recruiter screen
Hiring Manager interview
Peer programming session
System design interview
Final interviews with Engineering leadership
Ready to join? Blaze a new path into your career and take the next step
#J-18808-Ljbffr
Meer banen van je zoekopdracht
Monteur Technische Dienst Mechatronica | 2-ploegen | Smaakmakers
- Dronten
- 30 september 2026
Hybrid Data Scientist: Grow with AI & Cloud Solutions
- netherlands
- 30 september 2026
Teamleider Logistiek - for Netherland residents only
- Tilburg
- 30 september 2026
Facilitair medewerker afvalinzameling
- Arnhem
- 30 september 2026
Cad Tekenaar
- Emmen
- 30 september 2026
Geneesmiddelenonderzoek BR0078-2753-GRQ (Vergoeding van €6.829,-)
- Almelo
- 30 september 2026
Terreinmedewerker
- Groot-Ammers
- 30 september 2026
Senior Software Engineer - LLM Ops & Evals
- netherlands
- 30 september 2026
BHV-Toezichthouder COA Nunspeet - for Netherland residents only
- Nunspeet
- 30 september 2026
Productiemedewerker stansvormen (direct in dienst)
- Hengelo
- 30 september 2026
Horecamedewerker bijbaan La Place De Wuust
- Oirlo
- 30 september 2026
Geneesmiddelenonderzoek BA0680-0066-GRQ (Vergoeding tot €3.327,-)
- Harderwijk
- 30 september 2026
Reachtruckchauffeur
- Huissen
- 30 september 2026
Technisch TOPdesk Consultant
- Zoetermeer
- 30 september 2026
Assistent Sommelier Hemel & Aarde
- Utrecht
- 30 september 2026