Abridge
Machine Learning Infrastructure Engineer, Model Inference
About this role
Abridge seeks an ML Infrastructure Engineer to design and optimize the inference infrastructure powering our healthcare AI platform. You'll build scalable Kubernetes clusters, develop high-performance model serving systems, and collaborate with research and product teams to deploy and optimize AI models that transform clinical documentation.
What you'll do
- Design, deploy, and maintain scalable Kubernetes clusters for AI model inference and training
- Develop and optimize ML model serving infrastructure for high performance and low-latency delivery
- Collaborate with ML and product teams to scale backend infrastructure and optimize compute efficiency
- Optimize GPU utilization and compute-heavy workflows for ML workloads
- Build robust model API orchestration systems
- Define and implement infrastructure scaling strategies to support company growth
What they're looking for
- Kubernetes administration and container orchestration
- Distributed systems architecture and design
- ML model deployment and production experience
- API development and real-time/batch workload management
- Model serving frameworks (NVIDIA Triton Server, VLLM, TRT-LLM)
- GPU cluster management and CUDA optimization
- Infrastructure as code (Terraform, Ansible)
- PyTorch or TensorFlow experience
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Abridge
Abridge builds an AI platform that transforms healthcare conversations into clinical documentation using advanced natural language processing and generative AI. The company is hiring ML infrastructure engineers, implementation engineers, and machine learning scientists to develop model serving systems, manage customer deployments, and advance medical NLP research.
- Website
- abridge.com
Likely interview questions
- Walk us through your experience deploying and managing ML models in production. What were the biggest scaling challenges you faced, and how did you solve them?
- Describe your hands-on experience with Kubernetes. Have you built custom resource definitions or operators, and what problems were you solving?