Baseten
Software Engineer - GPU Networking & Distributed Systems
About this role
Baseten seeks a GPU networking engineer to architect the software infrastructure that optimizes communication across thousands of GPUs for distributed AI inference. You'll integrate RDMA/RoCE capabilities, optimize distributed serving patterns, and co-design networking alongside compute to achieve wire-speed data movement on cutting-edge hardware.
What you'll do
- Integrate RDMA, RoCE, and InfiniBand as first-class primitives in the inference stack
- Optimize networking for disaggregated KV cache offload and wide expert parallelism across GPU clusters
- Design observability tools to visualize packet flow, congestion, and bandwidth across interconnects
- Characterize performance on next-generation hardware (B200/B300, Blackwell, NVL72) and write acceptance tests
- Implement custom communication kernels and optimize NCCL/NVSHMEM for overlapped compute-transfer
- Enable sub-10-second cold starts for large language models via checkpointing and storage optimization
What they're looking for
- High-performance networking protocols (InfiniBand, RoCE v2)
- C++ and Python with hardware optimization experience
- NVIDIA GPU architecture knowledge (H100, Blackwell, memory hierarchy)
- NCCL, NVSHMEM, and UCX libraries
- Distributed systems and performance debugging
- TensorRT-LLM, vLLM, or similar inference frameworks
- GPUDirect Storage or high-performance filesystem experience
- Hardware benchmarking and cluster qualification
Benefits
- Work on bleeding-edge hardware including Blackwell and Rubin architectures
- Recently raised $1.5B Series F funding led by top-tier investors
- Impact AI deployment for companies like Cursor, Notion, and OpenAI partners
- Deep technical challenges at the intersection of networking and compute
- San Francisco location with access to cutting-edge GPU clusters
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Baseten
Baseten builds an AI inference platform that enables companies to deploy and manage machine learning models in production at scale. The company is hiring software engineers, infrastructure specialists, and product engineers to develop observability systems, frontend experiences, reliability infrastructure, developer tools, and enterprise deployment solutions.
- Website
- baseten.com
Likely interview questions
- Walk us through your experience with RDMA, RoCE, or InfiniBand. How have you optimized networking performance in high-performance computing or distributed systems?
- Describe a time you had to choose between using an off-the-shelf networking solution and building a custom one. What was the trade-off, and how did you make the decision?