Software Engineer at Haystack

Job Description

Software Development Engineer, AI/ML, AWS Neuron, Model Inference | Cupertino, California | Remote-Friendly | $129,300 – $223,600

We’re working with Annapurna Labs (U.S.) Inc. on this exciting opportunity.

Join the Annapurna Labs team at AWS and contribute to the cutting-edge AWS Neuron SDK, accelerating deep learning and GenAI workloads on custom ML accelerators. This role offers a unique opportunity to work at the intersection of machine learning, high-performance computing, and distributed architectures, shaping the future of AI acceleration technology.

Key Responsibilities:

• Architect and implement business critical features, and mentor a brilliant team of experienced engineers.
• Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
• Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
• Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize system-level performance across multiple generations of Neuron hardware.

What You’ll Need:

• Experience optimizing inference performance for both latency and throughput on large models across the stack from system level optimizations through to Pytorch or JAX.
• Strong software development using Python, System level programming and ML knowledge.
• Expertise in low-level optimization, system architecture, and ML model acceleration.
• Bachelor’s degree in computer science or equivalent.

What’s On Offer:

• Work in a startup-like development environment, where you’re always working on the most important initiative.
• A builder’s culture where experimentation is encouraged, and impact is measurable.
• An environment that celebrates knowledge-sharing and mentorship, with one-on-one mentoring and thorough code reviews.

Apply via Haystack today!