Design, build, and operate end-to-end training and inference infrastructure for large language and multimodal models. Improve efficiency (memory, parallelism, kernel optimizations), ensure robust scalable training and RL pipelines, optimize low-latency/high-throughput serving (quantization, caching, speculative decoding), manage multi-GPU and multi-cloud orchestration, and productionize new algorithms with strong observability and reproducibility.
About Goaly
About the Role
Key Responsibilities
Requirements
Bonus Points
At Goaly, our mission is to make custom AI affordable for every business. Our founding team comes from the front lines of top AI labs and tech giants (Meta MSL, TikTok AI, Google DeepMind, xAI, Microsoft Research, etc.), where we built large-scale training infrastructure powering trillion-parameter models and scaled GenAI models to a global user base. Now, we are building something we wish we had before: a platform that makes training and adapting custom AI affordable for all modern companies, not just Big Tech. Our north star is ambitious: for a domain-specific task, reach 90% of SOTA performance at less than 10% of the cost. To get a taste of what we are doing, see our first tech blog.
You will sit at the intersection of systems engineering and applied ML, building specialized infrastructure that keeps large language and multimodal models fast, reliable, and cost-effective. You will partner with research, product, and infra teams to ship production-ready platforms for training and serving AI at scale.
- Efficiency & performance: Improve LLM training and inference efficiency through better memory utilization, optimized parallelism, and kernel-level innovations (e.g. FlashAttention, CUDA/Triton).
- Training & RL robustness: Build scalable, stable training and RL pipelines with strong reproducibility, observability, and debuggability.
- Serving & inference optimization: Design and tune high-throughput, low-latency model serving systems, including quantization, caching, and speculative decoding.
- Scalability & infrastructure: Own end-to-end training and inference infrastructure — from data ingestion and checkpointing to multi-GPU and multi-cloud orchestration.
- Production enablement: Work closely with researchers and product engineers to turn new algorithms into reliable, production-ready systems.
- 5+ years building or operating ML infrastructure at scale, ideally supporting large language or multimodal models.
- Deep understanding of GPU architecture, distributed training frameworks (PyTorch, DeepSpeed, Megatron, Ray), and parallelism strategies.
- Hands-on experience running inference stacks (vLLM / SGLang, TGI, Triton) and optimizing them via low-level profiling.
- Strong software engineering fundamentals in Python and one of C++/Rust/Go, with clean, reliable code shipped to production.
- Working knowledge of modern data pipelines, feature stores, and vector databases used in production AI systems.
- Comfort automating infrastructure with Kubernetes, Terraform/Pulumi, and observability stacks (Prometheus, Grafana, OpenTelemetry).
- Experience deploying open-source LLMs (Llama 3, Qwen, DeepSeek) or training custom foundation models.
- Contributions to ML systems tooling (compilers, kernels, inference runtimes) or open-source infrastructure projects.
- Background in reinforcement learning, evaluation harnesses, or alignment tooling that hardens production AI systems.
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Manage end-to-end implementation and adoption of Samsara’s IoT solutions for mid-market customers. Responsibilities include creating launch plans, tracking implementation progress and account health, delivering product training, demonstrating technology, resolving challenges, and coordinating with Sales, Support, Sales Engineering, and Product. The role requires strong customer communication, consultative implementation skills, technical product experience, stakeholder management, and fluency in English and French.
Top Skills:
Equipment MonitoringIotSaaSSamsara Connected Operations CloudVehicle Telematics
Information Technology • Software • Travel
Build and maintain a mature SaaS platform across Rails backend, JSON APIs, MySQL, and Vue.js frontend. Develop travel features, partner integrations, AI-powered functionality, and document generation; optimize performance, write automated tests, upgrade dependencies, address security issues, investigate production incidents, review code, and document work. The role requires autonomous feature ownership, careful work in a large codebase, and collaboration through Git and pull requests.
Top Skills:
Amazon CloudfrontAmazon S3Aws EcsAws LambdaCapybaraDockerFactorybotGitGithub ActionsJavaScriptJson ApisMySQLRspecRubyRuby On RailsSentryVueVuetify
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Leads complex, cross-functional technology programs and projects from initiation through delivery. Establishes governance, delivery plans, milestones, dependencies, risk frameworks, and stakeholder cadences. Partners with Product and Engineering leadership to translate strategic roadmaps into executable release plans, manages critical-path issues and escalations, and drives release readiness across multiple teams. The role also improves program execution, release governance, transparency, quality, and delivery velocity in a healthcare technology environment.
Top Skills:
AgileAha!Hybrid Delivery ModelsJIRASoftware Development Lifecycle (Sdlc)
What you need to know about the Ottawa Tech Scene
The capital city of Canada and the nation's fourth-largest urban area, Ottawa has proven a rapidly growing global tech hub. With over 1,800 tech companies, many of which are leaders in their sectors, the city's tech talent now makes up more than 13 percent of its total workforce. This growth is driven not only by the big players like UL Solutions and Dropbox, but also by a thriving startup ecosystem, as new businesses emerge to follow in the footsteps of those that came before them.



