Senior Data Engineer (Azure Synapse)

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

About Quantiphi:

Quantiphi is an award-winning, AI-First global digital engineering company that helps the world’s leading Fortune 1000 organizations transform bold ideas into measurable business impact. We go beyond building innovative AI technologies, we solve the problems that matter most to our clients.

Since our founding in 2013, Quantiphi has built a proven track record of turning complex challenges into meaningful outcomes across industries.

Headquartered in Boston, with more than 4,000 professionals worldwide, we partner with global enterprises to deliver large-scale digital, cloud, and AI-driven transformation. #SolvingWhatMatters

We are an Elite and Premier partner to Google Cloud, AWS, NVIDIA, Snowflake, and other leading technology platforms, and our work has been recognized across the industry, including:

• 21 Google Cloud Partner of the Year awards in the past 10 years
• 3 AWS AI/ML Partner of the Year awards
• 3 NVIDIA Partner of the Year awards
• 3 Snowflake Partner of the Year awards
• Rated Leaders by Gartner, Forrester, IDC, ISG, Everest Group and other leading analyst firms

Quantiphi delivers First-in-class AI solutions across  Life Sciences, Healthcare, Banking, Financial Services, CPG, Manufacturing, Energy, High-Tech, Telecommunications, etc., powered by cutting-edge Generative AI and Agentic AI accelerators.

We are also proud to be certified as a Great Place to Work, reflecting our commitment to our people and our culture.

For more details, visit: Website or LinkedIn Page.

Experience Level: 6+  years of experience  

Work Location: Remote Canada

What you’ll do:  

We are seeking a highly skilled and experienced Senior Data Engineer with a strong focus on Azure Synapse Analytics to join our innovative team within the healthcare domain. In this critical role, you will be instrumental in designing and delivering robust big data pipelines for both structured and unstructured data, operating across multiple geographies. Your expertise will directly contribute to helping healthcare organizations achieve their business goals through advanced data ingestion technologies, cloud services, and DevOps practices. You will collaborate closely with Architects from various specialties, including Cloud Engineering, Software Engineering, and ML Engineering, to create cutting-edge platforms, solutions, and applications. These initiatives will address the latest trends in the healthcare industry, such as digital diagnosis, software as a medical product (SaMD), and AI marketplaces.

Role & Responsibilities:

• Azure Synapse Expertise: Work with cloud engineers and customers to solve complex big data problems by developing utilities for migration, storage, and processing specifically leveraging Azure Synapse Analytics.
• Cloud Migration Strategy: Design and build a comprehensive cloud migration strategy for both cloud and on-premise applications, with a strong emphasis on integrating with Azure Synapse.
• Troubleshooting & Optimization: Diagnose and troubleshoot complex distributed systems problems within the Azure ecosystem, developing solutions with a significant impact at massive scale, particularly within Synapse environments.
• Large-Scale Data Processing: Build tools to ingest and jobs to process several terabytes or petabytes per day, optimizing performance and cost within Azure Synapse.
• Next-Gen Storage & Compute: Design and develop next-generation storage and compute solutions for several large customers, utilizing the full capabilities of Azure Synapse.
• Cross-Functional Collaboration: Communicate effectively with a wide set of teams, including Infrastructure, Network, Engineering, DevOps, SiteOps teams, and cloud customers, ensuring seamless integration and delivery.
• Advanced Tooling: Build advanced tooling for automation, testing, monitoring, administration, and data operations across multiple Azure Synapse workspaces and associated cloud clusters.

Must Have:

• 3+ years of hands-on Data Engineering experience with strong knowledge of data structures, distributed systems, data modeling, and databases. 
• Strong proficiency in SQL and hands-on experience with Python/PySpark. 
• Extensive experience in developing Big Data solutions (migration, storage, processing) with a proven track record of implementing solutions on cloud platforms.
• Experience building and supporting large-scale systems in a production environment.
• Strong experience developing ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines. 
• Cloud Platforms: Deep expertise in Azure, specifically Azure Synapse Analytics (SQL Pools, Spark Pools, Data Explorer, Pipelines).
• Big Data Distributions: Experience with Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2).
• Distributed Processing Frameworks: Strong experience with Apache Spark (especially within Azure Synapse Spark Pools).
• Database/Warehouse: Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other relevant Azure data services.
• Orchestration Frameworks: Azure Data Factory, Apache Airflow, or similar.
• Message/Event Solutions: Azure Event Hubs, Azure Service Bus, Kafka.
• Container Orchestration (Good to have): Kubernetes (Azure Kubernetes Service - AKS).
• Leadership Qualities:
  • Ability to lead technology teams and provide them mentorship/support to accelerate performance.
  • Experience in leading multiple large projects as well as a deep understanding of Agile developments.
  • Effective communication with all stakeholders involved.
  • Communicate clearly about complex subjects and technical plans with technical and non-technical audiences.

Good to Have:

• Experience working with Insurance data/domain.
• Experience with Apache Airflow, Databricks, dbt, Microsoft Fabric, or Snowflake.
• Experience with Kubernetes / AKS.
• Knowledge of Terraform / Infrastructure as Code and Azure DevOps.
• Experience with data governance, data quality, lineage, and metadata management..
• Team management and mentoring.

Behavioral Skills:

• Effective communication with all stakeholders involved.
• Need to communicate clearly about complex subjects and technical plans.
• Ability to mentor and groom junior members of the team and provide them with guidance and roadmap.
• Must also have the ability to interact with other members of the team (Juniors, Seniors, ATA, TA, BA, etc.) to get their designs from concept to development.
• Keeping various audiences in mind, engineers must write their reports in clear language accessible to all.

• Make an impact at one of the world’s fastest-growing AI-first digital engineering companies.
• Upskill and discover your potential as you solve complex challenges in cutting-edge areas of technology alongside passionate, talented colleagues.
• Work where innovation happens - work with disruptive innovators in a research-focused organization with 60+ patents filed across various disciplines.
• Stay ahead of the curve—immerse yourself in breakthrough AI, ML, data, and cloud technologies and gain exposure working with Fortune 500 companies.