Movable Ink Logo

Movable Ink

Principal Data Engineer - Data Platform & Ingress

Reposted 9 Days Ago
Be an Early Applicant
Easy Apply
Remote
Hybrid
Hiring Remotely in Toronto, ON
Expert/Leader
Easy Apply
Remote
Hybrid
Hiring Remotely in Toronto, ON
Expert/Leader
As a Principal Data Engineer, you will drive the direction of the Data Warehouse, enabling data access across departments and designing data pipelines to handle massive data ingestion and ensure compliance with data regulations. You will mentor less experienced team members and optimize pipeline performance.
The summary above was generated by AI

Movable Ink scales content personalization for marketers through data-activated content generation and AI decisioning. The world’s most innovative brands rely on Movable Ink to maximize revenue, simplify workflow and boost marketing agility. Headquartered in New York City with close to 600 employees, Movable Ink serves its global client base with operations throughout North America, Central America, Europe, Australia, and Japan.

As a Principal Data Engineer, you will help drive the direction of our Data Warehouse and Hybrid Data Lake Infrastructure. You will be hands-on with teammates across all departments, enabling them to access the data and empowering teams to make data-driven decisions about the direction of our business. You will play a pivotal role in owning the infrastructure and code for our data pipelines within our Data Platform, with a focus on handling data at scale. You will be responsible for designing, implementing, and optimizing ingestion pipelines, enabling seamless extraction and loading of data from various sources such as customer warehouses, business systems, and event messaging buses. Movable Ink collects campaign data resulting from billions of requests served each day—come and help us manage and make sense of the massive amount of data we’re ingesting!

Responsibilities:

  • Partner with internal operations teams to identify, collect, and integrate data from various business systems, ensuring comprehensive and accurate data capture.
  • Design, implement, and maintain robust batch and real-time data pipelines, leveraging tools like Apache Airflow, Apache Flink, and Terraform for IaC.
  • Build and optimize Hybrid Data Lake / Data Warehouse infrastructure with solutions like Apache Iceberg for scalable and cost-effective storage.
  • Ensure data pipelines adhere to best practices and are optimized for performance, scalability, and reliability.
  • Conduct thorough testing of data pipelines to validate data accuracy and integrity.
  • Monitor data pipelines, implement telemetry and alerting, troubleshoot any issues that arise, and proactively improve system reliability.
  • Establish and track SLAs for data processing and delivery, ensuring timely and reliable access to data for all users.
  • Become a mentor for less experienced team members and establish patterns and practices that increase the quality, accuracy, and efficiency of solutions produced by the team.
  • Design and implement Change Data Capture (CDC) solutions to support real-time data replication and point-in-time data queries.
  • Work with other teams to ensure secure data access and compliance with regulatory requirements (e.g., GDPR, CCPA, etc.).

Qualifications:

  • 12+ years of professional experience in data engineering, software engineering, database administration, business intelligence, or related fields, with 8+ years as a Data Engineer focused on cloud-based Data Warehouses(Redshift, Snowflake, Firebolt, BigQuery). We currently use Redshift.
  • Deep experience working with multi-petabyte, mission-critical databases, optimizing for high availability, performance, and reliability, informed by a strong understanding of database internals.
  • Expert proficiency with Python and SQL, and significant experience building robust data pipelines with these languages.
  • Expert proficiency in deploying and managing data pipeline orchestration frameworks such as Apache Airflow or Prefect. We currently use Apache Airflow.
  • Significant experience with Infrastructure-as-Code (Terraform) and automating cloud infrastructure management.
  • Significant experience with stream processing technologies such as Apache Flink, Apache Kafka, or Apache Pulsar.
  • Significant experience in building telemetry, monitoring, and alerting solutions for large-scale data pipelines.
  • Significant experience in implementing Hybrid Data Lake / Data Warehouse architectures, with a focus on Apache Iceberg or similar technologies.
  • Significant experience in designing and implementing solutions that comply with regulatory requirements such as GDPR and CCPA.
  • Experience in Agile/Scrum environments, working with technical managers and product owners to break down high-level requirements into actionable work.
  • Excellent communication skills, with the ability to effectively collaborate across technical and business teams.

Studies have shown that women, communities of color, and historically underrepresented people are less likely to apply to jobs unless they meet every single qualification. We are committed to building a diverse and inclusive culture where all Inkers can thrive. If you’re excited about the role but don’t meet all of the abovementioned qualifications, we encourage you to apply. Our differences bring a breadth of knowledge and perspectives that makes us collectively stronger.

We welcome and employ people regardless of race, color, gender identity or expression, religion, genetic information, parental or pregnancy status, national origin, sexual orientation, age, citizenship, marital status, ethnicity, family or marital status, physical and mental ability, political affiliation, disability, Veteran status, or other protected characteristics. We are proud to be an equal opportunity employer.

What you need to know about the Ottawa Tech Scene

The capital city of Canada and the nation's fourth-largest urban area, Ottawa has proven a rapidly growing global tech hub. With over 1,800 tech companies, many of which are leaders in their sectors, the city's tech talent now makes up more than 13 percent of its total workforce. This growth is driven not only by the big players like UL Solutions and Dropbox, but also by a thriving startup ecosystem, as new businesses emerge to follow in the footsteps of those that came before them.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account