Seeking an experienced Data Engineer to design and implement data pipelines, collaborate with teams, and manage diverse data sets while ensuring data governance.
We are seeking a talented and experienced Data Engineer to join our team at Provectus. As part of our diverse practices, including Data, Machine Learning, DevOps, Application Development, and QA, you will collaborate with a multidisciplinary team of data engineers, machine learning engineers, and application developers. You will encounter numerous technical challenges and have the opportunity to contribute to Provectus’ open source projects, build internal solutions, and engage in R&D activities, providing an excellent environment for professional growth.
Requirements
- Experience in data engineering;
- Experience working with Cloud Solutions (preferably AWS, also GCP or Azure);
- Experience with Cloud Data Platforms (e.g., Snowflake, Databricks);
- Proficiency with Infrastructure as Code (IaC) technologies like Terraform or AWS CloudFormation;
- Experience handling real-time and batch data flow and data warehousing with tools and technologies like Airflow, Dagster, Kafka, Apache Druid, Spark, dbt, etc.;
- Proficiency in programming languages relevant to data engineering such as Python and SQL;
- Experience in building scalable APIs;
- Experience in building Generative AI Applications (e.g., chatbots, RAG systems);
- Familiarity with Data Governance aspects like Quality, Discovery, Lineage, Security, Business Glossary, Modeling, Master Data, and Cost Optimization;
- Advanced or Fluent English skills;
- Strong problem-solving skills and the ability to work collaboratively in a fast-paced environment.
Nice to Have:
- Relevant AWS, GCP, Azure, Databricks certifications;
- Knowledge of BI Tools (Power BI, QuickSight, Looker, Tableau, etc.);
- Experience in building Data Solutions in a Data Mesh architecture;
- Familiarity with classical Machine Learning tasks and tools (e.g., OCR, AWS SageMaker, MLFlow, etc.).
Responsibilities:
- Collaborate closely with clients to deeply understand their existing IT environments, applications, business requirements, and digital transformation goals;
- Collect and manage large volumes of varied data sets;
- Work directly with Data Scientists and ML Engineers to create robust and resilient data pipelines that feed Data Products;
- Define data models that integrate disparate data across the organization;
- Design, implement, and maintain ETL/ELT data pipelines;
- Perform data transformations using tools such as Spark, Trino, and AWS Athena to handle large volumes of data efficiently;
- Develop, continuously test and deploy Data API Products with Python and frameworks like Flask or FastAPI.
Top Skills
Airflow
Apache Druid
AWS
Aws Cloudformation
Azure
Dagster
Databricks
Dbt
GCP
Kafka
Python
Snowflake
Spark
SQL
Terraform
Similar Jobs
Fintech • HR Tech
The Senior Machine Learning Engineer will build and maintain ML infrastructure, develop MLOps solutions, and collaborate on API-enabled services and deployments.
Top Skills:
AWSAws SagemakerJavaKubeflowMlflowPythonRuby
Fintech • HR Tech
The Senior Staff Product Designer will lead the design of Gusto's new Corporate & Tax Compliance product line, collaborating with cross-functional teams to define product strategy and improve customer experiences. This leader will advocate for high design quality and contribute to a collaborative product design community.
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Develop systems and APIs for applications, debug issues, and work with modern technologies in a fast-paced environment.
Top Skills:
C#GitiOSJava
What you need to know about the Ottawa Tech Scene
The capital city of Canada and the nation's fourth-largest urban area, Ottawa has proven a rapidly growing global tech hub. With over 1,800 tech companies, many of which are leaders in their sectors, the city's tech talent now makes up more than 13 percent of its total workforce. This growth is driven not only by the big players like UL Solutions and Dropbox, but also by a thriving startup ecosystem, as new businesses emerge to follow in the footsteps of those that came before them.
