Principal Data Engineer

25WD84745

Position Overview  

Autodesk is looking for diverse engineering candidates to join Licensing and Compliance team, a collection of systems and teams focused on detecting and exposing non-compliant users of Autodesk software. This is a key strategic initiative for the company! 

As a Principal Data Engineer, you will be responsible for developing best practices and making architectural decisions to rapidly improve critical data processing & analytics pipelines. You will tackle hard problems to improve the platform’s reliability, resiliency, and scalability.

We are looking for someone who thrives on autonomy and has experience driving long-term projects to completion. You are detail and quality oriented, and excited about the prospects of having a big impact with data at Autodesk. Our tech stack includes Airflow, Hive, EMR, PySpark, Presto, Jenkins, Snowflake, Datadog and various AWS services. 

You will report to a Data Engineering Manager and work hybrid in the Toronto office.

Responsibilities 

• Modernizing and expanding a critical set of existing systems 

• Understand business requirements and architect systems that will scale and extend to accommodate those needs

• Break down complex problems, document technical solutions and sequence work to make fast, iterative improvements

• Build and scale data infrastructure that powers batch and real-time data processing of billions of records

• Develop CI/CD ETL pipelines using Jenkins for deployment and automation testing

• Take ownership of ETL pipelines, own data quality for your areas, and ensure SLAs are met

• Interface with data engineers, data scientists, product managers and all data stakeholders to understand their needs and promote best practices

• Lead the design and implementation of complex data processing systems, ensuring they meet business requirements and performance standards

• Manage and coordinate the efforts of a team of data engineers, ensuring alignment with project goals and timelines

• Provide technical guidance and mentorship to junior engineers, fostering a culture of continuous learning and improvement

Minimum Qualifications

• 9+ years of relevant industry experience in big data systems, data processing and SQL data warehouses

• At least 5+ years of experience on working with large Hadoop projects and building scalable applications using PySpark by leveraging Spark DataFrame and Spark SQL api.

• At least 2+ years of experience in optimizing Spark applications, spark memory management, and e2e spark process optimization

• Strong overall programming skills, able to write modular, maintainable code, preferably Python & SQL 

• Fundamental knowledge of distributed data processing systems, storage mechanisms, and compression techniques.

• Experience building code-driven infrastructure on public cloud platforms, preferably AWS 

• Understanding of SQL, dimensional modeling, and analytical data warehouses, like Snowflake, Redshift

• Knowledge of Data Modeling techniques and high volume ETL/ELT design.

• Familiar with workflow management tools, like Airflow and Oozie 

• Experience with CI and code management tools like Git, Jenkins

• Familiar with BI tools, like Power BI, Looker, Qlikview

• Problem solver with excellent written and interpersonal skills; ability to make sound, complex decisions in a fast-paced, technical environment 

• Bachelor’s degree in Computer Science, Engineering or related field, or equivalent training, fellowship, or work experience

• Product-focused mindset

#LI-JG1

Learn More

About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.

When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us!

Salary transparency

Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, we also have a significant emphasis on discretionary annual cash bonuses, commissions for sales roles, stock or long-term incentive cash grants, and a comprehensive benefits package.

Diversity & Belonging
We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging

Are you an existing contractor or consultant with Autodesk?

Please search for open jobs and apply internally (not on this external site).