Cloud Data Engineer

The purpose of the Cloud Data Ops Engineer is to develop and maintain analytical data flow solutions for multiple projects across businesses using multiple cloud technologies.

The Cloud Data Ops Engineer will work in a dynamic environment with a variety of ETL tools and Data Platforms which include Azure Data Factory, Synapse, Spark, Python and SQL. Part of the day-to-day activities would be to develop and monitor data flows using monitoring platforms and administer source and destination systems to ensure optimal information flow. Strong focus on Data Integrity and attention to detail.

Essential Responsibilities

• Provide data architecture leadership in guiding the development of conceptual, logical and physical data models to support BI, analytics, data science & machine learning models on a hosted cloud environment (Azure)
• Developing and maintaining Cloud ETL Processes on Azure (i.e. Python/Pyspark, SQL, Airflow, Data Factory, Synapse, Fabric, etc.)
• Develop a migration plan from existing on-premise infrastructure to cloud and document data architectures and data models
• Ability to work on Cloud based DB technology (Azure) to design and maintain data marts/delta lakes, integrating data from various on-prem databases and systems
• Administration of Cloud data platforms
• Troubleshooting and performance tuning of Python, Pyspark and SQL Code and design
• Writing functional and technical specifications for the design and implementation of analytical solutions
• Demonstrated ability to communicate and work well in an agile team environment
• Implement processes and automations that will ensure that the data is correct, reconciling data sources and identifying inconsistencies and data quality issues
• Establish processes that allow for cleaner integration and effective use of data within the business, ensuring final models are productionized in the data warehouse
  
  
  

Position Requirements

Education:

• Bachelor’s Degree and/or Masters Degree in Computer Science or STEM preferred
  
  
  

Experience:

• 4+ years of experience working as a Data and ETL Engineer, at least 2 of which in Azure
• 3+ years of experience supporting Data Operations
• 2+ years of customer interfacing experience – project oversight, customer presentation
• Experience in Data Analysis / Data Science is considered a plus
  
  
  

Skills/Knowledge:

• Solid experience in the Cloud, Data and Analytics space creating data lakes and data flows
• Support of Data Operations and related data platforms
• Applied experience with at least two of the following technologies: Synapse, Spark, Databricks, Fabric or similar
• Attention to detail and ability to focus on finding elegant solutions to existing data issues to promote data integrity
• Working with various types of data, ideally financial (SAP Environment ) / manufacturing data is considered a plus