<p><p>We are seeking a highly skilled and motivated Data Engineer to join our data platform team.<br/><br/> The ideal candidate will have hands-on experience in building and optimizing data pipelines and architectures using Databricks and Snowflake.<br/><br/> You will play a crucial role in managing our data ecosystem, ensuring data accessibility, quality, and performance to drive key business decisions.</p><br/><p><b>Key Responsibilities :</b><br/></p><p><br/></p><p>- Design & Development : Architect, build, and maintain robust and scalable ETL/ELT data pipelines using Databricks (PySpark/Scala) and Snowflake.
This includes ingesting, transforming, and loading data from various sources into our data platform.<br/></p><p><br/></p><p>- Data Modeling & Warehousing : Develop and manage dimensional and relational data models within Snowflake, ensuring efficient data organization and high-performance querying for analytics and reporting.<br/></p><p><br/></p><p>- Performance Optimization : Troubleshoot, debug, and optimize existing data workflows and SQL queries in Databricks and Snowflake to improve performance and reduce computational costs.<br/></p><p><br/></p><p>- Collaboration : Work closely with cross-functional teams, including data scientists, data analysts, and business stakeholders, to understand their data requirements and deliver reliable, well-structured datasets.<br/></p><p><br/></p><p>- Governance & Quality : Implement and maintain data governance frameworks, including data quality checks, lineage tracking, and security measures (e.g., RBAC, ACLs) within both Databricks and Snowflake.<br/></p><p><br/></p><p>- Automation & Orchestration : Utilize orchestration tools like dbt (Data Build Tool), Airflow, or Azure Data Factory to automate data pipelines and ensure seamless data flow.<br/></p><p><br/></p><p>- Innovation : Stay up-to-date with the latest features and best practices in Databricks and Snowflake, and propose new solutions to improve our data infrastructure.</p><br/><p><b>Required Skills & Qualifications :</b><br/></p><p><br/></p><p>- Bachelor's degree in Computer Science, Engineering, or a related technical field.<br/></p><p><br/></p><p>- 3-6 years of hands-on experience as a Data Engineer or in a similar role.<br/></p><p><br/></p><p>- Proficiency in Python and advanced SQL is a must.<br/></p><p><br/></p><p>- Strong experience with Databricks for data processing, pipeline orchestration, and developing Spark applications.<br/></p><p><br/></p><p>- Extensive experience with Snowflake as a cloud data warehouse platform, including a deep understanding of its architecture (Virtual Warehouses, Snowpipe, etc.).<br/></p><p><br/></p><p>- Solid understanding of ETL/ELT processes, data modeling (Kimball, Inmon), and data warehousing concepts.<br/></p><p><br/></p><p>- Experience with at least one major cloud platform (AWS, Azure, or GCP).<br/></p><p><br/></p><p>- Knowledge of Azure is a plus.<br/></p><p><br/></p><p>- Familiarity with data governance, metadata management, and observability frameworks.<br/></p><p><br/></p><p>- Excellent problem-solving, analytical, and communication skills.<br/></p><p><br/></p><p>- Ability to work independently and as part of a team in a fast-paced, agile environment.</p><br/><p><b>Good to Have Skills :</b><br/></p><p><br/></p><p>- Experience with dbt (Data Build Tool) for data transformation and data modeling.<br/></p><p><br/></p><p>- Knowledge of streaming technologies like Kafka or Kinesis.<br/></p><p><br/></p><p>- Familiarity with CI/CD tools such as Jenkins, Azure DevOps, or GitHub Actions.<br/></p><p><br/></p><p>- Relevant certifications, such as Databricks Certified Data Engineer or Snowflake SnowPro Core</p><br/></p> (ref:hirist.tech)