Data Engineer

Website-https://intraedge.com/
Job Title: Data Engineer
Experience: 5+Years
Location: [Remote]
Senior Data Engineer
The Senior Data Engineer will design, develop, monitor and maintain a robust and scalable data platform used by other data analyst and engineering teams to deliver powerful insights to both internal and external stakeholders
Responsibilities
● Design, build and maintain data infrastructure that powers both batch and realtime processing of billions of records a day.
● Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection
● Design, build and maintain a central data cataloging system to ease integration and discovery of datasets
● Develop data pipelines that provide fast, optimized, and robust end-to-end analytical solutions
● Automate manual processes and create a platform in favor of self-service data consumption
● Deploy and configure components to production environments.
● Participate in on-call schedule to provide emergency incident support
● Mentor and train teammates on design and operation of data platform
● Stay current with industry trends, making recommendations as needed to help the company excel
● Other job-related duties as assigned Skills,
Experience and Qualifications
● Bachelor’s Degree in Computer Science or Engineering a plus
● 5+ years of relevant industry experience in Data Engineering working with large scale data driven systems
● Deep knowledge of dimensional modeling and designing schemas and data sets optimized for OLAP environment
● Experience fine-tuning queries around large, complex data sets
● Extensive experience working with an MPP data platform (ie Snowflake, BigQuery, or Databricks)
● Experience designing and implementing workflows on a modern orchestration framework (ie Airflow, Prefect)
● Advanced Data Build Tool (DBT) experience (ie macro design, generalizing tests, using different incremental strategies)
● Deep understanding of SQL and data warehouse systems, especially Snowflake (SnowPro certification preferred)
● Experience working with and administering a BI tool (Tableau preferred)
● Expertise in object-oriented and/or functional programming languages (Python preferred)
● Strong programming skills, able to write modular, maintainable code
● Understanding of DevOps principles such as automating of CI/CD pipelines and Infrastructure as code
● Understanding of polyglot data persistence (relational, key/value, document, column)
● Excellent problem-solving skills and the ability to proactively solve issues
● Excellent communication and organizational skills and proven ability to complete tasks and meet deadlines
● Ability to be flexible with working in tandem with a team of engineers or alone, as required