Know ATS Score
CV/Résumé Score
  • Expertini Resume Scoring: Our Semantic Matching Algorithm evaluates your CV/Résumé before you apply for this job role: Infocepts Senior Data Engineer Azure Databricks.
India Jobs Expertini

Urgent! Infocepts - Senior Data Engineer - Azure Databricks Job Opening In Pune – Now Hiring InfoCepts Technologies Pvt. Ltd.

Infocepts Senior Data Engineer Azure Databricks



Job description

<p><p><b>Description : </b></p><br/><p><b>Key Responsibilities : </b></p><br/><p>Data Pipeline Development : </p><br/><p>- Design and implement robust batch and real-time streaming data pipelines leveraging Azure Databricks and Apache Spark.</p><br/><p>- Build scalable and maintainable ETL/ELT workflows to support data ingestion, transformation, and integration from diverse sources.</p><br/><p><b>Data Architecture Implementation : </b></p><br/><p>- Apply the Medallion Architecture to organize data into distinct layers raw, enriched, and curated ensuring clarity, quality, and usability of data assets.</p><br/><p>- Collaborate in designing data models and schemas that support analytics and business intelligence needs.</p><br/><p><b>Data Quality & Governance : </b></p><br/><p>- Define and enforce data quality rules and principles to maintain data accuracy and consistency across the pipeline.</p><br/><p>- Implement data governance policies and manage metadata using tools like Azure Purview and Unity Catalog.</p><br/><p>- Monitor data lineage and ensure compliance with organizational and regulatory standards.</p><br/><p><b>Performance Optimization : </b></p><br/><p>- Optimize Spark jobs, Delta Lake tables, and SQL queries for improved efficiency, reduced latency, and cost-effectiveness.</p><br/><p>- Identify and resolve bottlenecks in data processing workflows to ensure high availability and reliability.</p><br/><p><b>Collaboration & Delivery : </b></p><br/><p>- Work closely with data analysts, solution architects, business stakeholders, and other engineering teams to deliver end-to-end data solutions aligned with business goals.</p><br/><p>- Participate in Agile/Scrum ceremonies and contribute to continuous improvement initiatives within the team.</p><br/><p><b>Required Qualifications : </b></p><br/><p>- Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or related fields.</p><br/><p>- 5+ years of experience in data engineering, with strong expertise in Azure Databricks and Apache Spark.</p><br/><p>- Proficiency in data integration techniques, ETL/ELT processes, and data pipeline design.</p><br/><p>- Experience implementing Medallion Architecture or similar data layering concepts.</p><br/><p>- Knowledge of Delta Lake and experience managing large-scale data lakes.</p><br/><p>- Familiarity with data quality frameworks and governance tools such as Azure Purview and Unity Catalog.</p><br/><p>- Strong SQL skills and experience optimizing complex queries for performance.</p><br/><p>- Experience with cloud data storage, processing, and orchestration tools within Azure ecosystem.</p><br/><p>- Good understanding of data security, privacy, and compliance requirements.</p><br/><p>- Excellent communication, collaboration, and problem-solving skills.</p><br/></p> (ref:hirist.tech)


Required Skill Profession

Computer Occupations



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your Infocepts Senior Potential: Insight & Career Growth Guide