Data Architect - Governance & Security

Technical Skills

• Databricks Lakehouse Platform:

o Medallion Architecture o Unity Catalog (data governance, lineage)

o Delta Lake & DLT Pipelines

o PySpark Workbooks o Spark SQL & SQL Warehouse

• Programming: Python, SQL, PySpark

• AWS Cloud Services: IAM, S3, Lambda, EMR, Redshift, Bedrock

• Other:

o Familiarity with DevOps and CI/CD processes

o Experience with any Data Security tool

Key Responsibilities

• Establish, enforce, and monitor Data Engineering governance standards, best practices, and guidelines across enterprise and self-service environments.

• Develop and maintain documentation for data engineering processes

• Define and implement data security policies and role-based access controls (RBAC/ABAC) across all data engineering processes.

• Oversee data classification, lifecycle management, and comprehensive metadata management to ensure transparency, traceability, and compliance.

• Implement and manage robust change control processes for all data engineering activities

• Monitor, maintain, and optimize data pipelines and workflows, ensuring reliability, scalability, and efficiency.

• Continuously monitor and optimize the performance of data engineering processes and resource utilization.

• Promote a culture of data engineering governance, data security awareness, and operational excellence within the team and across the organization.