Data Architect

Key Responsibilities:

• Design, develop, and optimize scalable data pipelines and workflows using Databricks and Apache Spark on AWS.
• Collaborate with data scientists, analysts, and other stakeholders to transform data into actionable insights.
• Implement data ingestion processes from various sources including S3, RDS, Redshift, and external APIs.
• Optimize Spark jobs for performance and cost efficiency.
• Monitor and troubleshoot data workflows, ensuring data quality and integrity.
• Automate deployment and orchestration of data pipelines using AWS services like Lambda, Step Functions, or Glue.
• Maintain security and governance standards in cloud data environments.
• Document processes, data lineage, and architecture designs.