Data Engineer - Spark/Python

<p>Job Description : <br/></p><p><br/></p><p>- 5-10 years of meaningful experience in data engineering working on building ETL pipelines, big data tech like spark.<br/><br/>- Very good expertise in algorithms and data structures<br/><br/>- Excellent in at least one programming language: Python, Go, R<br/><br/>- Hands-on expertise big data technologies such as Apache Spark, Hadoop, message queues(Kafka), Apache Airflow, Azure ML, Elastic Search<br/><br/>- Databases: Hands-on expertise in data modeling, database schema definition, query optimization, usage of relevant data stores (blob storage, data warehouse, MongoDB, Timescale DB),<br/><br/>- Hands-on expertise working on analytics platforms such as Azure synapse or equivalent<br/><br/>- Good understanding of performance analysis and optimizations<br/><br/>- Hands-on experience in at least one cloud platform - Azure(preferred), AWS, or GCP<br/><br/>- B.E/B.Tech/M.E/M.Tech/M.S in Computer/Electronics Engineering or equivalent degrees with experience<br/><br/></p><p>Nice to have : <br/><br/>- Knowledge of data visualization tools such as Power BI, Tableau<br/><br/>- Working knowledge of the "Internet of Things" desired<br/><br/></p><p>- Knowledge of internet-scale cloud-based distributed systems<br/><br/>- Good understanding of MQTT, Web Sockets, Azure Service Bus, Apache Pulsar<br/><br/>- Good knowledge of Domain-Driven Design, Test Driven Development<br/><br/>- Experience developing in an Agile / SCRUM Environment<br/><br/></p><p>Soft skills : <br/><br/>- Self-driven, results-oriented, collaborative, comfortable with a fast-paced, always-on, highly ambiguous start-up like environment<br/><br/>- Good with rapid problem solving, a solid technical background, and a passion for continuous learning and improvement<br/><br/>- Good verbal and written communication skills</p> (ref:hirist.tech)