<p>The candidate must be excited by the prospect of optimizing- or even re-designing data pipelines to support the modernization of data platform</p><p><br/></p><p>- Be curious and eager to work across a variety and volume of data.<br/><br/></p><p>- Design and build scalable and reliable data pipelines with the latest tech stack.<br/><br/></p><p>- Facilitate real-life actionable use cases leveraging- our data with a user- and product-oriented mindset.<br/><br/></p><p>- Support teams without data engineers with building decentralized data solutions and product integrations.<br/><br/></p><p>- Conceptualize, design and implement improvements to ETL processes and data through independent communication with data-savvy stakeholders.<br/><br/></p><p>- Designing, building, and operating a Data Lake or Data Warehouse.<br/><br/></p><p>- Knowledge to ingest, cleanse, transform and load data from varied data sources in the above Azure Services (Databricks and DataFactory)<br/><br/></p><p>- Strong knowledge of Medallion architecture<br/><br/></p><p>- Consume data from source with different file format such as XML, CSV, Excel, Parquet, JSON<br/><br/></p><p>- Strong problem-solving skill such as backtracking of dataset, data analysis etc.<br/><br/></p><p>- Strong Knowledge of in advanced SQL techniques for carrying out data analysis as per client requirement.<br/><br/></p><p>Mandatory Skills :<br/><br/></p><p>The candidates need to understand different data architecture patterns and parallel data processing.<br/><br/></p><p>- S/he should be proficient in using the following services to create data processing solutions :<br/><br/></p><p>1.
Azure Databricks<br/><br/></p><p>2.
Azure Data Lake Storage<br/><br/></p><p>3.
Azure Data Factory<br/><br/></p><p>- Strong Knowledge in :<br/><br/></p><p>1.
PySpark<br/><br/></p><p>2.
SQL<br/><br/></p><p>- Should be familiar to be built data pipelines and data analysis using Python<br/><br/></p><p>Desired Skills :<br/><br/></p><p>- Ability to query the data from serverless SQL Pool in Azure Synapse Analytics.<br/><br/></p><p>- Knowledge of Azure DevOps.<br/><br/></p><p>- Knowledge to configure any dataset with Vnet, Subnet Networks<br/><br/></p><p>- Knowledge of Microsoft Entra ID, to create App registration for single and multitenant for security purpose.</p> (ref:hirist.tech)