Principal Data Engineer

Job description

Company Profile:

Quantiphi is an award-winning Applied AI and Big Data software and services company, driven by a deep desire to solve transformational problems at the heart of businesses.

Our signature approach combines groundbreaking machine-learning research with disciplined cloud and data-engineering practices to create breakthrough impact at unprecedented speed.

Some company highlights:

● Quantiphi has seen 2.5x growth YoY since its inception in 2013

● Winner of the Machine Learning Partner of the year award from Google for two consecutive years - 2017 and 2018

● Winner of the Social Impact Partner of the year award from Google for 2019

● Headquartered in Boston, with 700+ data science professionals across different offices

Role & Responsibilities:

● Data Modeler with strong Conceptual, Logical and Physical Data Modeling skills, having

experience in requirements gathering, creating data mapping documents, writing functional

specifications, queries for Data Warehouse, Data Lake, Lakehouse, DataMart, OLTP, OLAP

applications.

● Extensive experience in Entity-Relational, Data Vault and Dimensional Data modeling for creating

ER Diagrams using Erwin, ER/Studio, Enterprise Architect and PowerDesigner data modeling

tools.

● Integral part of the Data Management team performing Data Extraction, Data Analysis, Data

Cleansing, Data Mapping and Data Profiling exercises using Informatica Analyst and other

profiling tools.

● Worked on industry standard models such as ACORD for insurance,

● Involved in the Database Design for Operational Data Store working with DBAs and ETL

Architects using Data Vault Modeling approach.

● Design, develop, and implement highly scalable data capture and transformation processes

● Suggest best modeling approach to the client based on their requirement and target architecture

● Analyze and understand the Datasets and guide the team in creating Source to Target Mapping

and Data Dictionaries,capturing all relevant details

● Profile the Data sets to generate relevant insights

● Define Data Modelling Best Practices and ensure implementation of Best Practices across

projects

● Optimize the Data Models and work with the Data Engineers to define the Ingestion logic,

ingestion frequency and data consumption patterns

● Work with QE team to define the right testing strategy

● Drives automation in modeling activities

● Collaborate with Business Stakeholders, Data Owners, Business Analysts, Architects to design

and develop next generation data platform

● Closely monitor the Project progress and provide regular updates to the leadership teams on the

milestones,impediments etc

● Create effective ETLs/ELTs to move large volumes of data from various operational systems to

dimensional data models for analytics consumption

● Act as principal designer and reviewer for new data models, make data architectural decisions

● Work with the database engineering and DBAs to create optimal physical data models (transactions,

normalized and dimensional models etc.)

● Manage data modeling repository, and data modeling process and support model-driven

development

● Define and govern data modeling/design standards, tools, best practices, and related development

methodologies

● Champion Data Lineage, Metadata Management, and Data Quality Analysis processes

● Collaborate with Business Data Analysts to design and model Business Intelligence Semantic layers for

optimized information access

● Expand and grow data existing platform capabilities to solve new data problems and challenges

● Ensure all automated processes preserve data integrity by managing the alignment of data availability

and integration processes

● Identify opportunities for new data acquisition and new uses for existing data resources

● Research and make recommendations for new data management technologies and software

engineering practices.

Collaborate on decisions around the use of new tools and practices

● Define data retention policy, establish data governance best practices, and create automated anomaly

detection services

● Document and update business continuity and disaster recovery procedures

● Engage in ongoing collaboration with data architects, modelers, and other members to achieve

common goals

● Provide guidance to development teams regarding best practices and design patterns for analytics

solutions.

Coach and provide guidance to junior team members

● Produce and maintain support documentation for ongoing operations.

● Act as support to troubleshoot and resolve technical issues with production data models and services

● Excellent communication, influencing, and facilitation skills, in particular for

problem-solving/troubleshooting activities

● Experience in client-facing roles

● Assertiveness in dealing with people at all levels within and outside of the delivery organization,

including partner and client organizations

Required Skills:

● 10+ years of experience with hybrid data environments that leverage both distributed and relational database technologies to support analytics services (Oracle, IMB DB2, GCP)

● Solid understanding of data warehousing principles, architecture, and its implementation in complex environments.

● Good experience in OLTP and OLAP systems

● Excellent Data Analysis skills

● Good understanding of one or more ETL tools and data ingestion frameworks.

● Experience as a designer of complex Dimensional data models for analytics services

● Experience with various testing methodologies and user acceptance testing.

● Experience on one or more cloud platforms (e.G. AWS, Azure, GCP)

● Understanding of Data Quality and Data Governance

● Understanding of Industry Data Models

● Experience in leading the large teams

● Experience with processing large datasets from multiple sources.

● Ability to operate effectively and independently in a dynamic, fluid environment.

● Good understanding of agile methodology

● Strong verbal and written communications skills with experience in relating complex concepts to

non-technical users.

● Demonstrated ability to exchange ideas and convey complex information clearly and concisely

● Proven ability to lead and drive projects and assignments to completion

● Exposure to Data Modeling Tools

○ ERwin

○ Power Designer

○ Business Glossary

○ ER/Studio

○ Enterprise Architect,

○ MagicDraw

Required Skill Profession

Computer Occupations