Position Description:
Job Title: Data Engineer (Azure & Databricks)
Experience Required:4 5 Years
About the Role
We are looking for a motivated and detail oriented Data Engineer to join our growing analytics team. In this role, you will work closely with Lead Data Engineers and Architects to develop, maintain, and optimize data pipelines using modern cloud technologies.
You will get hands on experience building our Enterprise Data Warehouse and Lakehouse solutions using Python, Databricks, and Microsoft Azure. This is an excellent opportunity for an emerging data professional to grow their skills in cloud data engineering and distributed computing.
Key Responsibilities
. Pipeline Development: Build, test, and maintain robust ETL/ELT pipelines to move data from various sources into our Azure Data Lake and Data Warehouse.
. Coding & Transformation: Write clean, efficient PySpark and Python code in Azure Databricks to clean, transform, and aggregate data.
. Data Integration: Utilize Azure Data Factory (ADF) to orchestrate data movement and schedule automated workflows.
. SQL Development: Write and optimize SQL queries for data extraction, analysis, and creating views for downstream reporting tools.
. Monitoring & Support: Monitor daily batch jobs, troubleshoot failed pipelines, and resolve data quality issues in a timely manner.
. Collaboration: Work alongside senior engineers to understand architectural designs, participate in code reviews, and learn enterprise best practices for data engineering.
. Documentation: Maintain clear technical documentation for data pipelines, data dictionaries, and operational runbooks.
Required Skills & Qualifications
. Experience: 4 5 years of hands on experience in Data Engineering, ETL development, or Data Analytics.
. Python Programming: Strong proficiency in Python for data manipulation (e.g., using Pandas) and scripting.
. SQL Mastery: Solid understanding of relational databases and the ability to write complex SQL queries, joins, and aggregations.
. Big Data / Databricks: Hands on experience with Apache Spark (PySpark or Spark SQL) and familiarity with the Databricks environment.
. Azure Cloud: Exposure to the Azure data ecosystem, specifically working with Azure Data Lake Storage (ADLS Gen2) and creating pipelines in Azure Data Factory (ADF).
. Data Warehousing Concepts: Good foundational understanding of Data Warehousing principles (Star schema, facts, and dimensions).
. Version Control: Experience using Git for source code version control.
Preferred/Bonus Skills
. Understanding of Delta Lake concepts (Time travel, ACID transactions).
. Familiarity with CI/CD concepts and deploying code using Azure DevOps or GitHub Actions.
. Prior experience working in an Agile/Scrum environment.
. Certifications: Azure Data Fundamentals (DP 900) or Databricks Certified Data Engineer Associate.
Skills:
- Azure
- Big Data,Analytics&Operations
- Databricks
- Python
- Cloud Data Platform Design
What you can expect from us:
Together, as owners, let’s turn meaningful insights into action.
Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…
You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.
Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.
You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.
That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals.
Come join our team—one of the largest IT and business consulting services firms in the world.