3 CDI Data Engineering vacancies, differing mainly in where they sit within the customer data lifecycle.
The first is the most data-centric, the second is the most DevOps/MLOps-oriented, and the third is the most transformation-focused.
2-3 days a week in office based in Canary Wharf.

Client Details
Our client operates within the Oil and Gas sector. British Petroleum.

Description
Role 1: CDI Data Engineer
Implement and maintain SCI base, fact and curated pipelines.
Deliver Elavon/PAR/WRE/payment and loyalty matching.
Support ICE-to-EV data engineering and dashboard datasets.
Implement DO-site and market-specific logic.
Investigate volume drops, match-rate changes and data completeness defects.
Execute controlled reloads, production validation, DQ/freshness controls and BAU support.
Prepare PRs, tests, technical designs, CAB/release evidence and stakeholder

Role 2: BNA Production-Readiness, MLOps & Integration Engineer:
Automate end-to-end BNA orchestration
Reliable retry/recovery and idempotent processing
Production monitoring and alerting automated activation and acknowledgement handling
Reducing manual intervention Stable QA production deployment process
Runbooks and operational handover
Reducing runtime and production failures

Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer:
Repointing remaining CIS/SCI dependencies
Stabilising migrated pipelines
Reusable error-handling patterns
Reducing legacy technical debt
Removing dual-platform monitoring
Resolving SFMC/Comarch integration issues
Reducing production support burden
Clear handover into BAU

Profile
Role 1: CDI Data Engineer

Must Have worked with

Strong SQL, Python/Spark and Databricks pipeline engineering background.
Large-scale transaction, payment, loyalty, EPOS/WRE and identity-matching work history
Historical/full/incremental loads, high-water marks and reconciliation.
Azure DevOps PRs, unit testing, CI/CD and production release support.
Nice to have: Reporting/Power BI dataset exposure desirable. SFMC/CIS integration and multi-market SCI knowledge desirable.

Role 2: BNA Production-Readiness, MLOps & Integration Engineer

Must Have

Strong hands-on background in Databricks, PySpark, Python and SQL Production pipeline engineering and troubleshooting
Workflow orchestration and dependency management Idempotency, retry/restart and failure-recovery patterns CI/CD, version control and automated deployment
Worked across monitoring, logging and operational alerting
Completed data-quality and reconciliation controls API/file/SFTP integration patterns
Nice to have: Delta Lake / Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning / personalisation platforms

Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer

Must Have worked with

Strong hands-on background with Databricks, SQL, Python/PySpark ETL/ELT Data-pipeline migration
Legacy-to-cloud modernisation
Production troubleshooting
Dependency analysis
Pipeline repointing
Error handling and recovery
Data validation/reconciliation
Integration testing
Production support / L2-L3 troubleshooting
Nice to have: AWS-to-Databricks migration Informatica / SSIS / legacy ETL SFMC Comarch SFTP/file interfaces Azure/AWS CI/CD

Job Offer
£200-£400 day rate
2-3 days a week in office
12 months contract
Inside IR35 only.