Overview
DataOps Consultant, Lakehouse Support – Abu Dhabi, United Arab Emirates Jobs in Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates at Luxoft
Title: DataOps Consultant, Lakehouse Support – Abu Dhabi, United Arab Emirates
Company: Luxoft
Location: Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates
Project Description
- L2 DataOps Consultant – Lakehouse Support for an investment management environment
- Provide operational diagnostics and incident management for Lakehouse data platforms
- Own L2 incident resolution across data pipelines
- Collaborate with Platform Engineering, Data Engineers, and Data Domain teams
- Apply ITSM, change management, and problem management to minimize recurring issues
- Utilize investment management domain knowledge for faster incident resolution and root cause analysis (RCA)
Key Responsibilities
- Own end-to-end L2 incident resolution for Lakehouse data platforms
- Perform operational diagnostics across batch and streaming data pipelines
- Coordinate issue resolution with Platform Engineering, Data Engineering, and business teams
- Manage incidents using ITSM tools such as ServiceNow
- Troubleshoot issues using metadata, data lineage, and technical documentation
- Maintain clear ownership, escalation, and communication during incidents
- Conduct RCA and implement preventive measures through problem management
- Support change management activities and ensure operational stability
Mandatory Skills
- Strong experience in Data Operations (L2 Support)
- Hands-on experience with Lakehouse architectures (Databricks, Delta Lake, or equivalent)
- Expertise in incident management and ITSM processes
- Experience troubleshooting batch and streaming data pipelines
- Knowledge of metadata-driven operations, data lineage, and observability
- Strong skills in Root Cause Analysis (RCA), change management, and problem management
- Experience owning incidents from identification to resolution
- Working knowledge of investment management data:
- Portfolio & Positions
- Trade Lifecycle
- Market Data
- Risk & Performance Data
- Experience as a Data Engineer in Databricks and Delta Lake
- Hands-on knowledge of Delta Lake internals
Nice-to-Have Skills
- Palantir Foundry – DQC setup, pipeline diagnostics, build failure troubleshooting
- Monte Carlo – Data quality monitoring and observability
- Azure DevOps – Repository management and CI/CD
- Experience with Dynatrace or other observability tools
- Experience with enterprise-scale data platforms
- Knowledge of batch orchestration and workflow tools
- Familiarity with investment management systems such as Optima and Cardia
- Understanding of portfolio concepts such as Maturity Date and Value Date
- Strong understanding of data lineage, metadata, and observability concepts