Overview
Senior Data Engineer Jobs in Doha, Qatar at LanceSoft Middle East
2 days ago Be among the first 25 applicants
Direct message the job poster from Lance Soft Middle East
Job Title – Senior Data Engineer
Location – Dubai (Onsite)
Duration – 6+ Months
Job Description – Senior Data Engineer (Databricks)
A Databricks Data Engineer would design, build, and maintain data pipelines and workflows using Databricks, a cloud-based platform for big data processing. The role involves collaborating with data scientists and analysts to understand data needs, ensuring data quality, security, and compliance. Responsibilities include data integration, transformation, and loading processes, often leveraging tools like PySpark and SQL.
Key Responsibilities:
Develop and implement data pipelines using Databricks, including extraction from diverse sources, transformation with Spark, and loading into target systems.
Optimize data pipeline performance for efficiency and cost-effectiveness.
Utilize Databricks ecosystem tools such as SQL, Delta Lake, and Workflows to ensure data quality and pipeline reliability.
Integrate Databricks with cloud services like Azure, AWS, or GCP for storage, compute, and security.
Design and implement data models and architectures for data lakes and warehouses.
Implement data quality checks and ensure adherence to data governance policies.
Collaborate with data scientists, analysts, and stakeholders to understand requirements and provide technical expertise.
Skills Required:
Proficiency in Databricks Lakehouse Platform, Spark SQL, Python, and data pipeline development.
Experience with cloud services (Azure, AWS, GCP), especially Azure Data Factory and Data Lake.
Knowledge of Delta Lake, Unity Catalog, and MLOps concepts.
Strong understanding of data modeling, data governance, and optimization of Databricks workloads.
Experience with SQL, Python, Spark, and ML/AI tools like MLflow.
Ability to develop scalable ETL/ELT pipelines on Databricks.
Additional Details:
Seniority level:
Mid-Senior level
Employment type:
Contract
Job function:
Information Technology
Industries:
Staffing and Recruiting, Information Services
Referrals increase your chances of interviewing at Lance Soft Middle East by 2x.
Get notified about new Data Engineer jobs in Doha, Qatar.
#J-18808-Ljbffr
Title: Senior Data Engineer
Company: LanceSoft Middle East
Location: Doha, Qatar
Category: IT/Tech (Data Engineer, Data Science Manager, Cloud Computing, Data Analyst)
