Data Engineering

key Accountability

1. 1. Support Data pipeline monitoring:

• Monitor task status in GCP
• Validate data freshness and partition loads
• Review alerts in GCP Monitoring & Logging

2. 2. Learning & Data Engineering Development:

• Learn new and existing technologies
• Develop new data pipeline and tasks
• Maintain existing pipelines

3. 3. Assist in pipeline validation & Testing:

• Run BigQuery SQL validation checks
• Track anomalies in DQ rule outputs
• Notify teams of any critical deviations

4. 4. Support Documentation:

• Update runbooks and checklists
• Document recurring issues
• Create new troubleshooting articles
• Share and maintaing RCA documents

5. Team collaboration:

• Provide daily handover notes
• Participate in team meetings
• Share observations for pipeline improvement

skills required:

• Basic SQL
• Python fundamentals,
• Cloud basics (preferably GCP)
• understanding of monitoring concepts

• Basic SQL
• Python fundamentals,
• Cloud basics (preferably GCP)
• understanding of monitoring concepts

Bachelor’s degree in Computer Science, Artificial Intelligence, or a related field.