Senior Data Engineer (GovTech and Public Sector)

Senior-level Full Time Found 9d ago

Tasks

• Build secure data environments (Ranger, Atlas, SDX)
• Collaborate on data modeling and governance
• Configure data workflows with Airflow/NiFi
• Design data ingestion pipelines
• Develop SQL queries for analytics
• Implement CI/CD automation for data processes
• Maintain Data Lake architecture (HDFS, Iceberg, S3)
• Optimize Hive/Impala performance
• Support machine learning workflows (Spark MLlib, CML)

Perks/Benefits

• Continuous improvement
• Impactful projects
• Team collaboration

Skills/Tech-stack

Airflow | Atlas | Big Data | Data Governance | Data Modeling | Data Warehousing | Distributed Systems | ELT | ETL | HDFS | Hive | Iceberg | Impala | Kafka | ML frameworks | Nifi | Predictive Modeling | Python | Ranger | S3 | SDx | SQL | Scala | Spark