Sr. Data Engineer (Big Data & Analytics Engineering)
Tasks
- Build ETL/ELT pipelines
- Collaborate with data scientists on feature generation
- Design governed data models
- Develop PySpark Spark data processing jobs
- Implement CI/CD for data workflows
- Implement data quality checks and monitoring
- Maintain data lineage documentation
- Optimize pipeline performance and cost
- Support data governance and privacy for PII
- Troubleshoot production pipeline issues
- Write SQL for data processing
Perks/Benefits
Skills/Tech-stack
Access Controls | Apache Airflow | Apache Hudi | Apache Iceberg | Apache NiFi | Auditability | Automated testing | Azure Data | Azure Data Factory | Broadcast joins | Bucketing | CDC | CI/CD | Data Factory | Data Governance | Data Modeling | Delta Lake | Git | HDFS | Hadoop | Hive | Impala | ORC | Observability | Oozie | PII handling | Parquet | Partitioning | PySpark | Python | SCD Type 1 | SCD Type-2 | SQL | Shuffle Optimization | Talend | Type 1 | Type 2) | YARN
Education
Roles
Related jobs
-
Identity Security DevOps Engineer INR 1000K-1700KAPI Development | Access Management | Audit Logs | Bash | BeyondTrust PasswordSafeMid-level Full TimeIndia12h ago
-
Senior-level Full TimeThane, Maharashtra, India13h ago
-
Ansible | Automation | Bitbucket | CI/CD | Cause analysisSenior-level Full TimeHyderabad, Telangana, India14h ago
-
Senior-level Full TimePune, Maharashtra, India15h ago
-
Cybersecurity Engineer(Autosar + Cyber +SW dev) INR 2156K-4590KASPICE | AUTOSAR | Android | Cybersecurity | Embedded CCreche facility | Employee assistance program | Fitness club access | Food and beverage cards | Higher education opportunitiesSenior-level Full TimeIND - India Tech Center21h ago
-
Senior Engineer, Software Engineering INR 2520K-3380KAWS | AWS Athena | AWS EC2 | AWS EMR | AWS GlueContinuous learning | Family-friendly perks | Flexible time off | Health care coverage | Retirement planningSenior-level Full TimeIN - AHMEDABAD, India21h ago
-
Associate Engineer, Software Engineering (React, OOPS) INR 980K-1050K.NET | ASP.NET MVC | Agile | Behavior-Driven Development | C#Continuing education program | Continuous learning access | Family-friendly perks | Flexible time off | Health care coverageMid-level Full TimeIN - HYDERABAD SKYVIEW, India21h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agent/tool use | Apache Spark | Azure | ContainersContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India21h ago
-
Associate Engineer, Software Engineering (React, OOPS) INR 980K-1050K.NET | ASP.NET MVC | Agile | Behavior-Driven Development | C#Continuous learning | Family-friendly benefits | Flexible time off | Health insurance | Referral incentivesMid-level Full TimeIN - HYDERABAD SKYVIEW, India21h ago
-
Senior Engineer, Software Engineering INR 2520K-3380KAWS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaContinuing education support | Continuous learning resources | Family-friendly perks | Financial wellness programs | Flexible time offSenior-level Full TimeIN - AHMEDABAD, India21h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agentic AI | Azure | CI/CD | Code reviewContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India21h ago
-
Penetration Tester INR 2829K-3220KACL | API Security | API Top 10 | Access Control | Active DirectorySenior-level Full TimeIndia - Hyderabad21h ago
-
Sr. Principal Security Engineer INR 2092K-4500KAI Security | API Security | API discovery | Adversarial Testing | Artifact validationSenior-level Full TimeBengaluru, Karnātaka, India21h ago
-
Senior Staff Software Engineer Product Security INR 3000K-4500KAccess Management | Aquasec | Architecture | Bash | CI/CDCareer development | Employee resource groups | Flexible work from home | Paid time off | Paid volunteer timeSenior-level Full TimeIndia-Bangalore-Remote R21h ago
-
Senior Developer 2 INR 2156K-4600KAWS | Apache Athena | Azure | Bash | DatabricksEmployee assistance program | Equity for employees | Fertility support | Flexible annual leave | Life insuranceSenior-level Full TimeOffice - IND - Karnataka, Bengaluru, …21h ago
-
Staff Software Engineer INR 2542K-4000KALB | AWS | AWS Config | Amazon MQ | ArgoCDRelocation assistanceSenior-level Full TimeBengaluru, India21h ago
-
Software Engineer INR 1400K-2500KAWS Config | Amazon ECR | Amazon EKS | Amazon MQ | Amazon S3Relocation assistanceMid-level Full TimeBengaluru, India21h ago
-
Python Gen AI developer - Director- Cyber Security Engineering INR 2000K-6000KAPI Integration | Agile | Amazon EKS | BMC Remedy | C#Executive-level Full TimeRMZ Ecoworld - Campus 6A, India21h ago
-
Sr. Data Engineer (Big Data & Analytics Engineering) INR 3000K-3880KAWS | Apache Airflow | Apache Hudi | Apache Iceberg | Apache KafkaSenior-level Full TimePune, India21h ago
-
Analyst, Big Data Analytics & Engineering INR 2516K-3356KBusiness Intelligence | Dashboarding | Data Engineering | Data Visualization | Data analyticsSenior-level Full TimeGurgaon, India21h ago
-
Lead, Big Data Analytics & Engineering INR 3000K-4000KAWS | Apache Airflow | Apache NiFi | Azure | Azure DataSenior-level Full TimePune, India21h ago
-
IAM Identity Engineer INR 1971K-3285KAccess Control | Access Management | Automation | Azure | Azure DevOpsFlexible working | Hybrid working | Work-life balanceSenior-level Full TimeHyderabad Knowledge Park Tower 2, India21h ago
-
Mid-level Full TimeIndore, INDC1A, India21h ago
-
Entitlement Engineer INR 3000K-4800KAI | AI ready data platforms | AI-Ready Data | Access Control | AuthorizationSenior-level Full TimePune, Gera Commerzone SEZ, India21h ago
-
Senior Security Engineer INR 500K-800KAPI Gateway | API Security | AWS Lambda | Amazon EC2 | Amazon RDSSenior-level Full TimeKarnataka, Bengaluru, India1d ago