Data Engineer II
Tasks
- Build ETL/ELT pipelines
- Deploy and manage pipelines in cloud
- Design and build data pipelines
- Develop batch and near real time pipelines
- Develop microservices
- Ensure data quality and reliability
- Follow secure data processing best practices
- Implement infrastructure-as-code
- Optimize data processing workloads
- Perform code reviews and mentor junior engineers
- Set up monitoring and observability
- Support production deployments and troubleshooting
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Amazon S3 | Apache Airflow | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | BigQuery | CI/CD | Cloud Storage | Cloud platform | CloudFormation | Data Lake Storage | Data Lakes | Data Warehouses | Databricks | Docker | ELT | EMR | ETL | Gatling | GitHub Actions | Google Cloud | Google Cloud Platform | Google Cloud Storage | Grafana | Hibernate | JMeter | JUnit | Java | Jenkins | Kafka | Kubernetes | Performance Testing | Prometheus | Python | REST APIs | Redshift | SQL | Snowflake | Spark Structured Streaming | Splunk | Spring Boot | Spring Security | Structured Streaming | Terraform | Unit Testing
Education
Roles
Related jobs
-
Senior-level Full TimeBengaluru, India6h ago
-
Lead Infrastructure Engineering INR 3000K-5000KAAA | AI Assisted Development | Access Control | Ansible | Audit managementSenior-level Full TimeIndia10h ago
-
Software Engineer II - Entity Intelligence INR 1200K-1680KAirflow | Algorithms | DNS | Data Structures | DatabricksOn-call rotation compensation | Remote/distributed teamMid-level Full TimeHybrid - Bangalore, India R14h ago
-
DevOps Engineer / Site Reliability Engineer (SRE) INR 2200K-3500KBash | CI/CD | Capacity Planning | Cloud Monitoring | Cloud loggingMid-level Full TimeIndia14h ago
-
Data minimization | Differential Privacy | Distributed Systems | GDPR | GoAnnual meetups | Co-working allowance | Flexible schedule | Fully remote | Home office setup allowanceSenior-level Full TimeIndia R14h ago
-
API | Access Governance | Access Management | Access reviews | Active DirectoryCareer development resources | Hybrid work environment | Wellbeing programsSenior-level Full TimeHyderabad, TG, India15h ago
-
API | Access Governance | Access Management | Access Review | Active DirectoryCareer development resources | Innovation practices | Wellbeing programsSenior-level Full TimeHyderabad, TG, India15h ago
-
Senior-level Full TimeJohnson Controls India COEE119h ago
-
Senior Staff Developer - IN - IDR - Integrations INR 1567K-4600KAWS | Azure | Distributed Systems | Docker | DynamoDBEmployee assistance program | Equity | Flexible leave | Life insurance | Medical insuranceSenior-level Full TimeOffice - IND - Karnataka, Bengaluru, …19h ago
-
Senior-level Full TimeJohnson Controls India COEE119h ago
-
Senior-level Full TimePune, India19h ago
-
Mid-level Full TimePune, India19h ago
-
Senior-level Full TimeIndia-Mumbai (Kalpataru Square)19h ago
-
Mid-level Full TimePune, India19h ago
-
Entry-level Full TimePune, India19h ago
-
Sr. Engineer | - VMS |Onsite, Bangalore INR 2500K-3500KAWS | Agentless Scanning | Asset Criticality | Attack surface | Attack surface managementProfessional training resources | Remote work support | Volunteer opportunities | Work-life balanceSenior-level Full TimeBangalore, India R19h ago
-
AVP - Infrastructure Security Services INR 1500K-4000KAAA | Access Control | Access Management | Account lifecycle management | AnsibleOn call 24x7 supportExecutive-level Full TimeBCIT Bengaluru Office (MGS), India19h ago
-
Senior-level Full TimeBengaluru, India1d ago
-
Cybersecurity Engineer INR 1500K-4600KApache Kafka | Apache Spark | Automation workflows | Compliance | Correlation logicSenior-level Full TimeHybrid (Bengaluru, Karnataka, IN); Hybrid (Pune, … R1d ago
-
Principal Engineer (Java/PAM), Software Engineering INR 3000K-5000KAWS | Access Management | Ant | Azure | CI/CDSenior-level Full TimeBengaluru1d ago
-
Software Engineer - Security INR 2000K-3487KAccess Management | Active Directory | Azure Active Directory | Bash | BigFixSenior-level Full TimeMumbai, Maharashtra, India1d ago
-
Consultant - Consulting Implementation INR 800K-1200KAWS | Access Control | Authentication | Cloud Security | DLPMid-level Full TimeGurgaon, Haryana, India2d ago
-
Sr. Software Development Engineer (IAM) INR 3000K-4000KAI anomaly detection | Anomaly Detection | CI/CD | Docker | HTTPEducation reimbursement | Health plans | Hybrid work | Parental leave | Retirement optionsSenior-level Full TimeBangalore, IND2d ago
-
IAM Support Engineer INR 704K-968KAccess Governance | Certificate management | IBM Security | IBM Security Verify | IBM Security Verify AccessMid-level Full TimeHyderabad - BSR IT - SEZ …2d ago
-
DevSecOps Engineer INR 2000K-3500KAPI Security | ASVS | Agentic coding | Application Security | Artificial IntelligenceSenior-level Full TimePune, Maharashtra, India3d ago