Data Engineer
Tasks
- Build automated validation testing and observability for pipelines
- Design build optimize data pipelines for telemetry logs events
- Develop and maintain data lineage documentation and codebooks
- Implement data retention partitioning and storage optimization
- Implement schemas mappings ETL ELT data models
- Integrate new data sources and normalize enrichment
- Mentor junior engineers and improve data engineering practices
- Monitor data quality latency completeness and reliability
- Troubleshoot production incidents and support evidentiary exports
Perks/Benefits
Skills/Tech-stack
Apache Flink | Apache Spark | Batch Processing | CI/CD | Cloud Data | Cloud Data Warehouse | Data Lake | Data Lineage | Data Modeling | Data Quality | Data Warehouse | ELT | ETL | Elasticsearch | Infrastructure as Code | Java | Kafka | Logging | Metrics | Observability | Python | Scala | Schema Design | Splunk | Stream processing | Tracing | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Engineering | Master of Science | PhD
Roles
Related jobs
-
API Design | CI/CD | Component design | Distributed Systems | Frontend architectureDental insurance | Financial benefits | Medical insurance | Vision insuranceSenior-level Full TimeSanta Clara, California, United States R10h ago
-
API Integration | AWS | Ansible | Azure | CI/CDSenior-level Full TimeSanta Clara, California, United States10h ago
-
AWS | Ansible | Azure | CI/CD | CertificatesSenior-level Full TimeSanta Clara, California, United States11h ago
-
Ansible | Autoscaling | CI/CD | Cloud Native | ContainerizationSenior-level Full TimeSanta Clara, California, United States R11h ago
-
API Integration | CI/CD | Component design | Design Systems | GraphQLDental insurance | Financial benefits | Medical insurance | Other benefits | Vision insuranceSenior-level Full TimeSanta Clara, California, United States12h ago
-
Senior Machine Learning Engineer - Cybersecurity USD 80K-200KAnomaly Detection | Behavioral analytics | Cyber Threat | Cyber Threat Detection | CybersecuritySenior-level Full TimeSan Jose, CA, United States16h ago
-
Software Engineer SME (TS/SCI with Poly Required) USD 187K-318KAWS | Apache NiFi | CI/CD | DevOps | ETLSenior-level Full TimeDulles, Virginia, United States17h ago
-
Software Engineer SME (TS/SCI with Poly Required) USD 187K-318KAPI | API Key | Amazon Kinesis | Amazon Web Services | Apache AirflowMid-level Full TimeChantilly, Virginia, United States17h ago
-
Software Engineer SME (TS/SCI with Poly Required) USD 187K-318KAWS | Anaconda | Apache NiFi | Azure | Azure MicroservicesMid-level Full TimeMcLean, Virginia, United States17h ago
-
Mid-level Full TimeOmaha, NE17h ago
-
API Integration | CI/CD | Frontend Development | Frontend architecture | GraphQLSenior-level Full TimeSanta Clara, California, United States R17h ago
-
Staff Production Engineer (Cloud Platform & Reliability – Machine Identity Security) - hybrid USD 181K-226KAKS | AWS | Ansible | ArgoCD | AzureSenior-level Full TimeSanta Clara, California, United States R18h ago
-
API Design | AWS | Azure | CI/CD | Certificate managementSenior-level Full TimeSanta Clara, California, United States18h ago
-
Mid-level Full TimeFrisco, TX, United States R18h ago
-
APIs | Distributed Systems | Go | Integration Testing | JavaDental insurance | Financial benefits | Health insurance | Mentorship | Vision insuranceMid-level Full TimeSanta Clara, California, United States19h ago
-
DevSecOps and Cloud Integration Engineer, Senior USD 170K-170K.NET | AWS CodePipeline | AWS Lambda | Agile | Amazon ECS401k match | Dental coverage | Employee assistance program | Flexible spending account | Health insuranceSenior-level Full TimeUT, US19h ago
-
Senior-level Full TimePortland, Oregon, United States20h ago
-
ACME | API | Concurrency | Data Modeling | Distributed SystemsDental insurance | Equity benefits | Medical insurance | Vision insuranceSenior-level Full TimeSanta Clara, California, United States22h ago
-
Senior-level Full TimeNeedham, Massachusetts, United States22h ago
-
IAM Software Engineer USD 100K-130KAdaptiveAuthentication | AuthenticationPolicies | Bash | CI/CD | JSONMid-level Full TimeChicago, Illinois22h ago
-
IAM Software Engineer USD 110K-135KAccess Management | Adaptive Authentication | Bash | CI/CD | Identity and Access ManagementMid-level Full TimeAtlanta, Georgia22h ago
-
Senior Systems Engineer USD 130K-168KAcceptance Testing | DISA STIG | Firewalls | Hyper-V | Hyperconverged InfrastructureSenior-level Full TimeQuantico, VA22h ago
-
Senior Data Scientist/Engineer USD 130K-168KAWS GovCloud | Azure Government | Data Cleansing | Docker | ETLOn-site parkingSenior-level Full TimeChantilly, VA22h ago
-
Principal Platform & Security Engineer USD 136K-204KAWS | Azure | CI/CD | Cloud Cost Management | Cost ManagementSenior-level Full TimeSanta Monica, CA, United States23h ago
-
Bash | Cortex XSIAM | Linux | Mitre Attack | Powershell100 percent remoteSenior-level Full TimeColumbia, SC, United States R23h ago