Principal AI/ML Ops Engineer

The global order relies on a free and open internet, and cybercriminals have turned it into their playground. As they chase AI to increase the speed and scale of their attacks, Rapid7 has been leveraging it to supercharge our cybersecurity detections and triage alerts quickly. For decades, we’ve been using AI technology to power risk and threat analysis to detect attacks earlier and reduce response time.  

Rapid7 is making significant investments in our Belfast office with the formation of our new AI Centre of Excellence, encompassing the full range of AI, ML and data science.  As a leader in cybersecurity, we’re on a new mission to use AI to accelerate threat investigation, detection and response (D&R) capabilities of our Security Operations Centre (SOC) for both conventional networks and cloud environments.  The AI race to stay ahead of attackers is on and needs more than speed.  Attack surfaces are getting broader every day and it’s important to optimise D&R with AI that can identify signals in a sea of noise.  Further, we’re embracing generative AI and researching novel ways LLMs can add value for our customers in D&R.  

We’re seeking the best AI, ML and data science talent to build systems for detecting patterns and anomalies humans can't, and which rule-based detections miss.  In this Principal AI/ML Ops Engineer role you will help define and deliver an MLOps strategy for our growing AI team.  There’s a ton of work to do and we would love you to join us!

Rapid7’s AI Centre of Excellence

The AI CoE partners with our D&R teams in enabling customers to assess risk, detect threats and automate their security programs.  Rapid7 also have a best-of-breed managed SOC offering, known as MDR, where teams of analysts are retained to carry out crucial D&R work on behalf of our customers.  The positive impact of AI in D&R will be felt right across our customer base.  The AI CoE’s goals are ambitious and we need dynamic people with a desire to be part of something big. If you want a career move where you can grow and make an impact with AI, this is it.

We ensure AI, ML and data science are applied in a meaningful way to add customer value, best achieve business objectives and deliver ROI.  Unnecessary complexity is avoided and we adopt a creative, fast-fail, highly iterative approach to accelerate ideas from proof-of-concept to go or no-go.  Our current capabilities are built on 20+ years of threat analysis and open-source communities with 40 AI patents granted and 20+ pending.  However, we’re just getting started!  

The make-up of the group is such that our technical skills complement one another.  No one can be an expert in everything; we share our AI, ML and data science knowledge between ourselves plus are creating an in-house AI Learning & Development plan.  The AI CoE also contributes on occasion to new external AI policy initiatives with recognised bodies like NIST.  In fact, we’d be delighted if you’re open to publishing and presenting your best work with Rapid7 at top conferences around the world and in journals.  Our current AI team have a track record of publishing award-winning research at the likes of AISec at ACM CCS and with IEEE; we realise the benefits publishing can bring to an AI career and the confidence it inspires in our customers.

Role

Rapid7 is seeking a Principal AI Engineer to join our team as we expand and evolve our growing AI and MLOps efforts. You should have a strong foundation in applied AI R&D, software engineering, as well asand MLOps and DevOps systems and tools. Further, you’ll have a demonstrable track record of taking models created in the AI R&D process to production with repeatable deployment, monitoring and observability patterns. In this intersectional role, you will deftly combine your expertise in AI/ML deployments, cloud systems and software engineering to enhance our product offerings and streamline our platform's functionalities.

Key Responsibilities

AI Engineering Program Leadership

• Define and implement platform-wide standards and systems for AI research, deployment and maintenance.

• Lead a team of collaborators to deliver high quality, reusable, libraries, tools and modules to be shared throughout the company.

Interdisciplinary Collaboration

• Collaborate closely with engineers and researchers to refine key product and platform components, aligning with both user needs and internal objectives.

• Actively contribute to cross-functional teams, focusing on the successful building and deployment of AI applications.

Data Pipeline Construction and Lifecycle Management

• Develop and maintain data pipelines, manage the data lifecycle, and ensure data quality and consistency throughout.

Feature Engineering and Resource Management

• Oversee feature engineering processes and optimize resources for both offline and online inference requests.

Model Development, Validation, and Maintenance

• Build, validate, and continuously improve machine learning models, manage concept drift, and ensure the reliability of deployed systems.

System Design and Project Management

• Architect and manage the end-to-end design of ML production systems, including project scoping, data requirements, modeling strategies, and deployment.

Knowledge and Expertise Sharing

• Thoroughly document research findings, methodologies, and implementation details.

• Share expertise and knowledge consistently with internal and external stakeholders, nurturing a collaborative environment.

ML Deployment

• Implement, monitor, and manage ML services and pipelines within an AWS environment, employing tools such as Sagemaker and Terraform.

• Assure robust implementation of ML guardrails, leveraging frameworks like NVIDIA NeMo, and managing all aspects of service monitoring.

• Develop and deploy accessible endpoints, including web applications and REST APIs, while maintaining steadfast data privacy and adherence to security best practices and regulations.

Software Engineering

• Lead the development of core API components to enable interactions with LLMs.

• Craft and optimize conversational interfaces, capitalizing on the capabilities of LLMs.

• Conduct API and interface optimization with a product-focused approach, ensuring performance, robustness, and user accessibility are paramount.

Continuous Improvement

• Embrace agile development practices, valuing constant iteration, improvement, and effective problem-solving in complex and ambiguous scenarios.

You may be a good fit if you

• Have expertise in both ML deployment (especially in AWS) and software engineering.

• Have experience as a software engineer, notably in building APIs and/or interfaces, paired with adept coding skills in Python and TypeScript.

• Possess adeptness in containerization and DevOps.

• Demonstrate exemplary problem-solving capabilities, particularly in decomposing complex problems into manageable parts and devising innovative solutions.

• Are proficient with CI/CD tooling, Docker, Kubernetes, and have prior experience developing APIs with Flask or FastAPI.

• Have experience deploying LLMs, managing advanced compute resources like GPUs, and navigating data collection for metrics and fine-tuning from LLM-based systems.

• Showcase robust analytical, problem-solving, and communication skills, with the capacity to convey intricate ideas effectively.

• Maintain high standards of engineering hygiene, embracing best practices and an agile development mindset.

Strong candidates may also 

• Have previous experience with NLP and ML models, understanding their operational frameworks and limitations.

• Are versed with frontend frameworks like React. 

• Possess adeptness in containerization and DevOps.

• Possess proficiency in implementing model risk management strategies, including model registries, concept/covariate drift monitoring, and hyperparameter tuning.

• Experience in designing and integrating scalable AI/ML systems into production environments.

• Strong communication skills, with the ability to explain complex AI/ML concepts to both technical and non-technical stakeholders.

And you as a person:

• Bring a positive, can-do, solution-oriented mindset, welcoming the challenge of tackling the biggest problems.

• Are persistent and consistent, being able to systematically tackle complex use cases head-on.

• Enjoy working in a fast-paced environment, sometimes with multiple projects to juggle simultaneously.

• Understand the highly iterative nature of AI development and the need for rigour.

• Appreciate the importance of thorough testing and evaluation to avoid silent failures.

• Are a great teammate to help peers become stronger problem solvers, communicators, and collaborators.

• Have a curiosity and passion for continuous learning and self-development.

• Stay receptive to new ideas, listen to suggestions from colleagues, carefully considering and sometimes adopting them.

• Realise the importance of wider ethical and risk considerations with AI.

• Possess strong interpersonal and communication abilities, explaining hard-to-understand topics to different audiences, working to build consensus, and writing up work clearly.

• Exhibit a bias for action, without being careless.

We know that the best ideas and solutions come from multi-dimensional teams. That’s because these teams reflect a variety of backgrounds and professional experiences. If you are excited about this role and feel your experience can make an impact on our AI mission, please don’t be shy - apply today.