2023-12-24 Senior MLOps Engineer Kloud9

Tags: jd Kloud9LLC MLOpsRole

About the job Role Overview:

We are seeking a senior MLOps Engineer to join our MLOps team within the Data Science COE at Kloud9. As a senior MLOps Engineer, you will contribute to the deployment, monitoring, and management of machine learning models and data pipelines. You will work with a peer group of MLOps engineers to develop MLOps modules and engineering solutions.

In this role, you will play a pivotal role in implementing our machine learning operations, ensuring the seamless deployment, monitoring, and management of our machine learning models and data pipelines. You will be work closely in a world class AI ML team comprised of experts in AI ML modelling, ML engineers and data science and data engineering teams. You will contribute to engineering and developing solutions for ML operations and be a critical part of leading Kloud9 AI-driven transformation to drive value internally and for our customers.

In a nutshell,

Create scalable MLOps frameworks and infrastructure to support the full machine learning lifecycle, from data ingestion to model deployment and monitoring. Implement automation and CI/CD practices to streamline model deployment, version control, and testing, ensuring efficient and reliable updates and rollbacks. Develop and maintain monitoring and alerting systems to track model performance, data drift, and system health, enabling proactive issue detection and resolution. Work closely with data scientists and software engineers to seamlessly integrate machine learning models into production systems, prioritizing robustness, scalability, and performance. Optimize resource utilization and cost-efficiency by establishing scalable and efficient infrastructure for training and inference, leveraging cloud platforms.

Key Responsibilities:

Pipeline Design and Development:

• Collaborate with data scientists and data engineers to understand model requirements and design scalable ML pipelines.
• Utilize Google Cloud Platform tools and services to build and manage machine learning pipelines for training, hyperparameter tuning, and model evaluation.
• Implement Bigquery ML models, Vertex AI where applicable.

Infrastructure and Environment Setup:

• Set up and configure the required cloud infrastructure and service to support ML workloads.
• Leverage managed services (e.g., Vertex AI Pipelines) for automating ML workflows to ensure a robust and scalable environment for ML deployment.

Data Engineering and Management:

• Apply solid knowledge of data engineering and data management concepts, including SQL, ETL pipelines, data fabric, and data lake.
• Collaborate with data engineering teams to ensure data quality, availability, and compatibility with ML workflows.

Model Deployment and Monitoring:

• Deploy machine learning models using cloud platform’s model deployment capabilities.
• Implement monitoring and alerting systems to track model performance and data drift.
• Continuously improve model reliability and availability.

Automation and Version Control:

• Implement CI/CD pipelines for automating model deployment and testing.
• Utilize version control systems (e.g., Git) to manage code and model versions.

Collaboration and Documentation:

• Collaborate effectively with cross-functional teams, including data scientists, data engineers, and DevOps?, to ensure seamless integration of ML solution.
• Maintain clear and comprehensive documentation for ML pipelines and deployment processes.

Security and Compliance:

• Implement security best practices to safeguard data and models.
• Ensure compliance with data privacy regulations and company policies.

As a MLOps Engineer, you will be responsible for driving the execution of crucial infrastructure and platform initiatives related to AI/ML pipelines. These pipelines are designed for highly efficient and scalable model Training, & Inference. The responsibilities include building and developing tools, automation of redundant tasks, and CI/CD systems. This role requires someone with a strong collaborative and growth mindset.

Experience and Competencies:

• 4 to 7 years of work experience in end-to-end machine learning pipelines, including data preprocessing, model training, validation, deployment, and monitoring.
• Strong proficiency in programming languages such as Python, and experience with machine learning frameworks (e.g., TensorFlow?, PyTorch?) and cloud platforms (e.g., AWS/ Azure / GCP)
• Build and maintain end-to-end machine learning pipelines, including data preprocessing, model training, validation, deployment, and monitoring.
• Develop tools and automation frameworks to streamline the deployment and management of machine learning models in production environments.
• Expertise in MLOps frameworks for example containerization (Docker and Kubernetes), Automation (CI/CD), and model deployment (Kubeflow, MLFlow)