Data Engineer - ML

Insight Global is seeking a dynamic Data Engineer (ML) to join the Data Platform team at a large retail client. This team oversees the technical foundation of the organization’s enterprise data platform, enabling teams to build scalable data products and integrate ML/AI into their workflows.
As a Data Engineer with a focus on ML Ops and platform, you will be responsible for the development, support, configuration, and automation of the data tools, infrastructure, and workflows that support the ML Ops strategy. You will work closely with data engineers, analysts, and architects to establish and scale enterprise ML platform capabilities.

Responsibilities
Platform Ownership – Databricks & ML Ecosystem

Own and support the Databricks platform, with a focus on administration, optimization, and operational support
Partner with engineering teams to support workloads, troubleshoot platform issues, and improve usage patterns
Support a Unity Catalog migration, helping drive governance and standardization across the platform
Monitor and support ML platform operations and workflows to ensure consistency and reliability

ML Ops Development & Standardization

Provide technical leadership in the design, expansion, and optimization of ML Ops capabilities
Design enterprise standards for infrastructure, process automation, and observability across the ML lifecycle
Build and mature ML pipelines using MLflow, focusing on improving current-state processes and consolidation
Simplify and unify ML workflows across the enterprise into a scalable, standardized framework
Support the full ML lifecycle: model development, training, deployment, and productionization

Cloud, Infrastructure & Operations

Lead Azure cloud infrastructure administration and automation, leveraging Infrastructure as Code (IaC)
Provision and automate network configurations, access controls, services, and applications
Ensure cloud environments meet expectations for reliability, availability, performance, and security
Define and implement monitoring, alerting, and observability frameworks
Oversee and participate in production deployments and CI/CD improvements

Operations & Cross-Functional Collaboration

Coordinate recovery from incidents using well-defined operational procedures and communication frameworks
Manage outages, escalations, and production incidents across ML and platform systems
Interface with global vendors and internal stakeholders
Actively recommend and implement improvements to the continuous delivery toolset
Provide guidance and mentorship to junior engineers

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

- 4+ years building or supporting ML Ops pipelines, ideally using Azure services
(Azure Data Factory, Azure Databricks, Azure Functions, ADLS, Azure SQL, Fabric/OneLake, etc.)
- 3+ years of hands-on experience with Azure cloud infrastructure administration
- 3+ years working with Databricks + MLflow + Unity Catalog in ML/AI workflows
- Strong experience supporting Databricks platforms (administration + operations)
- Proven experience managing outages, escalations, and production incidents
- Experience with DevOps tools: Azure DevOps, Jenkins, GitLab, Terraform, Airflow
- Strong coding skills in Python, PySpark, SQL, Spark SQL
- Experience with Infrastructure as Code (IaC)
- Familiarity with Agile delivery methodologies

Experience with Snowflake (strong plus)
Exposure to other cloud platforms (AWS, GCP)
Experience working in a retail environment

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.