Job Description
Design, build, and maintain scalable data pipelines using Python and SQL to support analytics and downstream applications. Orchestrate workflows using Airflow while ensuring data reliability, quality, and performance across large datasets. Work closely with cross-functional teams to optimize data storage and processing within Snowflake or Redshift environments. Leverage AWS services such as S3 and SQS, along with Terraform for infrastructure as code. Continuously improve existing pipelines, troubleshoot issues, and support both batch and streaming data solutions where applicable.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
• 5+ years of professional experience as a Data Engineer
• Strong experience with Python and Bash (building data pipelines and managing shell script wrappers)
• Experience working with large-scale datasets, including performance optimization
• Hands-on experience with Airflow for workflow orchestration
• Experience with Snowflake or Redshift databases
• Working knowledge of AWS services (S3, SQS) and Terraform
• Strong SQL skills (ad hoc querying and pipeline development)
Nice to Have Skills & Experience
• Experience using AI tools for code generation (e.g., Claude, Codex, Copilot)
• Experience with both streaming and batch data processing (e.g., Spark, Kafka)
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.