Insight Global is looking for a Data Engineer to join the Data & Solutions Infrastructure team for one of BC's leading health authorities. This resource will be instrumental in maintaining and advancing data infrastructure. Take charge of both cloud-based and local data systems, ensuring their continuous improvement and operational stability.
This resource will be responsible for engineering scalable and efficient data integration systems that support extensive analysis, model crafting, validation, and deployment. Work alongside diverse teams to pinpoint and realize forward-thinking solutions that cater to the informational requisites of the health authority.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
[email protected].
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
Masters Degree in Business Administration, Information Systems or equivalent.
Five years' recent, related experience including technical experience in working with databases and decision support tools, preferably in a health care setting or equivalent industry/certification experience (e.g. Databricks Data Engineer Associate, Databricks Data Engineer Professional, Microsoft Azure Data Engineer Associate).
Five years experience in Information Technology with focus in database development, warehousing, and/or data engineering.
Expertise using the Databricks platform and understanding the concepts of Databricks SQL, Delta Lake, cluster management.
Experience in optimization of pipelines and processes of Databricks platform.
Strong proficiency in Databricks SQL, Python, Scala with an ability to write efficient and scalable code for data processing/transformation tasks.
Experience building and optimizing data pipelines (ETL/ELT).
Strong understanding of DevOps and MLOps practices focusing on version control (Git), CI/CD pipelines, and automated testing frameworks.
Experience using big data processing frameworks such as Apache Spark & Hadoop.
Strong understanding of Spark RDD's, data frames, and API's for data manipulation.
Knowledge of data modelling and data warehousing principles.
Knowledge using cloud data warehousing solutions (e.g. Azure SQL, Synapse Analytics, Snowflake).
Advanced analytical and problem solving skills.
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.