Job Search Results for big data

A healthcare client of ours is looking for multiple Data Engineers to join their growing team. This team is currently working on a project to modernize on prem SQL warehouses to snowflake. The data... More warehouse is built, so you will be supporting the build out of data marts on top of their Snowflake enterprise warehouse. This includes lifting and shifting existing data marts, building net new data marts, and further integrations. Additionally, the data engineering team designs, tests, deploys and maintains data pipelines and establishes a strong data reporting environment. This company operates within an AWS environment which is supported by their DevOps team; however, any cloud integration experience could be helpful. This position is currently slated as a 6-month contract, with high likelihood to extend for an additional 6+ months. Furthermore, this org has a large pipeline of project work for the next three years, so continuous extensions are likely. Insight Global offers full benefits while on contract. If you're interested, apply today!We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

The Data Engineer is responsible for architecting, developing, and maintaining enterprise-scale data platforms that support analytics, operational reporting, and machine learning initiatives. This... More role requires deep technical expertise in distributed systems, cloud platforms, and data modeling, combined with strong communication and leadership capabilities.Design and implement large-scale ETL/ELT pipelines using Python, Spark, and distributed processing frameworksDevelop and maintain big data infrastructure leveraging Hadoop, Spark, Kafka, and Kafka StreamsArchitect cloud-native data solutions on AWS or Azure, including serverless componentsBuild and optimize Snowflake data warehouses using dimensional modeling best practicesConduct data discovery and source analysis to support new integrations and transformationsModel complex datasets using normalized, denormalized, star, and snowflake schemasIntegrate external systems through RESTful APIs and automated ingestion frameworksImplement DevOps practices including CI/CD, containerization, and orchestrationDevelop real-time streaming applications using Kafka, Storm, Kinesis, or Pub/SubOperationalize machine learning models in collaboration with data science teamsOptimize performance across queries, pipelines, and distributed workloadsTroubleshoot and resolve complex data issues across upstream and downstream systemsMaintain comprehensive documentation for data pipelines, models, and integrationsWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Insight Global is seeking a hands-on Principal Data Architect to sit hybrid at a cybersecurity client in Newport Beach, San Jose, or Frisco Texas. You bring a strong background in big data... More engineering, modern lakehouse architectures, and decentralized data governance. This is a unique opportunity to define the strategic vision for their next-generation data platform, transition our ecosystem to a Data Mesh paradigm, and implement robust security standards across a petabyte-scale environment.(This role can pay 220,000-245,000 based on years of experience).We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

One of our clients within the Financial Services space is looking for a full-time Data Engineer to support their organization. This position is part of the Enterprise Data & Analytics Capability... More team under Global Technology Organization. In this role, you will lead the design, development, and optimization of large-scale data solutions on the Databricks platform. Day to day responsibilities include the following:?Design, build, and maintain scalable data pipelines on Databricks (using Spark, Delta Lake, etc.)?Write clean, efficient and maintainable PySpark or SQL code for data transformation?Design robust data models for analytics and reporting?Ensure data quality, consistency, and governance?Handle batch and streaming data workflows?Provide architectural guidance and support in platform usage?Drive best practices in data engineering across the team?Monitor and optimize performance of Spark jobs and cluster usage?Ensure compliance with security and data privacy standardsWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

? Design and maintain the overall data platform architecture integrating Snowflake, AtScale, and dbt components? Establish data quality frameworks and monitoring systems? Define platform scalability... More and performance optimization strategies? Design and implement Snowflake warehouse configurations and resource management policies? Manage user access controls, roles, and security policies within Snowflake? Optimize query performance through warehouse sizing, clustering, and partitioning strategies? Implement data sharing and collaboration features across business units? Monitor and control Snowflake costs through usage optimization and resource scheduling? Design and maintain data pipeline architectures for ingestion and transformation? Establish connection protocols between AtScale and various BI tools (Tableau, Power BI, etc.)? Optimize query performance through aggregate tables and caching strategies? Manage AtScale cluster configurations and scaling policies? Implement security models that align with organizational data access requirements dbt Development & Operations? Design and maintain dbt project structures and development workflows? Create and enforce dbt coding standards and documentation requirements? Implement dbt testing frameworks for data quality validation? Manage dbt deployment pipelines and environment promotion processes? Design modular dbt models that support reusability and maintainability? Establish version control and collaboration practices for dbt development teams Integration & Data Flow Management? Design end-to-end data pipelines connecting source systems to Snowflake via dbt transformations? Implement data lineage tracking across all three platforms? Create monitoring and alerting systems for data pipeline health and performance? Establish disaster recovery and backup procedures for the integrated platform? Design API integrations and data exchange protocols between platforms Team Leadership & Collaboration? Mentor and train data engineers and analysts on platform best practices? Collaborate with data science teams to support ML/AI workload requirements? Coordinate with DevOps teams on infrastructure automation and deployment practices? Lead architectural review sessions and technical decision-making processes Performance & Cost Optimization? Monitor and optimize compute resource utilization across all platforms? Implement cost allocation and chargeback models for different business units? Analyze query patterns and usage metrics to identify optimization opportunities? Design and implement automated scaling policies based on workload demands? Establish SLA monitoring and reporting for platform availability and performance Security & Compliance? Implement data encryption, masking, and anonymization policies? Ensure compliance with regulatory requirements (GDPR, HIPAA, SOX, etc.)? Design and maintain audit trails and data access logging? Coordinate security assessments and vulnerability management? Establish data classification and handling procedures across all platformsPay Rate: $16.00-$22.00 depending on skills and experienceWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Insight Global is looking for a Lead Data Engineer to come join their team. This candidate will be helping design and deliver modern data solutions using cloud-based platforms and AI-driven... More innovations. You will partner with business, technology, and governance teams to enable scalable, secure and compliant data platforms that power advanced analytics and AI. You will act as the enterprise expert for designing, developing, and optimizing data processing systems and pipelines used to populate data into transactional databases, data warehouses, data marts, or other data repositories. You will also coordinate with other IT teams to configure and implement security, monitoring, tooling, and related capabilities as they apply to their assigned systems. The Day-to-Day:?Use Agile practices to elicit and refine requirements through an iterative process of planning, defining acceptance criteria, prioritizing, developing and delivering enterprise data asset solutions in collaboration with business intelligence teams or other IT teams?Develop, configure, customize and manage integration tools, databases, warehouses and analytical systems?Design, build, and deploy data extraction, transformation, and loading processes and pipelines from various sources including databases, APIs, and data files?Develop and deploy structured, semi-structured, and unstructured data storage models such as data vault or dimensional modeling on a various platforms to meet business requirements?Coordinate security, application monitoring, development tooling, and related capabilities with responsible teams?Provide operational support and incident management for data pipelines and data flows, including proactively updating internal clients and relevant parties on status changes and issues encountered?Author and maintain accurate and up-to-date documentation for processes, procedures, and technical designs?Coach junior data and business intelligence personnel across the company, including developing material and guiding junior personnel on their technical skill growthWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

We?re seeking a highly skilled Data Engineer to design, build, and maintain production-grade data pipelines that process and transform terabytes of data. In this role, you?ll collaborate closely with... More data scientists and other SWEs to ensure that our data infrastructure is scalable, reliable, and cost-effective.Salary Range - $150,000 - $220,000We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Key Responsibilities? Design, build, and optimize scalable data pipelines using Python, Spark, and Airflow? Lead development of ETL/data transformation frameworks supporting AI/ML use cases? Partner... More with engineering and AI teams to enable data readiness for Copilot/AI applications? Write clean, high-quality, and well-tested code aligned with enterprise standards? Ensure solutions meet functional, non-functional, and compliance requirements? Drive performance optimization and scalability within Spark-based environments? Act as a technical SME?mentoring team members and influencing architecture decisions? Support integration of data solutions into enterprise systems with robust interfacesBank of America?s DREAM organization (Global Technology) is seeking a Senior Data Engineer (SME-level) to support a high-impact AI Copilot Agent initiative. This is a hands-on individual contributor role requiring deep technical expertise and the ability to lead from the front?guiding design, development, and best practices across the team.This is not a learning role?the team needs a strong SME who can immediately own complex data engineering efforts and elevate overall delivery.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

?Design, build, and maintain highly scalable and reliable data pipelines to move and transform data across large-scale, distributed systems at exabyte scale.?Architect, implement, and deploy new data... More models and data processing workflows in production, ensuring data quality, integrity, and compliance with data privacy regulations.?Collaborate with cross-functional teams ? including software engineers, data scientists, and product managers ? to understand data requirements and deliver effective, high-impact solutions.?Develop and optimize data-driven systems and solutions to enhance operational scale, efficiency, and generate actionable business insights aligned with evolving product and business needs.?Work closely with engineering teams to plan and execute on half-yearly roadmaps that align with business priorities and available engineering capacity.?Perform data quality monitoring, root cause analysis, and proactive resolution of data pipeline issues to ensure high reliability and trust in data assets.?Develop comprehensive documentation for all data pipelines, models, and related processes, and cross-train team members on new and evolving data systems.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Insight Global is seeking a Data Scientist in the Detroit, Michigan area. Responsible for translating business requirements into analytical constructs and using data to propose solutions for... More effective decision making. Collects, validates, transforms, and cleanses data, as well as performs quantitative analysis to derive insights. Runs analytical experiments in a methodical manner and regularly evaluates alternative models and techniques. Develops predictive models to forecast business performance metrics and provides recommendations for strategic decisions. Responsible for teaching others the tools, techniques and best practices in self-service reporting, data analysis, and predictive analytics. Insight Global is seeking a Data Scientist in the Detroit, Michigan area. Responsible for translating business requirements into analytical constructs and using data to propose solutions for effective decision making. Collects, validates, transforms, and cleanses data, as well as performs quantitative analysis to derive insights. Runs analytical experiments in a methodical manner and regularly evaluates alternative models and techniques. Develops predictive models to forecast business performance metrics and provides recommendations for strategic decisions. Responsible for teaching others the tools, techniques and best practices in self-service reporting, data analysis, and predictive analytics. Insight Global is seeking a Data Scientist in the Detroit, Michigan area. Responsible for translating business requirements into analytical constructs and using data to propose solutions for effective decision making. Collects, validates, transforms, and cleanses data, as well as performs quantitative analysis to derive insights. Runs analytical experiments in a methodical manner and regularly evaluates alternative models and techniques. Develops predictive models to forecast business performance metrics and provides recommendations for strategic decisions. Responsible for teaching others the tools, techniques and best practices in self-service reporting, data analysis, and predictive analytics. Compensation:$40/hrExact compensation may vary based on several factors, including skills, experience, and education. Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less