Data Engineer

7 days ago

Australia Green Light Full time

About the Company:

Green Light is a leading IT services business, renowned for its innovative approach and commitment to excellence. With a diverse customer base across Australia, New Zealand, SE Asia, and Europe, we offer a wide range of IT services in Security, infrastructure, and Systems Integration.

About the Engagement:

We are seeking a highly skilled Data Engineer to join our Sydney delivery team, working on delivering exceptional services to a prominent enterprise end customer. The successful candidate will possess a strong background in data analysis and construction of complex queries.

Key Responsibilities:

  • Design and develop data pipelines using Azure Databricks for ingestion, curation, enrichment, and transformation of data.
  • Transformation of data – SQL, Spark, and DataBricks.
  • Data modelling predominately with Power BI.
  • Work closely with project stakeholders to understand data needs.
  • Develop and implement data quality checks and ensure data integrity throughout the processing pipeline within Databricks.
  • Define, produce, and implement data strategy across data from various data sources.
  • Design effective solutions to process very large volumes of data within acceptable SLA timeframe.
  • Complete Data Analysis and data modelling to build data models and physical data models for the designed solution – align with architecture framework.
  • Conduct Data Governance.
  • Fully utilize knowledge of Azure services like Azure Data Lake Storage and Azure Data Factory to optimize data storage, access, and orchestration within the data platform.

Requirements:

  • Strong data modelling & analysis, data migration strategy, data alignment across multiple applications, heterogeneous data identification, data masking/transformation & data mining.
  • Proven experience in designing and developing data pipelines using Databricks / Spark / Python.
  • Strong proven experience in building data models.
  • Strong hands-on experience doing data analysis using different formats.
  • Strong understanding of database concepts such as referential integrity and data alignment, table partitions, data import-export mechanisms, data transfer and transformation mechanisms, data encryption / decryption & cleansing / harmonization.
  • Strong understanding of table structures, schema, users, and grants.
  • Strong skills in designing optimized solutions to process TBs of data.
  • Knowledge of query optimization techniques.
  • Good knowledge of Microsoft Azure cloud platform and broader Databricks ecosystem (Azure Data Lake Storage, Azure Data Factory, Databricks workflow, Unity Catalog, etc.).
  • Strong business knowledge and quantitative modelling capabilities.
  • Must have a strong aptitude for careful and considered impact analysis, risk assessment, change management, and planning of recovery options.

Culture:

Innovative and continually improving enterprise environment.

Collaborative work environment.

Benefits:

Attractive working environment – emphasis on training and flexibility.

  • GIS Data Engineer

    1 week ago

    Australia ECF Data, LLC Full time

    **About ECF Data, LLC**We are a leading provider of data-driven solutions, operating in the field of geospatial data analysis and engineering. Our team of experts partners with clients to transform businesses using cloud, data, AI, and digital engineering.**Job Summary**We are seeking a highly skilled GIS Data Engineer to join our team. As a GIS Data...

  • Geoscience Data Engineer

    1 week ago

    Australia Geoscience Data Management Full time

    About the RoleWe are seeking a highly skilled Geoscience Data Engineer to join our team at Geoscience Data Management. As a key member of our data management team, you will be responsible for designing, developing, and implementing data management systems and processes to support our business operations.Key Responsibilities:Design and implement data...

  • Data Architect

    1 week ago

    Australia ECF Data, LLC Full time

    About ECF Data, LLCECF Data, LLC is a forward-thinking organization that thrives on innovation and collaboration. We're currently seeking a skilled Data Engineer to join our team and contribute to the development of our cloud-based data engineering solutions.Key ResponsibilitiesDesign and Develop Enterprise Data Engineering SolutionsDevelop and implement...

  • Operations Engineer

    5 days ago

    Australia The Chemical Engineer Full time

    About the RoleThe Chemical Engineer is seeking a highly skilled Operations Engineer to join our Midstream operations/production support organization. As a key member of our team, you will play a critical role in ensuring the safe operation and optimization of our LNG facility, field assets, and equipment.Key ResponsibilitiesSupport the implementation and...

  • Data Analyst

    1 week ago

    Australia ECF Data, LLC Full time

    About ECF Data, LLCECF Data, LLC is a leading provider of data-driven solutions, dedicated to empowering businesses with actionable insights. We are seeking a highly skilled Data Scientist to join our team, reporting directly to the Head of Data Analytics.Key ResponsibilitiesCollaborate with the data team to identify business challenges addressed via...

  • Data Analyst

    1 week ago

    Australia ECF Data, LLC Full time

    About ECF Data, LLCECF Data, LLC is a leading provider of data-driven solutions, seeking a highly skilled Data Analyst to join their team in Sydney. As a pioneer in reimagining global movement, the company aims to elevate the performance of advertising campaigns through data-driven insights and strategic analysis.Key ResponsibilitiesAnalyze Post-Campaign...

  • Data Analyst

    7 days ago

    Australia ECF Data, LLC Full time

    NAB seeks a dynamic Data Analyst to support the Chief Climate Office in delivering cutting-edge data insights. The role involves exposure to commercial, agricultural, and residential portfolios, and focuses on managing and reporting financed emissions.Key Responsibilities:Engage actively in the entire lifecycle of data services, including data collection,...

  • Data Insights Specialist

    5 days ago

    Australia ECF Data, LLC Full time

    Job DescriptionECF Data, LLC is seeking a highly skilled Data Insights Specialist to join their team. As an integral part of the company's data-driven approach, you will play a key role in supporting data science projects and providing bespoke analytical solutions.Key Responsibilities:Data Science Process Management: Collaborate with senior team members and...

  • Data Insights Analyst

    1 week ago

    Australia ECF Data, LLC Full time

    About the RoleECF Data, LLC is seeking a highly skilled Data Insights Analyst to join our team. As an Associate Data Scientist, you will play a key role in supporting data science projects and providing bespoke analytical solutions. You will contribute to optimizing marketing performance, improving customer engagement, and driving commercial value.Key...

  • Senior Data Analyst

    1 week ago

    Australia DATA MART Consulting GmbH Full time

    About the RoleWe are seeking a highly skilled Data Scientist to join our team at DATA MART Consulting GmbH. As a key member of our organization, you will play a critical role in driving business growth and decision-making through data-driven insights.Key ResponsibilitiesCollaborate with Cross-Functional Teams: Work closely with product managers, engineers,...

  • Data Engineering Manager

    1 week ago

    Australia Calleo Australia Full time

    About the Role:Calleo Australia is seeking a highly skilled Data Engineering Manager to lead our data architecture and engineering efforts. As a key member of our team, you will be responsible for designing, developing, and implementing complex data solutions in the cloud using data engineering tools.Key Responsibilities:Lead the development of data...

  • Data Engineering Lead

    4 days ago

    Australia Randstad Full time

    About the Company: Randstad, a leading global recruitment agency, is seeking a highly skilled Data Engineering Lead to join their team. As a key member of the data engineering team, you will play a crucial role in shaping and driving the next evolution of their data and analytics platform.Job Role: As a Lead Data Engineer, you will be responsible for...

  • Data Engineer Specialist

    7 days ago

    Australia Insight Full time

    About the RoleWe are seeking a highly skilled Data Engineer Specialist to join our team at Insight, a leading provider of cloud, data, AI, and cybersecurity solutions. As a key member of our Data & AI Division, you will play a critical role in helping our clients transform their businesses through data-driven insights.Key ResponsibilitiesDesign and Implement...

  • Senior Data Engineer

    1 week ago

    Australia Discovery Village at the West End – IL Full time

    About the RoleThe Senior Analytics Engineer will play a pivotal role in designing and developing robust data products and analytics solutions for scalable data infrastructure at Discovery Village at the West End – IL. As a Senior Analytics Engineer, you will work closely with the Analytics Engineering team, collaborating on projects, optimizing the...

  • Senior Data Engineer

    20 hours ago

    Australia Tbwa ChiatDay Inc Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Data Infrastructure team at Flex. As a key member of our team, you will be responsible for designing, implementing, and maintaining high-quality data infrastructure services, including data lakes, Kafka, Amazon Kinesis, and data access layers.Key ResponsibilitiesDesign and...

  • Senior Data Engineer

    1 week ago

    Australia NCCR RNA & Disease Full time

    About NCCR RNA & DiseaseNCCR RNA & Disease is a leading research institution dedicated to advancing our understanding of RNA and its role in disease. We are driven by a mission to create innovative solutions for a better tomorrow.Job SummaryWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be...

  • Data Engineer

    1 week ago

    Australia Newconcord Oh Full time

    About Newconcord OhNewconcord Oh is a leading organization in the field of environmental conservation. We are committed to delivering high-quality data to our stakeholders and contributing to marine conservation efforts.Job SummaryWe are seeking a highly skilled Data Engineer to join our team. The successful candidate will be responsible for implementing new...

  • Data Engineer

    5 days ago

    Australia Infinite Talent Group Full time

    Job SummaryInfinite Talent Group is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems and architectures.Key ResponsibilitiesDesign and develop data warehouses and data lakes using MS SQL, Teradata, Snowflake, Redshift, and Databricks.Develop...

  • Data Engineer

    1 week ago

    Australia First Line Software, Inc Full time

    We are seeking a skilled Data Engineer to join our dynamic team at First Line Software, Inc. working on an international project for a leading real estate company in the United States. Our team is focused on transforming the real estate industry by helping our client make data-driven decisions for investment evaluation and property acquisition.As a Data...

  • Data Engineer

    7 days ago

    Australia Infinite Talent Group Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Infinite Talent Group. As a Data Engineer, you will play a critical role in designing, building, and maintaining large-scale data systems and architectures.Key ResponsibilitiesData Engineering: Design, develop, and deploy data pipelines, ETL processes, and data warehouses to...