Building Scalable Data Pipelines
2 days ago
We are seeking a skilled Data Engineer to join our organization.
- Data Engineers will be responsible for designing, building, and maintaining scalable data pipelines and workflows on the Databricks platform.
- Collaborate with data engineers, data scientists, and analysts to optimize data architecture and performance.
- Implement monitoring, alerting, and automation solutions to ensure the reliability and efficiency of Databricks clusters and jobs.
- Provide technical expertise and support for troubleshooting issues and optimizing data processing and analysis tasks.
Required Skills:
- Proficiency in Databricks, Apache Spark, and related big data technologies.
- Experience with designing and implementing ETL processes, data modeling, and data warehousing concepts.
- Strong programming skills in Python, Scala, or SQL.
- Familiarity with cloud platforms like AWS.
Certifications in Databricks or related technologies are desirable. Experience with Airflow concepts such as scheduling, task dependencies, retries, SLA monitoring is also required. Knowledge of CI/CD and DevOps practices, including version control (Git), is essential.
About the OrganizationWe are a technology and digital recruitment agency connecting professionals with clients across Asia-Pacific.
-
Building Scalable Data Solutions
7 days ago
Sydney, New South Wales, Australia beBeeData Full time $150,000 - $180,000Data Solutions EngineerWe are committed to building scalable data solutions by leveraging cloud infrastructure and collaborating with global teams.At our organization, diverse professionals come together to drive innovation in 34 markets.Key ResponsibilitiesDesign, develop, deploy, and maintain data assets on Google Cloud Platform (GCP).Coordinate project...
-
Building Scalable Infrastructure Pipelines
4 days ago
Sydney, New South Wales, Australia beBeeDeveloper Full time $120,000 - $140,000Job Title: Pipeline DeveloperJob DescriptionCrafty Apes is a global company that specializes in visual effects and infrastructure development. We are currently looking for an experienced Pipeline Developer to join our team.The ideal candidate will have a strong background in pipeline design, development, and maintenance, as well as expertise in tools like...
-
Sydney, New South Wales, Australia beBeeData Full time $120,000 - $180,000Data SpecialistThis role focuses on transforming existing proof-of-concept into scalable data solutions. Ideal candidate will have extensive knowledge of Python, especially working with PySpark.The successful Data Engineer will be responsible for building and refining complex data pipelines to support ongoing monitoring and management. They must collaborate...
-
Building Scalable AI Systems
2 days ago
Sydney, New South Wales, Australia beBeeArchitect Full time $180,000 - $190,000AI Architect RoleAs an AI Architect, you will be responsible for designing scalable and secure artificial intelligence systems that meet the needs of our clients.This is a critical role that requires expertise in software/data/cloud architecture, with a focus on AI/ML solutions. You will work closely with cross-functional teams to develop innovative...
-
Advanced Data Pipeline Specialist
4 days ago
Sydney, New South Wales, Australia beBeeDataEngineer Full timeSenior Data Engineer Job DescriptionWe are seeking a skilled Senior Data Engineer to design, build, and maintain scalable data pipelines and ETL processes using Azure Data Factory. This role requires expertise in data architecture, ETL processes, and data modeling on Azure.The ideal candidate will have strong analytical and problem-solving skills, with the...
-
Building Scalable Data Solutions
3 days ago
Sydney, New South Wales, Australia beBeeData Full time $180,000 - $220,000Job Title: Enterprise Data ArchitectCuscal is shaping the future of payments and pioneering innovative solutions that millions of Australians rely on every day. With over 59 years of experience, we're preparing for what's next.As an Enterprise Data Architect in our Data Governance and Data Platform team, you'll develop, maintain, and support ground-breaking...
-
Senior Data Pipeline Engineer
1 day ago
Sydney, New South Wales, Australia beBeeMachine Full time $120,000 - $140,000We are looking for an experienced engineer to join a team focused on scaling and maintaining machine learning platforms.Key Responsibilities:Design, build, and maintain end-to-end data pipelinesCollaborate with data scientists to deploy models into productionImplement automated workflows for model deploymentMonitor and manage model performance metricsDrive...
-
Build Scalable Data Systems
1 day ago
Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $140,000Data Engineer PositionWe are seeking a skilled Data Engineer to play a critical role in shaping the future of automotive in Australia.The Data Engineer will work collaboratively with technology squads, ETL Engineers, and the Head of Infrastructure to create flexible, scalable, and efficient data models that cater to diverse analytical and processing...
-
Data Pipeline Architect
2 days ago
Sydney, New South Wales, Australia beBeeEtldatabaseexpert Full time $120,000 - $170,000Job DescriptionAs a key member of our team, you will play a vital role in designing and implementing high-performance ETL systems that power insights across the organization. You will work with large-scale data in a batch processing environment, creating robust data pipelines and warehouse architectures.Design and implement high-performance ETL systemsWork...
-
Senior Data Pipeline Specialist
1 day ago
Sydney, New South Wales, Australia beBeeDataEngineer Full time $100,000 - $130,000Optimize Your Data Career with a High-Performing TeamJob OverviewWe're seeking a skilled Data Engineer to join our high-performing team. This role involves contributing to a data migration project, working on high-quality data streaming applications.This initial 6-month contract will require collaboration with the team to design and implement scalable data...
