Data Solution Designer

2 weeks ago

Melbourne, Australia XPT Software Australia Pty Ltd Full time
Job Description

We are seeking a talented 10+ years Solution Designer with experience in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer, you will play a key role in architecting and implementing scalable, efficient, and high-performance data solutions.

 

Responsibilities:

 

1.            Architectural Design: Design end-to-end data solutions using Databricks platform, considering scalability, performance, security, and reliability requirements.

2.            Data Modeling: Define data models, schemas, and structures to support efficient data processing, storage, and retrieval within Databricks environment.

3.            Data Ingestion: Implement data ingestion pipelines from various sources such as databases, streaming platforms, and external APIs into Databricks for analytics and processing.

4.            Data Transformation: Develop data transformation pipelines using Apache Spark and Databricks notebooks to clean, transform, and enrich raw data into meaningful insights.

5.            Data Governance: Implement data governance policies, access controls, and data lineage tracking mechanisms within Databricks to ensure data quality, compliance, and auditability.

6.            Performance Optimization: Optimize data processing workflows, queries, and jobs within Databricks to achieve optimal performance and resource utilization.

7.            Integration: Integrate Databricks with other data platforms, tools, and services within the data ecosystem to enable seamless data flows and interoperability.

8.            Documentation: Document solution designs, data flows, architectural decisions, and best practices for Databricks usage to facilitate knowledge sharing and collaboration within the team.

9.            Collaboration: Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver value-added insights.

 

Qualifications:

 

1.            Bachelor's degree in Computer Science, Information Technology, or related field.

2.            Proven experience as a Solution Designer or Data Engineer with expertise in Databricks platform.

3.            Deep understanding of big data technologies, distributed computing, and data engineering principles.

4.            Hands-on experience with Apache Spark, Scala/Python programming languages, and Databricks notebooks.

5.            Familiarity with cloud-based data platforms such as Azure Databricks, AWS EMR

6.            Experience in data modeling, ETL/ELT processes, and data integration techniques.

7.            Knowledge of data governance, security, and compliance practices in data management.

8.            Strong analytical, problem-solving, and communication skills.

9.            Ability to work collaboratively in a cross-functional team environment and drive results

 



Requirements
We are seeking a talented 10+ years Solution Designer with expertise in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer, you will play a key role in architecting and implementing scalable, efficient, and high-performance data solutions. Responsibilities: 1. Architectural Design: Design end-to-end data solutions using Databricks platform, considering scalability, performance, security, and reliability requirements. 2. Data Modeling: Define data models, schemas, and structures to support efficient data processing, storage, and retrieval within Databricks environment. 3. Data Ingestion: Implement data ingestion pipelines from various sources such as databases, streaming platforms, and external APIs into Databricks for analytics and processing. 4. Data Transformation: Develop data transformation pipelines using Apache Spark and Databricks notebooks to clean, transform, and enrich raw data into meaningful insights. 5. Data Governance: Implement data governance policies, access controls, and data lineage tracking mechanisms within Databricks to ensure data quality, compliance, and auditability. 6. Performance Optimization: Optimize data processing workflows, queries, and jobs within Databricks to achieve optimal performance and resource utilization. 7. Integration: Integrate Databricks with other data platforms, tools, and services within the data ecosystem to enable seamless data flows and interoperability. 8. Documentation: Document solution designs, data flows, architectural decisions, and best practices for Databricks usage to facilitate knowledge sharing and collaboration within the team. 9. Collaboration: Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver value-added insights. Qualifications: 1. Bachelor's degree in Computer Science, Information Technology, or related field. 2. Proven experience as a Solution Designer or Data Engineer with expertise in Databricks platform. 3. Deep understanding of big data technologies, distributed computing, and data engineering principles. 4. Hands-on experience with Apache Spark, Scala/Python programming languages, and Databricks notebooks. 5. Familiarity with cloud-based data platforms such as Azure Databricks, AWS EMR 6. Experience in data modeling, ETL/ELT processes, and data integration techniques. 7. Knowledge of data governance, security, and compliance practices in data management. 8. Strong analytical, problem-solving, and communication skills. 9. Ability to work collaboratively in a cross-functional team environment and drive results

  • Data Solution Designer

    2 weeks ago

    Melbourne, Australia XPT Software Australia Pty Ltd Full time

    We are seeking a talented 10+ years Solution Designer with experience in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer, you will play...

  • Data Bricks

    4 weeks ago

    Melbourne, Australia Pyramid Global Technologies Full time

    About the job Data Bricks - Solution Designer "Tech Stack Required:Required: 12+ Years· Experience in Data Lake, Azure Data Bricks· Experience Designing and deploying Databricks platforms for any of AWS, Azure or Google Cloud· Experience building and deploying data engineering pipelines into production, including...

  • Solution Designer

    4 weeks ago

    Melbourne, Australia Employment Hero Full time

    As a valued member of our development team based in Melbourne, you will be responsible for providing solution designs and technical guidance to the development team building the IRiS software. Utilising your big-picture thinking and commercial acumen, you’ll have the opportunity to have an impact as you proactively contribute to strategy and...

  • Senior Azure Data Engineer

    7 days ago

    Melbourne, Australia NTT DATA Full time

    Job Description NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network...

  • Security Solution Designer

    4 weeks ago

    Melbourne, Australia Capstone Full time

    Excellent Day Rate $$$ - 12 Month Contract - Solution Design - DLP - Security Monitoring - Threat Management - Melbourne CBD Location **Security Solution Designer** We currently seek a hands-on Solution Designer responsible for providing solution development and technical leadership across multiple domains (including identity, data loss protection, asset...

  • Solution Designer

    4 weeks ago

    Melbourne, Australia Capgemini Full time

    About Capgemini Capgemini is a diverse collective of more than 350,000 strategic and technological experts based across more than 50 countries, partnering with world-renowned clients to transform and manage their businesses. We are dedicated to leveraging cloud, data, AI, connectivity, software, digital engineering, and platforms to address the entire...

  • Solution Designer

    3 weeks ago

    Melbourne, Australia XPT Software Australia Pty Ltd Full time

    We are seeking a talented 10+ years Solution Designer with expertise in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer, you will play...

  • Solution Designer

    3 weeks ago

    Melbourne, Australia XPT Software Australia Pty Ltd Full time

    Job DescriptionWe are seeking a talented 10+ years Solution Designer with expertise in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer,...

  • Data Science and Analytics Consultant- Australia

    4 weeks ago

    Melbourne, Australia Data Science Group Full time

    As a Data Science and Analytics Consultant, you will work with clients to design their AI processes and governance procedures, develop innovative solutions to complex business challenges, provide strategic guidance on leveraging data to drive business outcomes, and design and implement data-driven solutions. You will be responsible for delivering successful...

  • Solution Designer

    4 weeks ago

    Melbourne, Australia RMIT VN Full time

    Overview: - Dynamic Solution Designer position available, 2 years Fixed Term position. Based at the Melbourne CBD campus (Hybrid role), and Flexible Working Arrangements- Deep Technical Skills and Technical leadership across large and complex Network & Cybersecurity solutions. Initial primary focus is Network Segmentation.- As a trusted advisor and key...

  • Salesforce Solution Designer

    7 days ago

    Melbourne, Australia Launch Recruitment Full time

    Flexible, hybrid working environment (CBD office location) - Technology: Salesforce, APIs, HTML & CSS, DevOps, Agile and more - Initial 12 month contract (initially) **About the Company** This client is responsible for regulating Victoria's building and plumbing industries. **The Role** You will be responsible for the Salesforce Solution Design and for...

  • Data Solution Lead

    4 weeks ago

    Melbourne, Australia ANZ Banking Group Full time

    **Req ID**: 46740 **Department**: Risk Strategy & Transformation - Reporting & Model **Division**: Risk **Location**: Melbourne About the role At ANZ our purpose is to shape a world where people and communities thrive. We’re making this happen by improving the financial wellbeing and sustainability of our customers so they can achieve incredible...

  • Solution Designer

    7 days ago

    Melbourne, Australia Davidson Recruitment Full time

    **About the Company** Davidson Technology have partnered with One of Australia's big four banks, that offers a wide range of banking and financial services in Australia, New Zealand, the United Kingdom, and Asia. They are looking for a Solution Designer to join their Fincrime team. Known for their innovative projects, this opportunity will offer career...

  • Solution Designer

    4 weeks ago

    Melbourne, Australia Paxus Australia Pty Ltd Full time

    Posted 25 January 2023 - SalaryAU$950 - AU$1000 per day - LocationMelbourne - Job type Contract - DisciplineSoftware Development + Testing - ReferenceBBBH259974_1674617834 **Job description**: We are looking for an experienced Solution Designer for a high profile financial services firm. The role is based in Melbourne and will be for 6 months. The role is...

  • Solution Designer

    4 weeks ago

    Melbourne, Australia SG Consulting Limited Full time

    SG Consulting is an IT consulting firm based in New Zealand, Australia, and India and the business leaders come to us when they want enduring results. We find value across boundaries, develop insights to act on, and energize teams to sustain success. We are looking for a Solution Designer for one of our clients in Melbourne, Australia. **Short...

  • Solutions Design Manager

    4 weeks ago

    Melbourne, Australia Zoom Recruitment Full time

    **THE COMPANY** Zoom Recruitment is excited to be partnering with the world’s leading integrated logistics service provider with a global network of some 2000 locations and over 76,000 employees worldwide. Their Asian Pacific region operations connects some 14 countries through land transport, air and ocean freight, contract logistics and Supply Chain...

  • Data Quality Solutions Analyst

    7 days ago

    Melbourne, Australia Nixil Full time

    **Work options**: Hybrid We're embarking on a crucial phase in our partner's digital transformation journey, and you could be a key player in elevating data quality solutions across the organization. **Your Impact**: As a Data Quality Solutions Analyst, you'll be at the forefront of our partner's Investment Data Management (IDM) unit, ensuring the...

  • Data Analytics Consulting Virtual Internship

    4 weeks ago

    Melbourne, Australia Data 2 Insight Consulting Full time

    Are you an established Data & Analytics Professional with experience in Cloud based Data Platforms including Snowflake, Azure & Databricks and looking for your next opportunity? Data to Insight (D2i) are excited to be expanding and looking to welcome a Data & Analytics Professional who has a passion for technology, solving complex data queries and...

  • Solution Designer

    4 days ago

    Melbourne City Centre, Australia Peoplebank Full time

    Location: - Melbourne CBD- Job Type: - Contract- Posted: - 34 minutes ago- Contact: - Tamara McMurtrie- Discipline: - General IT - Reference: - 257642- **Fulfil your professional development goals in a dynamic and support environment**: - ** Gain access to roles across diverse industry sectors, utilising various technologies in a wide range of IT...

  • Solution Designer

    6 days ago

    Melbourne City Centre, Australia Peoplebank Full time

    Our client a large Australian Government Department looking for a Solution Deisigner to join the team. Skills and experience include: - Create and maintain technical design documents and requirements including workflow design of organization & enterprise grade API's - Participate in design and planning meetings with company's API developers and Systems...