Urgent Search: Senior Data Engineer
3 weeks ago
Senior Data Engineer: Firewall Automation Program
- Contract: 6-12 months (Pty Ltd only)
- $1,150-$1,200/day + GST
- Sydney based (Hybrid)
- Urgent start - interviewing immediately
About the role
Build MS SQL ETL pipelines that consolidate 5 million network flows down to 5,000-10,000 automated firewall rules for a tier-1 bank's CoreMod migration. You'll enrich NetFlow data with CMDB/DNS/VMware metadata, produce JSON/CSV deployment feeds for APIC/CSM, and optimise performance at scale. This role suits a data engineer who thrives on genuinely interesting technical problems in collaborative mob programming environments.
About the engagement
Major infrastructure transformation migrating 400 applications across ~3,000 servers. Three-person specialist team (you, Python dev, network engineer) automating firewall implementation across Legacy and CoreMod infrastructure. Fast-tracked delivery with high autonomy and creative problem-solving.
What you'll own
Data Pipeline: Consolidate 5M network flows stored in MS SQL into contextualised, validated firewall rules using high-performance T-SQL ETL jobs with deduplication, enrichment and sliding-window aggregations.
Deployment Feeds: Produce controller-ready JSON/CSV exports conforming to CSM import formats and APIC object model payloads that enable the Python dev to automate rule implementation via Cisco APIs.
Orchestration: Create Tidal job definitions, retries, alerts and exception queues. Implement lineage tracking from aggregated tuples back to raw flows for audit purposes.
This is collaborative data engineering - mob programming setup where all three specialists work in lockstep. You'll optimise partitioning, indexing and query plans for very large NetFlow tables while navigating access problems and data challenges with a proactive mindset.
What we need
- Must have: Expert MS SQL Server (T-SQL, indexing, partitioning, window functions, query tuning), high-performance ETL pipelines for large-scale datasets (5M+ rows), controller-ready export formats (JSON/CSV) for network automation platforms, Tidal or similar orchestration tools, data engineering mindset (NOT DBA), collaborative team player, Pty Ltd contractor.
Nice to have
- Dask/Spark experience
- Power BI
- Understanding of network flows/NetFlow data
- Migration project experience
Won't work
- Pure DBA backgrounds
- Expecting cloud migration work
- Needing extensive banking experience
The opportunity
Real infrastructure impact enabling automated firewall deployment at enterprise scale. Strong likelihood of 12-month engagement.
#J-18808-Ljbffr
-
Clinic Manager
1 week ago
Newcastle West, Australia Heal Specialist Urgent Care Full time**At Heal, we’re on a mission to revolutionise the way urgent medical care is delivered in Australia by**: - Making healthcare easy - Delivering extraordinary care - Caring for people, not patients - Empowering people by growing health literacy We are building a team of clinical and support excellence, to challenge the status quo in healthcare and help...
-
Urgent: Senior Structural Engineer
2 weeks ago
Newcastle, Australia ClearCompany Full timeOverview It's in their Parramatta office where they're looking for an exceptional Senior Structural Engineer to lead a team of 5 and manage projects. Responsibilities - Managing the delivery of all design aspects of the project on time, on budget and to standards - Design and Project Management of medium to high rise building projects including...
-
(Urgent Search) Data Quality Officer
4 weeks ago
Newcastle, Australia Domain Group Full timeData Quality Officer Job Description Company Overview: CoStar Group (NASDAQ: CSGP) is a leading global provider of commercial and residential real estate information, analytics, and online marketplaces. Included in the S&P 500 Index and the NASDAQ 100, CoStar Group is on a mission to digitize the world’s real estate, empowering all people to discover...
-
▷ [Urgent] Senior Project Engineer - Earthworks
4 weeks ago
Newcastle, Australia John Holland Pty Ltd Full timeJob Number: 21098 Location: Mascot, NSW, AU, 2020 At John Holland, our purpose is simple – we transform lives with everything we do. We’ve always known that infrastructure is about people — our customers, our employees, and the communities in which we work every day. That’s our difference. Deep experience and capability with a genuine care about...
-
▷ Urgent Search: Senior Business Analyst
3 weeks ago
Newcastle, Australia ClearCompany Full timeOverview Senior Business Analyst - Merchant Payments Location: Sydney (Hybrid) Contract Type: Day Rate Contract Rate: Competitive daily rate (based on experience) Eligibility: Sydney-based candidates only We're seeking an experienced Senior Business Analyst to join a leading Australian financial institution on a Merchant Payments program. Key...
-
Senior Network Engineer
1 week ago
Newcastle, New South Wales, Australia VIS Global Pty Ltd Full time $120,000 - $180,000 per yearJob Title: Senior Network EngineerDesign, deploy, and manage LAN, WAN, WLAN, and cloud network infrastructures.Monitor network performance and troubleshoot complex issues related to connectivity, latency, throughput, and system integration.Plan and execute upgrades, maintenance, and expansion of existing infrastructure.Implement and enforce security...
-
Senior Algorithm Engineer
2 weeks ago
Newcastle, New South Wales, Australia Advanced Navigation Full time $120,000 - $180,000 per yearAdvanced Navigation - It's In The NameAdvanced Navigation is a global leader in navigation and autonomous systems. By leveraging capabilities in software-enhanced hardware, every solution delivers unrivaled capabilities and exceptional performance across land, air, sea and space applications where GPS is unreliable.Made possible with extensive research,...
-
Newcastle, Australia Notre Dame Full timeResearch Assistant, School of Medicine, Sydney Campus, Part‑time opportunity (0.6), fixed‑term contract for 12 months - Friendly and collaborative work environment - Rewarding opportunity offering career development - Level 4 from $80,208 + 14.50% super (pro‑rata) About the university The University of Notre Dame Australia is a private Catholic...
-
Senior Control Systems Engineer
7 days ago
Newcastle, Australia Alliance Automation Pty Full timeJoin Alliance Automation: Where Excellence Meets Opportunity! Transforming Australian Industry with Secure Digital Solutions, Smart Automation, and Electrical Engineering. We believe that our people are our most valuable asset. With a strong emphasis on integrity, trust, and honesty, we incorporate business ethics into everything we do. Join us and become...
-
Senior Data Analyst
1 day ago
Newcastle, Australia ClearCompany Full timeFeel empowered in this Innovative team with AI and data analytics in its DNA. You'll drive results that combines cutting edge advanced analytics and automated AI with an integrated suite of leading tools and technologies. We have an exciting opportunity for a Senior Data Analyst to join a Data Analytics practice within our Financial Services client where...