Technical Lead Data Engineering

About the role 

We are now seeking strategic and hands‑on data leader to guide our high‑performing team in delivering smarter, more efficient regulatory outcomes whilst playing a key role to uplift processes, strengthen governance and drive reliable data solutions.  

As Technical Lead, Data Engineering you will bring expertise across data architecture, data lifecycle management, AI software development, information security, and agile delivery.  
 
Key responsibilities of this role are:  

• Be accountable for the quality of data pipeline development and the flow of data on the NHVR Enterprise Data Platform. Quality is measured by data integrity, pipeline performance, uptime, efficient administration, cost, data security, data governance, and architectural integrity.
• Own and evolve the platform architecture and technical roadmap. This includes leading the transition from notebook- and job-based pipelines toward Lakeflow declarative pipelines and other modern Databricks capabilities.
• Own the design and continuous improvement of the metadata-driven ingestion and transformation framework. Ensure it stays scalable, maintainable, well-documented, and consistently applied.
• Define, document, and enforce engineering standards, patterns, and practices, covering coding standards, Delta Lake and medallion design, testing, CI/CD, and observability. Ensure compliance through design reviews and code reviews.
• Provide authoritative advice to Managers, Directors, and other senior staff across the organisation regarding architecture, augmentation, implementation, and manageability of the data on the NHVR Enterprise Data Platform.  
• Provide technical guidance, mentoring, and coaching to Data Engineers, including pairing on complex problems and building team capability in Databricks and modern engineering practice.  
• Lead technical planning within the Scrum team: break down epics, assess solution options, estimate technical effort, and identify technical risks and dependencies early.
• Act as the technical escalation point for complex incidents, leading root cause analyses and driving lasting improvements to monitoring, alerting, and resilience.
• Represent data engineering changes at CAB and ensure releases meet quality, security, and change management requirements.
• Remain hands-on, delivering complex or high-risk pipeline and framework components alongside the team.  

NHVR’s Data Science and Analytics Unit are currently undertaking growth and considerable transformation, and this is an exciting time to join the organisation. 

To learn more, the position description can be found on the NHVR Careers site -https://careers.nhvr.gov.au/jobs/search 

About You  

To be successful in this role you will possess a Master’s degree in information technology or computer science related or equivalent.  You will have a minimum of 10 years’ experience with demonstrated skills in Databricks and Azure cloud platform (certifications in either / both would be highly regarded) 

Your toolkit for success: 

• Expert Python and PySpark development skills, with advanced SQL (Spark SQL and T-SQL or similar).
• Deep Databricks expertise, including notebooks, Jobs/Workflows, cluster configuration and policies on classic compute, and advanced Spark performance tuning.
• Advanced Delta Lake knowledge, including MERGE and upsert patterns, schema evolution, OPTIMIZE and VACUUM, liquid clustering and partitioning strategy, and time travel.
• Demonstrated experience designing, building, or owning metadata-driven or configuration-driven pipeline frameworks.
• Demonstrated experience designing and implementing Unity Catalog security and governance models, including grants and ownership, row and column level security, external locations, storage credentials, and lineage.
• Proven experience establishing approaches for identifying, classifying, and protecting sensitive and PII data.
• Strong data architecture and modelling skills, covering relational, dimensional, and medallion architecture, normalisation, schemas, and data profiling.
• Proven code review, software analysis, and design review skills, with a track record of defining and enforcing engineering standards.
• Strong software engineering discipline, including modular design and design patterns, unit and data quality testing (e.g. pytest, data quality frameworks), Git-based workflows, CI/CD with Azure DevOps, and debugging.
• Extensive experience with ELT/ETL pipelines and data integration, including REST APIs, file-based exchange (e.g. SFTP), change data capture, and streaming or event-based patterns.
• Experience leading technical teams in Scrum and Agile methods, with the ability to mentor engineers and influence without direct authority.
• Excellent communication skills, including explaining complex technical matters and trade-offs to senior, non-technical stakeholders

Why join us?

The National Heavy Vehicle Regulator (NHVR) commenced operations in 2013 as the national entity responsible for regulating all vehicles in Australia over 4.5 tonnes gross vehicle mass. The NHVR minimises the compliance burden on the heavy vehicle transport industry, reduces duplication and inconsistences across state and territory borders, and ultimately for the heavy vehicle business with government in Australia.

We are headquartered in Brisbane and employ more than 1000 people across the ACT, New South Wales, Queensland, South Australia, Tasmania and Victoria.

We offer-

• A competitive salary package of $166,662.51 per annum + Superannuation.
• Monthly RDO and paid leave over Christmas closure.
• Hybrid working environment – up to 2 days from home.
• Up to15.25% superannuation through a co-contribution scheme
• Leave loading 17.5%
• Wellbeing programs – discounted fitness through Fitness Passport, Discount health insurance, flu vaccine, financial programs including discounted banking products, finance information sessions.
• Salary packaging options including novated lease.
• Discounted IT products including Microsoft and free family password security plan.
• Ongoing training and professional development support
• Professional membership

Application Process

Please apply with your CV and cover letter addressing the essential requirements of the role.

If you have any questions, please don’t hesitate to contact the NHVR recruitment team on recruitment@nhvr.gov.au.

Successful applicants will be required to complete standard pre-employment checks including a national police and work rights check.

Please Note: We will not require any support from Recruitment Agencies at this time and kindly request no contact is made. Only direct applications will be considered.

We are committed to being an inclusive, diverse and flexible workplace where differences are valued. We welcome applications from people of all backgrounds, experience and abilities.