Data Engineer

THE WEBFORM TO SUBMIT YOUR CV IS NOT WORKING – PLEASE EMAIL YOUR CV DIRECTLY TO INFO@ENCOREIT.COM.AU

The Data Engineer is an experienced practitioner who independently designs, builds and supports enterprise data solutions that enable trusted reporting, analytics and AI-driven insights. The role translates solution designs and business requirements into scalable, production-ready deliverables while ensuring solutions are performant, governed and aligned with business outcomes.

Key duties and responsibilities

Key Responsibilities

• Develop and maintain batch, micro-batch and real-time data ingestion, transformation and curation pipelines using event-driven architectures where appropriate.
• Translate solution designs, technical specifications and data contracts into scalable, production-ready data solutions.
• Apply data warehousing and governance principles, including data modelling, quality, metadata, lineage, classification, retention, security and auditability.
• Leverage enterprise frameworks, standards and reusable design patterns to deliver consistent solutions.
• Identify and close gaps between solution design and implementation, ensuring requirements are fully realised.
• Proactively identify risks, dependencies and design deficiencies, recommending sustainable improvements.
• Optimise solutions for performance, scalability, reliability and operational supportability.
• Collaborate with architects, business analysts, product owners and stakeholders throughout the delivery lifecycle.
• Contribute to estimation, technical analysis, solution planning and continuous improvement activities.
• Develop and maintain DataOps practices, including CI/CD, infrastructure automation, automated testing, monitoring, alerting, observability and release management for data pipelines and data products.
• Build and optimise trusted, high-quality datasets that support advanced analytics, machine learning and generative AI capabilities.
• Capture lessons learned and drive standardisation, automation and reusable delivery patterns.

Essential Experience and Qualifications

• Minimum three years’ data engineering experience, including at least two years designing, building and operating production data solutions on Google Cloud Platform (GCP).
• Demonstrated experience delivering and supporting production data pipelines and data solutions in enterprise environments.
• Experience applying data warehousing, dimensional modelling and data quality practices in enterprise environments.
• Must have Negative Vetting 1 Clearance

Technical Skills

• Google Cloud Platform (GCP)
  • BigQuery, including data modelling, partitioning, clustering, query and cost optimisation, materialised views and workload monitoring.
  • Dataflow and Apache Beam for developing scalable batch and streaming data pipelines.
  • Dataform, dbt or comparable SQL-based transformation frameworks, with Dataform experience preferred.
  • Cloud Storage (GCS).
  • Cloud Composer / Airflow.
  • Pub/Sub and event-driven data ingestion.
  • Dataplex, data cataloguing, metadata management, lineage and data quality controls.
  • Datastream, BigQuery Data Transfer Service or comparable tools for batch ingestion, migration and change data capture.
  • IAM, service accounts, least-privilege access, encryption and key management, secrets management and secure handling of sensitive information.
  • Cloud Logging, monitoring, alerting and observability.
  • Designing resilient and fault-tolerant data workloads, including disaster recovery and separation of development, test and production environments.
  • Working knowledge of GCP networking and secure connectivity patterns for data services.
  • Cloud Functions (desirable).
  • Cloud Run (desirable).
  • Dataproc (desirable).
• Advanced SQL and Python for designing, developing, testing and optimising production data pipelines and data solutions.
• Terraform for provisioning and managing cloud infrastructure through infrastructure-as-code practices.
• Git-based source control and CI/CD using Cloud Build, GitHub Actions or equivalent tooling.
• Automated unit, integration, reconciliation and data quality testing for pipelines and data products.

Key Capabilities

• Strong understanding of modern data architectures, cloud-based data platforms and data lifecycle management.
• Ability to use enterprise data frameworks while identifying opportunities for automation, standardisation and optimisation.
• Strong analytical and problem-solving skills.
• Demonstrated ability to bridge business requirements and technical implementation.
• Strong stakeholder engagement and communication skills.
• Ability to balance immediate delivery needs with long-term platform sustainability and capability uplift.
• Ability to design and deliver reusable data products.

Desirable Experience and Qualifications

• Google Cloud Certified – Professional Data Engineer.
• Experience implementing Data Mesh principles, including domain-aligned ownership, federated governance and reusable data products.

Success Measures

Successful Data Engineers:

• Deliver reliable, scalable and high-performing data solutions.
• Consistently convert solution designs into business-ready outcomes.
• Identify and resolve implementation gaps before they impact delivery.
• Improve data quality, trust and usability through sound data engineering and governance practices.
• Enable analytics-ready and AI-ready capabilities through well-designed and governed data solutions.
• Improve operational reliability and reduce overhead through engineering best practices, optimisation and automation.
• Contribute reusable patterns, automation capabilities and platform standards that support continuous improvement.

Remote/flexible work arrangements can be discussed with the Specified Personnel’s manager upon commencement, and a Working from Home Agreement must be formalised