Lead Data Engineer

The Australian Criminal Intelligence Commission (ACIC) requires an experienced Data Engineer/Analyst to the National Criminal Intelligence System (NCIS) Program.

Data Engineers prepare raw data for analytical or operational uses. They make data accessible so that it can be used for optimisation, delivery of services and enhanced analytics solutions.

Key duties and responsibilities

Specified personnel will be responsible for undertaking the following activities including but not limited to:

• Provide analytical support and reporting on data quality assessment across the National Criminal Intelligence (NCIS) system

• Engage with stakeholders and consult with the Principal Advisor, Director, Project Managers, Technical Team Leads, Project Teams and external police partners as required

• Assist in the development and analysis of database queries, data collection systems, data analytics, reporting mechanisms and other strategies that optimise statistical efficiency and quality with the aim of improving the provision of NCIS system

• Negotiate and acquire data from primary and secondary data sources (including external partners) and maintain databases I data systems

• Provide data specific design advice to projects that are aligned with the ACIC’s ICT Strategic Plan

• Develop project related documentation as required

• Interpret data, analyse results using statistical techniques and provide ongoing reports

• Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality

• Identify, analyse, and interpret trends or patterns in complex data sets and assemble large, complex data sets that meet business requirements

• Filter and “clean” data by reviewing reports, printouts, and performance indicators to locate and correct code problems

• Work with management to prioritize business and information needs

• Create and maintain optimal data pipeline architecture,

• Identify, design, and implement internal process improvements: automating manual processes, enhancing data delivery, re-designing infrastructure for greater scalability, etc

• Locate and define new process improvement opportunities

• Provide support for developers in implementing specified solutions, including review of technical specifications.

• Provide support for testers, including review of test plans and scripts

• Provide written and oral briefings to senior executives.

• Undertake other Systems Analysis duties as required.

Security requirements for the role

The ACIC requires specified personnel to:

• Obtain the ACIC’s Organisational Suitability Assessment (OSA) prior to commencement.

• Have an Australian Government security clearance (AGSVA) to the minimum of Negative Vetting Level 1 (NV1) or be willing to undergo and obtain and AGSVA NV1 clearance.

• As directed by the ACIC, complete a Deed of Confidentiality or Non-Disclosure Agreement prior to commencement or comply with processes required to engage the Specified Personnel under section 48 of the Australian Crime Commission Act (2002).

Note: Seller may be required to meet the cost of any security clearances.

About the project

The National Criminial Intelligence System (NCIS) is a joint project between the ACIC, Australian police agencies and the Department of Home Affairs. NCIS connects data from the ACIC and law enforcement to provide a comprehensive unified picture of criminal activity.

About the organisation

The ACIC provides mission critical intelligence to partners and government to reduce the harm from serious criminal threats impacting Australia. The ACIC is part of the Attorney-General’s portfolio and is a member of the Five Eyes Law Enforcement Group. The ACIC’s specialist capabilities include national criminal intelligence data holdings.

Essential criteria

1. 3+ years demonstrated experience successfully researching, selecting and integrating a Big Data technology stack including batch and stream processing tools such as Spark and indexing technologies such as Elastic and Solr

2. Experience modelling, extracting, loading and analysing data from a wide range of sources including flat files, SQL and nosql data and providing advice about ways to enhance scalability and system performance

3. Experience forming and maintaining strong working relationships with internal and external stakeholders including inter-disciplinary and inter-agency liaison.

4. Demonstrated relevant tertiary qualifications or equivalent/demonstrated experience

5. Flexibility of the proposal over the lifecycle of the procurement and environmental sustainability of the goods or services

___________________________________________________________

This role requires candidates to be onsite in Canberra. Working remotely up to 3 days per week may be negotiated after 6 months subject to approval and in accordance with the ACIC Home Based Work Policy. Seller will be required to meet any costs associated with travel and accommodation should they be incurred for remote work purposes.