This site uses cookies to provide a better user experience. For further information click here. To remove this message click here.
Top banner

National Pupil Database (NPD)

 

Dataset summary

Full title of dataset: National Pupil Database.

Period covered: From 2002.

Frequency of release: Annual.

Geography: England.

Data updated: Throughout the year.

Format output: SPSS, CSV or text file (other options may be available on request).

Size: The NPD holds a number of different datasets, holding thousands of records each.

Data quality: Generally of high standard. Please see specific section for more information.

Example variables: Postcode, ethnicity, attainment, free meals eligibility, special needs.

Weighted or unweighted: Unweighted.

Access: Yes.

Documentation: All documentation relating to the NPD is available to download from the Department for Education’s website here.  Alternatively, the ADLS has grouped this documentation into a single PDF pack, or you can download the individual documents in the ‘Access’ section towards the bottom of this page.

The ADLS provide regular updates on the release of the NPD data.  Follow us on Twitter to receive this information.

Contact Details: For applications and also general enquiries please contact the NPD Data Warehouse team at  NPD.requests@education.gsi.gov.uk or telephone 01325 735433. 

Overview

The National Pupil Database (NPD) is one of the richest education datasets in the world, holding a wide range of information about students who attend schools and colleges in England.  The NPD combines the examination results of pupils with information on pupil and school characteristics and is an amalgamation of a number of different datasets, including Key Stage attainment data and Schools Census data (formerly known as PLASC) which are linked using a unique identifier for each pupil.

The NPD provides detailed information about children’s education at different stages (pre-school, primary and secondary education and further education).  The data held includes detailed information about pupils’ test and exam results, prior attainment and progression at different key stages for all schools in the state sector in England.  Attainment data is also held for pupils and students in non-maintained special schools, sixth-form and FE colleges and (where available) independent schools.

The NPD also includes further information about pupils in the state sector and non-maintained special schools such as gender, ethnicity, first language, eligibility for free school meals, information about special educational needs (SEN) and detailed information about pupil absence and exclusions.

The Key Stage data contain the attainment scores for children taking key stage examinations in each year. Thus, each key stage (1 to 5) is contained on a separate file and each of these files contains around 700,000 records (although this depends upon the size of the cohort in each year).

Key Stage tests are taken at five different age points, typically at 7, 11, 14, 16 and 18.  Key Stage 1 and Key Stage 3 are now examined through teacher assessment, whilst Key Stage 2 is assessed through national tests. Key Stage 4 relates to attainment in GCSE and equivalent qualifications and covers all candidates who were at the end of Key Stage 4 in the current year and who sat their exams at institutions in England.  Key Stage 5 covers all 16-18 year old candidates who attempted a GCE/GCE Applied A Level or GCE Applied Double Award Level or Level 3 qualification equivalent in size to at least one A level in the current year and who sat their exams at institutions in England.

The Schools Census dataset contains approximately eight million records per year and includes variables on the pupil’s home postcode, gender, age, ethnicity, special educational needs (SEN), free school meals eligibility, and schooling history. It covers pupils in state-funded primary, secondary, nursery, special schools and pupil referral units. Schools that are entirely privately funded are not included.

Prior to 2007, the Schools Census dataset was known as the Pupil Level Annual Schools Census (PLASC). The only reason for this name change was because a decision was made to collect this data three times a year. The Key Stage attainment datasets are updated yearly.

Whilst the Key Stage attainment datasets and School Census dataset can be used independently, they are more valuable as a merged dataset in order to combine information on pupil characteristics with attainment scores. As such, the NPD has been widely used for research purposes. The linked dataset can include historic information from previous versions of the School Census/PLASC and for previous attainment exams.

PLUG is run by the University of Bristol and is a source for information, training and documentation about the NPD.  Further information is available from here.

There is also a dedicated NPD wiki page which provides useful information and an opportunity to share your thoughts and questions here.

Coverage

Geography
The National Pupils Database covers all pupils in state (or partially state-funded) schools in England. Similar systems operate across the rest of the UK.

  • For Wales see here.
  • For Scotland see here.
  • For Northern Ireland see here.

Time
The PLASC data was first collected in 2002 and was updated annually until 2007. In 2007, the PLASC was renamed the School Census. The same data are now collected three times a year in January, May and October.

The initial year for which Key Stage attainment data were first collected varies according to the examination of interest. For example Key Stage 2 data was first collected in 1996 and Key Stage 5 data was first collected in 2002.

Variables are updated at different points in the year.

Data updates
As the NPD is a combination of many datasets, each dataset is updated at different points throughout the year. As an example, the Schools Census is updated three times a year (available in June, August and January) and the Key Stage attainment dataset is updated annually.

For further information on NPD dataset releases download our administrative data timetable here.  You can also follow the ADLS on Twitter to receive information about when new NPD releases are available.

Suitability

Examples of usage

The National Pupil Database forms a significant part of the evidence base for the education sector and supports a number of key priorities around accountability and school improvement.  For example:

  • It supports schools’ operational decisions by enabling schools and inspectors to interrogate test and examination results in order to identify strengths and weaknesses and help them focus on those areas that need most improvement, including lesson planning and support for individual pupils.
  • It is the source for a wide range of analysis and statistics published in statistical first releases (SFRs) on the Department’s website. For example, each year the Department publishes information about National Curriculum assessment and GCSE attainment by key pupil characteristics such as ethnic group, special educational needs status and free school meals eligibility in England.
  • The Department uses the data to provide accurate targeting of funding for local authorities and schools, including the Pupil Premium and Revenue Support Grant. The data and statistical analyses are also used to inform, influence and improve education policy and to monitor the performance of the education service as a whole.
  • The data held in the NPD is also used in a wide range of research including being linked, with consent, to a variety of external datasets.

Research and Statistics Gateway
The NPD is also the source for analyses in Statistical First Releases (SFRs) published in the Department for Education Research and Statistics Gateway. For example, each year the Department for Education publishes an SFR on National Curriculum assessment and GCSE attainment by key pupil characteristics such as ethnic group, special educational needs status and free school meals take-up in England.

Research uses
The inclusion of postcode in the School Census enables various residency based analyses, e.g. showing pupil attainment and absence by residence, analyses of cross border movement, distances travelled to school and links to other data sets including IDACI and ACORN.

The NPD has allowed the calculation of Contextualised Value Added (CVA) measures, which take account not only prior attainment, but also a number of other pupil and school characteristics associated with performance differences which are outside schools’ control, such as gender, special educational needs, movement between schools, and family circumstances. KS1-2, KS2-3 and KS3-4 CVA measures were published for the first time in the 2007 Achievement and Attainment Tables (AATs).

Data in the NPD are also being used to match to previous surveys such as the Longitudinal Study of Young People in England (LSYPE) and as such have been used as a sampling frame for surveys carried out as part of research and evaluation of education policies. Linking data in this way has reduced the reporting burden placed on schools.

Some examples of work carried out using the Schools Census / PLASC can also be found on the PLUG website.

Publications using this research data are available from our Publication Hub here.

Technical guidance
The National Pupil Database can be used for comparison of attainment and pupil characteristics at school and local authority level as well as for detailed analysis at pupil level.

To understand how the NPD files from different years can be linked together please refer to the document ‘Structure Cohorts’ available from the PLUG website.  Examples of different file formats are also available there too.

The Education (Individual Pupil Information) (Prescribed Persons) (England) Regulations 1999 was amended in 2013 to broaden the sharing of pupil data from persons ‘conducting research into the educational achievement of pupils‘ to

‘persons who, for the purpose of promoting the education or well-being of children in England are conducting research or analysis, producing statistics, or providing information, advice or guidance, and who require individual pupil information for that purpose’.

This revised wording reflects the Department’s aims in widening the use of the NPD data beyond research into education attainment. It will enable the use of the data for wider research such as socio-economic analysis, or research into equality issues, including disability, gender or race.  This change will increase accountability and promote a range of social and economic benefits.

Prescription will also be made in the Regulations for religious bodies for schools designated as having a religious character. This will allow Dioceses and other religious bodies to access the data of pupils attending their schools, without placing an unnecessary administrative burden on what are often small schools. This data will help Dioceses and other religious bodies to provide appropriate support programmes and to assist with raising standards.

It is recommended that you fully familiarise yourself with the dataset, timelines and variables available prior to completing your application by studying the user guides available from the Department for Education website here or alternatively downloading them from our Department for Education PDF pack here. This also contains information on using and interpreting the attainment variables.

The National Pupil Database is large and can be complex to use, particularly when working with longitudinal data. Due to the size of the datasets it is recommended that each data file is reduced to the key variables and cases of interest prior to carrying out any analyses.

Data Definitions

  • Version 1 at Key Stage 1/3 is considered as ‘provisional data’ and is the data used to populate the Statistical First Release.
  • Version 2 at Key Stage 1/3 is considered as ‘final data’.
  • Unamended data at Key Stage 2/4/5 is considered as ‘raw data’.
  • Amended data at Key Stage 2/4/5 is data where the schools/LA’s have had an opportunity to inform the DfE of irregularities and errors found within the Unamended data.
  • Final data at Key Stage 2/4/5 may have had some fine tuning from the previous 2 processes.

Access

The Secretary of State has specific powers to allow the sharing of pupil data from the NPD, as described in he technical guidance section above.

To make an application, the Department for Education have a specialist Data Warehouse team that deal with enquiries and applications for the NPD data. To ensure that the sharing of data is proportionate and that the different users of the data can access the information they need, the Department has developed a number of standard extracts which they hope will fulfil many peoples’ needs and speed up access to the data. The extracts have been organised into four different tiers of access, each with their own governance arrangements as follows:

Tier 1: Individual pupil level data – identifiable and / or identifiable and highly sensitive
Individual pupil level extracts that include identifying and highly sensitive information about pupils and their characteristics including items described as ‘sensitive personal data’ within the Data Protection Act 1998.

Tier 2: Individual pupil level data – identifiable and sensitive
Individual pupil level extracts that include sensitive information about pupils and their characteristics including items described as ‘sensitive personal data’ within the Data Protection Act 1998 which have been recoded to become less sensitive.

Tier 3: Aggregate School level data – identifiable and sensitive
Aggregated extracts of school level data from the Department’s school level database which could include items described as ‘sensitive personal data’ within the Data Protection Act 1998 and could include small numbers and single counts.

Tier 4: Individual pupil level data – identifiable
Individual pupil level extracts that do not contain information about pupils and their characteristics which is considered to be identifying or described as sensitive personal data within the Data Protection Act 1998.

Bespoke extracts
Researchers can still request bespoke extracts of the data if needed and will likely be required to submit a business application with the main application.

To help decide which extract is needed, researchers should consider whether they require a dataset containing pupil level or aggregated data, whether sensitive data is required and whether any identifying and highly sensitive data items are needed.

Linked data
The National Pupil Database can also be linked to individual records from the further and higher education sectors, using data from the individualised learner record (ILR) and Higher Education Statistics Agency (HESA) student record.  If you are new to either of these datasets then you should contact them about any linkage you propose to do with their data first.  HESA can be contacted at 01242 211133 and there is more information available from here. The Data Service (ILR) can be contacted at 0870 2670001.

Note that there maybe a charge for linkage to HESA data and this should be discussed with HESA at the time of application.  A full list of the data that is available together with example extracts from the NPD, HESA Student and ILR datasets are available from here.

Researchers should fully familiarise themselves with the NPD user guide prior to completing an application.  Depending on the data required researchers will need to complete either the NPD data request application form or Linked data request application form together with the information security questionnaire.  Further details are available from here.  Researchers should provide as much information in the forms as possible, including the nature of the research and details of all colleagues involved in the analysis of the data. This will help the approvals panel to deal with your request in an efficient manner.

Researchers are required to demonstrate that they will comply with all relevant requirements of the Data Protection Act 1998. In particular that they:

  • Have appropriate security arrangements in place to process the data;
  • Intend to use the data only for the specified purpose;
  • Will keep the data only for the specified length of time; and
  • Will not further disclose the data.

Once completed the application should be sent to NPD.REQUESTS@education.gsi.gov.uk.

When you make a request for NPD data it will be considered for approval by the Education Data Division (EDD) with the exception of tier 1 data requests, which will be assessed by the department’s Data Management Advisory Panel. The EDD will inform you of the outcome of the decision.

Details of previous research applications together with whether they have been approved by the DfE is available from here.

The application process is free and is expected to be completed within six to eight weeks.  Successful applications for NPD data will either be sent as an SPSS, CSV or text file. If required in another format this can be advised in the application. Successful applications will require a data sharing agreement and declaration form to be signed. Examples of these forms are available from here.

Data files will be sent electronically as a password protected Zip file so you will need to have a program to unzip files, for example WinZip.

Fast track applications

The DfE can also fast track applications if any of the following criteria are met:

  • The new request for data is exactly in line with a previously approved decision and all required approvals and evidence have been provided.
  • The request is an update request for annual data where permission to use the data for longer than one year has been granted and is still active and all required approvals and evidence have been provided.
  • The request is made by an accredited requester who has already demonstrated their credentials.

Additional information
The NPD is released in various different versions throughout the year. For example, the majority of the Key Stage attainment datasets have un-amended, amended and final versions. It is always best to use the latest version of the dataset where possible. There is usually little difference between the amended and final versions and so should have little effect on research outcomes, but it is best to check with the Data Warehouse team first.  By following the ADLS on Twitter you can keep upto date with these releases.

The ADLS produce guidance to help researchers apply for and use administrative data responsibly, including legal and data security guidance.  Further information is available from here.

Data quality

The data recorded in the NPD has a high level of completeness and accuracy. Department for Education derived variables are included in the dataset to allow researchers to reproduce the methodology used to calculate published results on educational attainment.

However, there may be instances where pupil data is missing. For example, if a child did not attend school on the day of an assessment or when a child has previously been educated outside of the state system or moved to England from abroad.

Although the NPD is updated annually it is released in several versions as the individual datasets contained within it are updated at different times. This is because attainment results may be challenged by schools or individuals, and errors may subsequently be found in the data.

When a new version of the NPD becomes available for the first time it is regarded as ‘provisional’ and means it could be revised at a later date. The number of revisions that do occur is generally very small and in most cases would have a negligible impact on any research findings, although it is best to contact the Data Warehouse team for clarification.

Considerations

Each pupil’s home postcode is collected in the Schools Census/PLASC so it is possible to carry out analyses at small area level. However, postcode data is classified as a restricted access variable and you will have to provide a strong business case to access this information.

The complex nature of the NPD database makes it easy to duplicate requests. Double check the data that you require.

Read the NPD data user tables carefully to determine which data is sensitive and what you actually require access to.  Also ensure that you really need the data that you are applying for, as a single file of PLASC/Census data typically contains over seven million records.

If you are applying for sensitive data you must fully justify your reasons for it, otherwise your application is unlikely to be successful.

Linkage

The NPD is already a linked dataset. It consists of a number of different datasets including the Schools Census/PLASC datasets and the Key Stage Attainment datasets. These are linked using a unique identifier for each pupil.

Researchers can apply to link the NPD to either the HESA student record of the Individualised Learner Record (see access section above).

The NPD datasets can also be linked longitudinally to trace pupils over time and can also be linked with other surveys of young people. For example, data linkage work has already been undertaken, or is planned to link the NPD with the Longitudinal Study of Young People in England, the Millennium Cohort Study, and the Avon Longitudinal Study of Parents & Children.

For more information on this see Hansen, K. and Vignoles, A. (2007), The use of large scale data-sets in educational research. London: TLRP or visit the British Educational Research Association which has a section on the use of large scale datasets in educational research.  Further linkages to NPD data can also be seen within the Effective Provision of Pre-School Education (EPPE) project.

Research users / comments

Send us your views and comments
If you would like to send us any comments about your experiences of accessing or using this dataset please click the button below.

Comments posted
‘The NPD has been described as wide-ranging but at various points shallow. It lacks a number of variables common in school effectiveness research and parental occupational class, despite their potential relevance to educational improvement. That said, there is not even a near competitor, the NPD (to me) virtually cries out to be used in mixed quant/qual research, and with hardly any exception (what are now) DfE colleagues have been excellent to work with. ’

Posted by Greater London Authority (19/05/11)

Please contact us with any comments regarding this dataset.

Content image