Administrative data is the term used to describe everyday data about individuals collected by government departments and agencies. Examples include exam results, benefit receipt and National Insurance payments.
Attrition is the discontinued participation of study participants in a longitudinal study. Attrition can reflect a range of factors, from the study participant not being traceable to them choosing not to take part when contacted. Attrition is problematic both because it can lead to bias in the study findings (if the attrition is higher among some groups than others) and because it reduces the size of the sample.
Body mass index is a measure used to assess if an individual is a healthy weight for their height. It is calculated by dividing the individual’s weight by the square of their height, and it is typically represented in units of kg/m2.
Cohort studies are concerned with charting the lives of groups of individuals who experience the same life events within a given time period. The best known examples are birth cohort studies, which follow a group of people born in a particular period.
Complete case analysis is the term used to describe a statistical analysis that only includes participants for which we have no missing data on the variables of interest. Participants with any missing data are excluded.
Conditioning refers to the process whereby participants’ answers to some questions may be influenced by their participation in the study – in other words, their responses are ‘conditioned’ by their being members of a longitudinal study. Examples would include study respondents answering questions differently or even behaving differently as a result of their participation in the study.
Confounding occurs where the relationship between independent and dependent variables is distorted by one or more additional, and sometimes unmeasured, variables. A confounding variable must be associated with both the independent and dependent variables but must not be an intermediate step in the relationship between the two (i.e. not on the causal pathway).
For example, we know that physical exercise (an independent variable) can reduce a person’s risk of cardiovascular disease (a dependent variable). We can say that age is a confounder of that relationship as it is associated with, but not caused by, physical activity and is also associated with coronary health. See also ‘unobserved heterogeneity’, below.
Cross-sectional surveys involve interviewing a fresh sample of people each time they are carried out. Some cross-sectional studies are repeated regularly and can include a large number of repeat questions (questions asked on each survey round).
Data harmonisation involves retrospectively adjusting data collected by different surveys to make it possible to compare the data that was collected. This enables researchers to make comparisons both within and across studies. Repeating the same longitudinal analysis across a number of studies allows researchers to test whether results are consistent across studies, or differ in response to changing social conditions.
Data imputation is a technique for replacing missing data with an alternative estimate. There are a number of different approaches, including mean substitution and model-based multivariate approaches.
Data linkage simply means connecting two or more sources of administrative, educational, geographic, health or survey data relating to the same individual for research and statistical purposes. For example, linking housing or income data to exam results data could be used to investigate the impact of socioeconomic factors on educational outcomes.
Dummy variables, also called indicator variables, are sets of dichotomous (two-category) variables we create to enable subgroup comparisons when we are analysing a categorical variable with three or more categories.
General ability is a term used to describe cognitive ability, and is sometimes used as a proxy for intelligent quotient (IQ) scores.
Heterogeneity is a term that refers to differences, most commonly differences in characteristics between study participants or samples. It is the opposite of homogeneity, which is the term used when participants share the same characteristics. Where there are differences between study designs, this is sometimes referred to as methodological heterogeneity. Both participant or methodological differences can cause divergences between the findings of individual studies and if these are greater than chance alone, we call this statistical heterogeneity. See also: unobserved heterogeneity.
Household panel surveys collect information about the whole household at each wave of data collection, to allow individuals to be viewed in the context of their overall household. To remain representative of the population of households as a whole, studies will typically have rules governing how new entrants to the household are added to the study.
Kurtosis is sometimes described as a measure of ‘tailedness’. It is a characteristic of the distribution of observations on a variable and denotes the heaviness of the distribution’s tails. To put it another way, it is a measure of how thin or fat the lower and upper ends of a distribution are.
Longitudinal studies gather data about the same individuals (‘study participants’) repeatedly over a period of time, in some cases from birth until old age. Many longitudinal studies focus upon individuals, but some look at whole households or organisations.
Non-response bias is a type of bias introduced when those who participate in a study differ to those who do not in a way that is not random (for example, if attrition rates are particularly high among certain sub-groups). Non-random attrition over time can mean that the sample no longer remains representative of the original population being studied. Two approaches are typically adopted to deal with this type of missing data: weighting survey responses to re-balance the sample, and imputing values for the missing information.
Observational studies focus on observing the characteristics of a particular sample without attempting to influence any aspects of the participants’ lives. They can be contrasted with experimental studies, which apply a specific ‘treatment’ to some participants in order to understand its effect.
Panel studies follow the same individuals over time. They vary considerably in scope and scale. Examples include online opinion panels and short-term studies whereby people are followed up once or twice after an initial interview.
A percentile is a measure that allows us to explore the distribution of data on a variable. It denotes the percentage of individuals or observations that fall below a specified value on a variable. The value that splits the number of observations evenly, i.e. 50% of the observations on a variable fall below this value and 50% above, is called the 50th percentile or more commonly, the median.
In prospective studies, individuals are followed over time and data about them is collected as their characteristics or circumstances change.
Recall error or bias describes the errors that can occur when study participants are asked to recall events or experiences from the past. It can take a number of forms – participants might completely forget something happened, or misremember aspects of it, such as when it happened, how long it lasted, or other details. Certain questions are more susceptible to recall bias than others. For example, it is usually easy for a person to accurately recall the date they got married, but it is much harder to accurately recall how much they earned in a particular job, or how their mood at a particular time.
Record linkage studies involve linking together administrative records (for example, benefit receipts or census records) for the same individuals over time.
A reference group is a category on a categorical variable to which we compare other values. It is a term that is commonly used in the context of regression analyses in which categorical variables are being modelled.
Residuals are the difference between your observed values (the constant and predictors in the model) and expected values (the error), i.e. the distance of the actual value from the estimated value on the regression line.
Respondent burden is a catch all phrase that describes the perceived burden faced by participants as a result of their being involved in a study. It could include time spent taking part in the interview and inconvenience this may cause, as well as any difficulties faced as a result of the content of the interview.
In retrospective studies, individuals are sampled and information is collected about their past. This might be through interviews in which participants are asked to recall important events, or by identifying relevant administrative data to fill in information on past events and circumstances.
Sample is a subset of a population that is used to represent the population as a whole. This reflects the fact that it is often not practical or necessary to survey every member of a particular population. In the case of birth cohort studies, the larger ‘population’ from which the sample is drawn comprises those born in a particular period. In the case of a household panel study like Understanding Society, the larger population from which the sample was drawn comprised all residential addresses in the UK.
A sampling frame is a list of the target population from which potential study participants can be selected.
Skewness is the measure of how assymetrical the distribution of observations are on a variable. If the distribution has a more pronounced/longer tail at the upper end of the distribution (right-hand side), we say that the distribution is negatively skewed. If it is more pronounced/longer at the lower end (left-hand side), we say that it is positively skewed.
Study participants are the individuals who are interviewed as part of a longitudinal study.
Survey weights can be used to adjust a survey sample so it is representative of the survey population as a whole. They may be used to reduce the impact of attrition on the sample, or to correct for certain groups being over-sampled.
The term used to refer to a round of data collection in a particular longitudinal study (for example, the age 7 sweep of the National Child Development Study refers to the data collection that took place in 1965 when the participants were aged 7). Note that the term wave often has the same meaning.
The population of people that the study team wants to research, and from which a sample will be drawn.
Tracing (or tracking) describes the process by which study teams attempt to locate participants who have moved from the address at which they were last interviewed.
Unobserved heterogeneity is a term that describes the existence of unmeasured (unobserved) differences between study participants or samples that are associated with the (observed) variables of interest. The existence of unobserved variables means that statistical findings based on the observed data may be incorrect.
Variables is the term that tends to be used to describe data items within a dataset. So, for example, a questionnaire might collect information about a participant’s job (its title, whether it involves any supervision, the type of organisation they work for and so on). This information would then be coded using a code-frame and the results made available in the dataset in the form of a variable about occupation. In data analysis variables can be described as ‘dependent’ and ‘independent’, with the dependent variable being a particular outcome of interest (for example, high attainment at school) and the independent variables being the variables that might have a bearing on this outcome (for example, parental education, gender and so on).
The term used to refer to a round of data collection in a particular longitudinal study (for example, the age 7 wave of the National Child Development Study refers to the data collection that took place in 1965 when the participants were aged 7). Note that the term sweep often has the same meaning.
Each sweep of data collection for a longitudinal study covers a range of topics. This section looks at the different ways in which data is collected, and the tools used to capture information about different aspects of life.
This section looks at the instruments and tools used to capture information about different aspects of participants’ lives.
Study teams choose the best possible survey instrument to collect the information they are looking for. Because longitudinal studies collect a broad range of information at each sweep, they require more than one type of survey instrument each time.
Most data collection instruments are completed by the study participant, but as we learned in the introduction to longitudinal studies module, information is sometimes collected from important people in participants’ lives as well.
For instance, if the study participants are too young to answer questions themselves, the information will be collected from their parents. As the participants get older, the information they provide may be supplemented by information from their parents, siblings, teachers, school nurses, health visitors, partners or children.
Otis Dudley Duncan is credited with saying “if you want to measure change, don’t change the measure”. This is a particular challenge for longitudinal study teams, which often need to balance the appeal of maintaining consistent measures over time with the need to ensure they are collecting the highest quality and most relevant information possible.
For example, imagine a study team is particularly interested in looking at how blood pressure develops over someone’s lifetime and how it relates to different circumstances. In the early sweeps of data collection, they use the best technology available at the time, but at later sweeps they have the choice to move to a more advanced and accurate blood pressure reader that is now typically used on other studies.
Changing to the latter would improve their data quality and help make it comparable with other studies, but could involve new costs (if they have to purchase the new devices) and will make it harder to compare their new data with those collected earlier using the older devices.
A key way of minimising the impact of any change will be to identify or carry out calibration studies that compare measurements using the different machines.
Most longitudinal studies collect a considerable amount of information from participants through survey questionnaires. They might be administered by an interviewer (either face-to-face or over the phone), or completed by the participants on their own (either online or using a paper questionnaire). The topics covered in the questionnaire will vary considerably depending on the study and the participants’ stage of life.
A range of question formats can be used in a questionnaire, from simple yes/no questions to complex dietary diaries or calendars to capture the timing of different events.
Many questions in longitudinal study questionnaires are ‘closed’, which means that they ask participants to choose from a pre-set list of answer options.
Some questions are ‘open’ to allow participants to put their answer in their own words, either by writing it down themselves (if the questionnaire is being completed by the participant) or by the interviewer transcribing or recording them.
Cognitive assessments are exercises that test how participants think and how their minds are developing (or declining). They typically take the form of a series of questions or set of activities (a bit like games).
Study teams tend to use standardised assessments that have been thoroughly tested to ensure that they are capturing the right information. Using the same instruments over time or across studies can make comparisons easier.
For example, participants in the 1970 British Cohort Study took nine cognitive assessments at age 16, including a vocabulary test. The participants were given the same vocabulary test again at age 42. This allowed researchers to see whose vocabularies improved over time, and to determine what factors affected this.
One of the things they found was that readers of quality newspapers made more progress in vocabulary over time than people who did not read newspapers. Interestingly, tabloid readers actually made slightly less progress than those who never read newspapers at all.
Vocabulary is an example of an aspect of cognitive ability that needs to be measured with an age-appropriate instrument. For example, while 16- and 42-year-olds might be able to complete a written assessment, this might be a lot harder for a 5-year-old.
Many studies use the British Ability Scales (BAS) to measure cognition. The BAS are a set of age-appropriate standard tests of cognitive abilities and educational achievements. For example, the BAS Naming Vocabulary test assesses young children’s spoken vocabulary by showing them a series of coloured pictures of objects and asking them to name each one.
Some studies ask participants to keep a diary to record more detail about how they spend their time or what food they have eaten.
For example, the 1970 British Cohort Study and Millennium Cohort Study both included a time diary for participants (at age 16 and age 14 respectively). The diaries covered cohort members’ exercise, homework and social participation.
The Southampton Women’s Survey asked participants to keep a 24 hour food diary during pregnancy (at 11 weeks gestation) and a two day food diary when their child was three years old.
Many studies collect information about participants’ physical characteristics. Common examples include height, weight and blood pressure.
For example, the 1946 MRC National Survey of Health and Development has carried out the following assessments of participants since they were in the 30s:
Some health assessments are carried out by trained interviewers or nurses in the participant’s own home, while others are carried out in clinics.
Studies are interested increasingly in participants’ mental health and wellbeing. As with cognitive assessments, assessments of mental health and wellbeing typically take the form of a set of questions that have been developed and tested thoroughly.
The Malaise Inventory is a common assessment used to measure levels of psychological distress, or depression. It includes questions like these:
The participants’ answers are normally pooled together to create an aggregate score for psychological distress, however the questions can also be used separately.
Other common mental health assessments include the Strengths and Difficulties Questionnaire, which is used to identify mental health problems and measure common forms of psychopathology in children and young adults.
The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) is often used to assess wellbeing. It comprises 14 statements. Participants are asked to indicate how often each statement reflects how they have felt in the past two weeks, on a 5-item scale from ‘none of the time’ to ‘all of the time’.
Several studies collect biological samples, such as blood or saliva. Genetic information can be extracted from such samples, and can provide objective measures of people’s health (known as biological markers or biomarkers).
For example, the Avon Longitudinal Study of Parents and Children has collected biological samples from participants since it began. In addition to blood, hair and saliva, it has collected mother’s blood and placenta, as well as nail clippings and milk teeth from their children. It has genotype data about 1,500 children and their parents, which opens new routes of investigation into how genetic risk is passed from parent to child.
Some studies also collect qualitative information from participants, either through in-depth interviews or open-ended questions.
Qualitative instruments are often used when the information is difficult to collect accurately using fixed response options. For example, our understanding of participants’ social identities, their plans for the future, or their feelings about their neighbourhoods are all better understood through the participants’ own words.
As you can imagine, answering open-ended questions such as this takes considerably more time and effort for the participants. As such, these questions often have lower response rates than closed questions.
The 1958 National Child Development Study includes three substantial sets of qualitative information, collected at ages 11 and 50.
At the Age 11 Sweep in 1969, participants were asked to write an essay on the following topic: ‘Imagine you are now 25 years old. Write about the life you are leading, your interests, your home life and your work at the age of 25. (You have 30 minutes to do this).’
Just over 13,600 participants completed the essay. The average length of their responses was 204 words. The children wrote about a range of different themes, including family, friends, domestic labour, cars, money, jobs, working hours, relationships and football.
Further information about this study can be found at the UK Data Service.
At the Age 50 Sweep in 2008, participants were sent a paper questionnaire to complete themselves in advance of the face-to-face interview. The last question asked them to “Imagine that you are now 60 years old…please write a few lines about the life you are leading (your interests, your home life, your health and well-being and any work you may be doing).”
Nearly 7,400 participants answered the question. The average length of their responses was 57 words, but some participants wrote as few as three or four words, and some wrote more than 150.The answers covered a range of different themes, including work, money, travel, family, health and death.
Further information about this study can be found at the UK Data Service.
Just after the main interviews at age 50, 230 participants took part in a qualitative sub-study. In-depth interviews were carried out to capture information on neighbourhood and belonging, social participation and leisure activities, friendships, life story and trajectories, identities, and the experience of taking part in the study over time. The interviews were about 90 minutes long, on average.
Further information about this study can be found at the UK Data Service.