Methodology

Identification of domains and cohorts

Through a review of national and international reports on wellbeing (see Appendix A in the full Australia’s Social Pulse report for details), CSI identified the following domains that recur across national and international indexes: health and disability, education, employment, living standards, housing and homelessness, social cohesion, and life satisfaction. While international reports often group health and disability as one domain, Australia’s Social Pulse makes a distinction between health and disability because while they may be sometimes related, having a disability does not necessarily mean someone has poor health. In addition, Australia’s Social Pulse makes a distinction between physical and mental health. The methodology section in the full Australia’s Social Pulse report presents details about the final eight domains used and their intersection with national and international reports on wellbeing.

In addition to the cohorts of gender and age groups focused on by other reports, Australia’s Social Pulse also reports on the following groups:

  • Youth not in education, employment or training (NEET)
  • People with disability
  • Socio-Economic Indexes for Areas (SEIFA); specifically, the SEIFA Index of Relative Socio-Economic Disadvantage
  • Indigenous Australians
  • People with psychological distress
  • Geographical region (remoteness)

Including these cohorts allows us to unpack potential differences in how these various groups experience outcomes and ultimately make social progress. This provides a more holistic understanding of how the nation is faring, and enables far greater sophisticated analysis to answer questions around ‘why’ differences have occurred, and ‘how’ policies might respond.

Selecting and assessing indicators  

CSI’s systematic approach (Bennett et al., 2016) was used to select and assess the indicators included in the report. The process includes: establishing the purpose and principles to guide the process; a background review of existing indicators in the literature and practice; assessment of indicators against predefined selection criteria; and finalisation and selection of indicators. This approach involved assessing the indicators against ten selection criteria that examined subjective and technical aspects of the indicators, to determine whether they were: important, specific, validated, reliable, accessible, acceptable, appropriate, useable, and feasible.

Indicators were selected from the aforementioned national and international reports, and two researchers then independently assessed each indicator against the selection criteria above. Overall, there was a high level of agreement between the researchers; any differences in opinions were discussed and a consensus reached. To identify a shortlist of indicators we selected a subset of all identified indicators that met the criteria of being:  1) feasible, 2) important, and 3) usable. Detailed methodology about the indicator selection approach is described in Bennett et al. (2016).

Data and data analysis

The report predominantly uses national data from the Household, Income and Labour Dynamics in Australia Survey (HILDA)[1] and the Census of Population and Housing (census) as these datasets enable analysis of a broad range of indicators over time and disaggregated by population groups of interest. The HILDA survey is an ongoing national longitudinal survey of household labour market and dynamics that is representative of people in Australia who live in private dwellings, excluding very remote parts of Australia. See the methodology section in the full Australia’s Social Pulse report for more details about the HILDA survey.

Data from the 2001, 2006, and 2011 Census of Population and Housing surveys were also used in this report. Where possible, Time Series Profile data was used to examine aggregate-level changes over 2001, 2006, and 2011. Where Time Series Profile data was not available, data from TableBuilder Pro was used to examine changes between 2001 and 2006.

Supplementary data from the Australian Bureau of Statistics (ABS), and other national surveys were also used in some domains: ABS Labour Force Survey, National Drug Strategy Household Survey, ABS Household Income and Wealth and Specialist Homeless Services Collection demographics data cube.

HILDA Data analysis 

This report uses unbalanced panel data from HILDA waves 1, 6, 11 and 13 (2001, 2006, 2011 and 2013). Waves 1, 6 and 11 were selected to be consistent with Census collection years. At the time of data analysis, wave 13 was the most recent wave of HILDA data available and therefore provided the most recent insight into how Australia was faring. Data analysis was conducted using the combined HILDA data file, which consists of enumerated person-, household-, and responding person-level data. Enumerated person-level data includes responding- and non-responding adults and children less than 15 years.

Cross sectional weights were applied to the data to ensure that the results are representative of the relevant in-scope population for each wave of data analysed. Where descriptive comparisons were undertaken to examine changes in individual level indicators over time, variances were clustered (using the variable ‘xwaveid’) to correct for repeated measures on the same individual over time. There is no unique household ID across waves for household level analyses over time; however, these data were replicate weighted to control for survey design (i.e. area level clustering and stratification).[2]

Cross sectional regression analyses were also undertaken, using wave 13 data (year 2013), to investigate whether outcomes varied by particular individual or household demographic variables. Individual level regression analyses were clustered by household ID to correct for repeated measures for individuals within the same household. Logistic regressions were carried out for indicators with categorical outcomes, and indicators with more than two categories were recoded into dichotomous variables for ease of interpretation. Linear regressions were carried out for indicators with outcomes measured on a continuous scale (non-categorical). The table below displays the demographic variables and the categories within each variable that were included in the regression models. The results for each individual variable control for all the other variables listed (e.g., findings for gender control for differences in age, disability status, Indigenous status, K10 category, SEIFA quintile, and geographic region). Results for 0-14 year olds were based on enumerated person data and therefore do not control for K10 score, which is only available for individuals 15 years and over. Regression analysis results are graphed using colour coding: the base case or category is represented in blue, categories for which outcomes are statistically significantly ‘better’ than the base are graphed in green, significantly ‘worse’ off categories in red and categories for which no statistically significant differences in outcomes were found are graphed in yellow.

Demographics Table



[1] This paper uses unit record data from the Household, Income and Labour Dynamics in Australia (HILDA) Survey. The HILDA Project was initiated and is funded by the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) and is managed by the Melbourne Institute of Applied Economic and Social Research (Melbourne Institute). The findings and views reported in this paper, however, are those of the author and should not be attributed to either FaHCSIA or the Melbourne Institute.

[2] Note that in Stata the vce option for clustering and svy commands (used to apply replicate weights) cannot be combined. Greater correlation was expected between individual’s answers over time than between people from the same area. Therefore, for individual level analyses of change over time, variances were clustered but the data were not replicate weighted.