4. Measuring wellbeing Working in R

Download the code

To download the code chunks used in this project, right-click on the download link and select ‘Save Link As…’. You’ll need to save the code download to your working directory, and open it in RStudio.

Don’t forget to also download the data into your working directory by following the steps in this project.

R-specific learning objectives

In addition to the learning objectives for this project, in this section you will learn how to convert (‘reshape’) data from wide to long format and vice versa.

Getting started in R

For this project you will need the following packages:

• tidyverse, to help with data manipulation
• readxl, to import an Excel spreadsheet
• reshape2, to manipulate datasets.

You will also use the ggplot2 package to produce accurate graphs, but that comes as part of the tidyverse package.

If you need to install these packages, run the following code:

install.packages(c("readxl", "tidyverse", "reshape2"))

You can import these libraries now, or when they are used in the R walk-throughs below.

library(readxl)
library(tidyverse)
library(reshape2)

Part 4.1 GDP and its components as a measure of material wellbeing

Learning objectives for this part

• check datasets for missing data
• sort data and assign ranks based on values
• distinguish between time series and cross sectional data, and plot appropriate charts for each type of data.

The GDP data we will look at is from the United Nations’ National Accounts Main Aggregates Database, which contains estimates of total GDP and its components for all countries over the period 1970 to present. We will look at how GDP and its components have changed over time, and investigate the usefulness of GDP per capita as a measure of wellbeing.

To answer the questions below, download the data and make sure you understand how the measure of total GDP is constructed.

• Go to the United Nations’ National Accounts Main Aggregates Database website. On the right-hand side of the page, under ‘Data Availability’, click ‘Downloads’.
• Under the subheading ‘GDP and its breakdown at constant 2010 prices in US Dollars’, select the Excel file ‘All countries for all years – sorted alphabetically’.
• Save it in an easily accessible location, such as a folder on your Desktop or in your personal folder.

R walk-through 4.1 Importing the Excel file (.xlsx or .xls format) into R

First, use setwd to tell R where the datafile is stored. To avoid having to repeatedly use setwd to tell R where your files are, keep all the files you need in that folder, including the Excel sheet you just downloaded. Replace ‘YOURFILEPATH’ with the full filepath which points to the folder with your datafile. If you don’t know how to find the path to that folder, see the ‘Technical Reference’ section.

setwd("YOURFILEPATH")

Then use the function readxl (part of the tidyverse suite of packages) to import the datafile. Before importing the file into R, open the file in Excel to see how the data is organized in the spreadsheet, and note that:

• There is a heading that we don’t need, followed by a blank row.
• The data we need starts on row three.

# Load the library
library(tidyverse)
library(readxl)

# Excel filename
UN = read_excel("Download-GDPconstant-USD-countries.xls",
  # Sheet name
  sheet = "Download-GDPconstant-USD-countr", 
  # Number of rows to skip
  skip = 2)

head(UN)

## # A tibble: 6 x 50
##   CountryID Country  IndicatorName      `1970` `1971` `1972` `1973` `1974`
##       <dbl> <chr>    <chr>               <dbl>  <dbl>  <dbl>  <dbl>  <dbl>
## 1         4 Afghani~ Final consumption~ 5.56e9 5.33e9 5.20e9 5.75e9 6.15e9
## 2         4 Afghani~ Household consump~ 5.07e9 4.84e9 4.70e9 5.21e9 5.59e9
## 3         4 Afghani~ General governmen~ 3.72e8 3.82e8 4.02e8 4.21e8 4.31e8
## 4         4 Afghani~ Gross capital for~ 9.85e8 1.05e9 9.19e8 9.19e8 1.18e9
## 5         4 Afghani~ Gross fixed capit~ 9.85e8 1.05e9 9.19e8 9.19e8 1.18e9
## 6         4 Afghani~ Exports of goods ~ 1.12e8 1.45e8 1.73e8 2.18e8 3.00e8
## # ... with 42 more variables: `1975` <dbl>, `1976` <dbl>, `1977` <dbl>,
## #   `1978` <dbl>, `1979` <dbl>, `1980` <dbl>, `1981` <dbl>, `1982` <dbl>,
## #   `1983` <dbl>, `1984` <dbl>, `1985` <dbl>, `1986` <dbl>, `1987` <dbl>,
## #   `1988` <dbl>, `1989` <dbl>, `1990` <dbl>, `1991` <dbl>, `1992` <dbl>,
## #   `1993` <dbl>, `1994` <dbl>, `1995` <dbl>, `1996` <dbl>, `1997` <dbl>,
## #   `1998` <dbl>, `1999` <dbl>, `2000` <dbl>, `2001` <dbl>, `2002` <dbl>,
## #   `2003` <dbl>, `2004` <dbl>, `2005` <dbl>, `2006` <dbl>, `2007` <dbl>,
## #   `2008` <dbl>, `2009` <dbl>, `2010` <dbl>, `2011` <dbl>, `2012` <dbl>,
## #   `2013` <dbl>, `2014` <dbl>, `2015` <dbl>, `2016` <dbl>

1. You can see from the tab ‘Download-GDPconstant-USD-countr’ that some countries have missing data for some of the years. Data may be missing due to political reasons (for example, countries formed after 1970) or data availability issues.

• Make and fill a frequency table similar to Figure 4.1, showing the number of years that data is available for each country in the category ‘Final consumption expenditure’.

• How many countries have data for the entire period (1970 to the latest year available)? Do you think that missing data is a serious issue in this case?

R walk-through 4.2 Making a frequency table

We want to create a table showing how many years of Final consumption expenditure data are available for each country.

Looking at the dataset’s current format, you can see that countries and indicators (for example, Afghanistan and Final consumption expenditure) are row variables, while year is the column variable. This data is organized in ‘wide’ format (each individual’s information is in a single row).

For many data operations and making charts it is more convenient to have indicators as column variables, so we would like Final consumption expenditure to be a column variable, and year to be the row variable. Each observation would represent the value of an indicator for a particular country and year. This data is organized in ‘long’ format (each individual’s information is in multiple rows).

To change data from wide to long format, we use the melt command from the package reshape2. The melt command is very powerful and useful, as you will find many large datasets are in wide format. In this case, it takes the data in Column 4 to the last column (these columns indicate the years) and uses them to create two new columns: one column (variable) contains the name of the row variable (the year) and the other column (value) contains the associated value. Compare long_UN to wide_UN to understand how the melt command works. To learn more about organizing data in R, see the R for Data Science website.

library(reshape2)

wide_UN <- UN
# Keep all data except for column 1 (CountryID)
wide_UN = wide_UN[, -1]

# id.vars are the names of the column variables.
long_UN = 
  melt(wide_UN, id.vars = c("Country", "IndicatorName"),
  value.vars = 4:ncol(UN))

head(long_UN)

##       Country
## 1 Afghanistan
## 2 Afghanistan
## 3 Afghanistan
## 4 Afghanistan
## 5 Afghanistan
## 6 Afghanistan
##   IndicatorName
## 1 Final consumption expenditure
## 2 Household consumption expenditure (including Non-profit institutions serving households)
## 3 General government final consumption expenditure
## 4 Gross capital formation
## 5 Gross fixed capital formation (including Acquisitions less disposals of valuables)
## 6 Exports of goods and services
##   variable      value
## 1     1970 5559066266
## 2     1970 5065088737
## 3     1970  372478456
## 4     1970  984580895
## 5     1970  984580895
## 6     1970  112390156

Our new ‘long’ format dataset is called long_UN. During the reshaping process, a new variable called variable was created which contains years. We will use the names function to rename it as Year.

names(long_UN)[names(long_UN) == "variable"] <- "Year"

To create the required table, we only need Final consumption expenditure of each country, which we extract using the subset function.

cons = subset(long_UN,
  IndicatorName == "Final consumption expenditure")

Now we create the table showing the number of missing years by country, using the piping operator (%>%) from the tidyverse package. This operator allows us to perform multiple functions, one after another.

# Use the pipe operator (%>%) from the tidyverse package.
# This means: use the result of the current line
# as the first argument in the next line's function.

missing_by_country = cons %>%
  group_by(Country) %>%
  summarize(available_years=sum(!is.na(value))) %>%
  print()

## # A tibble: 220 x 2
##    Country             available_years
##    <chr>                         <int>
##  1 Afghanistan                      47
##  2 Albania                          47
##  3 Algeria                          47
##  4 Andorra                          47
##  5 Angola                           47
##  6 Anguilla                         47
##  7 Antigua and Barbuda              47
##  8 Argentina                        47
##  9 Armenia                          27
## 10 Aruba                            47
## # ... with 210 more rows

Translating the code in words: Take the variable cons (cons %>%) and group the observations by country (group_by(Country)), then take this result (%>%) and produce a table (summarize(...)) that shows the variable available_years (which is the sum (sum(...)) of the variable !is.na(value)).

To understand what !is.na(value) means, recall that value contains the numerical values for the variable of interest. When an observation is missing, it is recorded as NA. The function is.na(value) will return a value of 1 (or TRUE) if the value is missing and 0 (or FALSE) otherwise. We add a ! in front since we want the function to return a 1 if the observation exists and a 0 otherwise. For R, ! means ‘not’ so we get a 1 if the particular observation is not missing.

Now we can establish how many of the 220 countries in the dataset have complete information. A dataset is complete if it has the maximum number of available observations (max(missing_by_country$available_years)).

sum(missing_by_country$available_years == max(
  missing_by_country$available_years))

## [1] 179

If you add up the data on the right-hand side of this equation, you may find that it does not add up to the reported GDP value. The UN notes this discrepancy in Section J, item 17 of the ‘Methodology for the national accounts’: ‘The sums of com­ponents in the tables may not necessarily add up to totals shown because of rounding’.

There are three different ways in which countries calculate GDP for their national accounts, but we will focus on the expenditure approach, which calculates gross domestic product (GDP) as:

Gross capital formation refers to the creation of fixed assets in the economy (such as the construction of buildings, roads, and new machinery) and changes in inventories (stocks of goods held by firms).

1. Rather than looking at exports and imports separately, we usually look at the difference between them (exports minus imports), also known as net exports. Choose three countries that have GDP data over the entire period (1970 to the latest year available). For each country, create a variable that shows the values of net exports in each year.

R walk-through 4.3 Creating new variables

We will use Brazil, the US, and China as examples.

Before we select these three countries, we will calculate the net exports (exports minus imports) for all countries, as we need that information in R walk-through 4.4. We will also shorten the names of the variables we need, to make the code easier to read.

# Shorten the names of the variables we need
# When a string straddles two lines of code
# we need to wrap it into the 'strwrap' function

long_UN$IndicatorName[long_UN$IndicatorName == 
  strwrap("Household consumption expenditure (including 
    Non-profit institutions serving households)")] <- 
  "HH.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "General government final consumption expenditure"] <- 
  "Gov.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Final consumption expenditure"] <-
  "Final.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Gross capital formation"] <-
  "Capital"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Imports of goods and services"] <-
  "Imports"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Exports of goods and services"] <-
  "Exports"

long_UN still has several rows for a particular country and year (one for each indicator). We will reshape this data using the dcast function to ensure that we have only one row per country and per year. We then add a new column called Net.Exports containing the calculated net exports.

# We need to cast (reshape) the long_UN data to a dataframe
# We use the dcast function (used for dataframes)
table_UN <- dcast(long_UN, Country + Year ~ IndicatorName)

# Add a new column for net exports (= exports – imports)
table_UN$Net.Exports <- 
  table_UN[, "Exports"]-table_UN[, "Imports"]

Let us select our three chosen countries to check that we calculated net exports correctly.

sel_countries = c("Brazil", "United States", "China")

# Using our long format dataset, we get imports, exports, 
# and year for these countries.
sel_UN1 = subset(table_UN, 
  subset = (Country %in% sel_countries), 
  select = c("Country", "Year", "Exports",
    "Imports", "Net.Exports"))

head(sel_UN1)

##      Country Year     Exports     Imports  Net.Exports
## 1223  Brazil 1970 12337240060 20187929130  -7850689070
## 1224  Brazil 1971 13016975734 24162976191 -11146000457
## 1225  Brazil 1972 16162334455 29025977711 -12863643256
## 1226  Brazil 1973 18466228366 34950588132 -16484359766
## 1227  Brazil 1974 18897150611 44821362355 -25924211744
## 1228  Brazil 1975 21084064715 42838470795 -21754406080

Now we will create charts to show the GDP components in order to look for general patterns over time and make comparisons between countries.

1. Evaluate the components over time, for two countries of your choice.

• Create a new row for each of the four components of GDP (Household consumption expenditure, General government final consumption expenditure, Gross capital formation, Net exports). To make the charts easier to read, convert the values into billions (for example, 4.38 billion instead of 4,378,772,008). Round your values to two decimal places.

• Plot a separate line chart for each country, showing the value of the four components of GDP on the vertical axis and time (the years 1970–Present) on the horizontal. (Use more than one line chart per country if necessary, to show the data more clearly). Name each component in the chart legend appropriately.

• Which of the components would you expect to move together (increasing or decreasing together) or move in opposite directions, and why? Using your charts from Question 3(a), describe any patterns you find in the relationship between the components. Does the data support your hypothesis about the behaviour of the components?

• For each country, describe any patterns you find in the movement of components over time. What factors could explain the patterns that you find within countries, and any differences between countries (for example, economic or political events)? You may find it helpful to research the history of the countries you have chosen.

• Extension: For one country, add data labels to your chart to indicate the relevant events that happened in that year.

R walk-through 4.4 Plotting and annotating time series data

Extract the relevant data

We will work with the long_UN dataset, as the long format is well suited to produce charts with the ggplot package. In this example, we use the US and China (saved as the dataset comp).

# Select our chosen countries
comp = subset(long_UN, 
  Country %in% c("United States", "China"))

# value in billion of USD
comp$value = comp$value / 1e9

comp = subset(comp, 
  select = c("Country", "Year",
    "IndicatorName", "value"),
  subset = IndicatorName %in% c("Gov.Expenditure", 
  "HH.Expenditure", "Capital", "Imports", "Exports"))

Plot a line chart

We can now plot this data using the ggplot library.

library(ggplot2)

# ggplot allows us to build a chart step-by-step.
pl = ggplot(subset(comp, Country == "United States"),
  # Base chart, defining x (horizontal) and y (vertical) 
  # axis variables
  aes(x = Year, y = value))

# Specify a line chart, with a different colour for each 
# indicator name and line size = 1
pl = pl + geom_line(aes(group = IndicatorName, 
  color = IndicatorName), size = 1)

# Display the chart
pl

Figure 4.2 The US’s GDP components (expenditure approach).

There are plenty of problems with this chart:

• we cannot read the horizontal axis, because it labels every year
• the vertical axis label is uninformative
• there is no chart title
• the grey (default) background makes the chart difficult to read
• the legend is uninformative.

To improve this chart, we add features to the already existing figure pl.

pl = pl + scale_x_discrete(breaks=seq(1970, 2016, by = 10))
pl = pl + scale_y_continuous(name="Billion US$")
pl = pl + ggtitle("GDP components over time")

# Change the legend title and labels
pl = pl + scale_colour_discrete(name = "Components of GDP",
  labels = c("Gross capital formation", "Exports",
    "Government expenditure", "Household expenditure",
    "Imports")) 

pl = pl + theme_bw()

pl = pl + annotate("text", x = 37, y = 850,
  label = "Great Recession")

pl

Figure 4.3 The US’s GDP components (expenditure approach), amended chart.

We can make a chart for more than one country simultaneously by repeating the code above, but without subsetting the data:

# Repeat all steps without subsetting the data

# Base line chart
pl = ggplot(comp, aes(x = Year, y = value, 
  color = IndicatorName))
pl = pl + geom_line(aes(group = IndicatorName), size = 1)
pl = pl + scale_x_discrete(
  breaks = seq(1970, 2016, by = 10))
pl = pl + scale_y_continuous(name = "Billion US$")
pl = pl + ggtitle("GDP components over time")
pl = pl + scale_colour_discrete(name = "Component")   
pl = pl + theme_bw()

# Make a separate chart for each country
pl = pl + facet_wrap(~Country)
pl = pl + scale_colour_discrete(
  name = "Components of GDP",   
  labels = c("Gross capital formation",  
    "Exports",
    "Government expenditure",
    "Household expenditure",
    "Imports"))
pl

Figure 4.4 GDP components over time (China and the US).

1. Another way to visualize the GDP data is to look at each component as a proportion of total GDP. Use the same countries that you chose for Question 3.

• For each country used in Question 3, create a new column showing the proportion of each component of total GDP (in other words, as a value ranging from 0 to 1). (Hint: to calculate the proportion of a component, divide the value of that component by the sum of all four components.)

• Plot a separate line chart for each country, showing the proportion of the component of GDP on the vertical axis and time (the years 1970 to the latest year available) on the horizontal axis.

• Describe any patterns in the proportion of spending over time for each country, and compare these patterns across countries.

• Compared to the charts in Question 3, what are some advantages of this method for making comparisons over time and between countries?

R walk-through 4.5 Calculating new variables and plotting time series data

Calculate proportion of total GDP

We will use the comp dataset created in R walk-through 4.4. First we will calculate net exports, as that contributes to GDP. As the data is currently in long format, we will reshape the data into wide format so that the variables we need are in separate columns instead of separate rows (using the dcast function, as in R walk-through 4.3), calculate net exports, then transform the data back into long format using the melt function.

# Reshape the data to wide format (indicators in columns)
comp_wide <- dcast(comp, Country + Year ~ IndicatorName)

head(comp_wide)

##   Country Year  Capital   Exports Gov.Expenditure HH.Expenditure   Imports
## 1   China 1970 67.58221  5.305242        19.30034       107.8411  6.119649
## 2   China 1971 73.79977  6.318662        22.63929       112.3586  6.094361
## 3   China 1972 70.62638  7.740001        23.77126       118.3906  7.510857
## 4   China 1973 80.79658 10.810644        24.34177       126.7546 11.540975
## 5   China 1974 83.38207 12.856408        26.14306       129.3434 16.041397
## 6   China 1975 93.47130 13.309628        27.29335       134.5575 15.859461

# Add the new column for net exports = exports – imports
comp_wide$Net.Exports <- 
  comp_wide[, "Exports"] - comp_wide[, "Imports"]

head(comp_wide)

##   Country Year  Capital   Exports Gov.Expenditure HH.Expenditure   Imports
## 1   China 1970 67.58221  5.305242        19.30034       107.8411  6.119649
## 2   China 1971 73.79977  6.318662        22.63929       112.3586  6.094361
## 3   China 1972 70.62638  7.740001        23.77126       118.3906  7.510857
## 4   China 1973 80.79658 10.810644        24.34177       126.7546 11.540975
## 5   China 1974 83.38207 12.856408        26.14306       129.3434 16.041397
## 6   China 1975 93.47130 13.309628        27.29335       134.5575 15.859461
##   Net.Exports
## 1  -0.8144069
## 2   0.2243011
## 3   0.2291447
## 4  -0.7303314
## 5  -3.1849891
## 6  -2.5498330

# Return to long format with the HH.expenditure, Capital, and Net Export variables
comp2_wide <- 
  subset(comp_wide, select = -c(Exports, Imports))

comp2 <-
  melt(comp2_wide, id.vars = c("Year", "Country"))

Now we will create a new dataframe (props) also containing the proportions for each GDP component (proportion), using the piping operator to link functions together.

props = comp2 %>%
  group_by(Country, Year) %>%
  mutate(proportion = value / sum(value))

In words, we did the following: Take the comp2 dataframe and create groups by country and year (for example, all indicators for China in 1970). Then create a new variable (mutate) called proportion, which divides the variable value of an indicator by the sum of all value for that group (for example, all indicators for China in 1970). The result is then saved in props. Look at the props dataframe to confirm that the above command has achieved the desired result.

Plot a line chart

Now we redo the line chart from R walk-through 4.4 using the variable props.

# Base line chart
pl = ggplot(props, aes(x = Year, y = proportion, 
  color = variable))

pl = pl + geom_line(aes(group = variable), 
  size = 1)

pl = pl + scale_x_discrete(breaks = seq(1970, 2016,
  by = 10))

pl = pl + ggtitle("GDP component proportions over time")

pl = pl + theme_bw()

# Make a separate chart for each country
pl = pl + facet_wrap(~Country)

pl = pl + scale_colour_discrete(
  name = "Components of GDP",
  labels = c("Gross capital formation",
    "Government expenditure",
    "Household expenditure", 
    "Net Exports"))

pl

time series data

A time series is a set of time-ordered observations of a variable taken at successive, in most cases regular, periods or points of time. Example: The population of a particular country in the years 1990, 1991, 1992, … , 2015 is time series data.

cross-sectional data

Data that is collected from participants at one point in time or within a relatively short time frame. In contrast, time series data refers to data collected by following an individual (or firm, country, etc.) over a course of time. Example: Data on degree courses taken by all the students in a particular university in 2016 is considered cross-sectional data. In contrast, data on degree courses taken by all students in a particular university from 1990 to 2016 is considered time series data.

Figure 4.5 GDP component proportions over time (China and the US).

So far, we have done comparisons of time series data, which is a collection of values for the same variables and subjects, taken at different points in time (for example, GDP of a particular country, measured each year). We will now make some charts using cross-sectional data, which is a collection of values for the same variables for different subjects, usually taken at the same time.

1. Choose three developed countries, three countries in economic transition, and three developing countries (for a list of these countries, see Tables A–C in the UN country classification document).

• For each country, calculate each component as a proportion of GDP for the year 2015 only.

• Now create a stacked bar chart that shows the composition of GDP in 2015 on the horizontal axis, and country on the vertical axis. Arrange the columns so that the countries in a particular category are grouped together. (See the walk-through in Figure 3.8 of Economy, Society, and Public Policy for an example of what your chart should look like.)

• Describe the differences (if any) between the spending patterns of developed, economic transition, and developing countries.

R walk-through 4.6 Creating stacked bar charts

Calculate proportion of total GDP

This walk-through uses the following countries (chosen at random):

• developed countries: Germany, Japan, United States
• transition countries: Albania, Russian Federation, Ukraine
• developing countries: Brazil, China, India.

The relevant data are still in the table_UN dataframe. Before we select these countries, we first calculate the required proportions for all countries.

# Calculate proportions
table_UN$p_Capital <- 
  table_UN$Capital / 
  (table_UN$Capital +
    table_UN$Final.Expenditure +
    table_UN$Net.Exports)

table_UN$p_FinalExp <-
  table_UN$Final.Expenditure / 
  (table_UN$Capital +
    table_UN$Final.Expenditure +
    table_UN$Net.Exports)

table_UN$p_NetExports <-
  table_UN$Net.Exports /
  (table_UN$Capital +
    table_UN$Final.Expenditure + 
    table_UN$Net.Exports)

sel_countries <- 
  c("Germany", "Japan", "United States",
    "Albania", "Russian Federation", "Ukraine",
    "Brazil", "China", "India")

# Using our long format dataset, we select imports, 
# exports, and year for our chosen countries in 2015.

# Select the columns we need
sel_2015 <- 
  subset(table_UN, subset =
    (Country %in% sel_countries) & (Year == 2015),
    select = c("Country", "Year", "p_FinalExp",
      "p_Capital", "p_NetExports"))

Plot a stacked bar chart

Now let’s create the bar chart.

# Reshape the table into long format, then use ggplot
sel_2015_m <- melt(sel_2015, id.vars = 
  c("Year", "Country"))

g <- ggplot(sel_2015_m, 
  aes(x = Country, y = value, fill = variable)) + 
  geom_bar(stat = "identity") + coord_flip() +
  ggtitle("GDP component proportions in 2015") +
  scale_fill_discrete(name = "Components of GDP",
  labels = c("Final expenditure",
    "Gross capital formation",
    "Net Exports")) +
  theme_bw()

plot(g)

Figure 4.6 GDP component proportions in 2015.

Note that even when a country has a trade deficit (proportion of net exports < 0), the proportions will add up to 1, but the proportions of final expenditure and capital will add up to more than 1.

We have not yet ordered the countries so that they form the pre-specified groups. To achieve this, we need to explicitly impose an ordering on the Country variable using the factor function. The countries will be ordered in the same order we used to define sel_countries.

# Impose the order in the sel_countries object, then use ggplot
sel_2015_m$Country <- 
  factor(sel_2015_m$Country, levels = sel_countries)

g <- ggplot(sel_2015_m, 
  aes(x = Country, y = value, fill = variable)) +
  geom_bar(stat = "identity") + coord_flip() + 
  ggtitle("GDP component proportions in 2015 (ordered)") + 
  scale_fill_discrete(name = "Components of GDP", 
  labels = c("Final expenditure",
    "Gross capital formation",
    "Net Exports")) +
  theme_bw()

plot(g)

Figure 4.7 GDP component proportions in 2015 (ordered).

1. GDP per capita is often used to indicate material wellbeing instead of GDP, because it accounts for differences in population across countries. Refer to the following articles to help you to answer the questions:

   • ‘The Economics of Well-being’ in the Harvard Business Review
   • ‘Statistical Insights: What does GDP per capita tell us about households’ material well-being?’ in the OECD Insights.

• Discuss the usefulness and limitations of GDP per capita as a measure of material wellbeing.

• Based on the arguments in the articles, do you think GDP per capita is an appropriate measure of both material wellbeing and overall wellbeing? Why or why not?

Part 4.2 The HDI as a measure of wellbeing

Learning objectives for this part

• sort data and assign ranks based on values
• distinguish between time series and cross sectional data, and plot appropriate charts for each type of data
• calculate the geometric mean and explain how it differs from the arithmetic mean
• construct indices using the geometric mean, and use index values to rank observations
• explain the difference between two measures of wellbeing (GDP per capita and the Human Development Index).

In Part 4.1 we looked at GDP per capita as a measure of material wellbeing. While income has a major influence on wellbeing because it allows us to buy the goods and services we need or enjoy, it is not the only determinant of wellbeing. Many aspects of our wellbeing cannot be bought, for example, good health or having more time to spend with friends and family.

We are now going to look at the Human Development Index (HDI), a measure of wellbeing that includes non-material aspects, and make comparisons with GDP per capita (a measure of material wellbeing). GDP per capita is a simple index calculated as the sum of its elements, whereas the HDI is more complex. Instead of using different types of expenditure or output to measure wellbeing or living standards, the HDI consists of three dimensions associated with wellbeing:

• a long and healthy life (health)
• knowledge (education)
• a decent standard of living (income).

We will first learn about how the HDI is constructed, and then use this method to construct indices of wellbeing according to criteria of our choice.

The HDI data we will look at is from the Human Development Report 2016 by the United Nations Development Programme (UNDP). To answer the questions below, download the data and technical notes from the report:

• Go to the UNDP’s website.
• Click ‘Table 1: Human Development Index and its components’ on the left-hand side of the page. Then right-click on the ‘Download Data’ box on the top right-hand side of the page and select ‘Save Link As…’.
• Save the file in an easily accessible location, and make sure to give it a suitable name.
• The ‘Technical notes’ give a diagrammatic presentation of how the HDI is constructed from four indicators.

1. Refer to the technical notes and the spreadsheet you have downloaded. For each indicator, explain whether you think it is a good measure of the dimension, and suggest alternative indicators, if any. (For example, is GNI per capita a good measure of the dimension ‘a decent standard of living’?)

1. Figure 4.8 shows the minimum and maximum values for each indicator. Discuss whether you think these are reasonable. (You can read the justification for these values in the technical notes.)

We are now going to apply the method for constructing the HDI, by recalculating the HDI from its indicators. We will use the formula below, and the minimum and maximum values in the table in Figure 4.8. These are taken from page 2 of the technical notes, which you can refer to for additional details.

The HDI indicators are measured in different units and have different ranges, so in order to put them together into a meaningful index, we need to normalize the indicators using the following formula:

Doing so will give a value in between 0 and 1 (inclusive), which will allow comparison between different indicators.

1. Refer to Figure 4.8 and calculate the dimension index for each of the dimensions:

• Using the HDI indicator data in Column E of the spreadsheet, calculate the dimension index for a long and healthy life (health).

• Using the HDI indicator data in Columns G and I of the spreadsheet, calculate the dimension index for knowledge (education). Note that the knowledge dimension index is the average of the dimension index for expected years of schooling for those entering school, and mean years of schooling for adults aged 25 or older. When calculating the index for expected years of schooling you need to restrict the values to take a maximum value of 1.

• Using the HDI indicator data in Column K of the spreadsheet, calculate the dimension index for a decent standard of living (income). Note (from the technical notes) that you should calculate the GNI index using the natural log of the values. (See the ‘Find out more’ box below for an explanation of the natural log and how to calculate it in R.)

Find out more The natural log: What it means, and how to calculate it in R

The natural log turns a linear variable into a concave variable, as shown in Figure 4.9. For any value of income on the horizontal axis, the natural log of that value on the vertical axis is smaller. At first, the difference between income and log income is not that big (for example, an income of 2 corresponds to a natural log of 0.7), but the difference becomes bigger as we move rightwards along the horizontal axis (for example, when income is 100,000, the natural log is only 11.5).

Figure 4.9 Comparing income with the natural logarithm of income.

The reason why natural logs are useful in economics is because they can represent variables that have diminishing marginal returns: an additional unit of input results in a smaller increase in the total output than did the previous unit. (If you have studied production functions, then the shape of the natural log function might look familiar.)

When applied to the concept of wellbeing, the ‘input’ is income, and the ‘output’ is material wellbeing. It makes intuitive sense that a $100 increase in per capita income will have a much greater effect on wellbeing for a poor country compared to a rich country. Using the natural log of income incorporates this notion into the index we create. Conversely, the notion of diminishing marginal returns (the larger the value of the input, the smaller the contribution of an additional unit of input) is not captured by GDP per capita, which uses actual income and not its natural log. Doing so makes the assumption that a $100 increase in per capita income has the same effect on wellbeing for rich and poor countries.

The log function in R calculates the natural log of a value for you. To calculate the natural log of a value, x, type log(x). If you have a scientific calculator, you can check that the calculation is correct by using the ln or log key.

Now that you know about the natural log, you might want to go back to Question 3(c) in Part 4.1, and create a new chart using the natural log scale. The natural log is used in economics because it approximates percentage changes i.e. log(x) – log(y) is a close approximation to the percentage change between x and y. So, using the natural log scale, you will be able to ‘read off’ the relative growth rates from the slopes of the different series you have plotted. For example, a 0.01 change in the vertical axis value corresponds to a 1% change in that variable. This will allow you to compare the growth rates of the different components of GDP.

geometric mean

A summary measure calculated by multiplying N numbers together and then taking the Nth root of this product. The geometric mean is useful when the items being averaged have different scoring indices or scales, because it is not sensitive to these differences, unlike the arithmetic mean. For example, if education ranged from 0 to 20 years and life expectancy ranged from 0 to 85 years, life expectancy would have a bigger influence on the HDI than education if we used the arithmetic mean rather than the geometric mean. Conversely, the geometric mean treats each criteria equally. Example: Suppose we use life expectancy and mean years of schooling to construct an index of wellbeing. Country A has life expectancy of 40 years and a mean of 6 years of schooling. If we used the arithmetic mean to make an index, we would get (40 + 6)/2 = 23. If we used the geometric mean, we would get (40 × 6)^1/2 = 15.5. Now suppose life expectancy doubled to 80 years. The arithmetic mean would be (80 + 6)/2 = 43, and the geometric mean would be (80 × 6)^1/2 = 21.9. If, instead, mean years of schooling doubled to 12 years, the arithmetic mean would be (40 + 12)/2 = 26, and the geometric mean would be (40 × 12)^1/2 = 21.9. This example shows that the arithmetic mean can be ‘unfair’ because proportional changes in one variable (life expectancy) have a larger influence over the index than changes in the other variable (years of schooling). The geometric mean gives each variable the same influence over the value of the index, so doubling the value of one variable would have the same effect on the index as doubling the value of another variable.

Now, we can combine these dimensional indices to give the HDI. The HDI is the geometric mean of the three dimension indices (I_Health = Life expectancy index, I_Education = Education index, and I_Income = GNI index):

1. Use the formula above and the data in the spreadsheet to calculate the HDI for all the countries excluding those in the ‘Other countries or territories’ category. You should get the same values as those in Column C, rounded to three decimal places.

R walk-through 4.7 Calculating the HDI

We will use read_excel to import the data file, which we saved as ‘HDR_data.xlsx’ in our working directory. Before importing, look at the Excel file so that you understand its structure and how it corresponds to the code options used below. We will save the imported data as the dataframe HDR2018.

# File path
HDR2018 <- read_excel("HDR_data.xlsx",
  # Worksheet to import
  sheet = "Table 1",
  # Number of rows to skip
  skip = 2)

head(HDR2018)

## # A tibble: 6 x 15
##   X__1   X__2         `Human Development ~ X__3  `Life expectancy a~ X__4 
##   <chr>  <chr>        <chr>                <lgl> <chr>               <chr>
## 1 HDI r~ Country      Value                NA    (years)             <NA> 
## 2 <NA>   <NA>         2015                 NA    2015                <NA> 
## 3 <NA>   VERY HIGH H~ <NA>                 NA    <NA>                <NA> 
## 4 1      Norway       0.94942283449106446  NA    81.710999999999999  <NA> 
## 5 2      Australia    0.93867953564660933  NA    82.537000000000006  <NA> 
## 6 2      Switzerland  0.93913086905938037  NA    83.132999999999996  <NA> 
## # ... with 9 more variables: `Expected years of schooling` <chr>,
## #   X__5 <chr>, `Mean years of schooling` <chr>, X__6 <chr>, `Gross
## #   national income (GNI) per capita` <chr>, X__7 <chr>, `GNI per capita
## #   rank minus HDI rank` <chr>, X__8 <lgl>, `HDI rank` <chr>

str(HDR2018)

## Classes 'tbl_df', 'tbl' and 'data.frame':  264 obs. of 15 variables:
##  $ X__1                                  : chr  "HDI rank" NA NA "1" ...
##  $ X__2                                  : chr  "Country" NA "VERY HIGH HUMAN DEVELOPMENT" "Norway" ...
##  $ Human Development Index (HDI)         : chr  "Value" "2015" NA "0.94942283449106446" ...
##  $ X__3                                  : logi  NA NA NA NA NA NA ...
##  $ Life expectancy at birth              : chr  "(years)" "2015" NA "81.710999999999999" ...
##  $ X__4                                  : chr  NA NA NA NA ...
##  $ Expected years of schooling           : chr  "(years)" "2015" NA "17.671869999999998" ...
##  $ X__5                                  : chr  NA "a" NA NA ...
##  $ Mean years of schooling               : chr  "(years)" "2015" NA "12.746420000000001" ...
##  $ X__6                                  : chr  NA "a" NA NA ...
##  $ Gross national income (GNI) per capita: chr  "(2011 PPP $)" "2015" NA "67614.353480000005" ...
##  $ X__7                                  : chr  NA NA NA NA ...
##  $ GNI per capita rank minus HDI rank    : chr  NA "2015" NA "5" ...
##  $ X__8                                  : logi  NA NA NA NA NA NA ...
##  $ HDI rank                              : chr  NA "2014" NA "1" ...

Looking at the HDR dataframe, there are rows that have information that isn’t data (for example, all the rows with an ‘NA’ in the first column), as well as variables/columns that do not contain data (for example, most columns beginning with an ‘X_’, though columns labelled X_1 and X_2 contain the HDI rank and the country names respectively).

Cleaning up the dataframe can be easier to do in Excel by deleting irrelevant rows and columns, but one advantage of doing it in R is replicability. Suppose in a year’s time you carried out the analysis again with an updated spreadsheet containing new information. If you had done the cleaning in Excel, you would have to redo it from scratch, but if you had done it in R, you can simply rerun the code below.

Firstly, we eliminate rows that do not have any numbers in the HDI rank column (or X_1 column).

# Rename the first column, currently named X_1
names(HDR2018)[1] <- "HDI.rank"

# Rename the second column, currently named X_2 
names(HDR2018)[2] <- "Country"

# Rename the last column, which contains the 2014 rank
names(HDR2018)[names(HDR2018) == "HDI rank"] <- 
  "HDI.rank.2014"

# Eliminate the row that contains the column title
HDR2018 <- subset(HDR2018,
  !is.na(HDI.rank) & HDI.rank != "HDI rank")

Then we eliminate columns that contain notes in the original spreadsheet (names starting with ‘X_’).

# Check which variables do NOT (!) start with X_
sel_columns <- !startsWith(names(HDR2018), "X_")

# Select the columns that do not start with X_
HDR2018 <- subset(HDR2018, select = sel_columns)

str(HDR2018)

## Classes 'tbl_df', 'tbl' and 'data.frame':  188 obs. of 9 variables:
##  $ HDI.rank                              : chr  "1" "2" "2" "4" ...
##  $ Country                               : chr  "Norway" "Australia" "Switzerland" "Germany" ...
##  $ Human Development Index (HDI)         : chr  "0.94942283449106446" "0.93867953564660933" "0.93913086905938037" "0.9256689410716622" ...
##  $ Life expectancy at birth              : chr  "81.710999999999999" "82.537000000000006" "83.132999999999996" "81.091999999999999" ...
##  $ Expected years of schooling           : chr  "17.671869999999998" "20.43272" "16.040410000000001" "17.095939999999999" ...
##  $ Mean years of schooling               : chr  "12.746420000000001" "13.1751" "13.37" "13.18762553" ...
##  $ Gross national income (GNI) per capita: chr  "67614.353480000005" "42822.19627" "56363.957799999996" "44999.647140000001" ...
##  $ GNI per capita rank minus HDI rank    : chr  "5" "19" "7" "13" ...
##  $ HDI.rank.2014                         : chr  "1" "3" "2" "4" ...

Let’s change some of the long variable names (those in columns 3–8) to shorter ones.

names(HDR2018)[3] <- "HDI"
names(HDR2018)[4] <- "LifeExp"
names(HDR2018)[5] <- "ExpSchool"
names(HDR2018)[6] <- "MeanSchool"
names(HDR2018)[7] <- "GNI.capita"
names(HDR2018)[8] <- "GNI.HDI.rank"

Looking at the structure of the data, we see that R thinks that all the data are chr (character or text variables) because the original datafile contained non-numerical entries (these rows have now been deleted). Apart from the Country variable, which we want to be a factor variable (containing categories), all variables should be numeric.

HDR2018$HDI.rank <- as.numeric(HDR2018$HDI.rank)
HDR2018$Country <- as.factor(HDR2018$Country)
HDR2018$HDI <- as.numeric(HDR2018$HDI)
HDR2018$LifeExp <- as.numeric(HDR2018$LifeExp)
HDR2018$ExpSchool <- as.numeric(HDR2018$ExpSchool)
HDR2018$MeanSchool <- as.numeric(HDR2018$MeanSchool)
HDR2018$GNI.capita <- as.numeric(HDR2018$GNI.capita)
HDR2018$GNI.HDI.rank <- as.numeric(HDR2018$GNI.HDI.rank)
HDR2018$HDI.rank.2014 <- as.numeric(HDR2018$HDI.rank.2014)
str(HDR2018)

## Classes 'tbl_df', 'tbl' and 'data.frame':  188 obs. of 9 variables:
##  $ HDI.rank     : num  1 2 2 4 5 5 7 8 9 10 ...
##  $ Country      : Factor w/ 188 levels "Afghanistan",..: 125 9 163 64 47 151 120 81 76 32 ...
##  $ HDI          : num  0.949 0.939 0.939 0.926 0.925 ...
##  $ LifeExp      : num  81.7 82.5 83.1 81.1 80.4 ...
##  $ ExpSchool    : num  17.7 20.4 16 17.1 19.2 ...
##  $ MeanSchool   : num  12.7 13.2 13.4 13.2 12.7 ...
##  $ GNI.capita   : num  67614 42822 56364 45000 44519 ...
##  $ GNI.HDI.rank : num  5 19 7 13 13 -3 8 11 20 12 ...
##  $ HDI.rank.2014: num  1 3 2 4 6 4 6 8 9 9 ...

Now we have a nice clean dataset that we can work with.

We start by calculating the three indices, using the information given. For the education index we calculate the index for expected and mean schooling separately, then take the arithmetic mean to get I.Education. As some mean schooling observations exceed the specified ‘maximum’ value of 18, the calculated index values would be larger than 1. To avoid this, we use pmin to replace these observations with 18 to obtain an index value of 1.

HDR2018$I.Health <- 
  (HDR2018$LifeExp - 20) / (85 - 20)

HDR2018$I.Education <- 
  ((pmin(HDR2018$ExpSchool, 18) - 0) / 
  (18 - 0) + (HDR2018$MeanSchool - 0) / 
  (15 - 0)) / 2

HDR2018$I.Income <-
  (log(HDR2018$GNI.capita) - log(100)) /
  (log(75000) - log(100))

HDR2018$HDI.calc <- 
  (HDR2018$I.Health * HDR2018$I.Education * 
    HDR2018$I.Income)^(1/3)

Now we can compare the HDI given in the table and our calculated HDI.

HDR2018[, c("HDI", "HDI.calc")]

## # A tibble: 188 x 2
##   HDI   HDI.calc
##   <dbl> <dbl>
## 1 0.949 0.949
## 2 0.939 0.939
## 3 0.939 0.939
## 4 0.926 0.926
## 5 0.925 0.925
## 6 0.925 0.927
## 7 0.924 0.924
## 8 0.923 0.923
## 9 0.921 0.921
## 10 0.920 0.920
## # ... with 178 more rows

The HDI is one way to measure wellbeing, but you may think that it does not use the most appropriate measures for the non-material aspects of wellbeing (health and education).

Now we will use the same method to create our own index of non-material wellbeing (an ‘alternative HDI’), using different indicators. You can find alternative indicators to measure health and education on the UNDP website, in the file ‘Download 2018 Human Development Data Bank’ (left-hand side of the page). (The first column of the spreadsheet, ‘dimension’, tells you whether the indicators refer to ‘Health’, ‘Education’, or other categories).

1. Create an alternative index of wellbeing. In particular, propose alternative Education and Health indices in (a) and (b), then combine these with the existing Income index in (c) to calculate an alternative HDI. Examine whether the changes caused substantial changes in country rankings in (d).

• Choose two to three indicators to measure health, and two to three indicators to measure education. Explain why you have chosen these indicators.

• Carefully merge the data into your existing data. Choose a reasonable maximum and minimum value for each indicator and justify your choices.

• Calculate your alternative versions of the education and health dimension indices. Since you have chosen more than one indicator for this dimension, make sure to average the dimension indices as done in Question 3(b). Also ensure that higher indicator values always represent better outcomes. Now calculate the alternative HDI as done in Questions 3 and 4. Remember to combine your alternative education and health indices with the existing income index from Question 2.

• Create a new variable showing each country’s rank according to your alternative HDI, where 1 is assigned to the country with the highest value. Compare your ranking to the HDI rank. Are the rankings generally similar, or very different? (See R walk-through 4.8 on how to do this.)

R walk-through 4.8 Creating your own HDI

Merge data and calculate alternative indices

This example uses the following indicators:

• Education: Literacy rate, adult (% ages 15 and older); Gross enrolment ratio, tertiary (% of tertiary school-age population); Primary school teachers trained to teach (%)
• Health: Child malnutrition, stunting (moderate or severe) (% under age 5); Mortality rate, female adult (per 1,000 people); Mortality rate, male adult (per 1,000 people).

First, we import the data and check that it has been imported correctly. You can see that each row represents a different country and indicator (indicator_name), and each column represents a different year.

# Filename
allHDR2018 <- read_excel("2018_all_indicators.xlsx",
  # Sheet to import
  sheet = "Data")

head(allHDR2018)

## # A tibble: 6 x 34
##    dimension indicator_id indicator_name iso3 country_name `1990` `1991`
##    <chr>           <dbl> <chr>         <chr> <chr>     <dbl>  <dbl>
## 1  Composit-         14206 HDI rank      AFG   Afghanistan    NA   NA
## 2  Composit-         14206 HDI rank      ALB   Albania    NA   NA
## 3  Composit-         14206 HDI rank      DZA   Algeria    NA   NA
## 4  Composit-         14206 HDI rank      AND   Andorra    NA   NA
## 5  Composit-         14206 HDI rank    AGO   Angola     NA   NA
## 6  Composit-         14206 HDI rank    ATG   Antigua and~   NA   NA
## # ... with 27 more variables: `1992` <dbl>, `1993` <dbl>, `1993` <dbl>,
## #   `1994` <dbl>, `1995` <dbl>, `1996` <dbl>, `1997` <dbl>,`1998` <dbl>,
## #   `1999` <dbl>, `2000` <dbl>, `2001` <dbl>, `2002` <dbl>, `2003` <dbl>,
## #   `2004` <dbl>, `2005` <dbl>, `2006` <dbl>, `2007` <dbl>, `2008` <dbl>,
## #   `2009` <dbl>, `2010` <dbl>, `2011` <dbl>, `2012` <dbl>, `2013` <dbl>,
## #   `2014` <dbl>, `2015` <dbl>, `2016` <dbl>, `2017` <dbl>, `9999` <dbl>

str(HDR2018)

## Classes 'tbl_df', 'tbl' and 'data.frame':  25636 obs. of 34 variables:
##  $ dimension      : chr  "Composite indices" "Composite indices" "Composite indices" "Composite indices" ...
##  $ indicator_id   : num  146206 146206 146206 146206 ...
##  $ indicator_name : chr  "HDI rank" "HDI rank" "HDI rank" "HDI rank" ...
##  $ iso3           : chr  "AFG" "ALB" "DZA" "AND" ...
##  $ country_name   : chr  "Afghanistan" "Albania" "Algeria" "Andorra" ...
##  $ 1990           : num  NA NA NA NA ...
##  $ 1991           : num  NA NA NA NA ...
##  $ 1992           : num  NA NA NA NA ...
##  $ 1993           : num  NA NA NA NA ...
##  $ 1994           : num  NA NA NA NA ...
##  $ 1995           : num  NA NA NA NA ...
##  $ 1996           : num  NA NA NA NA ...
##  $ 1997           : num  NA NA NA NA ...
##  $ 1998           : num  NA NA NA NA ...
##  $ 1999           : num  NA NA NA NA ...
##  $ 2000           : num  NA NA NA NA ...
##  $ 2001           : num  NA NA NA NA ...
##  $ 2002           : num  NA NA NA NA ...
##  $ 2003           : num  NA NA NA NA ...
##  $ 2004           : num  NA NA NA NA ...
##  $ 2005           : num  NA NA NA NA ...
##  $ 2006           : num  NA NA NA NA ...
##  $ 2007           : num  NA NA NA NA ...
##  $ 2008           : num  NA NA NA NA ...
##  $ 2009           : num  NA NA NA NA ...
##  $ 2010           : num  NA NA NA NA ...
##  $ 2011           : num  NA NA NA NA ...
##  $ 2012           : num  NA NA NA NA ...
##  $ 2013           : num  NA NA NA NA ...
##  $ 2014           : num  NA NA NA NA ...
##  $ 2015           : num  NA NA NA NA ...
##  $ 2016           : num  NA NA NA NA ...
##  $ 2017           : num  NA NA NA NA ...
##  $ 9999           : num  NA NA NA NA ...

Then we follow the same process as in R walk-through 4.7—getting the data for the indicators we want, reshaping it so that each indicator is in a different column (instead of a different row), and giving each indicator a shorter name. We will save this data as HDR2018w. Note that the variable 9999 refers to the latest year available, or the average taken over a range of years (the Excel file contains information on which year(s) were used).

indicators <- c(
  "Literacy rate, adult (% ages 15 and older)", 
  "Gross enrolment ratio, tertiary (% of tertiary school-age population)", 
  "Primary school teachers trained to teach (%)", 
  "Child malnutrition, stunting (moderate or severe) (% under age 5)", 
  "Mortality rate, female adult (per 1,000 people)", 
  "Mortality rate, male adult (per 1,000 people)"

HDR2018l <- allHDR2018[
  allHDR2018$indicator_name %in% indicators, ]

HDR2018l <- subset(HDR2018l, 
  # Indicate which variables to keep.
  select = c("indicator_name", "country_name", "9999"))

HDR2018w <- dcast(HDR2018l, country_name ~ indicator_name, 
  value.var = "9999")
names(HDR2018.Edu)[1] <- "HDI.rank"

# Rename the second column, currently named 'Very high …'

names(HDR2018w)[1] <- "Country"
names(HDR2018w)[2] <- "Child.Malnu"
names(HDR2018w)[3] <- "Tert.Enrol"
names(HDR2018w)[4] <- "Adult.Lit"
names(HDR2018w)[5] <- "Mortality.Female"
names(HDR2018w)[6] <- "Mortality.Male"
names(HDR2018w)[7] <- "Prim.Teacher"

str(HDR2018w)

## 'data.frame': 194 obs. of 7 variables:
##  $ Country         : chr  "Afghanistan" "Albania" "Algeria" "Andorra" ...
##  $ Child.Malnu     : num 40.9 23.2 11.7 NA 37.6 NA NA 9.4 2 NA ...
##  $ Tert.Enrol      : num 8 61 43 NA 9 22 86 51 122 83 ...
##  $ Adult.Lit       : num 31.7 97.2 75.1 100 66 NA 98.1 99.7 NA NA ...
##  $ Mortality.Female: num 202 50 83 NA 203 106 74 74 45 46 ...
##  $ Mortality.Male  : num 245 79 106 NA 277 151 151 174 76 86 ...
##  $ Prim.Teacher    : num NA NA 100 100 47 65 NA NA NA NA ... 

Looking at the structure (str( )), we can see that all indicators are correctly in numerical (num) format.

Before we can calculate indices, we need to set minimum and maximum values, which we base on the minimum and maximum values in the sample.

summary(HDR2018w)

##    Country            Child.Malnu       Tert.Enrol        Adult.Lit   
##  Length:194         Min.   : 1.80   Min.   :  2.00   Min.   : 15.50  
##  Class :character   1st Qu.:10.78   1st Qu.: 13.50   1st Qu.: 68.55  
##  Mode  :character   Median :22.20   Median : 37.00   Median : 92.25  
##                     Mean   :22.84   Mean   : 40.36   Mean   : 80.11  
##                     3rd Qu.:32.65   3rd Qu.: 63.00   3rd Qu.: 97.35  
##                     Max.   :55.90   Max.   :122.00   Max.   :100.00  
##                     NA's   :62      NA's   :43         NA's   :60
##  Mortality.Female Mortality.Male   Prim.Teacher 
##  Min.   : 34.00    Min.   : 63.0   Min.   : 15.00  
##  1st Qu.: 66.75    1st Qu.:121.0   1st Qu.: 70.00  
##  Median :100.50    Median :184.0   Median : 86.00 
##  Mean   :130.86    Mean   :195.1   Mean   : 81.23  
##  3rd Qu.:181.50    3rd Qu.:253.5   3rd Qu.: 99.00  
##  Max.   :463.00    Max.   :555.0   Max.   :100.00  
##  NA's   :14       NA's    :14      NA's   :77

As we want the observations to be inside the [min, max] interval, we choose the following [min, max] pairs for the education indicators: Adult.Lit: [15, 100], Tert.Enrol: [2, 122], and Prim.Teacher: [15, 100]. You may want to research why there can be countries with a tertiary enrolment ratio larger than 100%.

Let’s calculate the alternative education index (I.Education.alt), taking an arithmetic average just as we did for I.Education in R walk-through 4.7.

HDR2018w$I.Adult.Lit <-
  (HDR2018w$Adult.Lit-15) / (100-15)

HDR2018w$I.Tert.Enrol<-
  (HDR2018w$Tert.Enrol-2) / (122-2) 

HDR2018w$I.Prim.Teacher <-
  (HDR2018w$Prim.Teacher-15) / (100-15)
  
HDR2018w$I.Education.alt <- 
  (HDR2018w$I.Adult.Lit +
    HDR2018w$I.Tert.Enrol + 
    HDR2018w$I.Prim.Teacher) / 3

summary(HDR2018w$I.Education.alt)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.1628  0.4222  0.5781  0.5862  0.7181  0.8973     113

You can see that we could not calculate this index for 113 countries, as at least one of the three values was missing.

We repeat this procedure to calculate an alternative health index (I.Health.alt). The [min, max] pairs we use are: Child.Malnu: [1, 56], Mortality.Female: [34, 463], Mortality.Male: [63, 555].

HDR2018w$I.Child.MalNu <-
  (HDR2018w$Child.MalNu - 1) /
  (56 - 1)

HDR2018w$I.Mortality.Female <-
  (HDR2018w$Mortality.Female - 34) /
  (463 - 34) 

HDR2018w$I.Mortality.Male <-
  (HDR2018w$Mortality.Male - 63) /
  (555 - 63)

HDR2018w$I.Health.alt <-
  (HDR2018w$I.Child.MalNu + 
  HDR2018w$I.Mortality.Female + 
  HDR2018w$I.Mortality.Male) / 3

# Note that these are all 'bad' indicators in the sense that higher numbers indicate worse outcomes.

# For all other indicators, larger numbers mean better outcomes. So we need to 'flip' the values of this indicator.

HDR2018w$I.Health.alt <- (1 - HDR2018w$I.Health.alt)

summary(HDR2018w$I.Health.alt)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.1370  0.5235  0.7088  0.6655  0.8274  0.9766      64

Now we use the merge function to merge this variable into our existing HDR2018 dataframe.

HDR2018 <- merge(HDR2018, HDR2018w)

Calculate an alternative HDI

Looking at HDR2018, you will see that alternative health and education indices have been added. Now we are in a position to calculate our own HDI (HDI.own).

HDR2018$HDI.own <- 
  (HDR2018$I.Health.alt *
    HDR2018$I.Education.alt *
    HDR2018$I.Income)^(1/3) 

summary(HDR2018$HDI.own)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.2647  0.4541  0.5705  0.5870  0.7460  0.8469     121

We have a substantial number of missing observations, leaving us with around 70 countries for which we could calculate the alternative HDI.

Calculate ranks

To compare the ranks of the two indices (the original HDI and our alternative HDI), we should only rank the countries that have observations for both indices. We will create a dataframe called HDR2018_sub that contains this subset of countries.

HDR2018_sub <- 
  subset(HDR2018, !is.na(HDI) & !is.na(HDI.own)) 

Let’s calculate the rank for our index. The rank function will assign rank 1 to the smallest index value, but we want the largest (best) index value to have the rank 1. We add - in front of the variable name to obtain the desired effect.

HDR2018_sub$HDI.own.rank <-
  rank(-HDR2018_sub$HDI.own, na.last = "keep") 
HDR2018_sub$HDI.rank <-
  rank(-HDR2018_sub$HDI, na.last = "keep") 

Now we will use the ggplot function to make a scatterplot comparing the rank of the HDI with that of our own index.

ggplot(HDR2018_sub, aes(x = HDI.rank, y = HDI.own.rank)) +
  # Use solid circles
  geom_point(shape = 16) +
  labs(y = "Alternative HDI rank", x = "HDI rank") +
  ggtitle("Comparing ranks between HDI and HDI.own") +
  theme_bw()

Figure 4.10 Scatterplot of ranks for HDI and alternative HDI index.

You can see that in general the rankings are similar. If they were identical, the points in the scatterplot would form a straight upward-sloping line. They do not form a straight line, but there is a very strong positive correlation. There are, however, a few countries where the alternative definitions have caused a change in ranking, so let’s use the head and tail functions to find out which countries these are.

temp <- HDR2018_sub[
  order(HDR2018_sub$HDI.rank -
  HDR2018_sub$HDI.own.rank),
  # Show selected variables 
  c("Country", "HDI.rank", "HDI.own.rank")]

# Show the countries with the largest fall in rank
head(temp, 5)

##        Country HDI.rank HDI.own.rank
## 98     Lesotho       52           70
## 164  Sri Lanka       13           29
## 104  Madagascar      54           68
## 57     Eswatini      41           51
## 154  Seychelles       7           17

# Show the countries with the largest increase in rank
tail(temp, 5)

##           Country HDI.rank HDI.own.rank
## 111    Mauritania       53           45
## 171      Tanzania       49           41
## 31       Cambodia       43           34
## 165         Sudan       59           50
## 120       Myanmar       45           33

1. Compare your alternative index to the HDI:

• The UN classifies countries into four groups depending on their HDI, as shown in Figure 4.11. Would the classification of any country change under your alternative HDI?

• Based on your answers to Questions 5(d) and 6(a), do you think that the HDI is a robust measure of non-material wellbeing? (In other words, does changing the indicators used in the HDI give similar conclusions about the non-material wellbeing of countries?)

We will now investigate whether HDI and GDP per capita give similar information about overall wellbeing, by comparing a country’s rank in both measures. To answer Question 7, first add (merge) this data to the dataframe that contains your HDI calculations, making sure to match the data to the correct country. (R walk-through 4.8 shows you how to extract the indicator(s) you need and merge them with another dataset.)

1. Evaluate GDP per capita and the HDI as measures of overall wellbeing:

• Create a new column showing each country’s rank according to GDP per capita, where 1 is assigned to the country with the highest value. Show this rank on a scatterplot, with GDP per capita rank on the vertical axis and HDI rank on the horizontal axis. (See R walk-through 4.8 for a step-by-step explanation of how to create scatterplots in R.)

• Does there appear to be a strong correlation between these two variables? Give an explanation for the observed pattern in your chart.

• Create a table similar to Figure 4.12 below. Using your answers to Question 7(a), fill each box with three countries. You can use the UN’s definition of ‘high’ for the HDI, as in Figure 4.11, and choose a reasonable definition of ‘high’ for GDP. Based on this table, which country or countries would you prefer to grow up in, and why?

• Explain the differences between HDI and GDP as measures of wellbeing. You may want to consider the way each measure includes income in its calculation (the actual value or a transformation), and the inclusion of other aspects of wellbeing.

1. The HDI is one way to measure wellbeing, but there are many other ways to measure wellbeing.

• What are the strengths and limitations of the HDI as a measure of wellbeing?

• Find some alternative measures of non-material wellbeing that we could use alongside the HDI to provide a more comprehensive picture of wellbeing. For each measure, evaluate the elements used to construct the measure, and discuss the additional information we can learn from it. (You may find it helpful to read Our World in Data’s page on happiness and life satisfaction, particularly the section ‘Correlates, Determinants, and Consequences’.)