Lab 3 - Data tidying and joining

Lab

Introduction

In this lab you’ll build the data wrangling and visualization skills you’ve developed so far and data tidying and joining to your repertoire.

Note

This lab assumes you’ve completed the labs so far and doesn’t repeat setup and overview content from those labs. If you have not yet done those, you should go back and review the previous labs before starting on this one.

Learning objectives

By the end of the lab, you will…

Be able to pivot/reshape data using tidyr
Continue developing your data wrangling skills using dplyr
Build on your mastery of data visualizations using ggplot2
Get more experience with data science workflow using R, RStudio, Git, and GitHub
Further your reproducible authoring skills with Quarto
Improve your familiarity with version control using Git and GitHub

Getting started

Click here if you prefer to see step-by-step instructions

Team member 1 connect to TV and log in to RStudio

http://turing.cornellcollege.edu:8787/

Clone the repo & start new RStudio project

Go to the course organization at github.com/dsc223-sb7-2024 organization on GitHub. Click on the repo with the prefix lab-1 and with your team number as the suffix It contains the starter documents you need to complete the lab.
I want the first member (and only the first team member) to clone this repo into their RStudio. We did this together in lab-0.
Click lab-1.qmd to open the template Quarto file. This is where you will write up your code and narrative for the lab.

Packages

In this lab we will work with the tidyverse package, which is a collection of packages for doing data analysis in a “tidy” way.

library(tidyverse)

Render the document which loads this package with the library() function.

Guidelines

As we’ve discussed in lecture, your plots should include an informative title, axes should be labeled, and careful consideration should be given to aesthetic choices.

In addition, the code should all the code should be be able to be read (not run off the page) when you render. Make sure that is the case, and add line breaks where the code is running off the page.¹

Note

Continuing to develop a sound workflow for reproducible data analysis is important as you complete the lab and other assignments in this course. There will be periodic reminders in this assignment to remind you to render, commit, and push your changes to GitHub. You should have at least 3 commits with meaningful commit messages by the end of the assignment.

Questions

Part 1

Inflation across the world

For this part of the analysis you will work with inflation data from various countries in the world over the last 30 years.

country_inflation <- read_csv("data/country-inflation.csv")

Question 1

Get to know the data.

glimpse() at the country_inflation data frame and answer the following questions based on the output. How many rows does country_inflation have and what does each row represent? How many columns does country_inflation have and what does each column represent?
Display a list of the countries included in the dataset.

Tip

A function that can be useful for part (b) is pull(). Check out its documentation for examples of usage.

Question 2

Which countries had the top three highest inflation rates in 2021? Your output should be a data frame with two columns, country and 2021, with inflation rates in descending order, and three rows for the top three countries. Briefly comment on how the inflation rates for these countries compare to the inflation rate for United States in that year.

Tip

Column names that are numbers are not considered “proper” in R, therefore to select them you’ll need to surround them with backticks, e.g. select( ` 1993 ` ).

Question 3

In a single pipeline,

calculate the ratio of the inflation in 2021 and inflation in 1993 for each country and store this information in a new column called inf_ratio,
arrange the data frame in decreasing order of inf_ratio, and
select the variables country and inf_ratio to display as the result of the pipeline.

Do not save this new variable in inf_ratio, only calculate and display it so you can answer the following question based on the output of the pipeline.

Which country’s inflation change is the largest over this time period? Did inflation increase of decrease between 1993 and 2021 in this country?

Tip

For this question you’ll once again need to use variables whose names are numbers (years) in your pipeline. Make sure to surround the names of such variables with backticks (`).

Question 4

Reshape (pivot) country_inflation such that each row represents a country/year combination, with columns country, year, and annual_inflation. Then, display the resulting data frame and state how many rows and columns it has.

Requirements:

Your code must use one of pivot_longer() or pivot_wider(). There are other ways you can do this reshaping move in R, but this question requires solving this problem by pivoting.
In your pivot_*() function, you must use names_transform = as.numeric as an argument to transform the variable type to numeric as you pivot the data so that in the resulting data frame the year variable is numeric.
The resulting data frame must be saved as something other than country_inflation so you (1) can refer to this data frame later in your analysis and (2) do not overwrite country_inflation. Use a short but informative name.

Important

The remaining questions in Part 1 require the use of the pivoted data frame from Question 4.

Question 5

Use a separate, single pipeline to answer each of the following questions.

Requirement: Your code must use the filter() function for each part, not arrange().

What is the highest inflation rate observed between 1993 and 2021? The output of the pipeline should be a data frame with one row and three columns. In addition to code and output, your response should include a single sentence stating the country and year.
What is the lowest inflation rate observed between 1993 and 2021? The output of the pipeline should be a data frame with one row and three columns. In addition to code and output, your response should include a single sentence stating the country and year.
Putting (a) and (b) together: What are the highest and the lowest inflation rates observed between 1993 and 2021? The output of the pipeline should be a data frame with two rows and three columns.

Question 6

a. Create a vector called countries_of_interest which contains the names of up tp five countries you want to visualize the inflation rates for over the years. For example, if these countries are Türkiye and United States, you can express this as follows:

countries_of_interest <- c("Türkiye", "United States")

If they are Türkiye, United States, and China, you can express this as follows:

countries_of_interest <- c(
  "Türkiye", "United States", "China (People's Republic of)"
)

So on and so forth… Then, in 1-2 sentences, state why you chose these countries.

Note

Your countries_of_interest should consist of no more than five countries. Make sure that the spelling of your countries matches how they appear in the dataset.

b. In a single pipeline, filter your reshaped dataset to include only the countries_of_interest from part (a), and save the resulting data frame with a new name so you (1) can refer to this data frame later in your analysis and (2) do not overwrite the data frame you’re starting with. Use a short but informative name. Then, in a new pipeline, find the distinct() countries in the data frame you created.

Tip

The number of distinct countries in the filtered data frame you created in part (b) should equal the number of countries you chose in part (a). If it doesn’t, you might have misspelled a country name or made a mistake in how to filter for these countries. Go back and check your code.

Question 7

Using your data frame from the previous question, create a plot of annual inflation vs. year for these countries. Then, in a few sentences, describe the patterns you observe in the plot, particularly focusing on anything you find surprising or not surprising, based on your knowledge (or lack thereof) of these countries economies.

Requirements for the plot:

Data should be represented with points as well as lines connecting the points for each country.
Each country should be represented by a different color line and different color and shape points.
Axes and legend should be properly labeled.
The plot should have an appropriate title (and optionally a subtitle).
Plot should be customized in at least one way – you could use a different than default color scale, or different than default theme, or some other customization.

If you haven’t yet done so, now is a good time to render, commit, and push. Make sure that you commit and push all changed documents and your Git pane is completely empty before proceeding.

Part 2

Inflation in the US

The OECD defines inflation as follows:

Inflation is a rise in the general level of prices of goods and services that households acquire for the purpose of consumption in an economy over a period of time.

The main measure of inflation is the annual inflation rate which is the movement of the Consumer Price Index (CPI) from one month/period to the same month/period of the previous year expressed as percentage over time.

Source: OECD CPI FAQ

CPI is broken down into 12 divisions such as food, housing, health, etc. Your goal in this part is to create another time series plot of annual inflation, this time for US only.

The data you will need to create this visualization is spread across two files:

us-inflation.csv: Annual inflation rate for the US for 12 CPI divisions. Each division is identified by an ID number.
cpi-divisions.csv: A “lookup table” of CPI division ID numbers and their descriptions.

Let’s load both of these files.

us_inflation <- read_csv("data/us-inflation.csv")
cpi_divisions <- read_csv("data/cpi-divisions.csv")

Question 8

a. How many columns and how many rows does the us_inflation dataset have? What are the variables in it? Add a brief (1-2 sentences) narrative summarizing this information.

b. How many columns and how many rows does the cpi_divisions dataset have? What are the variables in it? Add a brief (1-2 sentences) narrative summarizing this information.

c. Create a new dataset by joining the us_inflation dataset with the cpi_division_id dataset.

Determine which type of join is the most appropriate one and use that.
Note that the two datasets don’t have a common variable. Review the help for the join functions to determine how to use the by argument when the names of the variables that the datasets should be joined by are different.
Use a short but informative name for the joined dataset, and do not overwrite either of the datasets that go into creating it.

Then, find the number of rows and columns of the resulting dataset and report the names of its columns. Add a brief (1-2 sentences) narrative summarizing this information.

Question 9

a. Create a vector called divisions_of_interest which contains the descriptions or IDs of CPI divisions you want to visualize. Your divisions_of_interest should consist of no more than five divisions. If you’re using descriptions, make sure that the spelling of your divisions matches how they appear in the dataset. Then, in 1-2 sentences, state why you chose these divisions.

Tip

Refer back to the guidance provided in Question 6 if you’re not sure how to create this vector.

b. In a single pipeline, filter your reshaped dataset to include only the divisions_of_interest from part (a), and save the resulting data frame with a new name so you (1) can refer to this data frame later in your analysis and (2) do not overwrite the data frame you’re starting with. Use a short but informative name. Then, in a new pipeline, find the distinct() divisions in the data frame you created.

Question 10

Using your data frame from the previous question, create a plot of annual inflation vs. year for these divisions. Then, in a few sentences, describe the patterns you observe in the plot, particularly focusing on anything you find surprising or not surprising, based on your knowledge (or lack thereof) of inflation rates in the US over the last decade.

Data should be represented with points as well as lines connecting the points for each division.
Each division should be represented by a different color line and different color and shape points.
Axes and legend should be properly labeled.
The plot should have an appropriate title (and optionally a subtitle).
Plot should be customized in at least one way – you could use a different than default color scale, or different than default theme, or some other customization.
If your legend has labels that are too long, you can try moving the legend to the bottom and stack the labels vertically. Hint: The legend.position and legend.direction arguments of the theme() functions will be useful.

ggplot(...) +
  ... +
  theme(
    legend.position = "bottom", 
    legend.direction = "vertical"
  )

If you haven’t yet done so since Part 1, now is a good time to render, commit, and push. Make sure that you commit and push all changed documents and your Git pane is completely empty before proceeding.

Footnotes

Remember, haikus not novellas when writing code!↩︎