Hello! I’m trying to work on the exercises in the Forecasting: Principles and Practice textbook. In exercise 2.10, number 4, we are asked to:
Install the USgas package.
Create a tsibble from us_total with year as the index and state as the key.
Plot the annual natural gas consumption by state for the New England area (comprising the states of Maine, Vermont, New Hampshire, Massachusetts, Connecticut and Rhode Island).
I tried filter(state == c("Maine", "Vermont", "New Hampshire", "Massachusetts", "Connecticut", "Rhode Island")) which filtered the dataset to 23 observations, while filter(state %in% c("Maine", "Vermont", "New Hampshire", "Massachusetts", "Connecticut", "Rhode Island")) produces 138 observations.
My question is how do the two operators (== and %in%) produce different results during the filtering process?
Provide an example of your R code
# load packages
pacman::p_load(fpp3, USgas)
# create tsibble from us_total
us_tot <- as_tsibble(us_total,
index = year,
key = state)
# filter and plot using ==, this results in 23 observations
us_filter <- us_tot %>% filter(state == c("Maine", "Vermont", "New Hampshire",
"Massachusetts", "Connecticut", "Rhode Island"))
us_filter %>% autoplot(y/1e3) +
labs(y = "billion cubic feet")
# filter and plot using %in%, this results in 138 observations
us_filter2 <- us_tot |>
filter(state %in% c("Maine", "Vermont", "New Hampshire", "Massachusetts",
"Connecticut", "Rhode Island"))
us_filter2 %>% autoplot(y/1e3) +
labs(y = "billion cubic feet")
You should not be using == in this scenario, in fact, the only reason you are not receiving a warning is because the number of rows in the data is a multiple of 6. So R recycles across the state vector 6n times, where n is the number of rows. Instead, you should be using the %in% operator or fct_match from the forcats package to filter here.
See below:
#Loading tidyverse
library(tidyverse)
# Simulating data
sim_data <- tibble(id = seq_len(length.out = 100)) |>
rowwise() |>
mutate(letters = sample(x = letters, size = 1, replace = TRUE)) |>
ungroup()
# Testing both operators
# 100 is not a multiple of 3
sim_data |>
filter(letters == c("a", "b", "c"))
#> Warning: There was 1 warning in `filter()`.
#> ℹ In argument: `letters == c("a", "b", "c")`.
#> Caused by warning in `letters == c("a", "b", "c")`:
#> ! longer object length is not a multiple of shorter object length
#> # A tibble: 2 × 2
#> id letters
#> <int> <chr>
#> 1 12 c
#> 2 61 a
# Essentially, the vector you filter letters by is being repeated and then
# checked for equality at each row, as seen below
sim_data |>
mutate(filter = rep(x = c("a", "b", "c"), length.out = 100)) |>
filter(letters == filter)
#> # A tibble: 2 × 3
#> id letters filter
#> <int> <chr> <chr>
#> 1 12 c c
#> 2 61 a a
# 100 is a multiple of 4
sim_data |>
filter(letters == c("a", "b", "c", "d"))
#> # A tibble: 3 × 2
#> id letters
#> <int> <chr>
#> 1 60 d
#> 2 61 a
#> 3 76 d
# Essentially, the vector you filter letters by is being repeated and then
# checked for equality at each row, as seen below
sim_data |>
mutate(filter = rep(x = c("a", "b", "c", "d"), length.out = 100)) |>
filter(letters == filter)
#> # A tibble: 3 × 3
#> id letters filter
#> <int> <chr> <chr>
#> 1 60 d d
#> 2 61 a a
#> 3 76 d d
# %in% operator
sim_data |>
filter(letters %in% c("a", "b", "c"))
#> # A tibble: 9 × 2
#> id letters
#> <int> <chr>
#> 1 8 c
#> 2 12 c
#> 3 45 b
#> 4 50 a
#> 5 51 a
#> 6 61 a
#> 7 73 b
#> 8 80 c
#> 9 87 a
# fct_match
sim_data |>
filter(fct_match(f = letters, lvls = c("a", "b", "c")))
#> # A tibble: 9 × 2
#> id letters
#> <int> <chr>
#> 1 8 c
#> 2 12 c
#> 3 45 b
#> 4 50 a
#> 5 51 a
#> 6 61 a
#> 7 73 b
#> 8 80 c
#> 9 87 a
You should only use the == operator when you are dealing with scalar values, i.e., single values that are not vectors. The %in% operator or fct_match should be used when you are dealing with a vector, as in your problem.
See below:
#Loading tidyverse
library(tidyverse)
# Simulating data
sim_data <- tibble(id = seq_len(length.out = 100)) |>
rowwise() |>
mutate(letters = sample(x = letters, size = 1, replace = TRUE)) |>
ungroup()
# Filtering for a scalar value
sim_data |>
filter(letters == "z")
#> # A tibble: 5 × 2
#> id letters
#> <int> <chr>
#> 1 3 z
#> 2 14 z
#> 3 88 z
#> 4 91 z
#> 5 95 z
# Filtering for a vector value
## %in%
sim_data |>
filter(letters %in% c("a", "b", "c"))
#> # A tibble: 14 × 2
#> id letters
#> <int> <chr>
#> 1 10 c
#> 2 18 c
#> 3 20 c
#> 4 26 c
#> 5 29 a
#> 6 32 b
#> 7 51 b
#> 8 68 c
#> 9 73 a
#> 10 75 b
#> 11 76 a
#> 12 81 c
#> 13 90 a
#> 14 96 c
## fct_match
sim_data |>
filter(fct_match(f = letters, lvls = c("a", "b", "c")))
#> # A tibble: 14 × 2
#> id letters
#> <int> <chr>
#> 1 10 c
#> 2 18 c
#> 3 20 c
#> 4 26 c
#> 5 29 a
#> 6 32 b
#> 7 51 b
#> 8 68 c
#> 9 73 a
#> 10 75 b
#> 11 76 a
#> 12 81 c
#> 13 90 a
#> 14 96 c