How many 5 letter words where any string of letters is a word have at least two consecutive letters which are the same?

This chapter introduces you to string manipulation in R. You’ll learn the basics of how strings work and how to create them by hand, but the focus of this chapter will be on regular expressions, or regexps for short. Regular expressions are useful because strings usually contain unstructured or semi-structured data, and regexps are a concise language for describing patterns in strings. When you first look at a regexp, you’ll think a cat walked across your keyboard, but as your understanding improves they will soon start to make sense.

This chapter will focus on the stringr package for string manipulation, which is part of the core tidyverse.

You can create strings with either single quotes or double quotes. Unlike other languages, there is no difference in behaviour. I recommend always using ", unless you want to create a string that contains multiple ".

string1 <- "This is a string" string2 <- 'If I want to include a "quote" inside a string, I use single quotes'

If you forget to close a quote, you’ll see +, the continuation character:

> "This is a string without a closing quote + + + HELP I'M STUCK

If this happen to you, press Escape and try again!

To include a literal single or double quote in a string you can use \ to “escape” it:

double_quote <- "\"" # or '"' single_quote <- '\'' # or "'"

That means if you want to include a literal backslash, you’ll need to double it up: "\\".

Beware that the printed representation of a string is not the same as string itself, because the printed representation shows the escapes. To see the raw contents of the string, use writeLines():

x <- c("\"", "\\") x #> [1] "\"" "\\" writeLines(x) #> " #> \

There are a handful of other special characters. The most common are "\n", newline, and "\t", tab, but you can see the complete list by requesting help on ": ?'"', or ?"'". You’ll also sometimes see strings like "\u00b5", this is a way of writing non-English characters that works on all platforms:

x <- "\u00b5" x #> [1] "µ"

Multiple strings are often stored in a character vector, which you can create with c():

c("one", "two", "three") #> [1] "one" "two" "three"

Base R contains many functions to work with strings but we’ll avoid them because they can be inconsistent, which makes them hard to remember. Instead we’ll use functions from stringr. These have more intuitive names, and all start with str_. For example, str_length() tells you the number of characters in a string:

str_length(c("a", "R for data science", NA)) #> [1] 1 18 NA

The common str_ prefix is particularly useful if you use RStudio, because typing str_ will trigger autocomplete, allowing you to see all stringr functions:

How many 5 letter words where any string of letters is a word have at least two consecutive letters which are the same?

To combine two or more strings, use str_c():

str_c("x", "y") #> [1] "xy" str_c("x", "y", "z") #> [1] "xyz"

Use the sep argument to control how they’re separated:

str_c("x", "y", sep = ", ") #> [1] "x, y"

Like most other functions in R, missing values are contagious. If you want them to print as "NA", use str_replace_na():

As shown above, str_c() is vectorised, and it automatically recycles shorter vectors to the same length as the longest:

str_c("prefix-", c("a", "b", "c"), "-suffix") #> [1] "prefix-a-suffix" "prefix-b-suffix" "prefix-c-suffix"

Objects of length 0 are silently dropped. This is particularly useful in conjunction with if:

name <- "Hadley" time_of_day <- "morning" birthday <- FALSE str_c( "Good ", time_of_day, " ", name, if (birthday) " and HAPPY BIRTHDAY", "." ) #> [1] "Good morning Hadley."

To collapse a vector of strings into a single string, use collapse:

str_c(c("x", "y", "z"), collapse = ", ") #> [1] "x, y, z"

Above I used str_to_lower() to change the text to lower case. You can also use str_to_upper() or str_to_title(). However, changing case is more complicated than it might at first appear because different languages have different rules for changing case. You can pick which set of rules to use by specifying a locale:

# Turkish has two i's: with and without a dot, and it # has a different rule for capitalising them: str_to_upper(c("i", "ı")) #> [1] "I" "I" str_to_upper(c("i", "ı"), locale = "tr") #> [1] "İ" "I"

The locale is specified as a ISO 639 language code, which is a two or three letter abbreviation. If you don’t already know the code for your language, Wikipedia has a good list. If you leave the locale blank, it will use the current locale, as provided by your operating system.

Another important operation that’s affected by the locale is sorting. The base R order() and sort() functions sort strings using the current locale. If you want robust behaviour across different computers, you may want to use str_sort() and str_order() which take an additional locale argument:

x <- c("apple", "eggplant", "banana") str_sort(x, locale = "en") # English #> [1] "apple" "banana" "eggplant" str_sort(x, locale = "haw") # Hawaiian #> [1] "apple" "eggplant" "banana"

  1. In code that doesn’t use stringr, you’ll often see paste() and paste0(). What’s the difference between the two functions? What stringr function are they equivalent to? How do the functions differ in their handling of NA?

  2. In your own words, describe the difference between the sep and collapse arguments to str_c().

  3. Use str_length() and str_sub() to extract the middle character from a string. What will you do if the string has an even number of characters?

  4. What does str_wrap() do? When might you want to use it?

  5. What does str_trim() do? What’s the opposite of str_trim()?

  6. Write a function that turns (e.g.) a vector c("a", "b", "c") into the string a, b, and c. Think carefully about what it should do if given a vector of length 0, 1, or 2.

Regexps are a very terse language that allow you to describe patterns in strings. They take a little while to get your head around, but once you understand them, you’ll find them extremely useful.

To learn regular expressions, we’ll use str_view() and str_view_all(). These functions take a character vector and a regular expression, and show you how they match. We’ll start with very simple regular expressions and then gradually get more and more complicated. Once you’ve mastered pattern matching, you’ll learn how to apply those ideas with various stringr functions.

The simplest patterns match exact strings:

x <- c("apple", "banana", "pear") str_view(x, "an")

The next step up in complexity is ., which matches any character (except a newline):

But if “.” matches any character, how do you match the character “.”? You need to use an “escape” to tell the regular expression you want to match it exactly, not use its special behaviour. Like strings, regexps use the backslash, \, to escape special behaviour. So to match an ., you need the regexp \.. Unfortunately this creates a problem. We use strings to represent regular expressions, and \ is also used as an escape symbol in strings. So to create the regular expression \. we need the string "\\.".

# To create the regular expression, we need \\ dot <- "\\." # But the expression itself only contains one: writeLines(dot) #> \. # And this tells R to look for an explicit . str_view(c("abc", "a.c", "bef"), "a\\.c")

If \ is used as an escape character in regular expressions, how do you match a literal \? Well you need to escape it, creating the regular expression \\. To create that regular expression, you need to use a string, which also needs to escape \. That means to match a literal \ you need to write "\\\\" — you need four backslashes to match one!

x <- "a\\b" writeLines(x) #> a\b str_view(x, "\\\\")

In this book, I’ll write regular expression as \. and strings that represent the regular expression as "\\.".

  1. Explain why each of these strings don’t match a \: "\", "\\", "\\\".

  2. How would you match the sequence "'\?

  3. What patterns will the regular expression \..\..\.. match? How would you represent it as a string?

By default, regular expressions will match any part of a string. It’s often useful to anchor the regular expression so that it matches from the start or end of the string. You can use:

  • ^ to match the start of the string.
  • $ to match the end of the string.

x <- c("apple", "banana", "pear") str_view(x, "^a")

str_view(x, "a$")

To remember which is which, try this mnemonic which I learned from Evan Misshula: if you begin with power (^), you end up with money ($).

To force a regular expression to only match a complete string, anchor it with both ^ and $:

x <- c("apple pie", "apple", "apple cake") str_view(x, "apple")

str_view(x, "^apple$")

You can also match the boundary between words with \b. I don’t often use this in R, but I will sometimes use it when I’m doing a search in RStudio when I want to find the name of a function that’s a component of other functions. For example, I’ll search for \bsum\b to avoid matching summarise, summary, rowsum and so on.

  1. How would you match the literal string "$^$"?

  2. Given the corpus of common words in stringr::words, create regular expressions that find all words that:

    1. Start with “y”.
    2. End with “x”
    3. Are exactly three letters long. (Don’t cheat by using str_length()!)
    4. Have seven letters or more.

    Since this list is long, you might want to use the match argument to str_view() to show only the matching or non-matching words.

There are a number of special patterns that match more than one character. You’ve already seen ., which matches any character apart from a newline. There are four other useful tools:

  • \d: matches any digit.
  • \s: matches any whitespace (e.g. space, tab, newline).
  • [abc]: matches a, b, or c.
  • [^abc]: matches anything except a, b, or c.

Remember, to create a regular expression containing \d or \s, you’ll need to escape the \ for the string, so you’ll type "\\d" or "\\s".

A character class containing a single character is a nice alternative to backslash escapes when you want to include a single metacharacter in a regex. Many people find this more readable.

# Look for a literal character that normally has special meaning in a regex str_view(c("abc", "a.c", "a*c", "a c"), "a[.]c")

str_view(c("abc", "a.c", "a*c", "a c"), ".[*]c") str_view(c("abc", "a.c", "a*c", "a c"), "a[ ]")

This works for most (but not all) regex metacharacters: $ . | ? * + ( ) [ {. Unfortunately, a few characters have special meaning even inside a character class and must be handled with backslash escapes: ] \ ^ and -.

You can use alternation to pick between one or more alternative patterns. For example, abc|d..f will match either ‘“abc”’, or "deaf". Note that the precedence for | is low, so that abc|xyz matches abc or xyz not abcyz or abxyz. Like with mathematical expressions, if precedence ever gets confusing, use parentheses to make it clear what you want:

str_view(c("grey", "gray"), "gr(e|a)y")

  1. Create regular expressions to find all words that:

    1. Start with a vowel.

    2. That only contain consonants. (Hint: thinking about matching “not”-vowels.)

    3. End with ed, but not with eed.

    4. End with ing or ise.

  2. Empirically verify the rule “i before e except after c”.

  3. Is “q” always followed by a “u”?

  4. Write a regular expression that matches a word if it’s probably written in British English, not American English.

  5. Create a regular expression that will match telephone numbers as commonly written in your country.

The next step up in power involves controlling how many times a pattern matches:

  • ?: 0 or 1
  • +: 1 or more
  • *: 0 or more

x <- "1888 is the longest year in Roman numerals: MDCCCLXXXVIII" str_view(x, "CC?")

str_view(x, "CC+") str_view(x, 'C[LX]+')

Note that the precedence of these operators is high, so you can write: colou?r to match either American or British spellings. That means most uses will need parentheses, like bana(na)+.

You can also specify the number of matches precisely:

  • {n}: exactly n
  • {n,}: n or more
  • {,m}: at most m
  • {n,m}: between n and m
str_view(x, "C{2,}") str_view(x, "C{2,3}")

By default these matches are “greedy”: they will match the longest string possible. You can make them “lazy”, matching the shortest string possible by putting a ? after them. This is an advanced feature of regular expressions, but it’s useful to know that it exists:

str_view(x, 'C[LX]+?')

  1. Describe the equivalents of ?, +, * in {m,n} form.

  2. Describe in words what these regular expressions match: (read carefully to see if I’m using a regular expression or a string that defines a regular expression.)

    1. ^.*$
    2. "\\{.+\\}"
    3. \d{4}-\d{2}-\d{2}
    4. "\\\\{4}"
  3. Create regular expressions to find all words that:

    1. Start with three consonants.
    2. Have three or more vowels in a row.
    3. Have two or more vowel-consonant pairs in a row.
  4. Solve the beginner regexp crosswords at https://regexcrossword.com/challenges/beginner.

Earlier, you learned about parentheses as a way to disambiguate complex expressions. Parentheses also create a numbered capturing group (number 1, 2 etc.). A capturing group stores the part of the string matched by the part of the regular expression inside the parentheses. You can refer to the same text as previously matched by a capturing group with backreferences, like \1, \2 etc. For example, the following regular expression finds all fruits that have a repeated pair of letters.

str_view(fruit, "(..)\\1", match = TRUE)

(Shortly, you’ll also see how they’re useful in conjunction with str_match().)

  1. Describe, in words, what these expressions will match:

    1. (.)\1\1
    2. "(.)(.)\\2\\1"
    3. (..)\1
    4. "(.).\\1.\\1"
    5. "(.)(.)(.).*\\3\\2\\1"
  2. Construct regular expressions to match words that:

    1. Start and end with the same character.

    2. Contain a repeated pair of letters (e.g. “church” contains “ch” repeated twice.)

    3. Contain one letter repeated in at least three places (e.g. “eleven” contains three “e”s.)

When you use a pattern that’s a string, it’s automatically wrapped into a call to regex():

You can use the other arguments of regex() to control details of the match:

  • ignore_case = TRUE allows characters to match either their uppercase or lowercase forms. This always uses the current locale.

    bananas <- c("banana", "Banana", "BANANA") str_view(bananas, "banana")

    str_view(bananas, regex("banana", ignore_case = TRUE))
  • multiline = TRUE allows ^ and $ to match the start and end of each line rather than the start and end of the complete string.

  • comments = TRUE allows you to use comments and white space to make complex regular expressions more understandable. Spaces are ignored, as is everything after #. To match a literal space, you’ll need to escape it: "\\ ".

    phone <- regex(" \\(? # optional opening parens (\\d{3}) # area code [) -]? # optional closing parens, space, or dash (\\d{3}) # another three numbers [ -]? # optional space or dash (\\d{3}) # three more numbers ", comments = TRUE) str_match("514-791-8141", phone) #> [,1] [,2] [,3] [,4] #> [1,] "514-791-814" "514" "791" "814"

  • dotall = TRUE allows . to match everything, including \n.

There are three other functions you can use instead of regex():

  • fixed(): matches exactly the specified sequence of bytes. It ignores all special regular expressions and operates at a very low level. This allows you to avoid complex escaping and can be much faster than regular expressions. The following microbenchmark shows that it’s about 3x faster for a simple example.

    microbenchmark::microbenchmark( fixed = str_detect(sentences, fixed("the")), regex = str_detect(sentences, "the"), times = 20 ) #> Unit: microseconds #> expr min lq mean median uq max neval #> fixed 74.8 89.20 142.755 108.25 139.9 754.8 20 #> regex 293.6 304.85 377.650 322.60 360.3 1017.4 20

    Beware using fixed() with non-English data. It is problematic because there are often multiple ways of representing the same character. For example, there are two ways to define “á”: either as a single character or as an “a” plus an accent:

    a1 <- "\u00e1" a2 <- "a\u0301" c(a1, a2) #> [1] "á" "á" a1 == a2 #> [1] FALSE

    They render identically, but because they’re defined differently, fixed() doesn’t find a match. Instead, you can use coll(), defined next, to respect human character comparison rules:

    str_detect(a1, fixed(a2)) #> [1] FALSE str_detect(a1, coll(a2)) #> [1] TRUE

  • coll(): compare strings using standard collation rules. This is useful for doing case insensitive matching. Note that coll() takes a locale parameter that controls which rules are used for comparing characters. Unfortunately different parts of the world use different rules!

    # That means you also need to be aware of the difference # when doing case insensitive matches: i <- c("I", "İ", "i", "ı") i #> [1] "I" "İ" "i" "ı" str_subset(i, coll("i", ignore_case = TRUE)) #> [1] "I" "i" str_subset(i, coll("i", ignore_case = TRUE, locale = "tr")) #> [1] "İ" "i"

    Both fixed() and regex() have ignore_case arguments, but they do not allow you to pick the locale: they always use the default locale. You can see what that is with the following code; more on stringi later.

    stringi::stri_locale_info() #> $Language #> [1] "c" #> #> $Country #> [1] "" #> #> $Variant #> [1] "" #> #> $Name #> [1] "c"

    The downside of coll() is speed; because the rules for recognising which characters are the same are complicated, coll() is relatively slow compared to regex() and fixed().

  • As you saw with str_split() you can use boundary() to match boundaries. You can also use it with the other functions:

    x <- "This is a sentence." str_view_all(x, boundary("word"))

    str_extract_all(x, boundary("word")) #> [[1]] #> [1] "This" "is" "a" "sentence"

  1. How would you find all strings containing \ with regex() vs. with fixed()?

  2. What are the five most common words in sentences?

There are two useful function in base R that also use regular expressions:

  • apropos() searches all objects available from the global environment. This is useful if you can’t quite remember the name of the function.

    apropos("replace") #> [1] "%+replace%" "replace" "replace_na" "setReplaceMethod" #> [5] "str_replace" "str_replace_all" "str_replace_na" "theme_replace"

  • dir() lists all the files in a directory. The pattern argument takes a regular expression and only returns file names that match the pattern. For example, you can find all the R Markdown files in the current directory with:

    head(dir(pattern = "\\.Rmd$")) #> [1] "communicate-plots.Rmd" "communicate.Rmd" "datetimes.Rmd" #> [4] "EDA.Rmd" "explore.Rmd" "factors.Rmd"

    (If you’re more comfortable with “globs” like *.Rmd, you can convert them to regular expressions with glob2rx()):

stringr is built on top of the stringi package. stringr is useful when you’re learning because it exposes a minimal set of functions, which have been carefully picked to handle the most common string manipulation functions. stringi, on the other hand, is designed to be comprehensive. It contains almost every function you might ever need: stringi has 256 functions to stringr’s 49.

If you find yourself struggling to do something in stringr, it’s worth taking a look at stringi. The packages work very similarly, so you should be able to translate your stringr knowledge in a natural way. The main difference is the prefix: str_ vs. stri_.

  1. Find the stringi functions that:

    1. Count the number of words.
    2. Find duplicated strings.
    3. Generate random text.
  2. How do you control the language that stri_sort() uses for sorting?