The Packages: tidyverse and esquisse

We will rely on five. The tidyverse is Hadley Wickham’s collection of packages. It represents a different philosophy for the construction of exploratory data analysis with literate programming – code that you can read. We will rely on the %>% piping operators of the magrittr package that pipes something to a subsequent command as a core function of the tidyverse.

For everything that we want in a summary, there is the skimr function skim. For cross-tabulation the easy way, there is janitor. The other two are developmental pieces of software that have yet to deploy into the regular package system of R. esquisse and parts of that’s so random blog’s package for implementing ggmosaic.

pkgTest <- function(x)
  {
    if (!require(x,character.only = TRUE))
    {
      install.packages(x,dep=TRUE, type="binary")
        if(!require(x,character.only = TRUE)) stop("Package not found")
    }
}
pkgTest("tidyverse")
pkgTest("magrittr")
pkgTest("skimr")
pkgTest("janitor")
pkgTest("devtools")
devtools::install_github("EdwinTH/thatssorandom")
pkgTest("dreamRs/esquisse")

A first go

We have an embedded data object; if you look at the beginning of the document, I have used an R command called dput to embed the data in the document. Above, there is encoded text of Bond.Funds to use as an example. To get a sense of the data, I will load the skim function and put it to work. In R, we will use the library command to load functions into the namespace – the set of recognizable commands. R needs to know how a function is defined to use it. The function skim appears in the skimr library.

library(skimr)
skim(Bond.Funds)

Variable type: character

Variable type: numeric

# skimr::skim(Bond.Funds)

Numerical Summary

library(tidyverse)
Bond.Funds %>% group_by(Risk) %>% 
  summarise(Avg.Return = mean(`Return 2009`), SD.Return=sd(`Return 2009`), median.Return=median(`Return 2009`), IQR.Return=IQR(`Return 2009`))

## # A tibble: 3 × 5
##   Risk          Avg.Return SD.Return median.Return IQR.Return
##   <chr>              <dbl>     <dbl>         <dbl>      <dbl>
## 1 Above average       8.31      9.24           7.9      13.0 
## 2 Average             6.87      4.39           6         7.3 
## 3 Below average       6.31      2.71           6.1       3.18

I can also use skim.

Bond.Funds %>% group_by(Risk,Fees) %>% skim(`Return 2009`)

Variable type: numeric

I want to recreate a categorical pivot table also.

Categorical Descriptions Requires Tables [or Graphics]

There are numerous ways to build tables in R. Base R has a table function that works but it does not naturally work inside data environments; we have to provide them using $ or with environments [or %$% in magrittr]. This brief description of environments is part of a broader idea of scoping in R.

library(janitor)

## 
## Attaching package: 'janitor'

## The following objects are masked from 'package:stats':
## 
##     chisq.test, fisher.test

Bond.Funds %>% tabyl(Fees,Risk) %>% adorn_totals(c("row","col"))

##   Fees Above average Average Below average Total
##     No            37      49            44   130
##    Yes            22      20            12    54
##  Total            59      69            56   184

Bond.Funds %>% tabyl(Fees,Risk) %>% adorn_percentages("row")

##  Fees Above average   Average Below average
##    No     0.2846154 0.3769231     0.3384615
##   Yes     0.4074074 0.3703704     0.2222222

table(Bond.Funds$Fees,Bond.Funds$Risk)

##      
##       Above average Average Below average
##   No             37      49            44
##   Yes            22      20            12

xtabs(~Fees+Risk, data=Bond.Funds)

##      Risk
## Fees  Above average Average Below average
##   No             37      49            44
##   Yes            22      20            12

FR.Tab <- xtabs(~Fees+Risk, data=Bond.Funds)

table(Bond.Funds$Fees,Bond.Funds$Risk)

##      
##       Above average Average Below average
##   No             37      49            44
##   Yes            22      20            12

BRTab1 <- table(Bond.Funds$Fees,Bond.Funds$Risk)

with(Bond.Funds, table(Fees,Risk))

##      Risk
## Fees  Above average Average Below average
##   No             37      49            44
##   Yes            22      20            12

WBF1 <- with(Bond.Funds, table(Fees,Risk))

Conditional Frequency

If we think about conditional probability as measured in proportions of the table, we can ask R to calculate them. The command is prop.table and the inputs are a table and a margin here 1 is rows [conditioning on the first name you entered] and 2 is columns [the second name you entered]. Nothing specified is joint or total.

prop.table(FR.Tab)

##      Risk
## Fees  Above average    Average Below average
##   No     0.20108696 0.26630435    0.23913043
##   Yes    0.11956522 0.10869565    0.06521739

prop.table(FR.Tab, 1)

##      Risk
## Fees  Above average   Average Below average
##   No      0.2846154 0.3769231     0.3384615
##   Yes     0.4074074 0.3703704     0.2222222

prop.table(FR.Tab, 2)

##      Risk
## Fees  Above average   Average Below average
##   No      0.6271186 0.7101449     0.7857143
##   Yes     0.3728814 0.2898551     0.2142857

prop.table(WBF1)

##      Risk
## Fees  Above average    Average Below average
##   No     0.20108696 0.26630435    0.23913043
##   Yes    0.11956522 0.10869565    0.06521739

prop.table(WBF1, 1)

##      Risk
## Fees  Above average   Average Below average
##   No      0.2846154 0.3769231     0.3384615
##   Yes     0.4074074 0.3703704     0.2222222

prop.table(WBF1, 2)

##      Risk
## Fees  Above average   Average Below average
##   No      0.6271186 0.7101449     0.7857143
##   Yes     0.3728814 0.2898551     0.2142857

Pivot Plots [Mosaic]

Base R Graphics contain a mosaic with the same formula as the cross-tabulation above.

mosaicplot(~Risk+Fees, data=Bond.Funds)

I recently came across a nice plotter for tabular data on github. You can search for it as thatssorandom. We installed it above.

## Warning: `guides(<scale> = FALSE)` is deprecated. Please use `guides(<scale> =
## "none")` instead.

Notice it handles an implicit plotting of the set of conditional probabilities along the relevant margin. It is plotting \(Pr(Fees|Risk)\) as breaks along the y-axis defined by frequency/empirical probability. This would be the equivalent of taking the column marginal of the table of Fees and Risk that we saw before. Now it has a graphical representation.

prop.table(FR.Tab, 2)

##      Risk
## Fees  Above average   Average Below average
##   No      0.6271186 0.7101449     0.7857143
##   Yes     0.3728814 0.2898551     0.2142857

For Berkeley Admissions:

data("UCBAdmissions")
UCB <- DescTools::Untable(UCBAdmissions)
UCB$Gender <- as.character(UCB$Gender)
UCB$Gender[UCB$Gender=="Male"] <- "M"
UCB$Gender[UCB$Gender=="Female"] <- "F"
UCB$DeptMF <- paste(UCB$Dept,UCB$Gender, sep=":")
UCB <- UCB %>% select(Admit,DeptMF)
p1 <- ggmm(UCB, x=DeptMF, y=Admit, add_text = "n")

## Warning: `guides(<scale> = FALSE)` is deprecated. Please use `guides(<scale> =
## "none")` instead.

p1 + labs(x="Dept:M/F", y="Admitted") + theme(axis.text=element_text(size=10))

Barplots

Basic things like barplots can be accomplished in many ways. Using the R4DS approach, we have

ggplot(data = Bond.Funds) + 
  stat_count(mapping = aes(x = Risk))

Placed into densities. This is really only helpful with another dimension because the X is categorical so all bars will be height 1.

ggplot(data = Bond.Funds, aes(x = Risk, fill=Fees)) + geom_bar(position="fill")

Or in Base R

par(mfrow=c(1,2))
barplot(table(Bond.Funds$Risk))
barplot(table(Bond.Funds$Fees,Bond.Funds$Risk))

A legend would help.

par(mfrow=c(1,2))
barplot(table(Bond.Funds$Fees,Bond.Funds$Risk), legend=TRUE)
barplot(table(Bond.Funds$Fees,Bond.Funds$Risk), legend=TRUE, args.legend=list(bty="n"))

Simple Visualization: esquisse()

There is a wonderful tool for quickly succeeding with one of the most elegant and frustrating parts of R – ggplot2. Hadley Wickham’s Grammar of Graphics is brilliant when understood but is hard to comprehend initially and the programming structure of the package makes it hard for learners. Fortunately, a package called esquisse is available to make ggplot2 drag and drop to harness much of the power in an easy fashion. With code in the output, it also facilitates learning how to manipulate the code of ggplot2.

esquisse is quite powerful; we can explore this at length. Here is a simple graphic that I created with x and fills.

library(ggplot2)
plt1 <- ggplot(data = Bond.Funds) +
  aes(x = Type, fill = Fees) +
  geom_bar() +
  theme_minimal()
plt1

If you notice, esquisse directly outputs graphics to powerpoint. This feature is quite useful.