Data scientists, according to interviews and expert estimates, spend from 50 percent to 80 percent of their time mired in this more mundane labor of collecting and preparing unruly digital data, before it can be explored for useful nuggets.

For Big-Data Scientists, ‘Janitor Work’ Is Key Hurdle to Insight” - The New York Times, 2014

janitor


Travis-CI Build Status Coverage Status lifecycle CRAN_Status_Badge !Monthly Downloads!Downloads

janitor has simple functions for examining and cleaning dirty data. It was built with beginning and intermediate R users in mind and is optimized for user-friendliness. Advanced R users can already do everything covered here, but with janitor they can do it faster and save their thinking for the fun stuff.

The main janitor functions:

  • perfectly format data.frame column names;
  • create and format frequency tables of one, two, or three variables - think an improved table(); and
  • isolate partially-duplicate records.

The tabulate-and-report functions approximate popular features of SPSS and Microsoft Excel.

janitor is a #tidyverse-oriented package. Specifically, it plays nicely with the %>% pipe and is optimized for cleaning data brought in with the readr and readxl packages.

Installation

You can install:

Using janitor

Below are quick examples of how janitor tools are commonly used. A full description of each function can be found in janitor’s catalog of functions vignette.

Cleaning dirty data

Take this roster of teachers at a fictional American high school, stored in the Microsoft Excel file dirty_data.xlsx: All kinds of dirty.

Dirtiness includes:

  • Dreadful column names
  • Rows and columns containing Excel formatting but no data
  • Dates stored as numbers
  • Values spread inconsistently over the “Certification” columns

Here’s that data after being read in to R:

Excel formatting led to an untitled empty column and 5 empty rows at the bottom of the table (only 12 records have any actual data). Bad column names are preserved.

Clean it with janitor functions:

The core janitor cleaning function is clean_names() - call it whenever you load data into R.

Examining dirty data

Tabulating tools

A variable (or combinations of two or three variables) can be tabulated with tabyl(). The resulting data.frame can be tweaked and formatted with the suite of adorn_ functions for quick analysis and printing of pretty results in a report. adorn_ functions can be helpful with non-tabyls, too.

tabyl can be called two ways:

  • On a vector, when tabulating a single variable - e.g., tabyl(roster$subject)
  • On a data.frame, specifying 1, 2, or 3 variable names to tabulate : roster %>% tabyl(subject, employee_status).
    • Here the data.frame is passed in with the %>% pipe; this allows for dplyr commands earlier in the pipeline

tabyl()

Like table(), but pipe-able, data.frame-based, and fully featured.

One variable:

Two variables:

Three variables:

Adorning tabyls

The suite of adorn_ functions dress up the results of these tabulation calls for fast, basic reporting. Here are some of the functions that augment a summary table for reporting:

Pipe that right into knitr::kable() in your RMarkdown report!

These modular adornments can be layered to reduce R’s deficit against Excel and SPSS when it comes to quick, informative counts.

Contact me

You are welcome to: