How to Bootstrap in R: The Case of R-squared

I recorded a new video tutorial whose original intent was to demonstrate how to write a for loop. As I wanted to make the for loop count for something, I decided that my application would be to write some code that computes the bootstrap approximation to the sampling distribution. This is a common econometric application.

Here is the video where I describe the code and the method.



As with most of the videos here, I am merely describing the application. You should understand why you want to bootstrap before you consider running this code (unless you just want to see a for loop in action).

Here is the code I used:

	## ------------------------ ##
	## Introduction to Loops ##
	## and how to Bootstrap ##
	## ------------------------ ##
	set.seed(5151) ## Setting the Seed
	## ---------- ##
	## Read Data ##
	## ---------- ##
	ads.df = read.csv("C://R//ads.csv",header=T)
	summary(ads.df)
	N = length(ads.df[,1]) # Counts the number of observations
	B = 200 # Number of times to recompute estimate (user specified)
	## Sample with replacement ##
	sample(1:10, 7, replace=TRUE)
	## Bootstrap sample with replacement ##
	sample(1:10, 10, replace=TRUE)
	## --------------------------------- ##
	## Bootstrap sample with replacement ##
	## for this data set ##
	## --------------------------------- ##
	idx = sample(1:N, N, replace = TRUE)
	newdata.df = ads.df[idx,] ## Bootstrap Data
	## --------------------------------- ##
	## Calculations with Bootstrap Data ##
	## --------------------------------- ##
	clicks.lm = lm(ClicksPPC~ImpressionsPPC, data=ads.df) ## Clicks = a + b*Impressions
	clicks.lm2 = lm(ClicksPPC~ImpressionsPPC, data=newdata.df) ## Clicks = a + b*Impressions
	stor.r2 = rep(0,B) ## Storage Vector
	## The Loop that does the bootstrap on R-squared ##
	for(i in 1:B){
	idx = sample(1:N, N, replace = TRUE)
	newdata.df = ads.df[idx,]
	clicks.boot = lm(ClicksPPC~ImpressionsPPC, data=newdata.df) ## Clicks = a + b*Impressions
	stor.r2[i] = summary(clicks.boot)$r.squared
	}
	hist(stor.r2) ## Plot the histogram.

view raw BootstrappingR2.R hosted with ❤ by GitHub

I am not going to post the data on this one, but the code should be easy to modify to work with any data set you use.

An easier to use IV regression command in R

Update: I have added some functionality to my ivregress() command. Check out my newer post here.

After I posted my last video tutorial on how to use my IV regression function, I received a comment asking why I didn't write the command a different way to make the syntax easier to read.

The answer is that I didn't know how to write an easier to use function a year ago (when I wrote the ivreg() function). After some digging, I figured out how to work with "formula objects" in R and the result is an easier to use IV regression function (called ivregress()).

How to "install" ivregress()

Here's the code you need to run to define ivregress() and its companion summary command sum.iv(). This will provide instrumental variables regression estimates if you have one endogenous regressor with one or more instruments.



You only need to run the above code once to define the function object ivregress() for all future uses (as long as you don't write over it or clear your workspace).

How to use ivregress()

To create an "IV object" simply use the ivregress command as follows (the command relies on the car library):

library(car)
myivobject = ivregress(Y~X1+X2+...+Xk, Xj ~ Z1+Z2+...+Zl, dataframe)

where X1, X2,...Xk are second stage regressors, Xj is an endogenous regressor for which you would like to instrument, and Z1, Z2,...,Zl are instruments.

Then use sum.iv() from the previous posts to produce summary output. Sometime soon, I will post a video tutorial showing how to use ivregress() to perform IV regression.

Video Tutorial on Instrumental Variables in R

Update: I have replaced this video tutorial with a video tutorial on a newer, easier to use IV regression command. Check out that command here.

In this video, I show how to use my instrumental variables function in R, ivreg(), along with its companion summary command, sum.iv().



Here is a link to the synthetic "market share" data I used in the video.

IV Regression

Update: Since this post, I discovered a bunch of better ways to conduct 2SLS regression in R. These methods do not allow the user to look under the hood to see what is going on (so this post may be instructive anyway). If you're interested, ivreg() in AER, tsls() in sem and systemfit() in systemfit are good options. My package tonymisc also has a command, iv() that works when some or all of these fail. I'm working on making that function better (5/18/2011)

Here is my code from a previous post that performs IV regression. This may be easier to copy into an R script. I will post a video tutorial using this code shortly.

Basic Plots in R

Here's a tutorial I recorded on producing basic plots in R.



I lost the script file I used to create the video to a horrifying black screen of death, but I used the data from the previous post (available here). Hopefully, the video is clear enough that you can follow along with your own window of R.

Testing Multiple Linear Restrictions in R

This video shows a couple of things:

1. How to test (multiple) linear restrictions in R.
2. How to install a package in R.



Here is the code I use in the video (with some extra comments).



Here is the data set that I use in the video.

How Does R Deal with Categorical Explanatory Variables?

Here's a video tutorial where I demonstrate an answer to this question.



This video is designed to instill a baseline level of practical knowledge. There is more to how R treats factors in regression models. An interested reader should Google "R contrasts" for more information.

Here is the code I used in the video:



Here is a link to the data set.

Data from last post

Posting the code I used in the last post wasn't that useful unless I also posted the data set. Here's the data. These are made up data, but it is a nice data set for illustrating how to conduct a regression. Enjoy!