When do people worry about hurricanes?

I recently read an article in the December issue of Significance titled, “Does Christmas really come earlier every year?” by Nathan Cunningham of the University College of Dublin.  His premise was that, by using cluster analysis of Google Trends data, we can see how people have begun thinking about the holidays earlier and earlier each year.  It’s a good read: http://www.statslife.org.uk/significance/1892.  I should note that Nathan graciously answered my emails asking for clarification and saw real value in this technique for emergency management work.

I decided to replicate his results using a FEMA-related search term: “hurricane”. 

Google Trends

Google Trends (http://www.google.com/trends/) allows you to view the volume of searches on particular terms.  The units are percentage of total Google searches.  For example, the week that Hurricane Katrina made landfall, “hurricane” scored almost 100; almost all searches were hurricane related. If you sign-on with your Google ID, you can also download the data to CSV.  Cunningham used Google Trends to analyze search volumes on holiday-related terms (“Christmas”, “Santa Claus”, etc).  Here I’ve compared the search terms “hurricane” and “tornado”.  You can see that there is a somewhat repetitive pattern of increase mid-year.  I wanted to explore this pattern.

Cluster Analysis

Cluster Analysis looks at data and organizes it into groups that share similarities.  Once Cunningham had each year’s data, he used cluster analysis to determine in which week of the year the volume of holiday-related searches began to increase.  Similar analysis can be done on FEMA-related search terms; a cluster analysis of the Google Trend data for the search term “hurricane” reveals continuous periods of increased interest for the following weeks from 2004-2014.  This was simple to implement using R (see code below).  The accompanying graphic shows the “shape” of the cluster; the x-axis is the week number of the year, and the y-axis is the percentage of all Google searches for the term “hurricane”.  In hindsight, it is possible to find explanations for these clusters; for example, 2005 and 2012 had periods of exceptionally high interest corresponding to the hurricane activity of those years.  2009 and 2013 had little activity (look at the y-axis) corresponding to light years.

Further Investigation

This simple example shows how cluster analysis can illustrate the behavior of data that have more than one pattern.  This could find application in data that vary from Region to Region or JFO to JFO, or changes with disaster type.

Although Cunningham used cluster analysis to look at Google Trends data, it is easy to see that the data returned also lend themselves to Time Series Analysis.

R Code used in this example

## Crow's nest Clustering example – Tim Allen

# Adapted from http://www.statslife.org.uk/significance/1892

# Nathan Cunningham - Does Christmas really come earlier every year?

# Significance Magazine 11 November 2014

# Allow multiple plots (2 rows x 6 columns)

par(mfrow=c(2,6))

# You have to install and load the mclust package

library(mclust)

# Calculate clusters for each year

for (yr in 2007:2013) {

# 1) load this year's data in a matrix

observations <- span="">as.matrix(subset(gtrends, year==yr, select=c("week","hurricane")))

# 2) find clusters based on models' BIC

fit <- span="">Mclust(observations, 2)

# 3) Plot the clusters and print the model summary

plot(fit, what="classification", xlab=yr)

print(summary(fit))

}

Acknowledgement

My sincere appreciation to Nathan Cunningham of  the University College of Dublin for his kind help in preparation of this article.  Please read his article, "Does Christmas really come earlier every year?"

How to calculate a p-value for an ANOVA F-Statistic using R or a TI-84

At the end of calculating an Analysis of Variance (ANOVA), you have an F-statistic.  To get the p-value of the F-Statistic, you can use R or the TI-84:

For example, in an ANOVA with treatment degrees of freedom = 1 and error degrees of freedom = 10, you calculate the F-Statistic as 2.81 and want to know its p-value (that is, what is the probability of observing this F-Statistic under the Null Hypothesis?):

1) R
# The following calculates the p-value of an F-statistic
pf(q=2.81, df1=1, df2=10, lower.tail=FALSE)
 You'll get the answer:
[1] 0.1246126

2) TI-84
The syntax for the Fcdf command is:
Fcdf(lower limit, upper limit, numerator degrees of freedom, denominator degrees of freedom).
Fcdf(2.81, 99999, 1, 10)
 You'll get the answer:
[1] 0.1246126
I'm a student in the MS in Applied Statistics program and the University of the District of Columbia.

My new video: Median Polish - Exploratory Data Analysis

I recently completed a video about Median Polish, an Exploratory Data Analysis technique for analyzing two-way tables invented by John Tukey.  I had tried to find a Youtube video about this, without any luck.  So I thought I'd better create one to fill the gap.  Enjoy:

MS Word: How to delete all the comments in a document using Visual Basic for Applications (VBS)

I just got passed a document with a Microsoft Word document with a zillion comments in it.  I could have deleted all the comments by hand, but it would have taken forever and there would be a good chance I would miss one.  So I used Visual Basic for Applications to do it.  Here are the steps:

1. In MS Word, type Alt-F11 to bring up the Macro window
2. At the bottom of the Macro window, there is a pane labelled "Immediate".  Commands entered here are run immediately.
3. On a single line in the immediate pane, type
   
   for z = ActiveDocument.Comments.Count to 1 step -1: ActiveDocument.Comments(z).Delete : next
4. Hit Enter. All of the comments in the document will be deleted.

Parkour Flow video

I finally got around to filming a new parkour video.  This is a short video of some parkour flow work I did this afternoon near my work.  I filmed a lot and mean to edit together a longer clip when I have some time.  Enjoy!

Idea: The Shivery 30 Blanket Run

I visited National Cathedral today with Daniel. If you haven't had the guided tour, you really should do this. We learned a lot of great things. Daniel was terrific, really curious about everything; he asked great questions of the docent.

After a trip up to the observation deck, we made the obligatory visit to the Gift Shop. We got a couple things, and when we rang up, the sales person asked if we wanted to donate $15 to buy a blanket for a needy DC family. Since I'm not a heartless ogre, I went ahead and ponied up.

Then I started thinking: $15 for a blanket. Every time I run, I see homeless guys out there freezing in sweatshirts. And I'd already been playing with the idea of doing another ultramarathon and having people donate blankets. But here is a ready-made mechanism for getting blankets into peoples' hands... just waiting for a knucklehead like me to drive people to it.

I'm not thinking of doing another 46 miler for this... first off, Sònia wouldn't have it. And second off, Sònia wouldn't have it! But I already do 30 mile training runs pretty regularly. I could just do a 30 mile run, get people to sign up and send money to the Cathedral, and voila: cold homeless people get warm! A dollar a mile pledge would equal two blankets per donation. And the block around National Cathedral is just at one mile and a bit... I could actually run the thing around the Cathedral! So I'm giving some actual thought to this. I'm not saying it's a done deal... I'm just saying it sounds do-able. Really. Hmmm.

$1075 raised by Daddy's Crazy 46-Miler!

The Oyster-Adams Community Counsel folks have added up the donations made for the Daddy's Crazy 46-Miler, and so far we have raised at least $1075 for Oyster-Adams Bilingual School. You can still donate! Click here to make a donation. Thanks to all who have donated so far.

IMPORTANT: If you make a donation, please put "Daddy's Crazy 46" in the description. If not, the school won't know the DC46 was where the funds came from.

I ran 46 miles! Daddy's Crazy 46-Miler

Yesterday morning at 3am, I started running. Eight hours and 42 minutes later, I finished-- and I had covered 46 miles. We did it! And when I say "we", I really mean it. No way I would have completed this without a great deal of help from some very good friends.

Friendship can be defined in a lot of different ways. One of the most beautiful examples I can think of was Sancho Panza's friendship for Don Quijote. Sancho didn't ignore that his friend was crazy-- Sancho knew that Don Quijote was crazy. Sancho supported him anyway:

"God's will be done," said Sancho. "I'll believe all your worship says, but straighten yourself a bit in the saddle...."

No one tried to talk me out of tilting this windmill. On the other hand, plenty of people helped me stay straight in the saddle! Maybe I should call you all my "enablers" instead of my "supporters"!

Thanks to all of you. Andrew, Pedro, and Fabricio who met me at mile 31 and paced me for the next 15 miles. Claire and Marcia ran with me on Adams track, and way back in August did not ask me whether I was going to do this-- they just asked me "when?" And Phoebe, who helped me steer through the details of permissions, and who got me through the OCC board last year. Jerome, Dana, Joy, and Gina, who don't even have children in the school came and cheered me on. Ibis and Karen and Maria who cleared me to use the track for the last ten miles of the run-- and thank you, Ibis for opening up and cheering me on as well. Donna, David, Maricarmen, Paco, Susana, Scott, Elisa, Diego and Marta came and ran a few laps and helped out Sonia-- you guys are our constant friends and we're always grateful for you. And thank you to everyone who contributed to the OCC-- Oyster-Adams is a better school because of you. If you're still thinking about donating, you can click here to do so.

And of course, thank you to Sonia and Daniel. What kept me going was knowing you'd be at the finish line.

It went so much better this year than last. It mostly was just better training-- I came a lot better prepared this time, having done more and longer long runs, having tested all of my equipment better, and having been much more consistent in my training. Also, just having the experience of completing the distance last year was a big psychological help. But, whereas last year I had to walk a lot after the 30 mile point, this year I was able to run the entire 46 miles. I even was able to run the last three miles at a pace near 9 minutes per mile.

Now I get to think about what to do next. Sonia has already started talking about next year-- so I guess I have the most important permission already! Now that I've done two of these, I know that it's not just a fluke. If I do a third one, I suppose I can consider it a habit. With it going so well this time, I'm wondering if I shouldn't try to get in at least one official ultramarathon a year besides this one. Hmmm.

But for now, I guess I should concentrate on recovering to the point that I can go downstairs without gritting my teeth!

My checklist for the Daddy's Crazy 46-Miler

So much to remember. Here is my checklist of essentials for tomorrow's 46 mile run:

Fannypack:
IBUPROFEN
Lactic acid pills
Gels
Powerbars
Washcloth
Money
Keys
Lip balm
Batteries
Camera
Toilet paper
Whistle
Credit card, metro, license
Chalk or 16 spoons, whiteboard

Clothing:
Sweatshirt
Vamos T-shirt
Arm sleeves
Knee band
Camelback (Put Electrolyte in H2O)
Headlamp, tail-lamp
RoadID
Socks
Pants
Skivvies

Finish bag:
Water
Warm clothes for finish
Towel for finish
Potato chips
Something sweet

Tasks:
Plug in HRM tonight
HYDRATE tonight!
Breakfast morning
Take ibuprofen morning

Stock house for recovery:
Sonia, Daniel, Sparky
Ice cream, chili, pyjamas

Daddy's Crazy 46-Miler: One week to go!

I'm down to seven days until the Daddy's Crazy 46-Miler. I feel like I'm ready: my training has gone well, I have my route well thought out, I know what my strategy is for the run. I even got interviewed by the Washington Post Express-- it should be on the newstands next Tuesday, I think. And yet it doesn't feel very real for me yet. When I did this a year and a half ago... it just feels like my life has changed so much since that time. We bought a house and moved from Woodley Park to Glover Park. Sonia started working at the State Department and I started walking Daniel to school in the mornings. The big snow came. I got a big promotion at FEMA. We adopted Sparky.

My dad died.

I suppose life was just different. And last time, I really wasn't certain it was possible for me to finish the run. This time, I know it's likely I'll finish it. And I'm pretty sure I'll get through it in a lot better shape this time.

One thing that keeps running through my mind is the question Vicky Hallett asked me last week in the interview: "Why are you doing this?" I don't remember my exact answer, but it was something like, "because I have to... I can't imagine not doing this." Even when I told Vicky this, I knew I wasn't fully satisfied with the answer. I wish I had said that I love the feeling of running for hours in the dark by myself. Or that the way I lived my life sixteen years ago, I used to dream about doing things like this but never could have. Or that running a long way, for me, is kind of a celebration of the fact that I'm free to do this kind of thing now.

I did explain that my head clears when I'm out there running. That running is like meditating for me, and that when I'm having a good day, the monkeys in my head stop chattering for a while. I didn't tell her that it's like a long prayer for me; it is. I wish I could explain that better. But it certainly is true, and it's a big reason for my running.

Anyway, seven days to go. And really, I can't wait to get out there and start running.