Open 2017 Data

I just finished doing the download of the 2017 Open Results. It should have been done several weeks ago, but life happens. There was an extra challenge too in that redesigned their website right before the kickoff of week 1, meaning a whole new set of BeautifulSoup .find() calls to extract the results from the leaderboard. But after doing the same think in Chrome developer tools that I did to find Affiliate data before, I found the handy API calls and structure of the returned JSON from the leaderboard database.

Both the results for 2017 and the Python scripts from extracting straight from the new Games homepage are now up on Github. Anyways, enjoy!


Open Data from 2012-2016

Registration from the CrossFit Open 2017 has begun. Continuing from my previous post, I have finished pulling data for the Open events from 2012 to 2016. I excluded 2011 (the first official year) due to the low participation and data availability. Additionally, as I am looking for trends and additional divisions have been added each year, I only downloaded the data for the core Men and Women’s Rx’d divisions each year. That is, the Master’s and Teen divisions didn’t exist in 2012, so I didn’t download those for 2016. That will eliminate some performance data from individuals who crossed into other divisions, but it should provide consistent enough data points for those in the “core” divisions for me to conduct valid analysis.  Continue reading

Crossfit Open Data

Admittedly, I’m a CrossFitter. I even coach part-time at CrossFit Hierarchy Ivy City (stop by and check us out if you are in NE DC). Between my day job and my part-time job and my own workouts, I decided to take a break from the Android App I’ve been working on to play around with Python data analysis for something that my tie in to other areas of my life.

CrossFit + Data Analysis?

Every year, CrossFit HQ hosts the CrossFit Games. In order for athletes to qualify for the Games they have to complete the five-week CrossFit Open in the top 50 for their region then finish in the top five of their multi-day Regional event. Over 200K men and 140K women of all shapes and ages competed in the CrossFit Open in 2016. The beautiful thing for data nerds, is that all that data is freely available online. Additionally, every registered athlete has a profile page on where they can list their gym, height, weight, max lifts, and benchmark scores. Realizing this, and considering the number of open source data analysis libraries available for Python, I really want to see about playing with that data. In fact, several people have worked on CrossFit Open Data analysis over the years (,,, etc).

Continue reading

AdvRoutes, Inc 1

“No Battle Plan survives contact with the enemy.” -Helmuth von Moltke

I’ve managed to carve out enough time to hammer out an initial version of the previously described Android App to help motorcycle route planning (dubbed AdvRoutes). From the last post I refined the requirements as I see them and created a backlog of tasks that will need to be achieved before I reach my product. Using Agile terminology, this marks the end of a Sprint. Using Defense Acquisition terms the initial product would in turn be the first Increment using Spiral Development to incrementally add features in fully functional products. Continue reading

Be Prepared for the Unexpected

“There a known knowns, known unknowns, and unknown unknowns.”
–Hon. Donald Rumsfeld

Following up from the previous post, I built a simple Python program to read in our SampleEmailData.csv file. The file has two columns (Subj; Body) with the rest of the metadata removed. The data set is composed of some actual emails (sanitized) that are representative of the types of emails received in a real-world operating environment. To recap the previous post, our task is as follows:

Task: Automatically extract contact information from archived emails. Continue reading

Task, Purpose, Method, Effects

“In preparing for battle I have always found that plans are useless, but planning is indispensable.” –Gen. Dwight D. Eisenhower.

When planning for military operations, the Commander’s Intent is detailed in the Mission Statement. This mission statement is designed to emphasize the essential tasks, their purpose, the method for achieving them, and the intended end-state that may be observed in order to evaluate the achievement of effects. While this concept is stressed to junior officers learning about planning processes, it is something that may be (and often subconciously is) applied to any project. Continue reading