From OpenOffice noob to control freak: A love story with R, LaTeX and knitr

Lately I had to write a seminar paper for a class and I decided to overdo it.
But let's start at the very beginning. Here is my evolution of how I used to write stuff and how I got from this:


to that:



School: 
OpenOffice - I guess everyone has some youthful indiscretions.
I remember how much time i spent trying to position each figure correctly and trying to make every line in the table of contents to start at the same position.


1. Semester: 
I heard of this "LaTeX" - thing and there was a rumor that this might be useful throughout the whole time at university, at the latest for the bachelor thesis. So I decided to learn LaTeX from the very beginning and chose to write a formulary for descriptive statistics as a first project.
The result was very neat. So proud of myself.
Started to missionize other students.

2.- 5. Semester:
Then I continued to use LaTeX for almost every document. Mostly I did a lot of presentations with the beamer class. Most of the creation time spent choosing the best themes and colors, and creating the perfect title slide.
I discovered Sweave, but did not want to use it and still copied R output into my documents and included figures manually.
Also switched from gedit to emacs about that time.

6. Semester:
Bachelor thesis time. The first time I needed to embed more R-Code into a document. I wanted my R-Code to look good. So I searched and found: "Hm ... what's knitr? ... want to meet my bachelor thesis?"
At the same time I learned subversion. Felt odd to use, but at the same time I felt there was a deeper power in revision tools.

1. Master Semester: 
Revision Control: Level Up! Now working with git.
Facebook update: Now in a bigamous relationship with knitr and emacs.


But now back to my latest paper. I used different tools to create it. Here are the ingredients plus why I used them:


LaTeX

In my opinion, there is no alternative to LaTeX for scientific paper including at least one number or one figure. There have been many discussions about LaTeX vs. Word (or Word-alikes). I like the personal summary in this blog.


Tufte book documentclass

A very fancy documentclass for LaTeX I tried for the first time. Most Latex-adversaries I know complain about every slide show or paper looking the same and I have to admit that's true. Of course you can build your own documentclasses and styles from scratch, but I really don't feel like it. Fortunately there are already some other classes around, like the fancy Tufte package, which I like very much. Edward Tufte (statistician) has done a lot in the fields information design and data visualization.
The Tufte book document class is based on the style Tufte wrote his books in.
The most significant impression is the broad margin, which can be used for notes, references and pictures.


R

Any analysis we are doing at university (I am studying statistics) is done with R. There are of course alternatives like Julia, Matlab, Python, ...
R might not be the fastest language and from my subjective feelings it is a little messy.
But I definitely love the huge ecosystem of packages, which are making R a pleasure to use. Also the plot functionality is amazing (I am kind of a ggplot2 fanboy)


knitr

Speaking of the ecosystem, knitr is one of the best examples for useful add-on packages. knitr makes integrating R code and output in documents simple and even fun. It is based on Sweave, but eliminates some problems and extends the functionality. I also like the fact that you are not tied to LaTeX documents, but you can, for example, also write markdown files and convert them to html.


emacs

Personally I am using emacs a lot. Though I think RStudio evolved very nice as well. The integration of knitr is very neat, better than in emacs. But I feel very comfortable in emacs, that's why I haven't switched to RStudio yet.


git

A revision control system. Mostly used for coding, but this time I used it for my seminar paper. It is not necessary to use revision control for a paper, but it definitely has some benefits. First of all it is a good feeling to be able to revert back to older revisions, for example in case you accidentally deleted something. One cool side effect is, that you automatically begin to split bigger tasks in small steps, because every time you commit something, you are encouraged to write a short text about what changed.


That's basically my my status quo of tools I use for papers and presentations.
My repository for the paper is public, just visit my github account. Here is the PDF.

By the way, the paper I wrote is about conditional inference trees, where I also did a presentation.

Predictive analytics: Some ways to waste time

I am starting to take part at different competitions at kaggle and crowdanalytics. The goal of most competitions is to predict a certain outcome given some covariables.  It is a lot of fun trying out different methods like random forests, boosted trees, gam boosting, elastic net and other models. Although I still feel like not being very good with my predictions, I already learned a lot.

As I am a student I have quite some time for these competitions, which is really great, but the time I can spend is nonetheless limited. So it is in my interest to work efficiently. At least the technical part like pre processing of the data or creation of the folder structure is often the same and can be done quickly. I rather want to spent most of the time learning new methods.  In my first competitions I wasted some time with different technical things.
As an exemplary statistician I naturally use R.  So the short code snippets are in R.

Here are my tops of how I wasted time:

1. Forget to remove the ID variable from your trainings data set.

Especially for random forests (with packages randomForest or party) this is a pain, because the ID variable has a lot of different values, so there are a lot of possible values to do the split for a decision tree and the variable will always have a high importance in your random forest. So just using
train[-which(names(train) == "ID")], where train is your training data set, helps out of this situation.


2. Do data preprocessing separately for test and training data set, because copy and paste of  code was and always will be a good programming style.

This seems to be very obvious, but it can spare some time to do a call to
all <- rbind(train, test)
after reading in test and training data set. After binding both together you can impute your data, do variable transformation and so on.  Afterwards you can simply make a subset from your dataset and assign all rows from all, which have NA in the response variable, to the test data frame and all with values different from NA to the training data set.
Maybe there is  a reason (which I do not know yet) why it should be done separatly, but as long as I do not know this reason, I will do it like I described.


3. Do not check your predicted values for plausibility, but submit blindly

It is easy to make a technical mistake in fitting your model or making your prediction. A classical one is to forget to choose type = "response" when you, for example, have fitted a logistic regression model and want to predict your test data. In this case your prediction will be the linear predictor and the values will not make much sense. You can easily see with a quick summary() of your predicted values, if they are plausibel. In the logistic regression case you would recognize, that the values are not in the interval between 0 and 1. This can save you a precious submission slot. In some competitions you can make one submission per day and it is really annoying if you wasted one and it could have been easily avoided.


4. Don't save your final model, but only the predicted values

I find it useful to save every reasonable model you have fitted, random forest you have computed or regression you have boosted. Not only the predicted values are interesting. Especially for random forests or boosting methods the important/ chosen variables can be of major interest. So don't forget to save the model which maybe took some hours to calculate.  Don't throw hours of computing time easily away.



5. An algorithm is very slow? Makes sure you run it with all variables and maximum possible iterations

This is a quite general remark. If you have a e.g. for-loop, which fits models and saves some values in, let's say a data frame and you have a little mistake in your code, which first throws an error at the very end of the procedure, you are going to have a bad time. It is better to test an algorithm with just a few iterations and/or a subset of variables at first to get a feeling of how much time it will take and to see if your code works as expected.

6. Don't document your models or anything, because you are just trying things out

I recently started to save each model with a talking filename (for example glmnet-alpha-0.1-lambda-0.01-reduced-data.RData), cleanly commented my R-Code and also gave my predicted values a talking filename. That was a huge improvement to my sloppy working style I had at the beginning, because it's much easier to keep track on what you already have done. Especially for longer lasting competitions the amount of different data sets, models, R-codes and submissions can become large. So it is good practice to have a consequent folder structure and give every file a self-explaining name. Even if you are just trying one thing out, take the time, write the piece of program properly and don't produce countless messy files.



These are just a few things which improved and speeded up my workflow. My future plan is to become friends with git and github.

I hope this post can be a help for other people, who are getting started with predictive statistics, to save some time.

 If you have things to add to the list, please comment.