PML-WFDJ.Rmd

---
title: "PML Project: Weight Lifting Activity Recognition"
author: "WhitefishDontJump"
date: "June 16, 2015"
output: 
  html_document: 
    highlight: tango
    keep_md: yes
    theme: journal
    toc: yes
---

The source of data for this project is:

Velloso, E.; Bulling, A.; Gellersen, H.; Ugulino, W.; Fuks, H. **Qualitative Activity Recognition of Weight Lifting Exercises.** Proceedings of 4th International Conference in Cooperation with SIGCHI (Augmented Human '13) . Stuttgart, Germany: ACM SIGCHI, 2013

Read more: http://groupware.les.inf.puc-rio.br/har#ixzz3MCvfqKcP


# Summary

The objective is to predict 'classe', a factor with 5 levels, from the reference data.  The data from JHU/Coursera course is contained in a file 'training.csv', that will be used for building, validating and testing models. The second file 'testing.csv' lacks any classe values and will only be used for the submission test on coursera.

1. Initial cleaning:  Clean data, removing various flavors of NA and columns containing values that are not relevant to current project to correctly predict classe variable.

2. Determining variable importance: Initial rough model on 20% of 'training' set, using randomForest() with 3-fold cross-validation, with chief purpose to determine relative importance of features.

3. Second model: use a reduced set of predictors, based on varImp(), applying randomForest with 3-fold CV to 50% of data. Examine results and initial "out of bag" errors.

4. Perform additional validation by predicting and evaluating the second model accuracy against remaining 30% of data from 'training' set.

5. Predict 20 cases for the submission portion of project and report results.

Note: I recorded and report the system time required to run each model, and the time to process all the code chunks in this report. Both my hardware and software configuration are also reported for reference.


***Summary Results***
The second model, a 24 predictor Random Forest, validated with 3-fold cross validation, and an estimated out of sample error of ~1%, achieved 99% accuracy on an additional out of sample test. It also scored 20 of 20 correct on the course submission test. 

# Data Cleaning & Exploration

## Data Cleaning

```{r getcleandata}

      startknit <- Sys.time()  ## start time, see end of report total time

      sourceData <- read.csv("pml-training.csv", 
                         na.strings = c("NA", "", "#DIV/0!"))

      submitTestSet <- read.csv("pml-testing.csv", 
                        na.strings = c("NA", "", "#DIV/0!"))
      # View(sourceData)
      
# remove NA columns both sets
      cols2get <- colSums(is.na(sourceData))==0
      cols2get2 <- colSums(is.na(submitTestSet))==0
      cleanData <- sourceData[, cols2get]
      cleanSubmit <- submitTestSet[,cols2get2]
      
      ## View(cleanData)

# remove row "X", as well as time date window columns which 
# are not related to prediction of 'classe'. I will keep 
# 'user_name' in the set - it may or may not be important 
# in the model response.
      
      cleanData <- cleanData[,c(2,8:60)]
      cleanSubmit <- cleanSubmit[,c(2,8:60)]

```

## Data Exploration

Based on cursory examination of the first few columns of the data with exploratory plots, it is evident that several may be important predictors.
Note: these are a small sample of plots that were done and the code for all plots is printed in the Appendix.

```{r exploratory,echo=FALSE,message=FALSE,fig.width=8,fig.height=3}
      require(ggplot2)
      qplot(roll_belt,pitch_belt, data=cleanData, color = user_name,
            main="Plot1: Roll Belt and Pitch Belt by user_name")

      qplot(pitch_arm,roll_arm, data=cleanData, color = classe,
            main="Plot 2: Roll Arm and Pitch Arm by classe")
      
      qplot(pitch_arm,roll_arm, data=cleanData, color = user_name,
            main="Plot 3: Roll Arm and Pitch Arm by user_name")
      
      qplot(pitch_arm,yaw_arm, data=cleanData, color = classe,
            main="Plot 4: Yaw Arm and Pitch Arm by classe")

# the following plots were not shown for the report
#      boxplot(cleanData)
#      boxplot(roll_belt~classe,data=cleanData, main="Roll Belt by Classe")
#      boxplot(pitch_belt~classe,data=cleanData, main="Pitch Belt by Classe")
#      boxplot(yaw_belt~classe,data=cleanData, main="Yaw Belt by Classe")
#      qplot(yaw_arm,roll_arm, data=cleanData, color = classe,
#            main="Roll Arm and Yaw Arm by classe")
#      qplot(roll_belt,pitch_belt, data=cleanData, color = classe,
#           main="Roll and Pitch Belt by classe")
#      qplot(roll_belt,yaw_belt, data=cleanData, color = classe,
#            main="Roll and Yaw Belt by classe")
#      qplot(yaw_belt,pitch_belt, data=cleanData, color = classe,
#            main="Yaw Belt and Pitch Belt by classe")
```

Comments about plots 1-4: In plot 1, the differences between users are clear, so I will retain user_name as a feature in the initial rough model. In plot 2, it appears that classe may be more responsive to the absolute values and a non-linear relationship between the two variables. Plot 3, like plot 1, shows differences by user_name. In plot 4, there is some separation of classes D and E, together, from the other classes.

# Modeling

## Pre-modeling (partitions and train control)

```{r createpartitions,message=FALSE}

# doParallel with 2 cores reduced runtime ~30%.      
      require(parallel)  
      require(doParallel)  
      mc <- makeCluster(detectCores())
      registerDoParallel(mc)

      
      require(caret)
      
# create 3 data partitions: 
#     20% initial model(trainset1) 
#     50% second model(trainset2)
#     30% validation(validset)
      
      set.seed(616) # for repeatability
      
      trndx20 <- createDataPartition(cleanData$classe,
                                     p = 0.2, list = FALSE)
       
      trainset1 <- cleanData[trndx20,] # contains 20 % of rows
      tempset1 <- cleanData[-trndx20,] # contains 80 % of rows
       
      trndx50 <- createDataPartition(tempset1$classe,
                                     p = (0.5/0.8), list = FALSE)


      trainset2 <- tempset1[trndx50,] # contains 50% of original rows
      validset <- tempset1[-trndx50,] # contains 30% of original rows
      
# common train control parameters will be used on all train() . . .
      
      controla <- trainControl(method="cv", 
                             number = 3, 
                             allowParallel = TRUE)
  
```

## Initial Random Forest Model, all predictors, 20% of observations

```{r initialmodel, message=FALSE}
      
      start1 <- Sys.time()
      
      model1 <- train(classe~.,data=trainset1, 
                    method="rf",
                    trControl = controla)
      
      round(Sys.time() - start1,2) # model1 run time
      
      model1
      
      model1$finalModel

      varImp(model1)
      

```



```{r VarImpPlot,echo=FALSE,fig.height=9,fig.width=7}
    
        plot(varImp(model1), main="Ranked variable importance in model 1")

```

The predictor user_name isn't important in this model, so my hypothesis that the user_name would be important is rejected.

Given model1 high accuracy (confusionMatrix), I will use the varImp() results from model1 to reduce the feature set and run a second model on the 50% partition named 'trainset2'. I will use scaled importance value of 10 as the cut-off, which will remove user_name and many other predictors.


## Model 2:  Random Forest on 50% partition with reduced feature set

```{r model2,message=FALSE}
      
# reduce feature set to those with imp >= 10.0
      
      importance <- varImp(model1)$importance # get varImp()
      importance$vars <- rownames(importance) 
      importance <- importance[order(importance$Overall, 
                                     decreasing=TRUE),]
      impCols <- importance[(importance$Overall >= 10.0),2] # subset
      
      impCols
      
# create second model
      
      trainset2 <- trainset2[,c(impCols,"classe")]
      
      start2 <- Sys.time()
      
      model2 <- train(classe~.,data=trainset2, 
                    method="rf", 
                    trControl = controla)
      
      round(Sys.time()-start2,2) # model2 run time 
      
      model2
      
      model2$finalModel
      
# validation of model 2 on balance of data
      
      validate <- predict(model2,validset)
      confusionMatrix(validate,validset$classe)
      
```

Model 2, a 3 fold cross validated Random Forest, using 1/2 of the data, was validated, with 99% accuracy on an out of sample test, using 30% of the data.
This model used 24 variables as features which are listed earlier in the report as 'ImpCols'.

## Submission Test

```{r submission}
      
      answer20 <- predict(model2,cleanSubmit)
      answer20
      
# results were submitted and scored 20 of 20 correct.

```

# Appendix

This appendix has:

1. code for plots.

2. hardware & session information for reproducibility.

3. total elapsed time to knit the RMD, including all models, text and charts.

## Plot Code

```{r plotcodeonly,ref.label="exploratory",eval=FALSE}

```

Variable Importance plot:

```{r plotcodeonly2,ref.label="VarImpPlot", eval=FALSE}

```


## Hardware and Session Info

My hardware: HP/Compaq C770US.
Pentium Dual CPU T2390 @ 1.86GHz with 3 GB DDR2 ram. 

```{r config}

      print(sessionInfo(),locale=FALSE)  
      
      
      
```

## Elapsed Time to knit all chunks

```{r elapsedtime}
      
      round(Sys.time() - startknit,2)  ## time  to knit all chunks

```


---
end of report.  Thank you for your review and feedback.