Weighted least squares

... or Why can't we just be ordinary squares?

In fitting your linear model, you may be interested in generating a prediction line that describes the relationship between your predictor(s) and your outcome. If you have constant variance in the errors (homoskedasticity), an ordinary least square (OLS) approach is used to fit the model to the data and generate a best fit line. A best fit line essentially minimizes the distance between the observed data and the predictions made by the model. If your data shows constant variance in the errors AND the errors are normally distributed, then OLS is the maximum likelihood estimator.

However, in spatial statistics (the analysis of data with a spatial component that considers spatial dependency) we often use data that violate the rule of error constant variance (heteroskedasticity). In this case, we use weighted least square (WLS) to fit the model to the data and generate a best fit line. In WLS, the error assumptions are that errors are normally distributed with mean vector 0 and nonconstant variance-covariance matrix σ2W, where W is a diagonal matrix. See this post from Penn State for a short intro to the nonconstant variance-covariance matrix.

Interpretting linear models in R

If you're new to R and stats, check out this awesome post over at the yhatq blog. It walks you through everything, from the code to the analysis, in simple, straight-forward language with code output.

On residuals:

If our residuals are normally distributed, this indicates the mean of the difference between our predictions and the actual values is close to 0 (good) and that when we miss, we're missing both short and long of the actual value, and the likelihood of a miss being far from the actual value gets smaller as the distance from the actual value gets larger.

On variable p-values:

Probability the variable is NOT relevant. You want this number to be as small as possible. If the number is really small, R will display it in scientific notation. In or example 2e-16 means that the odds that parent is meaningless is about ¹⁄_{5000000000000000}

On R-squared:

Metric for evaluating the goodness of fit of your model. Higher is better with 1 being the best. Corresponds with the amount of variability in what you're predicting that is explained by the model.

On the F-test and resulting F-stat:

This takes the parameters of our model (in our case we only have 1) and compares it to a model that has fewer parameters (sic). In theory the model with more parameters should fit better. If the model with more parameters (your model) doesn't perform better than the model with fewer parameters, the F-test will have a high p-value (probability NOT significant boost). If the model with more parameters is better than the model with fewer parameters, you will have a lower p-value.

All quoted text from post "Fitting & Interpreting Linear Models in R" by yhat, published May 18, 2013 at http://blog.yhathq.com/posts/r-lm-summary.html

Fun with GLM links

I can never keep this straight: When to use link(log) vs. link(logit) in generalized linear models:

Use link(log) when you are after relative risks.
Use link(logit) when you are after odds ratios.

Pissing off Heidi Klum

"Essentially, all models are wrong, but some are useful"

- George EP Box, Empirical Model-Building and Response Surfaces

Simply Statistics

Blog recommendation: simplystatistics.tumblr.com

Graphing in R

Frank McCown outlines a quick and simple intro exercise for graphing in R:

http://www.harding.edu/fmccown/r/

A few notes to keep in mind as you run through the exercise:

• If you need a refresher on creating tab delimited files, ehow has you covered:
http://www.ehow.com/how_5079116_create-tab-delimited-files.html
• If you save your tabbed file with .tab per the ehow instructions, make sure to edit Frank's code:
autos_data <- read.table("C:/R/autos.dat", header=T, sep="\t")
#becomes
autos_data <- read.table("C:/R/autos.tab", header=T, sep="\t"
• Make sure you're using forward slashes when you set the file path! This post discusses that pesky character string error.

On models

(Not of the Vogue persuassion...)

From Wikipedia:

Deterministic vs. probabilistic (stochastic): A deterministic model is one in which every set of variable states is uniquely determined by parameters in the model and by sets of previous states of these variables. Therefore, deterministic models perform the same way for a given set of initial conditions. Conversely, in a stochastic model, randomness is present, and variable states are not described by unique values, but rather by probability distributions.

So for a coin toss, the probabilistic modeler would flip a coin 10 times, record the outcomes, and predict the outcome of the next coin toss based on that probability calculation. The deterministic modeler, on the other hand, would create a model based on data from the physical environment to predict a coin toss. The model may consider air pressure, air movement, the force of the toss, the weight of the coin, the color of the coin tosser's shirt, etc... any available data that may influence the outcome of a coin toss.

In related news, I'm now receiving unsolicited stats jokes. Here's the latest:

By the time Ted arrived at the football game, the first quarter was almost over. "Why are you so late?" his friend asked.

"I had to toss a coin to decide between going to church and coming to the game."

"How long could that have taken you?"

"Well, I had to toss it 14 times."

Now, what are the chances...