GB Exam 2

regroups methods to attempt to learn about distributions where the variables that can be split into categories

explanatory variables, predictors, regressors, independent

outcomes, response variables, labels, dependent

quantifies a linear relationship between two variables, y= intercept + slope * X

higher is better, discrete y cases, P(Y test/ X test)

lower, (1/n)E I Yi - Yhat I

lower, (1/n)E I (Yi-Yhat)/(Yi) I

lower, greatly prenalized by outliers, sqr root((1/n)E(Yi-Yhat)²)

how much accurately we can estimate the outcome variable with the explanatory variable, R²= 1-(SSErr-SSTot)

sum of squared error from the regression, Represents the total amount of variation that we can’t explain with our regression, SSE trend line were plotted at the average = SST (Sum of Squares in Total)

minimize the SSE loss

• Closer to 1 = explain a lot of the variations in Y with our regression

• Closer to 0 = can’t explain the variations in Y better with our regression

On average, an increase in study time by 1 hour is associated with an increase in grade by 5.2 points, everything else being equal.

On average, when a student spent 0 hours studying and skipped 0 classes, we expect their grade to be 57 points, everything else being equal.

how likely our data has no effect/relationship, low p-value = more confidence

ordinary lease square regression, relationship between X & Y is linear, estimates are predictions are denoted with a hat, coefficient are obtained by minimizing the sum of squared residuals

could be outside range of data or unrealistic, or both then extrapolate

Statistically significant at a confidence level if p-value < alpha

Extends the linear regression approach by allowing the distribution to be non-normal

becomes % and standard deviations

we can explain 24.5% of the variations in grades by looking at the variations in both the number of hours of study and in the number of class skipped

linearity, independence, normality (errors), equal variance

allows distribution to be non-normal, the mean Y to be function of a linear combination of Xs

link function

linear relationships

when the mean needs to be positive

cured relationships

a lot of averages, bell shaped, can be negative

a lot of times, potentiall skewed, always positive

probability of an event happening, binary, either 0 or 1

used for a lot of counts, positive integers

For cases with different number of variables across models, lower is better

the model is too flexible, great fit on training data, poor fit on new data

not flexible enough, poor fitting on training and new data

bias, poor prediction performance, inability to capture the complexity of some patterns

restricting the flexibility of a model

estimate on a training set, adjust on a validation set, test prediction performance with a test set.

use dimension reduction, solve overfitting issues, interpretation is still difficult, keep extra variables with variables selection

Method where variable selection is performed through regularization. It shrinks the coefficients towards 0

the strength of regularization, if 𝜆 is large the coefficient will be different from 0 𝜆 controls𝜆 controls

sensitive to x, issues with small datasets, scale sensitivity, loss of interpretability, bias

create groups based on thresholds on X values

don’t need to specify the relation between x and y, works for regression and classification, very easy to explain, mirrors decision making, graphs

don’t have the same prediction accuracy as other methods