Regression Analysis

PRESENTATION
ON

REGRESSION ANALYSIS

Introduction to Regression Analysis
• Regression analysis is the most often applied technique of
•

•

•

statistical analysis and modeling.
If two variables are involved, the variable that is the basis of
the estimation, is conventionally called the independent
variable and the variable whose value is to be estimated+ is
called the dependent variable.
In general, it is used to model a response variable (Y) as a
function of one or more driver variables (X1, X2, ..., Xp).
The functional form used is:

Yi = 0 + 1X1i + 2X2i + ... + pXpi + 
• The dependent variable is variously known as
explained variables, predictand, response and
endogenous variables.
• While the independent variable is known as
explanatory, regressor and exogenous variable.

Definition
The Regression Analysis is a technique of studying the dependence of one variable
(called dependant variable), on one or more variables (called explanatory variable),
with a view to estimate or predict the average value of the dependent ariables in terms
of the known or fixed values of the independent variables.

The regression technique is primarily used to :
• Estimate the relationship that exists, on the average, between the dependent variable
and the explanatory variable
• Determine the effect of each of the explanatory variables on the dependent variable,
controlling the effects of all other explanatory variables

• Predict the value of dependent variable for a given value of the explanatory variable

HISTORY
The term "regression" was coined by Francis Galton in the nineteenth century
to describe a biological phenomenon. The phenomenon was that the heights
of descendants of tall ancestors tend to regress down towards a normal average
(a phenomenon also known as .egression towards the mean] For Galton,
regression had only this biological meaning, but his work was later extended
by Uday Ule and Karl Pearson to a more general statistical context.I n the work
of Yule and Pearson, the joint distribution of the response and explanatory
variables is assumed to be Gaussian. This assumption was weakened by R.A
Fisher in his works of 1922 and 1925 Fisher assumed that the conditional
distribution of the response variable is Gaussian, but the joint distribution
need not be. In this respect, Fisher's assumption is closer to Gauss's
formulation of 1821.

Assumptions of the Linear
Regression Model
1.
2.
3.
4.

5.
6.
7.
8.
9.

Linear Functional form
Fixed independent variables
Independent observations
Representative sample and proper specification of
the model (no omitted variables)
Normality of the residuals or errors
Equality of variance of the errors (homogeneity of
residual variance)
No multicollinearity
No autocorrelation of the errors
No outlier distortion

Derivation of the
Intercept
y  a  bx  e
e 

y  a  bx

n

e
i 1



i

n


i 1

yi

n

a



i 1

i

n

 b  xi
i 1

n

Because by definition

e
i 1

0 

n


i 1

n



ai 

i 1

n



yi

 b

i 1



y

i

n

 xi
n

yi

 b  xi

 bx

i 1

n

 b  xi

i 1

n



a
i 1

i 1

na 
a



yi

n

i 1

i

 0

The term  in the model is referred to as a

“random error term” and may reflect a number of
things including the general idea that knowledge
of the driver variables will not ordinarily lead to
perfect reconstruction of the response.

• If there is only one driver variable, X, then we
usually speak of “simple” linear regression analysis.
• When the model involves
• (a) multiple driver variables,
• (b) a driver variable in multiple forms, or
• (c) a mixture of these, the we speak of “multiple linear
regression analysis”.

• The “linear” portion of the terminology refers to
the response variable being expressed as a “linear
combination” of the driver variables.

EXAMPLE
An agronomist may be interested in studying the dependence of paddy on
temperature, rainfall, amo0nt of fertilizer and soil fertility. such a dependency
analysis maybe enable the forecasting of the average yield, given information
about the explanatory variables

In regression analysis, the data used to describe the relationship
between the variables are primarily measured on interval scale.

the chief advantage of using the interval level of measurement is
that, with such data it is possible to describe the relationship
between variables more exactly employing mathematical
equation. This in turn allows more accurate prediction of one
variable from the knowledge of the other variables, which is one
of the most important objectives of regression analysis.

It is important to note that if the relationship between

X and Y is curvilinear , the regression line will be a
curved line rather than straight line. The greater the
strength of relationships between X and Y the better
is the prediction.

The problem is presented to the mathematician
as follows: "The values of a and b in the linear
model Y'i = a + b Xi are to be found which
minimize the algebraic expression ."
The mathematician begins as follows:

The result becomes:

Using a similar procedure to find the value of a yields:

Yi

XiYi

13

23

169

299

20

18

400

360

10

35

100

350

33

10

1089

330

15

SUM

Xi2

27

225

405

91

113

1983

1744

THE REGRESSION MODEL
The situation using the regression model is analogous to that of the
interviewers, except instead of using interviewers, predictions are made by
performing a linear transformation of the predictor variable. Rather than
interviewers in the above example, the predicted value would be obtained by
a linear transformation of the score. The prediction takes the form

where a and b are parameters in the regression model.

EXAMPLE USES OF REGRESSION MODELS
Pregnancy
A woman in the first trimester of pregnancy has a great deal of
concern about the environmental factors surrounding her pregnancy
and asks her doctor about what to impact they might have on her
unborn child. The doctor makes a "point estimate" based on a
regression model that the child will have an IQ of 75. It is highly
unlikely that her child will have an IQ of exactly 75, as there is always
error in the regression procedure. Error may be incorporated into the
information given the woman in the form of an "interval estimate."
For example, it would make a great deal of difference if the doctor
were to say that the child had a ninety-five percent chance of having
an IQ between 70 and 80 in contrast to a ninety-five percent chance
of an IQ between 50 and 100. The concept of error in prediction will
become an important part of the discussion of regression models.
It is also worth pointing out that regression models do not make
decisions for people. Regression models are a source of information
about the world. In order to use them wisely, it is important to
understand how they work.

Types of regression analysis:
Regression analysis is generally classified into two kinds: simple and
multiple. Simple
regression involves only two variables, one of which is dependent variable
and the other
Is explanatory(independent) variable. The associated model in the case of
simple
regression will be a simple regression model.
•A regression analysis may involve a linear model or a nonlinear model.
The term linear can be interpreted in two different ways:
1. Linear in variable
2. Linearity in the parameter

Regression Analysis: Model Assumptions
Model assumptions are stated in terms of the random
errors, , as follows:
the errors are normally distributed,
with mean = zero, and
constant variance 2, that does not depend on the settings
of the driver variables, and
the errors are independent of one another.

This is often summarized symbolically as:  is NID(0,
2)

LINEAR REGRESSION
In linear regression, the model specification is that the dependent variable,
yi is a linear combination of the parameters (but need not be linear in the
independent variables). For example, in simple linear regression for
modeling n data points there is one independent variable: xi, and two
parameters, β0 and β1:

Fig: Illustration of linear regression on a data set

In the case of simple regression, the formulas for
the least squares estimates are

Regression Analysis

More Related Content

What's hot

Viewers also liked

Similar to Regression Analysis

More from Salim Azad

Recently uploaded

Regression Analysis