Miskolci Egyetem Gazdaságtudományi Kar Üzleti Információgazdálkodási és Módszertani Intézet. Correlation & Regression

Átírás

1 Correlation & Regression

2 Types of dependence association between nominal data mixed between a nominal and a ratio data correlation among ratio data

3 Correlation describes the strength of a relationship, the degree to which one variable is linearly related to another Regression shows us how to determine the nature of a relationship between two or more variables X (or X 1, X,, X p ): known variable(s) / independent variable(s) / predictor(s) Y: unknown variable / dependent variable causal relationship: X causes Y to change

4 Correlation Measures 1. Covariance. Coefficient of correlation 3. Coefficient of determination 4. Coefficient of rank correlation

5 1. Covariance A measure of the joint variation of the two variables; An average value of the product of the deviations of observations on random variables from their sample means. C x, y x xy y n 1 ranges from - to +; C = 0, when X and Y are uncorrelated; its sign shows the direction of correlation it doesn t measure the degree of relationship!!!

6 . Linear correlation coefficient r C s s x y = Σd d x x d d y y Pearson correlation A measure of how closely related two data series are. Its sign shows the direction of correlation It measures the strength of correlation 0 < r < 1 statistical dependence r = 0 X and Y are uncorrelated r = -1 negative r = 1 positive You can use only in case of linear relationship!

7 3. Coefficient of determination r The square of the sample correlation coefficient between the dependent and independent variables. Measures the degree of correlation in percentage (%) Shows how many percent of the variance of dependent variable is explained by the independent variable. Varies from 0 to 1. r S S yˆ y =1- S S e y

8 No relationship 4000 Number of births Number of storks

9 Independence Y = E X 3 R - S q = 3. 4 % N i n c s k o r r e lá c i ó

10 Positive correlation Y = E X 3 R -S q = 6. 5 % P o z i t ív k o r r e l á c i ó

11 Negative correlation Y = E X 3 R - S q = % N e g a t ív k o r r e lá c i ó

12 Curvilinear relation Y = X X * * 4 0 R - S q = % N e m l i n e á r i s k o r r e lá c i ó

13 Scatter diagrams linear S a l e s i n $ Advertising in $ S e l l i n g p r i c e Age of a house (year) curvilinear w a s t a g e S e l l i n g p r i c e Production (number of products per day) Age of a car (year) direct relationship positive slope inverse relationship negative slope

14 Example A firm administers a test to sales trainees before they go into the field. The management of the firm is interested in determining the relationship between the test scores and the sales made by the trainees at the end of one year in the field. The following data were collected for 45 sales personnel who have been in the field one year. Calculate different correlation measures!

15 X Y independent dependent variable Salesperson Test score Number of units sold x i d x K. A L. Z B. E G. P S. G J. T V. P T. L Total d x d y = x y i y d xi xyi y dxd y y

16 Number of observed pairs: n = 45 x 16 y 166 s s x y C dxd n 1 y Positive correlation

17 r s x C s y r 6.36 % There is a strong & positive relation between test scores and number of units sold. The test scores explain 6.36 percent of the variation of number of units sold.

18 4. Coefficient of rank correlation 6di n (n 1) ρ Spearman correlation Measure of the relationship between two ordinal data n = number of paired observations, d = difference between the ranks for each pair of observations. perfect correlation ρ= 1 perfect inverse correlation ρ = -1 in case of independence ρ = 0

19 Example Ten students were ranked by their mathematical and musical ability: Ability Student A B C D E F G H I J Total Mathematics Music d i = x i - y i d i di strong relationship n (n 1) 10(10-1) ρ

20 Simple Linear Regression Model We model the relationship between two variables, X and Y as a straight line. The model contains two parameters: an intercept parameter, a slope parameter. E (y) y = β 0 + β 1 x + ε where: β 1 = slope β 0 = y-intercept x y dependent or response variable (the variable we wish to explain or predict) x independent or predictor variable ε random error component β 0 y-intercept of the line, i.e. point at which the line intercept the y-axis β 1 slope of the line

21 Assumptions of the Linear Regression Model Assumptions for Error term: Normally distributed; Expected value = 0 (E(ε)=0); The variance is the same for all observations (Homoscadasticity); Uncorrelated across observations (there isn t any autocorrelation). Assumptions for the Independent Variables: Not random, etc.

22 y Deterministic component ŷ i = b 0 + b i x i Random error x y = deterministic component + random error We always assume that the mean value of the random error equals 0 the mean value of y equals the deterministic component. It is possible to find many lines for which the sum of the errors is equal to 0, but there is one (and only one) line for which the SSE (sum of squares of the errors) is a minimum: Least squares line / regression line.

23 The method of least squares gives us the best linear unbiased estimators of the regression parameters: β 0, β 1. The least-squares estimators: b 0 estimates β 0 b 1 estimates β 1 The regression line: y caret ( hat ): Ŷ = b 0 + b 1 X The normal equations (with 1 x) Σy = nb 0 + b 1 Σx Σxy = b 0 Σx + b 1 Σx

24 Interpretation b 0 : when x=0, y=b 0 b 1 : for every 1 unit increase in x we expect y to change by b 1 units

25 Elasticity % change in x demanded % change in y E(y,x) b 1 b 0 x b 1 x E(y, x) = b x 1 y Elasticity at the mean

26 Estimation in Regression Regression estimation is a technique used to replace missing values in data. If we know: 1. The estimated parameter value;. The hypothesized value of the parameter; 3. Confidence interval around the estimated parameter. The number of degrees of freedom equals the number of observations minus the number of parameters estimated. = n-

27 Estimation in Regression Parameter Estimated value Standard error 0 b 0 1 b 1 0 ŷ 0 s e s e n(x x i s e i x) (xi x) 1 ( x0 n ( x i x) x) b b 0 1 yˆ t yˆ t Y 0 t t s s b s yˆ yˆ b 1 s 0 ŷ 0 s e 1 1 n ( x0 + (x i x) x) = n- In case of average Y values In case of discrete Y values

28 Analysis of Variance in Regression Analysis Sum of Squares Df Regression Sŷ = (ŷi y) 1 Mean Sum of Squares S Residual S = (y ŷ n- se Se /( n ) e i ) ŷ F = S F e S ŷ /(n - ) Total S y = (y i y) n-1 S y n -1 S y S y ˆ S e n i=1 n n i y) (ŷi y) + (yi y) i=1 i=1 (y

29 Model testing H 0 : β 1 = 0 H 1 : β 1 0 (linear model) Test statistic: F = S s Pr ŷ e 0 H 1 : 1 S 1 F1 ( ; 1) /(n - F-statistic tests whether all the slope coefficients in a linear regression are equal to 0. Measures how well the regression equation explains the variation in the dependent variable. e S ŷ ) F Pr H 0 1 F ( ; 1) 1 1 : 1 H 0 F ( 1; ) 1 F

30 H1 : m 0 H 0 : β 1 = 0 H 1 : β 1 0 Parameter testing Pr H 1 : m 0 Pr H 0 t 1 0 Test statistic: t b 1 s( b 1 ) t 0 1 / t1 / where: b 1 is the least square estimate of the regression slope s(b 1 ) is the standard error of b 1

31 Seminar

32 Exercise 1 Book: p185 e44 In a bar waiters believe that there is a relationship between the amount of consumption of cola and the average daily temperature. To test it a sample of 0 days was drawn and they examined the amount of consumption and the temperature in these days:

33 Results: y 1,19; x 537; xy 330,159; x 14,597; y 7,505,555; d 179 x d 149,93; d d 4495 y x y Determine the relationship between the temperature and the consumption in case of linear and curvilinear relationship. Day The amount of consumption (l) The maximum daily temperature ( C)

34 Exercise (p188 e48) The export and import of Hungary with European countries are the following: d d 1,195,957; x,948 x y y y 3, 071; x, 084, 046; 1, 68,345 Characterize the trade with European countries. Country Export (X) Import (Y) Austria Belgium Czech Republic France Holland Poland Great-Britain Germany Italy Russia Switzerland 7 49 Sweden Slovakia 54 1 Slovenia Ukraine

35 Exercise 3 p188 e48 The table shows the inflation rate (x) and the unemployment rate (y) of Germany between 197 and Results: x 9.4; y171.8 d 94.54; d x y xy 51.9 Determine the relationship between unemployment and inflation rate. Year Inflation rate (%) Unemployment rate (%)

36 Thanks for your attention!