Correlation – Introduction COMP20008 School of Computing and Information Systems Chris…
- User
Correlation – Introduction COMP20008 School of Computing and Information Systems Chris Ewin & Geela Chee 1 Plan • Discuss correlations between pairs of features in a dataset • Why useful and important • Pitfalls • Methods for computing correlation • Pearson correlation • Mutual information (another method to compute correlation) 2 Correlations in life As a child grows, so does his/her clothing size. As you drink more coffee, the number of hours you stay awake increases. The longer amount of time you spend in the bath, the more wrinkly your skin becomes. The older a man gets, the less hair that he has. https://towardsdatascience.com/3-beautiful-real-life-correlations-fed9e855da52, https://examples.yourdictionary.com/positive-correlation-examples.html, https://examples.yourdictionary.com/negative-correlation-examples.html 3 What is Correlation? Correlation is used to detect pairs of variables that might have some relationship. AND How strong is the relationship? https://www.mathsisfun.com/data/correlation.html 4 What is Correlation? Visually can be identified via inspecting scatter plots https://www.mathsisfun.com/data/correlation.html 5 What is Correlation? Linear relations https://www.mathsisfun.com/data/correlation.html 6 What is Correlation? Correlation strength https://images.app.goo.gl/zZXtjBLR2BcjRpK79 7 Example of non-linear correlation It gets so hot that people aren't going near the shop, and sales start dropping https://www.mathsisfun.com/data/correlation.html 8 Example of non-linear correlation It gets so hot that people aren't going near the shop, and sales start dropping https://www.mathsisfun.com/data/correlation.html 9 Example of Correlated Variables • Greater understanding of data • Can hint at potential causal relationships • Business decision based on correlation: increase electricity production when temperature increases 10 Example of Correlated Variables Correlation does not necessarily imply causality! 11 Example rank correlation “If a university has a higher-ranked football team, then is it likely to have a higher-ranked basketball team?” Football ranking University team 1 Melbourne 2 Monash 3 Sydney 4 New South Wales 5 Adelaide 6 Perth Basketball ranking University team 1 Sydney 2 Melbourne 3 Monash 4 New South Wales 5 Perth 6 Adelaide 12 Biology example - Microarray data Each chip contains thousands of tiny probes corresponding to the genes (20k - 30k genes in humans). Each probe measures the activity (expression) level of a gene Gene 1 expression Gene 2 expression … Gene 20K expression 0.3 1.2 … 3.1 13 Microarray dataset Gene 1 Gene 2 Gene 3 … Gene n Time 1 2.3 1.1 0.3 … 2.1 Time 2 3.2 0.2 1.2 … 1.1 Time 3 1.9 3.8 2.7 … 0.2 … … … … … … Time m 2.8 3.1 2.5 … 3.4 • Each row represents measurements at some time • Each column represents levels of a gene 14 Correlation analysis on Microarray data Can reveal genes that exhibit similar patterns ⇨ similar or related functions ⇨ Discover functions of unknown genes 15 Genetic network Connect genes with high correlation – understand behaviour of groups of genes http://www.john.ranola.org/?page_id=116 16 Salt Causes High Blood Pressure50 60 70 80 90 0 50 100 150 200 250 Median diastolic blood pressure (mmHg) Median urinary sodium excre on (mmol/24hr) Intersalt: an international study of electrolyte excretion and blood pressure. Results for 24 hour urinary sodium and potassium excretion. British Medical Journal; 297: 319-328, 1988. 17 Or Does It!?50 60 70 80 90 0 50 100 150 200 250 Median diastolic blood pressure (mmHg) Median urinary sodium excre on (mmol/24hr) Intersalt: an international study of electrolyte excretion and blood pressure. Results for 24 hour urinary sodium and potassium excretion. British Medical Journal; 297: 319-328, 1988. If we exclude these four ‘outliers’, which are non-industrialised countries with non-salt diets, we get a quite different result! 18 Spurious Correlation Correlation ≠ Causality https://images.app.goo.gl/FVr8BhxWmQMxCB5f7 https://assets.businessinsider.com/real-maps-ridiculous- correlations-2014-11?jwsource=cl 19 Why is correlation important? • Discover relationships • One step towards discovering causality A causes B Example: Smoking causes lung cancer Feature ranking: for building better predictive models A good feature to use, is a feature that has high correlation with the outcome we are trying to predict 20 Break 21 Pearson Correlation COMP20008 School of Computing and Information Systems 22 Correlation 23 • Objects can be represented with different measure scales d(temp,ice-cr)= 540324 d(temp,elect)= 12309388 • Euclidean distance: does not give a clear intuition about how well variables are correlated Problems of Euclidean distance Day 1 Day 2 Day 3 … Day m Temperature 20 22 16 … 33 #Ice-creams 50223 55223 45098 … 78008 #Electricity 102034 105332 88900 … 154008 24 Problems of Euclidean distance Cannot discover variables with similar behaviours/dynamics but at different scale 25 Problems of Euclidean distance Cannot discover variables with similar behaviours/dynamics but in the opposite direction (negative correlation) 26 Assessing linear correlation – Pearson correlation • We will define a correlation measure rxy, assessing samples from two features x and y • Assess how close their scatter plot is to a straight line (a linear relationship) Pearson Product Moment Correlation • Range of rxy lies within [-1,1]: • 1 for perfect positive linear correlation • -1 for perfect negative linear correlation • 0 means no correlation • Absolute value |r| indicates strength of linear correlation 27 Pearson's correlation coefficient (r) 𝑟 = 𝑟𝑥𝑦 = σ𝑖=1 𝑛 (𝑥𝑖 − ҧ 𝑥)(𝑦𝑖 − ത𝑦) (σ𝑖=1 𝑛 (𝑥𝑖 − ҧ 𝑥)2) × (σ𝑖=1 𝑛 (𝑦𝑖 − ത𝑦)2) ҧ 𝑥 = 1 𝑛 𝑖=1 𝑛 𝑥𝑖 ത𝑦 = 1 𝑛 𝑖=1 𝑛 𝑦𝑖 𝑟 = 𝑟𝑥𝑦 = σ𝑖=1 𝑛 𝑥𝑖 ∗ × 𝑦𝑖 ∗ (σ𝑖=1 𝑛 𝑥𝑖 ∗2) × (σ𝑖=1 𝑛 𝑦𝑖 ∗2) 𝑥𝑖 ∗ = 𝑥𝑖 − ҧ 𝑥 𝑦𝑖 ∗ = 𝑦𝑖 − ത𝑦 28 Pearson coefficient example Height (x) Weight (y) 1.6 50 1.7 66 1.8 77 1.9 94 • How do the values of x and y move (vary) together? • Big values of x with big values of y? • Small values of x with small values of y? 29 Pearson coefficient example 30 Interpreting Pearson correlation values In general it depends on your domain of application. Jacob Cohen has suggested • 0.5 is large • 0.3-0.5 is moderate • 0.1-0.3 is small • less than 0.1 is trivial 31 Properties of Pearson's correlation • Range within [-1,1] • Is sensitive to outliers • Can only detect linear relationships y = a.x + b + noise • Cannot detect non-linear relationships y = x3 + noise • Scale invariant: r(x,y)= r(x, Ky) • Multiplying a feature’s values by a constant K makes no difference • Location invariant: r(x,y)= r(x, K+y) • Adding a constant K to one feature’s values makes no difference 32 Pearson correlation examples https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient 33 2017 Exam question 3a 34 2016 exam question 2a) 35 • Correlation <> Causality • http://tylervigen.com/spurious-correlations • Google trends correlation • https://trends.google.com/trends/ 36 Break 37 Mutual Information COMP20008 School of Computing and Information Systems This topic is at the more difficult end of material covered in this subject 38 Recap: Pearson correlation – assess linear correlation between two features https://www.mathsisfun.com/data/correlation.html 39 What about non-linear correlation? Pearson correlation is not suitable for this scenario (value less than 0.1) https://www.mathsisfun.com/data/correlation.html 40 Entropy Mutual Information 41 Entropy – intuition • Entropy is a measure used to quantify the amount of uncertainty in an outcome • Randomly select an element from A. {1,1,1,1,1,1,1,2} versus B. {1,1,2,2,3,3,4,4} • In which case are you more uncertain about the outcome of the selection? Why? • More uncertain, less predictable => high entropy (b.) • Less uncertain, more predictable => low entropy (a.) 42 Another example Consider the sample of all people in this subject. Each person is labelled young (<30 years) or old (>=30 years) • Randomly select a person and inspect whether they are young or old. • How surprised am I likely to be by the outcome? • Suppose I repeat the experiment using a random sample of people catching the train to the city in peak hour? • How surprised am I likely to be by the outcome? 43 A recap on logarithms (to the base 2) • y = log2x (y is the solution to the question “To what power do I need to raise 2, in order to get x?’’) • 222*2=16 which means log216 = 4 (16 is 2 to the power 4) • log2 32 = 5 • log2 30 = 4.9 • log2 1.2 = 0.26 • log2 0.5 = -1 In what follows, we’ll write log instead of log2 to represent binary log 44 Entropy Given a feature 𝜲, assuming 𝜲 has k number of categories (bins), then 𝐻 𝜲 is its entropy. 𝐻 𝜲 = − 𝑖=1 𝑘 𝑝𝑖 log2 𝑝𝑖 X : Flipping a special coin, head: 75%, tail: 25% 𝐻 𝑋 = − 𝑝𝐻 log2 𝑝𝐻 + 𝑝𝑇 log2 𝑝𝑇 = −[0.75 log2 0.75 + 0.25 log2 0.25] = 𝟎. 𝟖𝟏 𝑝𝑖: proportion of points in the 𝑖-th category (bin) 45 Entropy practice example A B B A C C C C A We have 3 categories (A,B,C) for a feature 𝜲 9 objects, each object is in one category. What is the entropy of this sample data of 9 objects? P_A=3/9 P_B=2/9 P_C=4/9 Answer: 𝐻 𝜲 = 1.53 𝐻 𝜲 = − 𝑖=1 𝑘 𝑝𝑖 log2 𝑝𝑖 46 How would you compute the entropy for the “Likes to sleep” feature? Person Likes to sleep 1 Yes 2 No 3 Maybe 4 Never 5 Yes 6 Yes 7 Never 8 Yes 𝜲: Likes to sleep 𝐻 𝜲 = − 𝑖=1 𝑘 𝑝𝑖 log2 𝑝𝑖 47 Properties of entropy • H(X) ≥ 0 • Entropy is maximized for uniform distribution (highly uncertain what value a randomly selected object will have) • Entropy – when using log base 2 – measures uncertainty of the outcome in bits. This can be viewed as the information associated with learning the outcome 48 Variable discretisation • Pre-processing: continuous features are first discretised into bins (categories). E.g. small [0,1.4], medium (1.4,1.8), large [1.8,3.0] Object Height Discretised Height 1 2.03 large 2 1.85 large 3 1.23 small 4 1.31 small 5 1.72 medium 6 1.38 small 7 0.94 small 49 Variable discretisation: Techniques • Equal-width bin Divide the range of the continuous feature into equal length intervals (bins). If speed ranges from 0-100, then the 10 bins are [0,10), [10,20), [20,30), …[90,100] • Equal-frequency bin Divide range of continuous feature into equal frequency intervals (bins). Sort the values and divide so that each bin has roughly same number of objects • Domain knowledge: assign thresholds manually Car speed: • 0-40: slow • 40-60: medium • >60: high 50 Discretisation example • A 3-bin equal width discretization • Bin-width: 21−2 3 = 6.333 • [2, 8.333), [8.333, 14.666), [14.666, 21] Given the values 2, 2, 3, 10, 13, 15, 16, 17, 19, 19, 20, 20, 21 51 Discretisation example • A 3-bin equal frequency discretization • [2, 13), [13, 19), [19, 21] – by hand • (1.999, 13], (13, 19], (19, 21] – pandas Given the values 2, 2, 3, 10, 13, 15, 16, 17, 19, 19, 20, 20, 21 52 Break 53 Mutual Information COMP20008 School of Computing and Information Systems 54 Conditional entropy - intuition Suppose I randomly sample a person. I check if they wear glasses – how surprised am I by their age given their wearing glasses status? Person WearGlasses(X) Age (Y) 1 No young 2 No young 3 No young 4 No young 5 Yes old 6 Yes old 7 Yes old 55 Conditional entropy H(Y|X) 𝐻 𝑌 𝑋 = 𝑥∈𝑋 𝑝 𝑥 𝐻(𝑌|𝑋 = 𝑥) Measures how much information is needed to describe outcome Y, given that outcome X is known. Suppose X is Height and Y is Weight. Object Height (X) Weight (Y) 1 big light 2 big heavy 3 small light 4 small light 5 small light 6 small light 7 small heavy 56 Conditional entropy H(Y|X) 𝐻 𝑌 𝑋 = 𝑥∈𝑋 𝑝 𝑥 𝐻(𝑌|𝑋 = 𝑥) 𝐻 𝑌 𝑋 = ൗ2 7 ∗ 𝐻 𝑌 𝑋 = 𝑏𝑖𝑔 + ൗ5 7 ∗ 𝐻 𝑌 𝑋 = 𝑠𝑚𝑎𝑙𝑙 𝐻 𝑌 𝑋 = 𝑥 = − 𝑦∈𝑌 𝑝 𝑦 𝑥 log2 𝑝 𝑦 𝑥 𝐻 𝑌 𝑋 = 𝑏𝑖𝑔 = − ൗ1 2 log2 ൗ1 2 + ൗ1 2 log2 ൗ1 2 = 1 𝐻 𝑌 𝑋 = 𝑠𝑚𝑎𝑙𝑙 = − ൗ4 5 log2 ൗ4 5 + ൗ1 5 log2 ൗ1 5 = 0.721 𝐻 𝑌 𝑋 = ൗ2 7 ∗ 1 + ൗ5 7 ∗ 0.721 = 0.801 Object Height (X) Weight (Y) 1 big light 2 big heavy 3 small light 4 small light 5 small light 6 small light 7 small heavy 57 𝑀𝐼 𝑋, 𝑌 = 𝐻 𝑌 − 𝐻 𝑌 𝑋 𝑀𝐼 𝑋, 𝑌 = 𝐻 𝑋 − 𝐻 𝑋 𝑌 𝐻 𝑌 𝑋 = 𝑥∈𝑋 𝑝 𝑥 𝐻(𝑌|𝑋 = 𝑥) where X and Y are features (columns) in a dataset • MI (mutual information) is a measure of correlation • The amount of information about Y we gain by knowing X, or • The amount of information about X we gain by knowing Y Mutual information definition 58 Mutual information example 1 𝑀𝐼 𝑋, 𝑌 = 𝐻 𝑌 − 𝐻 𝑌 𝑋 𝐻 𝑌 = − 𝑦∈𝑌 𝑝(𝑦) log2 𝑝(𝑦) 𝐻 𝑌 𝑋 = 𝑥∈𝑋 𝑝 𝑥 𝐻(𝑌|𝑋 = 𝑥) 𝐻 𝑌 = − ൗ5 7 log2 ൗ5 7 + ൗ2 7 log2 ൗ2 7 = 0.8631 𝐻 𝑌 𝑋 = ൗ6 7 ∗ 𝐻 𝑌 𝑋 = 𝑠𝑚𝑎𝑙𝑙 + ൗ1 7 ∗ 𝐻 𝑌 𝑋 = 𝑏𝑖𝑔 𝐻 𝑌 𝑋 = ൗ6 7 ∗ − ൗ5 6 log2 ൗ5 6 − ൗ1 6 log2 ൗ1 6 + ൗ1 7 ∗ −1 ∗ log21 = 0.5571 𝑀𝐼 𝑋, 𝑌 = 0.8631 − 0.5571 = 0.3059 Object Height (X) Weight (Y) 1 small light 2 big heavy 3 small light 4 small light 5 small light 6 small light 7 small heavy 59 Mutual information example 2 𝑀𝐼 𝑋, 𝑌 = 𝐻 𝑌 − 𝐻 𝑌 𝑋 𝐻 𝑌 = − 𝑦∈𝑌 𝑝(𝑦) log2 𝑝(𝑦) 𝐻 𝑌 𝑋 = 𝑥∈𝑋 𝑝 𝑥 𝐻(𝑌|𝑋 = 𝑥) 𝐻 𝑌 = 1.379 𝐻 𝑌 𝑋 = 0.965 𝑀𝐼 𝑋, 𝑌 = 𝐻 𝑌 − 𝐻 𝑌 𝑋 = 0.414 Object Height (X) Weight (Y) 1 big light 2 big heavy 3 small light 4 small jumbo 5 medium light 6 medium light 7 small heavy 60 Properties of Mutual Information • The amount of information shared between two variables X and Y • MI(X,Y) • large: X and Y are highly correlated (more dependent) • small: X and Y have low correlation (more independent) • 0 ≤ MI(X,Y) ≤ ∞ • Sometimes also referred to as ‘Information Gain’ 61 Mutual information: normalisation • MI(X,Y) is always at least zero, may be larger than 1 • In fact, one can show it is true that • 0 ≤ MI(X,Y) ≤ min(H(X),H(Y)) • Thus if want a measure in the interval [0,1], we can define normalised mutual information (NMI). • NMI(X,Y) = MI(X,Y) / min(H(X),H(Y)) • NMI(X,Y) = MI(X,Y) / max(H(X),H(Y)) • NMI(X,Y) = MI(X,Y) / mean(H(X),H(Y)) • NMI(X,Y) • large: X and Y are highly correlated (more dependent) • small: X and Y have low correlation (more independent) 62 Normalised Mutual Information Example 1: 𝑀𝐼 = 0.3059 𝑁𝑀𝐼 = 0.517 Example 2: 𝑀𝐼 = 0.414 𝑁𝑀𝐼 = 0.300 Object Height (X) Weight (Y) 1 small light 2 big heavy 3 small light 4 small light 5 small light 6 small light 7 small heavy Object Height (X) Weight (Y) 1 big light 2 big heavy 3 small light 4 small jumbo 5 medium light 6 medium light 7 small heavy 63 Pearson correlation=-: -0.0864 Normalised mutual information (NMI)= 0.43 (3-bin equal spread bins) 64 Pearson correlation=-0.1 Normalised mutual information (NMI)=0.84 65 Pearson correlation=-0.05 Normalised mutual information (NMI)=0.35 66 Examples • Pearson? • NMI? 67 Examples • Pearson: 0.08 • NMI: 0.009 68 Computing MI with class features Identifying features that are highly correlated with a class feature Compute MI(HoursSleep, Happy), MI(HoursExercise, Happy), and MI(HoursStudy, Happy), MI(HairColour, Happy). Retain most predictive feature(s) HoursSleep HoursExercise HairColour HoursStudy Happy (class feature) 12 20 Brown low Yes 11 18 Black low Yes 10 10 Red medium Yes 10 9 Black medium Yes 10 10 Red high No 7 11 Red high No 6 15 Brown high No 2 13 Brown high No 69 Advantages and disadvantages of MI • Advantages • Can detect both linear and non-linear dependencies (unlike Pearson) • Applicable and very effective for use with discrete features (unlike Pearson correlation) • Disadvantages • If feature is continuous, it first must be discretised to compute mutual information. This involves making choices about what bins to use. • This may not be obvious. Different bin choices will lead to different estimations of mutual information. 70 Choose the statement that is not true • Mutual information can't detect the existence of linear relationships in the data, but Pearson correlation can • Mutual information can be used to assess how much one feature is associated with another • Mutual information can detect the existence of non-linear relationships in the data • Mutual information doesn't indicate the "direction" of a relationship, just its strength 71 Question 2ai) from 2016 exam a) Richard is a data wrangler. He does a survey and constructs a dataset recording average time/day spent studying and average grade for a population of 1000 students: Student Name Average time per day spent studying Average Grade … …. …. i) Richard computes the Pearson correlation coefficient between Average time per day studying and Average grade and obtains a value of 0.85. He concludes that more time spent studying causes a student’s grade to increase. Explain the limitations with this reasoning and suggest two alternative explanations for the 0.85 result. 72 Question 2aii) from 2016 exam a) Richard is a data wrangler. He does a survey and constructs a dataset recording average time/day spent studying and average grade for a population of 1000 students: Student Name Average time per day spent studying Average Grade … …. …. ii) Richard separately discretises the two features Average time per day spent studying and Average grade, each into 2 bins. He then computes the normalised mutual information between these two features and obtains a value of 0.1, which seems surprisingly low to him. Suggest two reasons that might explain the mismatch between the normalised mutual information value of 0.1 and the Pearson Correlation coefficient of 0.85. Explain any assumptions made. 73 End of Correlation topic 74 Acknowledgements • Materials are partially adopted from … • Previous COMP2008 slides including material produced by James Bailey, Pauline Lin, Chris Ewin, Uwe Aickelin and others • See also • Correlation and Causality – Don’t Confuse Them (13-2): https://www.youtube.com/watch?v=dDlgAiTHNqI 75 Create 5 exam style questions from the above. The questions should have multiple parts ie parts a,b, and c,d and e and be a mix of 80% applicaiton of relevant principles. Inlcude relevant images, formulas and other resources needed for the question aswell. Write the questions using overleaf code. Do not include the answers
- Assistant
\documentclass[12pt]{article} \usepackage{amsmath} \usepackage{graphicx} \usepackage{hyperref} \usepackage{enumitem} \title{COMP20008: Correlation – Exam-Style Questions} \author{} \date{} \begin{document} \maketitle \section*{Question 1: Pearson Correlation Coefficient} \textbf{a.} Given the following dataset of heights (in meters) and weights (in kilograms) for 7 individuals: \begin{center} \begin{tabular}{|c|c|c|} \hline \textbf{Person} & \textbf{Height (m)} & \textbf{Weight (kg)} \\ \hline 1 & 1.60 & 50 \\ 2 & 1.70 & 66 \\ 3 & 1.80 & 77 \\ 4 & 1.90 & 94 \\ 5 & 2.00 & 100 \\ 6 & 2.10 & 110 \\ 7 & 2.20 & 130 \\ \hline \end{tabular} \end{center} Calculate the Pearson correlation coefficient between height and weight. Show all your work. \textbf{b.} Interpret the value of the Pearson correlation coefficient obtained in part (a). What does it indicate about the relationship between height and weight in this dataset? \textbf{c.} Suppose you add an outlier to the dataset: Person 8 with a height of 1.65 m and weight of 200 kg. Recalculate the Pearson correlation coefficient including this outlier. \textbf{d.} Discuss how the outlier in part (c) affects the Pearson correlation coefficient. What does this imply about the sensitivity of Pearson's correlation to outliers? \textbf{e.} Based on your calculations and interpretations in parts (a) to (d), what conclusions can be drawn about using Pearson correlation in real-world data analysis? \newpage \section*{Question 2: Mutual Information and Entropy} \textbf{a.} Consider the following dataset representing the relationship between two categorical variables, $X$ (Weather) and $Y$ (Picnic): \begin{center} \begin{tabular}{|c|c|} \hline \textbf{Weather (X)} & \textbf{Picnic (Y)} \\ \hline Sunny & Yes \\ Sunny & Yes \\ Rainy & No \\ Sunny & Yes \\ Rainy & No \\ Cloudy & Maybe \\ Cloudy & Maybe \\ Sunny & Yes \\ Rainy & No \\ Cloudy & Maybe \\ \hline \end{tabular} \end{center} Calculate the entropy $H(Y)$ of the Picnic variable. \textbf{b.} Calculate the conditional entropy $H(Y|X)$. \textbf{c.} Using your results from parts (a) and (b), compute the Mutual Information $MI(X, Y)$ between Weather and Picnic. \textbf{d.} Interpret the Mutual Information value obtained in part (c). What does it say about the relationship between Weather and Picnic? \textbf{e.} Explain how Mutual Information differs from Pearson correlation in assessing the relationship between two variables. \newpage \section*{Question 3: Comparing Pearson Correlation and Mutual Information} \textbf{a.} Explain why Pearson correlation may not be suitable for detecting non-linear relationships between two variables. Provide a numerical example illustrating this limitation. \textbf{b.} Given two variables, $X$ and $Y$, where $Y = X^2$ and $X$ is uniformly distributed between -1 and 1, compute the Pearson correlation coefficient and the Mutual Information between $X$ and $Y$. \textbf{c.} Based on your computations in part (b), discuss which measure better captures the dependency between $X$ and $Y$, and why. \textbf{d.} Describe a real-world scenario where Mutual Information would be a more appropriate measure of association than Pearson correlation. \textbf{e.} Discuss the computational challenges associated with calculating Mutual Information for continuous variables and how discretization can address these challenges. \newpage \section*{Question 4: Discretization and Its Impact on Mutual Information} \textbf{a.} Define what is meant by discretization of a continuous variable. Provide two common techniques for discretizing data. \textbf{b.} Consider the continuous variable "Age" with values: 23, 25, 31, 35, 45, 47, 52, 58, 60, 65. Demonstrate how to discretize this variable into three equal-width bins. \textbf{c.} Using the discretized Age from part (b) and the following categorical variable "Health Status": \begin{center} \begin{tabular}{|c|c|} \hline \textbf{Age} & \textbf{Health Status} \\ \hline 23 & Good \\ 25 & Good \\ 31 & Fair \\ 35 & Good \\ 45 & Poor \\ 47 & Poor \\ 52 & Fair \\ 58 & Poor \\ 60 & Fair \\ 65 & Poor \\ \hline \end{tabular} \end{center} Calculate the Mutual Information between discretized Age and Health Status. \textbf{d.} Discuss how different discretization strategies (e.g., equal-width vs equal-frequency) might affect the Mutual Information calculated in part (c). \textbf{e.} Explain why discretization is a necessary step when computing Mutual Information for continuous variables and discuss any potential drawbacks. \newpage \section*{Question 5: Examining Correlation and Causality} \textbf{a.} Richard conducts a survey and collects data on the average time per day spent studying and the average grade for 1,000 students. He computes the Pearson correlation coefficient between these two variables and obtains a value of 0.85. Can Richard conclude that increased study time causes higher grades? Justify your answer. \textbf{b.} Identify two alternative explanations for the high Pearson correlation coefficient observed by Richard that do not imply causation. \textbf{c.} Suppose Richard also collects data on students' attendance rates and includes it in his analysis. How might attendance rates confound the relationship between study time and grades? \textbf{d.} Propose an additional statistical method or analysis that Richard could use to investigate the potential causal relationship between study time and grades. \textbf{e.} Discuss the importance of distinguishing between correlation and causation in data analysis, providing an example from real-life scenarios. \end{document}