Http lib stat cmu edu datasets

STAT 706, Fall 2014, Prof. Slavov INSTRUCTIONS FOR FINAL (PROJECT DUE: Wednesday, December 17) A) Choice of dataset to analyze:

• Pick a dataset you think would be interesting, relevant and fun to work with.

• Datasets discussed either in the book, in class, or in your homework assignments such as:

gala, savings, …, fat

cannot be used!

• Good online sources of datasets grouped by a topic:

http://archive.ics.uci.edu/ml/datasets.html (select “Regression” from Task menu on left) http://www.fueleconomy.org/feg/download.shtml http://lib.stat.cmu.edu/DASL (kind of old)

• The R libraries Ecdat and AER also have good data (e.g. VietNamI, Caschool,

Medicaid1986). Load package (library(Ecdat)) and then use data(package=”Ecdat”) and help(VietNamI) to learn more.

• I need to approve the dataset you want to work with. In case you can’t find a satisfactory

dataset, let me know. I’ll offer you a dataset.

• Make sure to edit out any extra text information that is sometimes included with the data file (e.g. name of the dataset, sources, etc.)

• Once you decide on a dataset and have successfully imported it into R, spend at least 5-10 minutes doing some preliminary background research on its source, its importance, why it was collected, and whether any external information (important events, dates, etc.) could be relevant to its analysis.

B) Data Analysis Follow the guidelines in Chapter 10.1 and the complete example in Chapter 11 of the book:

1) Plot and summarize your data and inspect it.

2) Fit a model to your data. Run diagnostics to check assumptions – constant variance, linearity, normality, outliers, influential points, serial correlation and colinearity 3) Transform: a) Box-Cox for the response; b) polynomial regressions (interaction terms), log() for the predictors 4) Variable selection: testing- and criterion-based methods (regularization methods) Repeat the steps if necessary but be aware of too much analysis. So, avoid complex models for small datasets.

C) Writing the report: The report should not be more than 10-12 pages long. It should contain a SUMMARY of your analysis, not a screen dump from R. Only include tables and figures that you directly refer to in the body of report. The report should generally be structured as follows:

INTRODUCTION: Description of problem, data and data sources, brief description of analyses, principal results. DATA: Sources of data, external evidence of data quality, description of data, examination of data for quality and discrepancies, standard descriptive summaries. ANALYSES: Reasons for choosing analysis, idea and theory of analyses, algorithms used, results. Summarize steps 1-4 described above. Very big tables or plots not necessary to the main conclusions should be avoided. PLOTS AND TABLES: Make sure plots have plenty of textual information (such as titles and labels) so that they can be understood by themselves. Likewise with tables. Avoid especially too many digits, which you get from the software output: 2-3 significant digits are often more than enough. CONCLUSIONS: Make succinct persuasive arguments connecting your data and analysis to the original problem. Your background research about the data and the underlying issues are especially useful in packaging your report into a standalone document.

CODE/APPENDIX: Key calls in R need to be included.

D) Submission is both electronic and a paper copy:

• name your document this way: STAT706_firstname_lastname_exam • use ONLY these formats: *.doc or *.pdf • send code/project file (*.r) and data • don’t forget to include your name in the document’s body