Cross-validation is a technique that help one to estimate the error precisely in terms of statistics. Two common types:

– leave-one-out cross validation: use only observation for testing, and the other for training –> use in data set with sparse data.

– k-fold cross validation (we often use k = 10): divide the dataset into k parts equally, then iteratively use one part for testing and the other parts for training. Finally, compute the average error.