The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics)

This review is written from the perspective of a programmer who has sometimes had the chance to choose, hire, and work with algorithms and the mathematician/statisticians that love them in order to get things done for startup companies. I don't know if this review will be as helpful to professional mathematicians, statisticians, or computer scientists. The good news is, this is pretty much the most important book you are going to read in the space. It will tie everything together for you in a way that I haven't seen any other book attempt. The bad news is you're going to have to work for it. If you just need to use a tool for a single task this book won't be worth it; think of it as a way to train yourself in the fundamentals of the space, but don't expect a recipe book. Get something in the "using R" series for that. When it came out in 2001 my sense of machine learning was of a jumbled set of recipes that tended to work in some cases. This book showed me how the statistical concepts of bias, variance, smoothing and complexity cut across both fields of traditional statistics and inference and the machine learning algorithms made possible by cheaper cpus. Chapters 2-5 are worth the price of the book by themselves for their overview of learning, linear methods, and how those methods can be adopted for non-linear basis functions. The hard parts: First, don't bother reading this book if you aren't willing to learn at least the basics of linear algebra first. Skim the second and third chapters to get a sense for how rusty your linear algebra is and then come back when you're ready. Second, you really really want to use the SQRRR technique with this book. Having that glimpse of where you are going really helps guide you're understanding when you dig in for real. Third, I wish I had known of R when I first read this; I recommend using it along with some sample data sets to follow along with the text so the concepts become skills not just abstract relationships to forget. It would probably be worth the extra time, and I wish I had known to do that then. Fourth, if you are reading this on your own time while making a living, don't expect to finish the book in a month or two.

There is no other book I know of in this space with the same combination of thorough detailed math, intuition, application to real-world data, and excellent graphics. It's also very well-written. Their notation can be a bit weird, but whatever. Maybe I'm weird for finding their notation weird. Enough praise. Just buy it and study it. I personally like it better than the comparable books by Barber, Bishop, Murphy, and others, but to each their own. These three are excellent books in their own right, and maybe some would prefer them, especially if one does a lot of Bayesian modeling. But usually, one doesn't. And if you're a beginner in machine learning, my opinion is that studying Bayesian inference as a default can be confusing. Reading advice, if you're not a mathematician (if you are, you don't need my advice): I highly recommend going through a book on standard statistical inference first, else you might be a bit lost, and subtle points that Hastie et al make might be missed (I often pick up details on a second reading - lots of "aha" moments to be had). Not to mention the fact that some of their derivations will seem impenetrable; that one for bias and variance of the linear model in chapter 2 nonplussed me for a while. Luckily there are the accompanying notes by Weatherwax et al (google it), which are seriously helpful. Good options for background are Casella & Berger (the standard), the book "Statistical rethinking from scratch" by Edge (such a good book!), the book "Probability and mathematical statistics" by Meyer (this looks excellent but I don't know it well), and many others (the number of books written on statistical inference asymptotically approaches infinity). Some people like the book by Wasserman but I find it so "skeletal" (as one reviewer said) that one has to go elsewhere for the details anyway. So why not just read a less skeletal book? Anyway, back to ESL. Reading this has made me a less dumb person, even though I've only read in detail the first 3 chapters. I hope it will do the same for you.

I have been using The Elements of Statistical Learning for years, so it is finally time to try and review it. The Elements of Statistical Learning is a comprehensive mathematical treatment of machine learning from a statistical perspective. This means you get good derivations of popular methods such as support vector machines, random forests, and graphical models; but each is developed only after the appropriate (and wrongly considered less sexy) statistical framework has already been derived (linear models, kernel smoothing, ensembles, and so on). In addition to having excellent and correct mathematical derivations of important algorithms The Elements of Statistical Learning is fairly unique in that it actually uses the math to accomplish big things. My favorite examples come from Chapter 3 "Linear Methods for Regression." The standard treatments of these methods depend heavily on respectful memorization of regurgitation of original iterative procedure definitions of the various regression methods. In such a standard formulation two regression methods are different if they have superficially different steps or if different citation/priority histories. The Elements of Statistical Learning instead derives the stopping conditions of each method and considers methods the same if they generate the same solution (regardless of how they claim they do it) and compares consequences and results of different methods. This hard use of isomorphism allows amazing results such as Figure 3.15 (which shows how Least Angle Regression differs from Lasso regression, not just in algorithm description or history: but by picking different models from the same data) and section 3.5.2 (which can separate Partial Least Squares' design CLAIM of fixing the x-dominance found in principle components analysis from how effective it actually is as fixing such problems). The biggest issue is who is the book for? This is a mathy book emphasizing deep understanding over mere implementation. Unlike some lesser machine learning books the math is not there for appearances or mere intimidating typesetting: it is there to allow the authors to organize many methods into a smaller number of consistent themes. So I would say the book is for researchers and machine algorithm developers. If you have a specific issue that is making inference difficult you may find the solution in this book. This is good for researchers but probably off-putting for tinkers (as this book likely has methods superior to their current favorite new idea). The interested student will also benefit from this book, the derivations are done well so you learn a lot by working through them. Finally- don't buy the kindle version, but the print book. This book is satisfying deep reading and you will want the advantages of the printed page (and Amazon's issues in conversion are certainly not the authors' fault).

Math books, at least data science texts, can usually be divided into those which are easy to read but contain little technical rigor and those which are written with a scientific approach to methodology but are so equation dense that it’s hard to imagine them being read outside an advanced academic setting. Fortunately, The Elements of Statistical Learning proves the exception. The text is full with the equations necessary to root the methodology without engaging the reader with long proofs that would tax those of us employing these techniques in the business world. The visual aspects of the text seem to have been written with John Tukey or Edward Tufte in mind. Though their frequent use makes the book some seven hundred pages long, reading and comprehension is made much easier. And, though it’s been almost ten years since the book was published, the techniques described remain, for the most part, at the cutting edge of data science. I was told by some other analysts I know that this was their bible for data science. I was somewhat skeptical of this kind of hyperbole but was pleasantly surprised that the book matched these high expectations. If you have an undergraduate degree in a mathematically related discipline, The Elements of Statistical Learning will prove to be an invaluable reference to understand the rapidly advancing avalanche of data mining techniques.

This is a quite interesting, and extremely useful book, but it is wearing to read in large chunks. The problem, if you want to call it that, is that it is essentially a 700 page catalogue of clever hacks in statistical learning. From a technical point of view it is well-ehough structured, but there is not the slightest trace of an overarching philosophy. And if you don't actually have a philosophical perspective in place before you start, the read you face might well be an even harder grind. Be warned. Some of the reviews here complain that there is too much math. I don't think that is an issue. If you have decent intuitions in geometry, linear algebra, probability and information theory, then you should be able to cruise through and/or browse in a fairly relaxed way. If you don't have those intuitions, then you are attempting to read the wrong book. There were a couple of things that I expected (things I happen to know a bit about), but that were missing. On the unsupervised learning side, the discussion of Gaussian mixture clustering was, I thought, a bit short and superficial, and did not bring out the combination of theoretical and practical power that the method offers. On the supervised learning side, I was surprised that a book that dedicates so much time to linear regression finds no room for a discussion of Gaussian process regression as far as I could see (the nearest point of approach is the use of Gaussian radial basis functions [oops: having written that, I immediately came across a brief discussion (S5.8.1) of, essentially, GP regression - though with no reference to standard literature]).

I have finished reading the book, but I feel somewhat confused... Although I think I did pretty good with math in graduate school (Physics PhD from a good school), I find it hard to follow some of the derivations. The book specified the prerequisite level of one elementary course of statistics and some linear algebra. But I feel the actual requirement for math is much higher, especially in the multivariate statistics and matrix representations. I often find myself staring at the formulas and trying to get the big picture. This book is particularly good in terms of grabbing the insights and intuitions underneath the different methods. It covers a vast majority of different techniques, and trying to present the ideas and intuitions why this method works or how this method is an implementation of a simpler idea, e.g.various obvious or hidden implementations of lasso regularization. This is impressive. No one can do better because the authors are actually leading researchers who either invented these ideas or made significant contributions to their development. Unfortunately, this book is not a textbook for outsiders of machine learning research. The actual implementations/algorithm are not covered in detail, provided often as-is, and have weak links to the presented formulas. The indices or notations in the formulas are sometimes not mentioned, and some of the key concepts for understanding the principles are not introduced. I presume researchers in this field would usually have a common sense of what they are, but for someone like me, it is sometimes very difficult to follow. Anyway, it is great to finish this book. I don't know how much I could get from this book, but at least I have some feelings of how deep and how broad this field is, and know where to look if later I encounter problems. I would recommend an outside learner to step away from this book as a principle textbook and only use it as a reference. On the other hand, the book introduction to statistical learning by the same authors are highly recommended for new comers.

This is a graduate level overview of key statistical learning techniques. I bought it as I am currently a student in Hastie's graduate level stats course at Stanford. The book is fairly deep given it's breadth, which is why it's so long. It's a very good book and an amazing reference but I do not recommend it unless you have strong mathematical maturity. If you do not know linear algebra / probability well, you will be lost. If you're like me, you'll also have to do some googling around for some of the stats stuff he talks about (confidence intervals, chi squared distribution...). I personally don't mind this. The coverage of this book is truly amazing. Trevor talks about everything from the lasso and ridge regression to knn, SVM's, ensemble methods, and random forests. It's fun to just flip to a random page and just start reading. If you want to be an expert in this area, you need to have this book. On the other hand if you just want to know basic techniques and then apply them, this book might be overkill and you may be better served by a more elementary text.

ESL is certainly a challenging read in the sense that you need to be comfortable with the topics of linear algebra, statistical theory, and the basics of probability theory. I would highly recommend supplementing this text with "A Solution Manual and Notes for: The Elements of Statistical Learning by Jerome Friedman, Trevor Hastie, and Robert Tibshirani" (Weatherwax and Epstein) as they do a nice job supplying derivations and explanations to several of the equations presented in-text. There are certain regions of this book that are more mathematically intensive than others. Overall, I would say this book provides a solid overview of many of the important fundamental topics of statistical inference and machine learning. Deep neural networks are only given a single and rather light chapter so this book is not the best resource in that regard. The organization and presentation of the material is very pleasing and logical. I read the book cover-to-cover, but the authors provide some guidance on their recommendation for what parts of the book should be read first before skipping around. I would highly recommend this book if you are involved in the fields of machine learning and artificial intelligence, but be prepared to wrestle with the material in order to understand it fully.

Great looking book

Livre parfait pour les personnes avec un bon background statistique, sinon je vous recommande pattern recognition and machine learning de Christopher M. Bishop qui repars de la base mais tend à suggérer une pensée plus bayesienne. Lire les deux vous donnera une vision Clair d'un peut prêt tout sur le machine learning hors réseau de neurones.

Questo volume è fondamentale per chiunque voglia approfondire le proprie basi (teoriche...) sull'apprendimento statistico. Scritto dai titani del campo, è un libro omnicomprensivo che, partendo dalle basi (nei primi capitoli, probabilmente per introdurli in maniera strumentale alla trattazione sviluppata, vengono descritte le tecniche base di regressione e classificazione) arriva a descrivere concetti molto più complessi e avanzati, come le varie tecniche di regolarizzazione (Ridge, LASSO), il metodo di Benjamini-Hochberg, le SVM etc.

Uno de los mejores libros que he leido sobre la tematica de machinelearning. Muy interesante y ligero de leer. Todo perfectamente explicado

Great book! Yet you wont learn much purely by reading it if you are newbie / student in statistics and or data science! Authors are the most revered alive researchers in the field of statistics!