On Arxiv, about 100 Machine Learning (ML) papers are published every day. That is equivalent to ~30K papers per year. Hence it becomes a daunting task to even follow what the major innovations or breakthroughs are.

In this case, it is important to learn some strategies to detect low quality research papers and start studying ML by reading the most fundamental papers. I will not go into details of detecting “low quality” papers but here’s the choice of Mike Tipping, a Professor of Data Science and former consultant on many Microsoft ML research projects at Cambridge on where to start reading if you are looking for some really important papers in the field.

• Stochastic variational inference by Hoffman, Wang, Blei and Paisley

  http://arxiv.org/abs/1206.7051

  A stochastic variational inference method, a scalable algorithm for approximating posterior distributions is developed to do approximate inference for probabilistic models with potentially billions of data.

• Austerity in MCMC Land: Cutting the Metropolis Hastings by Korattikara, Chen and Welling

  http://arxiv.org/abs/1304.5299

  Probabilistic models for again big data.

• Practical Bayesian Optimization of Machine Learning Algorithms by Snoek, Larochelle and Adams

  http://arxiv.org/abs/1206.2944

  Machine learning algorithms frequently require careful tuning of model hyperparameters, regularization terms, and optimization parameters. Unfortunately, this tuning is often a “black art” that requires expert experience, unwritten rules of thumb, or sometimes brute-force search. Much more appealing is the idea of developing automatic approaches which can optimize the performance of a given learning algorithm to the task at hand. In this work, the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm’s generalization performance is modeled as a sample from a Gaussian process (GP) is considered.

• Kernel Bayes Rule by Fukumizu, Song, Gretton

  http://arxiv.org/abs/1009.5736

  A nonparametric kernel-based method for realizing Bayes’ rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces.

• ImageNet Classification with Deep Convolutional Neural Networks

  http://www.cs.toronto.edu/~hinton/absps/imagenet.pdf

  This paper has started it all! It triggered the revolution that’s been happening in the last decade in the AI world. It is widely regarded as one of the most influential publications in the field.