Lectures - Dr. Valeriya Naumova - 16 / 24 October
Regularization Algorithms in Learning Theory
Objective
Making accurate predictions is a crucial factor in many systems (such as in medical treatments and prevention, geomathematics, social dynamics, financial computations) for cost savings, efficiency, health, safety, and organizational purposes. Learning theory and Machine learning provide a suitable framework and effective algorithms for a broad spectrum of real-life applications. The set of techniques based on these research areas already became a key technology to extract information from a vast amount of unstructured data around us and make sense of it.Approaches developed in the framework of learning theory are very much dependent on the quality of measured data. However, the situation mostly encountered in real-life applications is to have only at disposal incomplete or rough high-dimensional data, and extracting a predictive model from them is an impossible task unless one can rely on some a-priori knowledge of properties of the expected model. The impossibility of making a reliable prediction is the result of the combination of different factors, the most relevant being the incompleteness of the data, the roughness/noisiness of the data, and their intrinsic high-dimensional nature.
In order to break simultaneously all of these negative factors playing against us in a predictive attempt, we shall use regularization methods in learning theory.
The main goals of the proposed course are to discuss the connections between regularization theory and learning theory by reviewing the state of the art machine learning algorithms and discussing strategies of further development.
Topics to be covered within the Course
- Introduction: Statistical Learning and Machine Learning
- Supervised Learning in Reproducing Kernel Hilbert Spaces: learning as an inverse problem
- Tikhonov Regularization and the Representer Theorem
- Regularization Algorithms in Learning Theory. Multi-parameter regularization in Learning Theory
- Sparsity-Based Regularization / Multitask learning
- Learning Theory Approach to the Adaptive Regularization
- Meta-learning approach to regularization -- case study and possible applications