Eugenio Fco. Sánchez Úbeda

Supervised automatic learning models: A new perspective.

Huge amounts of data are available in many disciplines of Science and Industry. In order to extract useful information from these data, a large number of apparently very different learning approaches have been created during the last decades. There are learning methods developed in the context of machine learning, neural networks, statistics or neurofuzzy systems, to name a few. Each domain uses its own terminology (often incomprehensible to outsiders), even though all approaches basically attempt to solve the same generic learning tasks.

Nowadays hundreds of papers related to the approach of “learning from data” are published each year. Although they are essential for the progress of Science, this also produces an increasing fragmentation of knowledge in automatic learning methods.
The aim of this paper is to present a new perspective on the main existing automatic learning strategies, by providing a general framework to handle and unify many of the existing supervised learning models. Furthermore, the paper focuses on the underlying ideas and their practical implications, rather than on purely theoretical aspects.
First the supervised learning problem is stated using an adequate level of generality. Although this problem can be easily declared, no completely accepted method exists to solve it. Reasons for this difficulty are provided.
Next common one-dimensional supervised models are discussed. According to the proposed taxonomy, these basic models are classified into two main different groups: structured and unstructured models. Although this review is incomplete and necessarily brief, the main purpose is to provide a simple framework to fit the ideas underlying most of the existing onedimensional supervised models. Furthermore, it allows highlighting the similarity of some models whose original motivation comes from different fields, like engineering, statistics or mathematics. These different origins explain their markedly different presentation in the 'classical' literature.

High-dimensional models are shown as a composition of one-dimensional models, using the latter as elementary building blocks. These multidimensional models have also been classified into two major groups: structured and unstructured models. Classical neural networks such as multilayer perceptrons or Radial Basis Functions Networks are examples of the first type, whereas the standard K-Nearest Neighbors model is a basic unstructured model. Three main techniques for dealing with the multidimensional problem are described. Multidimensional models are based on some of these techniques.

Due to space limitations, a minority of the vast array of different supervised models is covered. These models have been developed in fields traditionally considered as completely different, like statistics, machine learning or neural network. However, the set of described models is large enough to provide a rich overview of the main principles and methods underlying most of the supervised models proposed in the growing literature on automatic learning. In order to clarify ideas, practical examples are provided.