Professor Ricardo Gabrielli Barreto Campello

Clustering genes based on their expression profiles is usually the first step in gene-expression data analysis. Among the many algorithms that can be applied to gene clustering, the k-means algorithm is one of the most popular techniques. This is mainly due to its ease of comprehension, implementation, and interpretation of the results. However, k-means suffers from some problems, such as the need to define a priori the number of clusters (k) and the possibility of getting trapped into local optimal solutions. Evolutionary algorithms for clustering, by contrast, are known for being capable of performing broad searches over the space of possible solutions and can be used to automatically estimate the number of clusters. This work elaborates on an evolutionary algorithm specially designed to solve clustering problems and shows how it can be used to optimize the k-means algorithm. The performance of the resultant hybrid approach is illustrated by means of experiments in several bioinformatics datasets with multiple measurements, which are expected to yield more accurate and more stable clusters. Two different measures (Euclidean and Pearson) are employed for computing (dis)similarities between genes. A review of the use of evolutionary algorithms for gene-expression data processing is also included. © 2007 Springer-Verlag Berlin Heidelberg.

HDBSCAN*, a state-of-the-art density-based hierarchical clustering method, produces a hierarchical organization of clusters in a dataset w.r.t. a parameter mpts. While small changes in mpts typically lead to a small change in the clustering structure, choosing a “good” mpts value can be challenging: depending on the data distribution, a high or low mpts value may be more appropriate, and certain clusters may reveal themselves at different values. To explore results for a range of mpts values, one has to run HDBSCAN* for each value independently, which can be computationally impractical. In this paper, we propose an approach to efficiently compute all HDBSCAN* hierarchies for a range of mpts values by building upon results from computational geometry to replace HDBSCAN*'s complete graph with a smaller equivalent graph. An experimental evaluation shows that our approach can obtain over one hundred hierarchies for the computational cost equivalent to running HDBSCAN* about twice, which corresponds to a speedup of more than 60 times, compared to running HDBSCAN* independently that many times. We also propose a series of visualizations that allow users to analyze a collection of hierarchies for a range of mpts values, along with case studies that illustrate how these analyses are performed.

Grouping data points is one of the fundamental tasks in data mining, which is commonly known as clustering if data points are described by attributes. When dealing with interrelated data, that is represented in the form a graph wherein a link between two nodes indicates a relationship between them, there has been a considerable number of approaches proposed in recent years for mining communities in a given network. However, little work has been done on how to evaluate the community mining algorithms. The common practice is to evaluate the algorithms based on their performance on standard benchmarks for which we know the ground-truth. This technique is similar to external evaluation of attribute-based clustering methods. The other two well-studied clustering evaluation approaches are less explored in the community mining context; internal evaluation to statistically validate the clustering result and relative evaluation to compare alternative clustering results. These two approaches enable us to validate communities discovered in a real-world application, where the true community structure is hidden in the data. In this article, we investigate different clustering quality criteria applied for relative and internal evaluation of clustering data points with attributes and also different clustering agreement measures used for external evaluation and incorporate proper adaptations to make them applicable in the context of interrelated data. We further compare the performance of the proposed adapted criteria in evaluating community mining results in different settings through extensive set of experiments.

This paper provides an overview of system identification using orthonormal basis function models, such as those based on Laguerre, Kautz, and generalised orthonormal basis functions. The paper is separated in two parts. The first part of the paper approached issues related with linear models and models with uncertain parameters. Now, the mathematical foundations as well as their advantages and limitations are discussed within the contexts of non-linear system identification. The discussions comprise a broad bibliographical survey of the subject and a comparative analysis involving some specific model realisations, namely, Volterra, fuzzy, and neural models within the orthonormal basis functions framework. Theoretical and practical issues regarding the identification of these non-linear models are presented and illustrated by means of two case studies. Copyright © 2012 Inderscience Enterprises Ltd.

This paper provides an overview of system identification using orthonormal basis function models, such as those based on Laguerre, Kautz, and generalised orthonormal basis functions. The paper is separated in two parts. In this first part, the mathematical foundations of these models as well as their advantages and limitations are discussed within the context of linear and robust system identification. The second part approaches the issues related with non-linear models. The discussions comprise a broad bibliographical survey of the subjects involving linear models within the orthonormal basis functions framework. Theoretical and practical issues regarding the identification of these models are presented and illustrated by means of a case study involving a polymerisation process. Copyright © 2011 Inderscience Enterprises Ltd.

Many different relative clustering validity criteria exist that are very useful in practice as quantitative measures for evaluating the quality of data partitions, and new criteria have still been proposed from time to time. These criteria are endowed with particular features that may make each of them able to outperform others in specific classes of problems. In addition, they may have completely different computational requirements. Then, it is a hard task for the user to choose a specific criterion when he or she faces such a variety of possibilities. For this reason, a relevant issue within the field of clustering analysis consists of comparing the performances of existing validity criteria and, eventually, that of a new criterion to be proposed. In spite of this, the comparison paradigm traditionally adopted in the literature is subject to some conceptual limitations. The present paper describes an alternative, possibly complementary methodology for comparing clustering validity criteria and uses it to make an extensive comparison of the performances of 40 criteria over a collection of 962,928 partitions derived from five well-known clustering algorithms and 1080 different data sets of a given class of interest. A detailed review of the relative criteria under investigation is also provided that includes an original comparative asymptotic analysis of their computational complexities. This work is intended to be a complement of the classic study reported in 1985 by Milligan and Cooper as well as a thorough extension of a preliminary paper by the authors themselves. © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 209-235, 2010.

In this paper, an overview about the identification of dynamic systems using orthonormal basis function models, such as those based on Laguerre and Kautz functions, is presented. The mathematical foundations of these models as well as their advantages and limitations are discussed within the contexts of linear, robust, and nonlinear identification. The discussions comprise a broad bibliographical survey on the subject and a comparative analysis involving some specific model realizations, namely, linear, Volterra, fuzzy, and neural models within the orthonormal basis function framework. Theoretical and practical issues regarding the identification of these models are also presented and illustrated by means of two case studies related to a polymerization process.

This paper presents an overview about predictive control schemes based on orthonormal basis function models. Different predictive control schemes based on such models are discussed, namely, linear controllers with terminal (stabilizing) constraints, robust controllers, and non-linear controllers. The discussions comprise a broad bibliographical survey on the subject as well as two case studies involving a simulated dynamic system and a real process.

User profiling is an important aspect of recommender systems. It models users' preferences and is used to assess an item's relevance to a particular user. In this paper we propose a profiling approach which describes and enriches the users' preferences using multiple types of interactions. We show in our experiments that the enriched version of users' profiles is able to provide better recommendations.

Recommender systems were created to represent user preferences for the purpose of suggesting items to purchase or examine. However, there are several optimizations to be made in these systems mainly with respect to modeling the user profile and remove the noise information. This paper proposes a collaborative filtering approach based on preferences of groups of users to improve the accuracy of recommendation, where the distance among users is computed using multiple types of users' feedback. The advantage of this approach is that relevant items will be suggested based only on the subjects of interest of each group of users. Using this technique, we use a state-of-art collaborative filtering algorithm to generate a personalized ranking of items according to the preferences of an individual within each cluster. The experimental results show that the proposed technique has a higher precision than the traditional models without clustering.

Outlier detection and ensemble learning are well established research directions in data mining yet the application of ensemble techniques to outlier detection has been rarely studied. Building an ensemble requires learning of diverse models and combining these diverse models in an appropriate way. We propose data perturbation as a new technique to induce diversity in individual outlier detectors as well as a rank accumulation method for the combination of the individual outlier rankings in order to construct an outlier detection ensemble. In an extensive evaluation, we study the impact, potential, and shortcomings of this new approach for outlier detection ensembles. We show that this ensemble can significantly improve over weak performing base methods. © Copyright 2014 ACM 978-1-4503-2722-0/14/06¿$15.00.

One of the most challenging aspects of clustering is validation, which is the objective and quantitative assessment of clustering results. A number of different relative validity criteria have been proposed for the validation of globular, clusters. Not all data, however, are composed of globular clusters. Density-based clustering algorithms seek partitions with high density areas of points (clusters, not necessarily globular) separated by low density areas, possibly containing noise objects. In these cases relative validity indices proposed for globular cluster validation may fail. In this paper we propose a relative validation index for density-based, arbitrarily shaped clusters. The index assesses clustering quality based on the relative density connection between pairs of objects. Our index is formulated on the basis of a new kernel density function, which is used to compute the density of objects and to evaluate the within- and between-cluster density connectedness of clustering results. Experiments on synthetic and real world data show the effectiveness of our approach for the evaluation and selection of clustering algorithms and their respective appropriate parameters.

Although there is a large and growing literature that tackles the semi-supervised clustering problem (i.e., using some labeled objects or cluster-guiding constraints like \must-link" or \cannot-link"), the evaluation of semi-supervised clustering approaches has rarely been discussed. The application of cross-validation techniques, for example, is far from straightforward in the semi-supervised setting, yet the problems associated with evaluation have yet to be addressed. Here we summarize these problems and provide a solution. Furthermore, in order to demonstrate practical applicability of semi-supervised clustering methods, we provide a method for model selection in semi-supervised clustering based on this sound evaluation procedure. Our method allows the user to select, based on the available information (labels or constraints), the most appropriate clustering model (e.g., number of clusters, density-parameters) for a given problem.

Many different relative clustering validity criteria exist that are very useful as quantitative measures for assessing the quality of data partitions. These criteria are endowed with particular features that may make each of them more suitable for specific classes of problems. Nevertheless, the performance of each criterion is usually unknown a priori by the user. Hence, choosing a specific criterion is not a trivial task. A possible approach to circumvent this drawback consists of combining different relative criteria in order to obtain more robust evaluations. However, this approach has so far been applied in an ad-hoc fashion only; its real potential is actually not well-understood. In this paper, we present an extensive study on the combination of relative criteria considering both synthetic and real datasets. The experiments involved 28 criteria and 4 different combination strategies applied to a varied collection of data partitions produced by 5 clustering algorithms. In total, 427,680 partitions of 972 synthetic datasets and 14,000 partitions of a collection of 400 image datasets were considered. Based on the results, we discuss the shortcomings and possible benefits of combining different relative criteria into a committee. Categories and Subject Descriptors H.2.8 [Database Applications]: Data Mining General Terms Measurement, Experimentation Keywords clustering validation, relative validity criteria, combinations of validity criteria. Copyright © 2013 ACM.

Outlier detection and ensemble learning are well established research directions in data mining yet the application of ensemble techniques to outlier detection has been rarely studied. Here, we propose and study subsampling as a technique to induce diversity among individual outlier detectors. We show analytically and experimentally that an outlier detector based on a subsample per se, besides inducing diversity, can, under certain conditions, already improve upon the results of the same outlier detector on the complete dataset. Building an ensemble on top of several subsamples is further improving the results. While in the literature so far the intuition that ensembles improve over single outlier detectors has just been transferred from the classification literature, here we also justify analytically why ensembles are also expected to work in the unsupervised area of outlier detection. As a side effect, running an ensemble of several outlier detectors on subsamples of the dataset is more efficient than ensembles based on other means of introducing diversity and, depending on the sample rate and the size of the ensemble, can be even more efficient than just the single outlier detector on the complete data.

We propose a theoretically and practically improved density-based, hierarchical clustering method, providing a clustering hierarchy from which a simplified tree of significant clusters can be constructed. For obtaining a "flat" partition consisting of only the most significant clusters (possibly corresponding to different density thresholds), we propose a novel cluster stability measure, formalize the problem of maximizing the overall stability of selected clusters, and formulate an algorithm that computes an optimal solution to this problem. We demonstrate that our approach outperforms the current, state-of-the-art, density-based clustering methods on a wide variety of real world data. © Springer-Verlag 2013.

One of the challenges for clustering resides in dealing with huge amounts of data, which causes the need for distribution of large data sets in separate repositories. However, most clustering techniques require the data to be centralized. One of them, the k-means, has been elected one of the most influential data mining algorithms. Although exact distributed versions of the k-means algorithm have been proposed, the algorithm is still sensitive to the selection of the initial cluster prototypes and requires that the number of clusters be specified in advance. This work tackles the problem of generating an approximated model for distributed clustering, based on k-means, for scenarios where the number of clusters of the distributed data is unknown. We propose a collection of algorithms that generate and select k-means clustering for each distributed subset of the data and combine them afterwards. The variants of the algorithm are compared from two perspectives: the theoretical one, through asymptotic complexity analyses, and the experimental one, through a comparative evaluation of results obtained from a collection of experiments and statistical tests. © 2012 IEEE.

Cluster analysis is usually the first step adopted to unveil information from gene expression data. One of its common applications is the clustering of cancer samples, associated with the detection of previously unknown cancer subtypes. Although guidelines have been established concerning the choice of appropriate clustering algorithms, little attention has been given to the subject of proximity measures. Whereas the Pearson correlation coefficient appears as the de facto proximity measure in this scenario, no comprehensive study analyzing other correlation coefficients as alternatives to it has been conducted. Considering such facts, we evaluated five correlation coefficients (along with Euclidean distance) regarding the clustering of cancer samples. Our evaluation was conducted on 35 publicly available datasets covering both (i) intrinsic separation ability and (ii) clustering predictive ability of the correlation coefficients. Our results support that correlation coefficients rarely considered in the gene expression literature may provide competitive results to more generally employed ones. Finally, we show that a recently introduced measure arises as a promising alternative to the commonly employed Pearson, providing competitive and even superior results to it. © 2012 Springer-Verlag.

An improved approach to determine exact search directions for the optimization of Volterra models based on Generalized Orthonormal Bases of Functions (GOBF) is proposed. The proposed approach extends the work in [7], where a novel, exact technique for optimizing the GOBF parameters (poles) for Volterra models of any order was presented. The proposed extensions take place in two different ways: (i) the formulation here is derived in such a way that each multidimensional kernel of the model is decomposed into a set of independent orthonormal bases (rather than a single, common basis), each of which is parameterized by an individual set of poles intended for representing the dominant dynamic of the kernel along a particular dimension; and (ii) a novel, more computationally efficient method to analytically and recursively calculate the search directions (gradients) for the bases poles is derived. A simulated example is presented to illustrate the performance of the proposed approach. A comparison between the proposed method, which uses asymmetric kernels with multiple orthonormal bases, and the original method, which uses symmetric kernels with a single basis, is presented. © 2011 IEEE.

The features describing a data set may often be arranged in meaningful subsets, each of which corresponds to a different aspect of the data. An unsupervised algorithm (SCAD) that performs fuzzy clustering and aspects weighting simultaneously was recently proposed. However, there are several situations where the data set is represented by proximity matrices only (relational data), which renders several clustering approaches, including SCAD, inappropriate. To handle this kind of data, the relational clustering algorithm CARD, based on the SCAD algorithm, has been recently developed. However, CARD may fail and halt given certain conditions. To fix this problem, its steps are modified and then reordered to also reduce the number of parameters required. The improved CARD is assessed over hundreds of real and artificial data sets. © 2011 IEEE.

Simplified Silhouette Filter (SSF) is a recently introduced feature selection method that automatically estimates the number of features to be selected. To do so, a sampling strategy is combined with a clustering algorithm that seeks clusters of correlated (potentially redundant) features. It is well known that the choice of a similarity measure may have great impact in clustering results. As a consequence, in this application scenario, this choice may have great impact in the feature subset to be selected. In this paper we study six correlation coefficients as similarity measures in the clustering stage of SSF, thus giving rise to several variants of the original method. The obtained results show that, in particular scenarios, some correlation measures select fewer features than others, while providing accurate classifiers. © 2010 IEEE.

Many different relative clustering validity criteria exist that are very useful in practice as quantitative measures for evaluating the quality of data partitions, and new criteria have still been proposed from time to time. These criteria are endowed with particular features that may make each of them able to outperform others in specific classes of problems. Then, it is a hard task for the user to choose a specific criterion when he or she faces such a variety of possibilities. For this reason, a relevant issue within the field of cluster analysis consists of comparing the performances of existing validity criteria and, eventually, that of a new criterion to be proposed. In spite of this, there are some conceptual flaws in the comparison paradigm traditionally adopted in the literature. The present paper presents an alternative methodology for comparing clustering validity criteria and uses it to make an extensive comparison of the performances of 4 well-known validity criteria and 20 variants of them over a collection of 142,560 partitions of 324 different data sets of a given class of interest.

This paper presents a Learning Object in the domain of Computational Intelligence that can be used in graduate and undergraduate courses. Additionally, it can be reused in other contexts and scenarios, such as a distance learning course on Artificial Intelligence. © 2006 IEEE.

OBF (Orthonormal Basis Function) Fuzzy models have shown to be a promising approach to the areas of nonlinear system identification and control since they exhibit several advantages over those dynamic model topologies usually adopted in the literature. Although encouraging application results have been obtained, no automatic procedure had yet been developed to optimize the design parameters of these models. This paper elaborates on the use of a genetic algorithm (GA) especially designed for this task, in which a fitness function based on the Akaike information criterion plays a key role by considering both model accuracy and parsimony aspects. The use of linear (actually affine) and nonlinear local models is also investigated. The proposed methodology is evaluated in the modeling of a real nonlinear magnetic levitation system. © 2006 IEEE.

This paper elaborates on the improvement of an evolutionary algorithm for clustering (EAC) introduced in previous work. Four new features are proposed and empirically assessed in seven datasets, using two fitness functions. Statistical analyses allow concluding that two proposed features lead to significant improvements on the original EAC. Such features have been incorporated into the EAC, resulting in a more computationally efficient algorithm called F-EAC (Fast EAC). We describe as an additional contribution a methodology for evaluating evolutionary algorithms for clustering in such a way that the influence of the fitness function is lessened in the assessment process, what yields analyses specially focused on the evolutionary operators. © 2006 IEEE.

The present paper involves the approximation of nonlinear systems using Wiener/Volterra models with Kautz orthonormal functions. It focuses on the problem of selecting the free complex pole which characterizes these functions. The problem is solved by minimizing an upper bound of the error arising from the truncated approximation of Volterra kernels using Kautz functions. An analytical solution for the optimal choice of one of the parameters related to the Kautz pole is thus obtained, with the results valid for any-order Wiener/Volterra models. An example illustrates the application of the mathematical results derived. Copyright © 2005 IFAC.

This paper is concerned with the optimization of Laguerre bases for the orthonormal series expansion of discrete-time Volterra models. Fu and Dumont (1993) approached this problem in the context of linear systems by minimizing an upper bound for the error resulting from the truncated Laguerre expansion of impulse response models, which are equivalent to first-order Volterra models. The present work generalizes the work mentioned above to Volterra models of any order. The main result is the derivation of analytic strict global solutions for the optimal expansion of the Volterra kernels either using an independent Laguerre basis for each kernel or using a common basis for all the kernels.

This paper is concerned with the selection of optimal Laguerre bases for the orthonormal series expansion of discrete-time Volterra models. The aim is to minimize the number of functions needed to provide a given approximation accuracy, thus simplifying the modeling and control problems associated with these models. This issue was addressed by Fu and Dumont [8] focusing on linear systems, which are equivalent to a first-order Volterra model. The present work is a generalization of the work mentioned above focusing on second-order models. The main result is an analytic strict global solution for the optimal expansion of the second-order Volterra kernel. An example is provided to illustrate some theoretical aspects of the mathematical results presented in the paper.

This work presents a novel framework for fuzzy modeling of dynamic systems using orthonormal basis functions in the representation of the model input signals. The main objective of using orthonormal bases is to overcome the task of estimating the order and time delay of the process. The result is a nonlinear moving average fuzzy model which, consequently, has no feedback of prediction errors. Although any technique of fuzzy modeling can be used in the proposed framework, a relational approach is considered. The performance of fuzzy models with orthonormal basis functions is illustrated by examples and the results are compared with those provided by conventional fuzzy models and Volterra models.