Loading...
Scalable mining for classification rules in relational databases
Wang, Min Iyer, Bala Vitter, Jeffrey Scott
Wang, Min
Iyer, Bala
Vitter, Jeffrey Scott
Citations
Altmetric:
Abstract
Data mining is a process of discovering useful patterns (knowledge)
hidden in extremely large datasets. Classification is a fundamental data mining
function, and some other functions can be reduced to it. In this paper we
propose a novel classification algorithm (classifier) called MIND (MINing in
Databases). MIND can be phrased in such a way that its implementation is
very easy using the extended relational calculus SQL, and this in turn allows
the classifier to be built into a relational database system directly. MIND is
truly scalable with respect to I/O efficiency, which is important since scalability
is a key requirement for any data mining algorithm.
We have built a prototype of MIND in the relational database management
system DB2 and have benchmarked its performance. We describe the working
prototype and report the measured performance with respect to the previous
method of choice. MIND scales not only with the size of datasets but also
with the number of processors on an IBM SP2 computer system. Even on
uniprocessors, MIND scales well beyond dataset sizes previously published for
classifiers.We also give some insights that may have an impact on the evolution
of the extended relational calculus SQL.
Description
doi:10.1214/lnms/1196285404
Date
2004
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Mathematical Statistics
Research Projects
Organizational Units
Journal Issue
Keywords
Citation
M. Wang, B. Iyer, and J. S. Vitter. “Scalable Mining for Classification Rules in Relational Databases,” Herman Rubin Festschrift, Lecture Notes Monograph Series, 45, Institute of Mathematical Statistics, Hayward, CA, Fall 2004. An extended abstract appears in Proceedings of the International Database Engineering & Application Symposium (IDEAS ’98), Cardiff, Wales, July 1998, 58–67. A shorter version appears in Proceedings of the ACM SIGMOD Data Mining and Knowledge Discovery Workshop (DMKD ’98), Seattle, WA, June 1998. http://dx.doi.org/10.1214/lnms/1196285404