Scalable mining for classification rules in relational databases

View/ Open
Issue Date
2004Author
Wang, Min
Iyer, Bala
Vitter, Jeffrey Scott
Publisher
Institute of Mathematical Statistics
Type
Article
Article Version
Scholarly/refereed, publisher version
Metadata
Show full item recordAbstract
Data mining is a process of discovering useful patterns (knowledge)
hidden in extremely large datasets. Classification is a fundamental data mining
function, and some other functions can be reduced to it. In this paper we
propose a novel classification algorithm (classifier) called MIND (MINing in
Databases). MIND can be phrased in such a way that its implementation is
very easy using the extended relational calculus SQL, and this in turn allows
the classifier to be built into a relational database system directly. MIND is
truly scalable with respect to I/O efficiency, which is important since scalability
is a key requirement for any data mining algorithm.
We have built a prototype of MIND in the relational database management
system DB2 and have benchmarked its performance. We describe the working
prototype and report the measured performance with respect to the previous
method of choice. MIND scales not only with the size of datasets but also
with the number of processors on an IBM SP2 computer system. Even on
uniprocessors, MIND scales well beyond dataset sizes previously published for
classifiers.We also give some insights that may have an impact on the evolution
of the extended relational calculus SQL.
Description
doi:10.1214/lnms/1196285404
Collections
Citation
M. Wang, B. Iyer, and J. S. Vitter. “Scalable Mining for Classification Rules in Relational Databases,” Herman Rubin Festschrift, Lecture Notes Monograph Series, 45, Institute of Mathematical Statistics, Hayward, CA, Fall 2004. An extended abstract appears in Proceedings of the International Database Engineering & Application Symposium (IDEAS ’98), Cardiff, Wales, July 1998, 58–67. A shorter version appears in Proceedings of the ACM SIGMOD Data Mining and Knowledge Discovery Workshop (DMKD ’98), Seattle, WA, June 1998. http://dx.doi.org/10.1214/lnms/1196285404
Items in KU ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
We want to hear from you! Please share your stories about how Open Access to this item benefits YOU.