Mining unknown patterns in data when the features are correlated

Robert S. Lynch Jr.; Peter K. Willett

doi:10.1117/12.719423

9 April 2007 Mining unknown patterns in data when the features are correlated

Proceedings Volume 6570, Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2007; 657008 (2007) https://doi.org/10.1117/12.719423
Event: Defense and Security Symposium, 2007, Orlando, Florida, United States

Abstract

In this paper, a previously introduced data mining technique, utilizing the Mean Field Bayesian Data Reduction Algorithm (BDRA), is extended for use in finding unknown data clusters in a fused multidimensional feature space. In extending the BDRA for this application its built-in dimensionality reduction aspects are exploited for isolating and automatically mining all points contained in each unknown cluster. In previous work, this approach was shown to have comparable performance to the classifier that knows all cluster information when mining up to two features containing multiple unknown clusters. However, unlike results shown in previous work based on lower dimensional feature spaces, the results in this paper are based on utilizing up to twenty fused features. This is due to improvements in the training algorithm that now mines for candidate data clusters by processing all points in a quantized cell simultaneously. This is opposed to the previous method that processed all points sequentially. This improvement in processing has resulted in a substantial reduction in the run time of the algorithm. Finally, performance is illustrated and compared with simulated data containing multiple clusters, and where the relevant feature space contains both correlated and uncorrelated classification information.

Citation Download Citation

Robert S. Lynch Jr. and Peter K. Willett "Mining unknown patterns in data when the features are correlated", Proc. SPIE 6570, Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2007, 657008 (9 April 2007); https://doi.org/10.1117/12.719423

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available