Improving the text classification using clustering and a novel HMM to reduce the dimensionality
DATE:
2016-11
UNIVERSAL IDENTIFIER: http://hdl.handle.net/11093/6105
EDITED VERSION: https://linkinghub.elsevier.com/retrieve/pii/S016926071530050X
UNESCO SUBJECT: 1203.17 Informática
DOCUMENT TYPE: article
ABSTRACT
In text classification problems, the representation of a document has a strong impact on the performance of learning systems. The high dimensionality of the classical structured representations can lead to burdensome computations due to the great size of real-world data. Consequently, there is a need for reducing the quantity of handled information to improve the classification process. In this paper, we propose a method to reduce the dimensionality of a classical text representation based on a clustering technique to group documents, and a previously developed Hidden Markov Model to represent them. We have applied tests with the k-NN and SVM classifiers on the OHSUMED and TREC benchmark text corpora using the proposed dimensionality reduction technique. The experimental results obtained are very satisfactory compared to commonly used techniques like InfoGain and the statistical tests performed demonstrate the suitability of the proposed technique for the preprocessing step in a text classification task.
Files in this item
![pdf [PDF]](/xmlui/themes/Mirage2/images/thumbnails/mimes/pdf.png)
- Name:
- 2016_seara_improving_text.pdf
- Size:
- 496.4Kb
- Format:
- Description:
- Manuscrito aceptado