《基于自然语言处理的文本分类分析与研究》由会员分享，可在线阅读，更多相关《基于自然语言处理的文本分类分析与研究（57页珍藏版）》请在金锄头文库上搜索。
1、分类号：密级：公开学号：2008179单位代码： 1 0 4 0 7硕士学位论文论文题目:基于自然语言处理的文本分类分析与研究研专究业方名向称数据挖掘计算机应用技术研究生姓名导师姓名、职称张春燕刘发升教授2011 年 6 月 3 日江西赣州摘要当今，随着信息技术不断向前发展，在机器学习这门计算机学科中，有关模式识别的理论知识已经日趋成熟，同时应用到了许多领域，其中一个重要的研究方向是基于统计的自然语言处理。由于互联网的兴起，基于自然语言表述的电子文本信息越来越多，在这么多的电子文本信息中，基于自然语言的信息处理的一个最大目标是怎样有效地获取和管理这些信息。对于提出的这些问题需要对自然语言进行研
2、究和其相关应用，文本分类在这里显得特别重要，它是作为信息检索等问题的基础。文本分类主要分为两个阶段，分别采用了自然语言处理、机器学习、模式识别、文本挖掘技术来实现。因此，文本分类在理论研究上的价值体现在对这些技术的推动。文本分类能够有效的提高网上信息检索的效果，不仅是改进信息获取模式的重要方面，也是内容安全的基础部分。因此分类性能的好坏已经成为关注的焦点，研究文本分类任务的理论和工程应用，将具有重要意义。在现有的研究成果上，本论文对文本分类及其相关技术做了一些研究。首先介绍了文本分类技术的研究现状及该课题研究意义；接着介绍了文本分类的过程和在这个过程中所用到的相关技术，主要对中文分词方法、特征
3、选择方法和文本分类算法做了研究；然后介绍文本分类设计的整个过程，在预处理的过程中为了对三字长交集型歧义字段消除歧义及对停用词的处理，对最大匹配分词法进行了改进，同时在基于KL散度的特征选取法基础上并且结合了特征项的TFIDF权值，这样选取出的特征项能比较准确的表达文本内容，为分类打好基础，最后对贝叶斯算法、简单向量距离分类法和 KNN（K最近邻居）算法进行了比较，找出其分类效果最佳并结合时间复杂度选出一种实用性比较好的算法。关键词：自然语言处理；文本分类；数据挖掘；特征选取IIAbstractNowadays, with the information technology developme
4、nt, machine learning and patternrecognition in computer science are more and more mature and widely applied to many areas,one of the important research direction is based on the statistics of natural languageprocessing. Due to the rise of the Internet, the electronic text information based on natura
5、llanguage description is exploding, the information processing based on natural language isone of the biggest target how effective these information acquisition and management. Thesequestions raised a lot of research and applications on natural language, of which textcategorization as the basis for
6、information retrieval problems, is especially paid attention.Text categorization mainly divided into two stages, using natural language processing,machine learning, pattern recognition, text mining technology to realize. Therefore, thevalue of text classification theory research reflected in these t
7、echnologies. Textclassification can effectively improve the effect of online information retrieval, not only toimprove the information of acquisition modes, but also an important aspect of contentsecurity. Therefore classified the performance has become the focus of attention, theresearch of text cl
8、assification task and engineering application, will be having theimportant meaning.In the existing research results, text categorization and related technologies have beendone some research. The beginning of the thesis introduces the status of the text classificationand the research significance; Th
9、en it introduces text classification process andthe relatedtechnologies in the process, it also has researched the Chinese word segmentation method,feature selection method, text classification algorithm ; the thesis introduces the design of thetext categorization, the process in order to eliminate
10、ambiguity for three characters longambiguous phrases of overlap type and process stop words, the best match points of lexicalwas improved, meanwhile based on KL dispersion degree feature selection method andcombining the characteristics of TFIDF weights, such a feature selection can compare toaccura
11、tely express the text, lay a good foundation for classification, Finally to the Bayesalgorithm, simple vector distance classification and KNN (K nearest neighbors) algorithm,the thesis had found out the classification results compared with time complexity andselected a better practical algorithm.IIIKey words: Nature Language Processing；Text Classification；Data Mining；FeatureSelection and ExtractionIV目录硕士学位论文 . i摘要.IIAbstract . III第一章绪论. 11.1课题研究背景与意义. 11.2文本分类研究现状. 21.3课题研究内容及方法. 4第二章自然语
