摘 要
随着互联网的发展,以及计算机和信息技术的不断更新换代,网络上存储的信息越来越丰富。文本作为信息的有效表现形式,数量也增长迅速。近年来,随着云计算和大数据的兴起,使得海量的文本信息得到有效的组织和管理。 如何高效、准确地获取到文本中的有效信息成为当前文本挖掘、信息检索、网络舆情分析等行业首要解决的问题。
本文文本处理预处理选用词袋模型,词袋模型将一段文本作为一个个分离的词进行处理,通过不同类文本中可能出现词的差异对文本进行分类。使用one-hot编码来处理离散型特征;特征提取选用IF-IDF算法,实现提取文章中关键词的目的;聚类算法的选取:选用DBSCAN算法,由于并不清楚聚类的簇数量,并且海量文本迭代的效率较低。系统实现基于Matlab GUI,系统初步设计完成后,对系统进行性能测试及可行性分析。
关键词:MATLAB;GUI;文本处理;文本分类;词袋模型
Abstract
With the development of the Internet, and the continuous upgrading of computers and information technology, the information stored on the network is more and more rich. As an effective form of information, text is also growing rapidly. In recent years, with the rise of cloud computing and big data, massive text information has been effectively organized and managed. How to efficiently and accurately obtain the effective information in the text has become the primary problem to be solved in the current text mining, information retrieval, network public opinion analysis and other industries. In this paper, the bag model is selected in the text processing pretreatment.
The bag model processes a paragraph of text as a separate word, and classifies the text through the difference of possible words in different types of texts. The one-hot code is used to handle discrete features; IF-IDF algorithm is used for feature extraction to extract the key words in the article; clustering algorithm:DBSCAN algorithm, because the number of clusters is not clear, and the efficiency of massive text iteration is low. The system implementation is based on Matlab GUI, after the preliminary system design, the system performance test and feasibility analysis.
Key words:MATLAB; GUI; text processing; text classification; word bag model
目录
摘 要
Abstract
1绪论
1.1课题研究的目的和意义
1.1.1课题研究的目的
1.1.2课题研究的意义
1.2研究现状
1.3发展趋势
1.4研究主要内容及方法
1.4.1研究内容
1.4.2研究方法
2模型算法
2.1词袋模型
2.2IF-IDF算法
2.3DBSCAN算法
2.4MATLAB软件
参考文献
致谢