《R语言文本挖掘》由会员分享，可在线阅读，更多相关《R语言文本挖掘（12页珍藏版）》请在金锄头文库上搜索。
1、中文及英文的文本挖掘R语言所需要的包 tm(text mining) rJava,Snowball,zoo,XML,slam,Rz, RWeka,matlab1文本挖掘概要文本挖掘是从大量的文本数据中抽取隐含的，求和的，可能有用的信息。通过文本挖掘实现Associate：关联分析，根据同时出现的频率找出关联规则Cluster:将相似的文档（词条）进行聚类Categorize：将文本划分到预先定义的类别里文档自动摘要：利用计算机自动地从原始文档中提取全面准确地反映该文档中心内容的简 单连贯描述性短文。文本挖掘的运用主要有如下几方面智能信息检索同义词，简称词，异形词，同音字、赘字移除网络内容安全内
2、容监控内容过滤内容管理自动分类检测和追踪市场监测口碑监测竞争情报系统市场分析2英文文本挖掘实例实现多个英文文档的聚类分析2.1文本预处理2.1.1读取文本内容#取得tm内部文件texts/crude/下的文件目录。library(tm,lib.loc=d:/ProgramFiles/R/R-3.0.3/library) vignette(tm)#获取相关帮助文档的内容，pdf格式 reut-system.file(texts,crude,package=tm)#用Corpus命令读取文本并生成语料库文件由于要读取的是xml文件，所以需要xml包 reuters 1nspecT(renters
3、1:2)A corpus 2 text documentsTh-e metadata consn sts of 2 tag-val ue pai fs and a data FraAval 1 able tags are:create_datc creatorAval 1abl variables in the data frame are: MetalD$ r eut-00001.xmldi amond shamrock (dia) cuts crude pr 1 csFew yoric, f eb 26 -d i amond shamrock corp said effect i ve t
4、oday cut contract pp5T e d price west texas 1 nter mediare 1C.00 dfr5 barrel , c III MtlMlIUIBMB另外可以从本地文件中读取用户txt文件 inputtest-read.csv(file.choose()2.1.2文本清理对于xml格式的文档用tm_map命令对语料库文件进行预处理，将其转为纯文本并去除多余 空格，转换小写，去除常用词汇、合并异形同意词汇，如此才能得到类似txt文件的效果 需要用到的包SnowballCreuters - tm_map(reuters, PlainTextDocument
5、)#去 除标签reuters - tm_map(reuters, stripWhitespace)#去 多余空白reuters - tm_map(reuters, tolower)#转换小写reuters query tm_filter(reuters, FUN = sFilter, query)找到 1 Iter (reuters, fun = sfi Iter , query) inspecx(temp)A corpus with 1 text documentThe metadata consists of 2 tag-value pairs and a data frameAval 1
6、able tags are:create_date creatorAval 1able variables in the data frame are:MetalD$reut-OOOOE.xml1ndonesia seen crossroads economic changejeremy clift, rentersJakarta, march 1 -indonesia appears nearing political crossroads measures deregulate pr u.s. embassy says new report. counter falling oi1 rev
7、enues, government measures past nine months boost exports outside cdl sector attract n&由于语料库已经将大小写转换以及将介词类删除所以对应的语句只是特有单词的组合。2.1.4生成词频矩阵并查看内容dtm inspect(dtm10:15,110:120)A document-term matrix (6 documents, 11 terms)Non-/sparse entries: 6/60Sparsity: 91%Maximal term length: 9Weighting: term frequenc
8、y (tf)TermsDocs activity. add added added. address addressed adherence adhering advantage advisers agency1,000000110022,000000000003,000000000014,000000010025,000000000006,000000000002.1.5查看含有特定词的文档若要考察多个文档特定词汇的出现频率或以手工生成字典，并将其作为生成阵的参数 inspect(tdmc(price”, texas),c(127”,”144”,”191”,194”)A term-docum
9、ent matrix (2 terms, 4 documents)Non-/sparse entries: 6/2Sparsity: 25%Maximal term length: 5Weighting: term frequency (tf)DocsTerms 127 144 191 194price2122texas1002 inspect(DocumentTermMatrix(reuters,+list(dictionary = c(prices”, crude, oil)A document-term matrix (20 documents, 3 terms)Non-/sparse
10、entries: 41/19Sparsity: 32%Maximal term length: 6Weighting: term frequency (tf)TermsDocs crude oil prices12735414401141913201944102110202361722370302.1.6元数据操作(词元素)2.1.6.1查看词条出现次数大于某个具体值的词findFreqTerms(dtm,5)#查看出现频大于等于5的词 fl ndFreqTrnis (dtm, 515.Brc6 agreementIt arabbpd,r21J PadTyZB energyabdul-aziz
11、 Fl”arabn a budgetdGmand exchangeabilitybarrel.eKpected accord analysxs barrels company economlc exportsagency r,april,r T,b-ilHon crude emergency rifebrc 2.1.6.2查看与某一词相关度大于0.8的词条 findAssocs(dtm,opec,0.8) opecmeeting0.8815.80.85oil 0.85emergency 0.83analysts 0.82buyers 0.802.1.7处理词频矩阵 dtm2 dtm2A doc
12、ument-term natrix (20 docuraents, 349 terras)Non-/sparse entries: 1120/5860Sparsity:84%Maximal ti整即 length: 14Weightlng:term frequency Ctfa temp=a5.data, frame(i n3pect(dtn2)A document-term matrix (20 docuinentss 349 terms)NDn-/spar5e entr i es SparsityMaximal term length Wei ghting1120/586014terra frequency Ctf)Ternsddcs rrnone (bpd)il n n(opec)1.21. 5013-nation15-1615.01905.
