Feasibility Project - Text Processing and Category Classification

-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Feasibility-Project-Text-Processing-and-Category-Classification-tp24493.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: Feasibility Project - Text Processing and Category Classification

Load the textFile as an RDD. Something like this: val file = sc.textFile(/path/to/file) After this you can manipulate this RDD to filter texts the way you want them : val a1 = file.filter( line = line.contains([ERROR]) ) val a2 = file.filter( line = line.contains([WARN]) ) val a3 =

Re: Feasibility Project - Text Processing and Category Classification

! Best Regards, Darksu -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Feasibility-Project-Text-Processing-and-Category-Classification-tp24493.html Sent from the Apache Spark User List mailing list archive at Nabble.com