Hi Carlos, In my opinion, you would need to properly segment that "sentence". It is virtually impossible the parser will get anything right if you pass it such sentences. Perhaps you can use the newlines in your "cleaned" text to create shorter more grammatical sentences. Also, if I had to deal with such text, and depending what the aim is, I would ask myself whether I actually need to use constituent parsing.... :)
HTH, Rodrigo On Tue, Jul 15, 2014 at 4:38 AM, Carlos Scheidecker <nando....@gmail.com> wrote: > Hello all, > > I have an interesting problem here. More of a challenge. > > I have been doing text cleansing for bad characters and all. > > Then I have another interesting problem. > > Extracted a public PDF with Tika does not necessary mean you will get clean > text because the original PDF might have different fonts within a section > that will cause weird behaviors. > > If you then divide it into Senteces via OpenNLP you will then get some > interesting sentences. > > Trying to parse those sentences then it gets worse. > > I am showing an example bellow and I would like to ask about solutions to > it, considering the text can be noisy. > > I do not think that it will be easy to fix the Sentence Parser. Here is > what I think on approaching it: > > Instead, the best way to do is to look at the sentences poorly parsed, > parse them and extract the inner (S) from the parse as separate sentences. > > What would you suggest? > > Here is an example of a piece of text extracted with Tika from a public > pdf. This part is what OpenNLP considered to be a sentence: > > ---- > > related research DocumentsBrief: Your Next Portal should Be An Engagement > WorkplaceFebruary 3, 2014Microsoft Aims sharePoint To The CloudJanuary 27, > 2014setting The Technology Foundation For Your social Business And > Collaboration strategyJuly 29, 2013The Forrester wave : enterprise social > Platforms, Q2 2014The 13 Providers That Matter Most And How They stack Upby > rob Koplowitzwith Peter Burris and Nancy Wang2257913JUNE 5, 2014For CIos > The Forrester Wave : Enterprise social Platforms, Q2 2014 2 2014, > Forrester Research, Inc. Reproduction Prohibited June 5, 2014 eNTeRPRIse > sOCIaL PLaTFORM MaRKeT MaTuRes aMID CONsOLIDaTION aND INTegRaTIONThe > enterprise social platform is no longer in its infancy as offerings become > increasingly functional. > > ---- > > It is now parsed as follows: > > > (S (S (S (NP (VBN related) (NN research) (NNP DocumentsBrief:) (NNP Your) > (NNP Next) (NNP Portal)) (VP (MD should) (VP (VB Be) (NP (NP (DT An) (NNP > Engagement) (NNP WorkplaceFebruary) (CD 3,) (JJ 2014Microsoft) (NNP Aims) > (NN sharePoint)) (PP (TO To) (NP (DT The) (NNP CloudJanuary))))))) (VP (VBD > 27,) (S (VP (VBG 2014setting) (NP (NP (NP (DT The) (NNP Technology) (NNP > Foundation)) (PP (IN For) (NP (PRP$ Your) (JJ social) (NNP Business)))) (CC > And) (NP (NNP Collaboration))) (PP (RB strategyJuly) (NP (CD 29,) (CD > 2013The) (NNP Forrester) (NN wave))))))) (: :) (S (VP (VB enterprise) (NP > (JJ social) (NN Platforms,)) (PP (IN Q2) (NP (NP (DT 2014The) (CD 13) (NNS > Providers)) (NP (NP (DT That) (NNP Matter) (JJS Most)) (SBAR (S (CC And) > (SBAR (WHADVP (WRB How)) (S (NP (PRP They)) (VP (VBP stack) (PP (IN Upby) > (NP (NP (NN rob)) (PP (IN Koplowitzwith) (NP (NP (NNP Peter) (NNP Burris)) > (CC and) (NP (NP (NNP Nancy) (NNP Wang2257913JUNE) (CD 5,)) (PP (IN > 2014For) (NP (NP (NNP CIos)) (NP (DT The) (NNP Forrester) (NNP Wave)) (: :) > (S (NP (NP (NP (NP (NN Enterprise) (JJ social) (NN Platforms,)) (PP (IN Q2) > (NP (CD 2014) (CD 2) (JJ 2014,) (NNP Forrester) (NNP Research,) (NNP Inc.) > (NNP Reproduction) (NNP Prohibited) (NNP June) (CD 5,) (CD 2014) (NN > eNTeRPRIse) (NN sOCIaL)))) (NP (NNP PLaTFORM) (NNP MaRKeT) (NNP MaTuRes) > (NN aMID) (NN CONsOLIDaTION))) (PP (IN aND) (NP (DT INTegRaTIONThe) (NN > enterprise) (JJ social) (NN platform)))) (VP (VP (VBZ is) (ADVP (RB no) (RB > longer)) (PP (IN in) (NP (PRP$ its) (NN infancy))) (SBAR (IN as) (S (NP > (NNS offerings)) (VP (VBP become) (ADVP (RB increasingly)))))) (VBG > functional.))))))))))))))))))))) > > > Notice that I have more than one (S (S (S > > And then I have the first correct structure as (S (NP ..... (VP..... > > What is the best way to deal with it?
