Revision: 19790
http://sourceforge.net/p/gate/code/19790
Author: markagreenwood
Date: 2016-11-24 15:03:23 +0000 (Thu, 24 Nov 2016)
Log Message:
-----------
merged in trunk changes
Modified Paths:
--------------
gate/branches/sawdust2/plugins/Twitter/LICENSE.txt
gate/branches/sawdust2/plugins/Twitter/build.xml
gate/branches/sawdust2/plugins/Twitter/creole.xml
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/english.jaspell
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/orth.en.csv
gate/branches/sawdust2/plugins/Twitter/resources/tokeniser/twitter.jape
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/Normaliser.java
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/pos/POSTaggerEN.java
Added Paths:
-----------
gate/branches/sawdust2/plugins/Twitter/lib/jaspell-0.3.1.jar
gate/branches/sawdust2/plugins/Twitter/lib/opencsv-3.8.jar
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/abbreviations.en.csv
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/misspellings.csv
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/slang.en.csv
gate/branches/sawdust2/plugins/Twitter/resources/twitie-pos-tagger.xgapp
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/apps/TwitIEPOS.java
Removed Paths:
-------------
gate/branches/sawdust2/plugins/Twitter/lib/jaspell.jar
Property Changed:
----------------
gate/branches/sawdust2/plugins/Twitter/
Index: gate/branches/sawdust2/plugins/Twitter
===================================================================
--- gate/branches/sawdust2/plugins/Twitter 2016-11-24 14:40:06 UTC (rev
19789)
+++ gate/branches/sawdust2/plugins/Twitter 2016-11-24 15:03:23 UTC (rev
19790)
Property changes on: gate/branches/sawdust2/plugins/Twitter
___________________________________________________________________
Modified: svn:mergeinfo
## -2,4 +2,4 ##
/gate/branches/release-7.0/plugins/Twitter:15335-15398
/gate/branches/sawdust/plugins/Twitter:17055-17079
/gate/tags/release-7.0/plugins/Twitter:15399-15407
-/gate/trunk/plugins/Twitter:3-390,19180-19294
+/gate/trunk/plugins/Twitter:3-390,19180-19789
\ No newline at end of property
Modified: gate/branches/sawdust2/plugins/Twitter/LICENSE.txt
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/LICENSE.txt 2016-11-24 14:40:06 UTC
(rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/LICENSE.txt 2016-11-24 15:03:23 UTC
(rev 19790)
@@ -1,4 +1,6 @@
-The GATE components within the Twitter plugin are licensed under the LGPL
(version 3).
+The GATE components within the Twitter plugin are licensed under the LGPL
+(version 3). The jaspell library uses a BSD licence and the opencsv library is
+released under the Apache licence version 2.
Note, however, that the TwitIE application is licensed under the GPL (version
3)
due to it containing components from the Standford_CoreNLP plugin.
Modified: gate/branches/sawdust2/plugins/Twitter/build.xml
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/build.xml 2016-11-24 14:40:06 UTC
(rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/build.xml 2016-11-24 15:03:23 UTC
(rev 19790)
@@ -22,13 +22,6 @@
<include name="*.zip" />
</fileset>
- <!-- we also need to pull in the Tagger_Stanford jars -->
- <fileset id="stanford.libs" dir="../Stanford_CoreNLP">
- <include name="*.jar" />
- <include name="*.zip" />
- </fileset>
-
-
<fileset id="gatelib.classpath" dir="${gate.home}/lib">
<include name="*.jar" />
<include name="*.zip" />
@@ -41,7 +34,6 @@
<path id="compile.classpath">
<fileset refid="libs" />
- <fileset refid="stanford.libs" />
<fileset refid="gatelib.classpath" />
<fileset refid="gate.classpath" />
</path>
Modified: gate/branches/sawdust2/plugins/Twitter/creole.xml
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/creole.xml 2016-11-24 14:40:06 UTC
(rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/creole.xml 2016-11-24 15:03:23 UTC
(rev 19790)
@@ -1,7 +1,7 @@
<?xml version="1.0"?>
<CREOLE-DIRECTORY>
<JAR SCAN="true">twitter.jar</JAR>
- <JAR>lib/jaspell.jar</JAR>
+ <JAR>lib/jaspell-0.3.1.jar</JAR>
+ <JAR>lib/opencsv-3.8.jar</JAR>
<REQUIRES>../Format_Twitter</REQUIRES>
- <REQUIRES>../Stanford_CoreNLP</REQUIRES>
</CREOLE-DIRECTORY>
Copied: gate/branches/sawdust2/plugins/Twitter/lib/jaspell-0.3.1.jar (from rev
19789, gate/trunk/plugins/Twitter/lib/jaspell-0.3.1.jar)
===================================================================
(Binary files differ)
Deleted: gate/branches/sawdust2/plugins/Twitter/lib/jaspell.jar
===================================================================
(Binary files differ)
Copied: gate/branches/sawdust2/plugins/Twitter/lib/opencsv-3.8.jar (from rev
19789, gate/trunk/plugins/Twitter/lib/opencsv-3.8.jar)
===================================================================
(Binary files differ)
Copied:
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/abbreviations.en.csv
(from rev 19789,
gate/trunk/plugins/Twitter/resources/normaliser/abbreviations.en.csv)
===================================================================
---
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/abbreviations.en.csv
(rev 0)
+++
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/abbreviations.en.csv
2016-11-24 15:03:23 UTC (rev 19790)
@@ -0,0 +1,158 @@
+adad,another day another dollar
+adih,another day in hell
+adip,another day in paradise
+aeap,as early as possible
+afaicr,as far as i can recall
+afaics,as far as i can see
+afaict,as far as i can tell
+afaik,as far as i know
+afair,as far as i remember
+afap,as far as possible
+bbiab,be back in a bit
+bbl,be back later
+bbs,be back soon
+bff,best friends forever
+bfn,bye for now
+bofh,bastard operator from hell
+brb,be right back
+bsod,blue screen of death
+btdt,been there done that
+btw,by the way
+cmiiw,correct me if i'm wrong
+cu,see you
+dftt,don't feed the trolls
+dgaf,don't give a fuck
+diaf,die in a fire
+dnd,do not disturb
+doa,dead on arrival
+eta,estimated time of arrival
+ffs,for fuck's sake
+fmcdh,from my cold dead hands
+fml,fuck my life.
+foad,fuck off and die
+foaf,friend of a friend
+ftfy,fixed that for you
+ftl,for the loss
+ftw,for the win
+fwiw,for what it's worth
+fyi,for your information
+g2g,got to go
+gbtw,get back to work
+gf,great fight
+gfu,good for you
+gfy,go fuck yourself
+gg,good game
+ggs,good games
+gj,good job
+gl,good luck
+gmta,great minds think alike
+gtfo,get the fuck out
+gtg,got to go
+hf,have fun
+hth,hope this helps
+ianal,i am not a lawyer
+ibtl,in before the lock
+idk,i don't know
+idk,I don't know
+iht,i had to
+iirc,if i remember correctly
+iiuc,if i understand correctly
+ily,i love you
+imao,in my arrogant opinion
+imho,in my humble opinion
+imnsho,in my not so humble opinion
+imo,in my opinion
+iono,i don't know
+iow,in other words
+irc,internet relay chat
+irl,in real life
+itym,i think you mean
+iwsn,i want sex now
+iykwim,if you know what i mean
+jas,just a sec
+jftr,just for the record
+kos,kill on sight
+lfg,looking for group
+lfm,looking for more
+lmirl,let's meet in real life
+lmk,let me know
+ltns,long time no see
+lylab,love you like a brother
+lylas,love you like a sister
+mfw,my face when
+motd,message of the day
+mtfbwy,may the force be with you
+myob,mind your own business
+n1,nice one
+ngl,not gonna lie
+nifoc,naked in front of computer
+nm,never mind
+np,no problem
+nsfw,naughty
+nsoh,no sense of humor
+nvm,never mind
+ofn,old freaking news
+oic,oh i see
+omw,on my way
+onoz,oh no
+op,original poster
+os,operating system
+otb,off to bed
+ot,off topic
+otoh,on the other hand
+otp,on the phone
+pita,pain in the ass
+plmk,please let me know
+pmsl,pissing myself laughing
+pov,point of view
+ptkfgs,punch the keys for god's sake
+rl,real life
+rms,ride me sideways
+rofl,haha
+rsn,real soon now
+rtfb,read the fucking binary
+rtfs,read the fucking source
+sfw,safe for work
+smh,shaking my head
+sohf,sense of humor failure
+stfu,shut the fuck up
+tbf,time between failures
+tbh,to be honest
+tgif,thank god it's friday
+tg,that's great
+tia,thanks in advance
+timwtk,this inquiring mind wants to know
+tins,there is no spoon
+tl;dr,too long; didn't read
+tmi,too much information
+tos,terms of service
+ttbomk,to the best of my knowledge
+ttfn,ta ta for now
+ttyl,talk to you later
+ttyn,talk to you never
+ttys,talk to you soon
+ty,thank you
+tyt,take your time
+tyvm,thank you very much
+ugo,you got owned
+utfse,use the fucking search engine
+wb,welcome back
+wrt,with respect to
+wtb,want to buy
+wtb,want to buy
+wtf,what the fuck
+wtg,way to go
+wth,what the hell
+wts,want to sell
+wtt,want to trade
+wug,what you got?
+yagni,you ain't gonna need it.
+yagtoh,you are going to own him.
+yec,young earth creationism
+ygm,you've got mail.
+yhbt,you have been trolled.
+ykw,you know what?
+ymmv,your mileage may vary.
+yoyo,you're on your own
+ytmnd,you're the man now dog
+yw,you're welcome.
\ No newline at end of file
Modified:
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/english.jaspell
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/resources/normaliser/english.jaspell
2016-11-24 14:40:06 UTC (rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/resources/normaliser/english.jaspell
2016-11-24 15:03:23 UTC (rev 19790)
@@ -37272,6 +37272,7 @@
oxlips : 1
profiling : 3
bullock : 3
+bollock : 3
uncommercial : 1
macron : 2
tarweeds : 1
@@ -42818,6 +42819,7 @@
base : 3
reaper : 3
bullocks : 1
+bollocks : 1
restoring : 3
modified : 3
papists : 1
Copied:
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/misspellings.csv
(from rev 19789,
gate/trunk/plugins/Twitter/resources/normaliser/misspellings.csv)
===================================================================
---
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/misspellings.csv
(rev 0)
+++
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/misspellings.csv
2016-11-24 15:03:23 UTC (rev 19790)
@@ -0,0 +1,242 @@
+absense,absence
+absance,absence
+acceptible,acceptable
+accidentaly,accidentally
+accomodate,accommodate
+acommodate,accommodate
+acheive,achieve
+acknowlege,acknowledge
+aknowledge,acknowledge
+acquaintence,acquaintance
+aquaintance,acquaintance
+aquire,acquire
+adquire,acquire
+aquit,acquit
+acrage,acreage
+acerage,acreage
+adress,address
+adultary,adultery
+adviseable,advisable
+advizable,advisable
+agression,aggression
+agressive,aggressive
+allegaince,allegiance
+allegience,allegiance
+alegiance,allegiance
+allmost,almost
+amatuer,amateur
+amature,amateur
+anually,annually
+annualy,annually
+apparant,apparent
+aparent,apparent
+artic,arctic
+arguement,argument
+athiest,atheist
+aweful,awful
+awfull,awful
+becuase,because
+becomeing,becoming
+beginning,begining
+beleive,believe
+bouy,buoy
+bouyant,buoyant
+buisness,business
+calender,calendar
+camoflage,camouflage
+camoflague,camouflage
+Carribean,Caribbean
+catagory,category
+cauhgt,caught
+caugt,caught
+cemetary,cemetery
+cematery,cemetery
+changable,changeable
+cheif,chief
+collaegue,colleague
+collegue,colleague
+collectable,collectible
+colum,column
+comming,coming
+commited,committed
+comitted,committed
+conceed,concede
+congradulate,congratulate
+consciencious,conscientious
+concious,conscious
+consious,conscious
+concensus,consensus
+contraversy,controversy
+cooly,coolly
+dacquiri,daiquiri
+daquiri,daiquiri
+decieve,deceive
+definate,definite
+definit,definite
+definitly,definitely
+definately,definitely
+defiantly,definitely
+desparate,desperate
+diffrence,difference
+dilema,dilemma
+disapoint,disappoint
+disasterous,disastrous
+drunkeness,drunkenness
+dumbell,dumbbell
+embarass,embarrass
+equiptment,equipment
+excede,exceed
+exilerate,exhilarate
+existance,existence
+experiance,experience
+extreem,extreme
+facinating,fascinating
+firey,fiery
+flourescent,fluorescent
+foriegn,foreign
+freind,friend
+fullfil,fulfil
+guage,gauge
+gratefull,grateful
+greatful,grateful
+garantee,guarantee
+garentee,guarantee
+garanty,guarantee
+guidence,guidance
+harrass,harass
+heighth,height
+heigth,height
+heirarchy,hierarchy
+humerous,humorous
+hygene,hygiene
+hygine,hygiene
+hiygeine,hygiene
+higeine,hygiene
+hygeine,hygiene
+hipocrit,hypocrite
+ignorence,ignorance
+immitate,imitate
+imediately,immediately
+independant,independent
+indispensible,indispensable
+innoculate,inoculate
+inteligence,intelligence
+intelligance,intelligence
+jewelery,jewellery
+kernal,kernel
+liesure,leisure
+liason,liaison
+libary,library
+liberry,library
+lisence,licence
+lightening,lightning
+maintainance,maintenance
+maintnance,maintenance
+medeval,medieval
+medevil,medieval
+mideval,medieval
+momento,memento
+millenium,millennium
+milennium,millennium
+miniture,miniature
+miniscule,minuscule
+mischievious,mischievous
+mischevous,mischievous
+mischevious,mischievous
+mispell,misspell
+misspel,misspell
+neccessary,necessary
+necessery,necessary
+neice,niece
+nieghbour,neighbour
+noticable,noticeable
+occassion,occasion
+occasionaly,occasionally
+occassionally,occasionally
+occurrance,occurrence
+occurence,occurrence
+occured,occurred
+ommision,omission
+omision,omission
+orignal,original
+outragous,outrageous
+parliment,parliament
+passtime,pastime
+pasttime,pastime
+percieve,perceive
+perseverence,perseverance
+personell,personnel
+personel,personal
+plagerize,plagiarize
+playright,playwright
+playwrite,playwright
+posession,possession
+possesion,possession
+potatos,potatoes
+preceed,precede
+presance,presence
+privelege,privilege
+priviledge,privilege
+professer,professor
+promiss,promise
+pronounciation,pronunciation
+prufe,proof
+publically,publicly
+quarentine,quarantine
+questionaire,questionnaire
+questionnair,questionnaire
+readible,readable
+realy,really
+recieve,receive
+reciept,receipt
+recomend,recommend
+reccommend,recommend
+refered,referred
+referance,reference
+refrence,reference
+relevent,relevant
+ revelant,relevant
+religous,religious
+religius,religious
+repitition,repetition
+restarant,restaurant
+restaraunt,restaurant
+rime,rhyme
+ryme,rhyme
+rythm,rhythm
+rythem,rhythm
+secratary,secretary
+secretery,secretary
+sieze,seize
+seperate,separate
+sargent,sergeant
+similer,similar
+skilfull,skilful
+speach,speech
+succesful,successful
+successfull,successful
+sucessful,successful
+supercede,supersede
+suprise,surprise
+surprize,surprise
+tomatos,tomatoes
+tommorrow,tomorrow
+twelth,twelfth
+tyrany,tyranny
+underate,underrate
+untill,until
+upholstry,upholstery
+usible,usable
+vaccuum,vacuum
+vaccum,vacuum
+vacume,vacuum
+vehical,vehicle
+visious,vicious
+wierd,weird
+wellfare,welfare
+welfair,welfare
+wether,whether
+wilfull,wilful
+withold,withhold
+writting,writing
+writeing,writing
\ No newline at end of file
Modified:
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/orth.en.csv
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/resources/normaliser/orth.en.csv
2016-11-24 14:40:06 UTC (rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/resources/normaliser/orth.en.csv
2016-11-24 15:03:23 UTC (rev 19790)
@@ -1,363 +1,3 @@
-10q,thank you
-10x,thanks
-1337,elite
-2day,today
-2gva,together
-2moro,tomorrow
-2morrow,tomorrow
-2moz,tomorrow
-2night,tonight
-2nite,tonight
-<3,love
-4eva,forever
-5-0,police
-5o,police
-5-O,police
-abt,about
-adad,another day another dollar
-ad,had
-adih,another day in hell
-adip,another day in paradise
-aeap,as early as possible
-afaicr,as far as i can recall
-afaics,as far as i can see
-afaict,as far as i can tell
-afaik,as far as i know
-afair,as far as i remember
-afap,as far as possible
-afta,have to
-agen,again
-aint,ain't
-anyways,anyway
-apr,April
-atm,now
-aug,August
-awol,away
-b4,before
-bbiab,be back in a bit
-bbl,be back later
-bbq,barbecue
-bbs,be back soon
-bby,baby
-b/c,because
-bc,because
-bcuz,because
-b-day,birthday
-bday,birthday
-bff,best friends forever
-bfn,bye for now
-birfday,birthday
-blate,blatently
-blates,blatently
-bles,bless
-blu,blue
-bofh,bastard operator from hell
-boi,boy
-brb,be right back
-bringin,bringing
-bro,brother
-bsod,blue screen of death
-bt,but
-btdt,been there done that
-btw,by the way
-cmiiw,correct me if i'm wrong
-cnt,can't
-cont,continued
-convo,conversation
-cryin,crying
-cum,come
-cus,curse
-cu,see you
-cussin,cursing
-cusz,because
-cuz,because
-da,the
-dat,that
-dec,December
-deez,these
-def,definitely
-dey,they
-dftt,don't feed the trolls
-dgaf,don't give a fuck
-diaf,die in a fire
-dint,didn't
-dis,this
-dl,download
-dnd,do not disturb
-dnt,don't
-doa,dead on arrival
-doin,doing
-dossin,dossing
-dun,don't
-ep,episode
-eps,episodes
-es,he's
-eta,estimated time of arrival
-every1,everyone
-everytim,every time
-evry1,everyone
-evry,every
-ez,easy
-f9,fine
-fakin,faking
-fallin,falling
-fb,Facebook
-fck,fuck
-fckn,fucking
-feb,February
-ffs,for fuck's sake
-fkn,fucking
-fmcdh,from my cold dead hands
-fml,fuck my life.
-foad,fuck off and die
-foaf,friend of a friend
-fo,for
-fo,for
-frm,from
-ftfy,fixed that for you
-ftl,for the loss
-ftw,for the win
-fucken,fucking
-fu,fuck you
-fwiw,for what it's worth
-fyi,for your information
-g2g,got to go
-gbtw,get back to work
-gd,good
-gen,again
-getn,getting
-gf,great/good fight
-gfu,good for you
-gfy,go fuck yourself
-gg,good game
-ggs,good games
-gj,good job
-gl,good luck
-gmta,great minds think alike
-goin,going
-govt,government
-gr8,great
-gtfo,get the fuck out
-gt,get
-gtg,got to go
-gud,good
-hella,very
-hf,have fun
-hpe,hope
-hse,house
-hses,houses
-hth,hope this helps
-ianal,i am not a lawyer
-ibtl,in before the lock
-idk,i don't know
-idk,I don't know
-iht,i had to
-iirc,if i remember correctly
-iiuc,if i understand correctly
-ily,i love you
-i'ma,i'll
-ima,I'll
-imao,in my arrogant opinion
-imho,in my humble opinion
-imma,I'll
-imnsho,in my not so humble opinion
-imo,in my opinion
-incl,including
-inna,in a
-innit,isn't it
-int,isn't
-iono,i don't know
-iow,in other words
-irc,internet relay chat
-irl,in real life
-itym,i think you mean
-ive,I've
-iwsn,i want sex now
-iykwim,if you know what i mean
-jan,January
-jas,just a sec
-jftr,just for the record
-j/k,joking
-jk,joking
-jul,July
-jun,June
-jus,just
-kkk,ok
-kk,ok
-kno,know
-kos,kill on sight
-kthx,thanks
-l8r,later
-lfg,looking for group
-lfm,looking for more
-lik,like
-lmao,haha
-lmfao,haha
-lmirl,let's meet in real life.
-lmk,let me know
-lst,last
-ltns,long time no see
-lurv,love
-lurve,love
-luv,love
-luvz,loves
-lv,love
-lyk,like
-lylab,love you like a brother.
-lylas,love you like a sister.
-m8,mate
-mar,March
-mfw,my face when
-mor,more
-motd,message of the day
-mtfbwy,may the force be with you
-mud,multi-user dungeon
-mush,multi-user shared hallucination
-myob,mind your own business
-n1,nice one
-nd,and
-ne1,anyone
-ngl,not gonna lie
-nifoc,naked in front of computer
-niggaz,people
-nite,night
-nm,never mind
-nothin,nothing
-nov,November
-np,no problem
-nsfw,naughty
-nsoh,no sense of humor
-ntn,nothing
-nuff,enough
-nutn,nothing
-nvm,nevermind
-nw,now
-nyt,night
-nyts,nights
-obvblates,assuredly
-obv,obviously
-obvs,obviously
-oct,October
-ofn,old freaking news
-oic,oh i see
-omw,on my way
-onoz,oh no
-op,original poster
-os,operating system
-otb,off to bed
-ot,off topic
-otoh,on the other hand
-otp,on the phone
-paw,parents are watching
-pic,picture
-pics,pictures
-pita,pain in the ass
-plez,please
-plmk,please let me know
-pls,please
-plz,please
-pmsl,pissing myself laughing
-pots,plain old telephone service
-pov,point of view
-ppl,people
-ptkfgs,punch the keys for god's sake
-pwned,owned
-pw,password
-pz,peace
-rl,real life
-rms,ride me sideways
-rofl,haha
-rsn,real soon now
-rtfb,read the fucking binary
-rtfs,read the fucking source
-sep,September
-sfw,safe for work
-sho,sure
-sis,sister
-smh,shaking my head
-smiley,another name for emoticons
-snail mail,normal paper mail service
-sohf,sense of humor failure
-sq,square
-stfu,shut the fuck up
-sth,something
-str8,straight
-sumn,something
-takin,taking
-tbf,time between failures
-tbh,to be honest
-tbh,to be honest
-tellin,telling
-tgif,thank god it's friday
-tg,that's great
-tha,the
-thn,then
-thnx,thanks
-thro,through
-thru,through
-tht,that
-thx,thanks
-tia,thanks in advance
-timwtk,this inquiring mind wants to know
-tins,there is no spoon
-tl;dr,too long; didn't read
-tmi,too much information
-tnx,thanks
-tos,terms of service
-totes,totally
-tryna,trying to
-ttbomk,to the best of my knowledge
-ttfn,ta ta for now
-ttyl,talk to you later
-ttyn,talk to you never
-ttys,talk to you soon
-twtr,twitter
-twt,tweet
-tymz,times
-ty,thankyou
-ty,thank you
-tyt,take your time
-tyvm,thank you very much
-ugo,you got owned
-urs,yours
-ur,you're
-utfse,use the fucking search engine
-vids,videos
-v.,very
-w8,wait
-wanna,want to
-wb,welcome back
-w/e,weekend
-w/e,whatever
-whts,what's
-wif,with
-wil,will
-wit,with
-w/o,without
-w/o,without
-wrt,with respect to
-wtb,want to buy
-wtb,want to buy
-wtf,what the fuck
-wtg,way to go
-wth,what the hell
-wts,want to sell
-wtt,want to trade
-wug,what you got?
-w/,with
-yagni,you ain't gonna need it.
-yagtoh,you are going to own him.
-ya,your
-yea,yeah
-yec,young earth creationism
-ygm,you've got mail.
-yhbt,you have been trolled.
-yhuu,you
-ykw,you know what?
-ymmv,your mileage may vary.
-yoyo,you're on your own
-ytmnd,you're the man now dog
-yuhuu,yes
-yuhu,yes
-yu,you
-yw,you're welcome.
+abbreviations.en.csv
+slang.en.csv
+misspellings.csv
\ No newline at end of file
Copied:
gate/branches/sawdust2/plugins/Twitter/resources/normaliser/slang.en.csv (from
rev 19789, gate/trunk/plugins/Twitter/resources/normaliser/slang.en.csv)
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/resources/normaliser/slang.en.csv
(rev 0)
+++ gate/branches/sawdust2/plugins/Twitter/resources/normaliser/slang.en.csv
2016-11-24 15:03:23 UTC (rev 19790)
@@ -0,0 +1,198 @@
+10q,thank you
+10x,thanks
+1337,elite
+2day,today
+2gva,together
+2moro,tomorrow
+2morrow,tomorrow
+2moz,tomorrow
+2night,tonight
+2nite,tonight
+<3,love
+4eva,forever
+5-0,police
+5o,police
+5-O,police
+abt,about
+afta,have to
+agen,again
+aint,ain't
+anyways,anyway
+apr,April
+atm,now
+aug,August
+awol,away
+b4,before
+bbq,barbecue
+bby,baby
+b/c,because
+bc,because
+bcuz,because
+b-day,birthday
+bday,birthday
+birfday,birthday
+blate,blatently
+blates,blatently
+bles,bless
+blu,blue
+boi,boy
+bringin,bringing
+bro,brother
+bt,but
+cnt,can't
+cont,continued
+convo,conversation
+cryin,crying
+cum,come
+cus,curse
+cussin,cursing
+cusz,because
+cuz,because
+da,the
+dat,that
+dec,December
+deez,these
+def,definitely
+dey,they
+dint,didn't
+dis,this
+dl,download
+dnt,don't
+doin,doing
+dossin,dossing
+dun,don't
+ep,episode
+eps,episodes
+es,he's
+every1,everyone
+everytim,every time
+evry1,everyone
+evry,every
+ez,easy
+f9,fine
+fakin,faking
+fallin,falling
+fb,Facebook
+fck,fuck
+fckn,fucking
+feb,February
+fkn,fucking
+fo,for
+fo,for
+frm,from
+fucken,fucking
+gd,good
+gen,again
+getn,getting
+goin,going
+govt,government
+gr8,great
+gt,get
+gud,good
+hella,very
+hpe,hope
+hse,house
+hses,houses
+i'ma,i'll
+ima,I'll
+imma,I'll
+incl,including
+inna,in a
+innit,isn't it
+int,isn't
+ive,I've
+jan,January
+j/k,joking
+jk,joking
+jul,July
+jun,June
+jus,just
+kkk,ok
+kk,ok
+kno,know
+kthx,thanks
+l8r,later
+lik,like
+lmao,haha
+lst,last
+lurv,love
+lurve,love
+luv,love
+luvz,loves
+lv,love
+lyk,like
+m8,mate
+mar,March
+mor,more
+n't,n't
+nd,and
+ne1,anyone
+nigga,nigger
+nite,night
+nothin,nothing
+nov,November
+nsfw,naughty
+ntn,nothing
+nuff,enough
+nutn,nothing
+nw,now
+nyt,night
+nyts,nights
+obvblates,assuredly
+obv,obviously
+obvs,obviously
+oct,October
+onoz,oh no
+pic,picture
+pics,pictures
+plez,please
+pls,please
+plz,please
+ppl,people
+pwned,owned
+pw,password
+pz,peace
+sep,September
+sho,sure
+sis,sister
+sq,square
+sth,something
+str8,straight
+sumn,something
+takin,taking
+tellin,telling
+tha,the
+thn,then
+thnx,thanks
+tho,though
+thro,through
+thru,through
+tht,that
+thx,thanks
+tnx,thanks
+totes,totally
+tryna,trying to
+twtr,twitter
+twt,tweet
+tymz,times
+urs,yours
+ur,you're
+vids,videos
+v.,very
+w8,wait
+wanna,want to
+w/e,weekend
+w/e,whatever
+whts,what's
+wif,with
+wil,will
+wit,with
+w/o,without
+w/o,without
+w/,with
+ya,your
+yea,yeah
+yhuu,you
+yuhuu,yes
+yuhu,yes
+yu,you
\ No newline at end of file
Modified:
gate/branches/sawdust2/plugins/Twitter/resources/tokeniser/twitter.jape
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/resources/tokeniser/twitter.jape
2016-11-24 14:40:06 UTC (rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/resources/tokeniser/twitter.jape
2016-11-24 15:03:23 UTC (rev 19790)
@@ -125,8 +125,19 @@
}
*/
-Rule: RecombineEmoticon
+Rule: NotRecombineEmoticon
+Priority: 100
+/* Do not recombine emoticon if it's preceded directly by a number, as it's
probably not an emoticon, e.g. 8:30 */
(
+ {Token.kind == number}
+ {Lookup.majorType=="emoticon"}
+ )
+ -->
+ {}
+
+
+ Rule: RecombineEmoticon
+(
{Lookup.majorType=="emoticon"}
):match
-->
@@ -188,7 +199,8 @@
// add spanning userid annotation
try {
outputAS.add(start+1, end, "UserID", userid);
- }
+ outputAS.add(start, end, "UserMention", userid);
+ }
catch (InvalidOffsetException e) {
e.printStackTrace();
}
Copied:
gate/branches/sawdust2/plugins/Twitter/resources/twitie-pos-tagger.xgapp (from
rev 19789, gate/trunk/plugins/Twitter/resources/twitie-pos-tagger.xgapp)
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/resources/twitie-pos-tagger.xgapp
(rev 0)
+++ gate/branches/sawdust2/plugins/Twitter/resources/twitie-pos-tagger.xgapp
2016-11-24 15:03:23 UTC (rev 19790)
@@ -0,0 +1,59 @@
+<gate.util.persistence.GateApplication>
+ <urlList class="gate.util.persistence.CollectionPersistence">
+ <localList>
+ <gate.util.persistence.PersistenceManager-URLHolder>
+ <urlString>$relpath$../../Stanford_CoreNLP</urlString>
+ </gate.util.persistence.PersistenceManager-URLHolder>
+ </localList>
+ <collectionType>java.util.ArrayList</collectionType>
+ </urlList>
+ <application
class="gate.util.persistence.ConditionalSerialAnalyserControllerPersistence">
+ <strategiesList class="gate.util.persistence.CollectionPersistence">
+ <localList>
+ <gate.util.persistence.AnalyserRunningStrategyPersistence>
+ <runMode>1</runMode>
+ <pr class="gate.util.persistence.LanguageAnalyserPersistence">
+ <runtimeParams class="gate.util.persistence.MapPersistence">
+ <mapType>gate.util.SimpleFeatureMapImpl</mapType>
+ <localMap/>
+ </runtimeParams>
+ <resourceType>gate.stanford.Tagger</resourceType>
+ <resourceName>Tweet PoS Tagger (EN)</resourceName>
+ <initParams class="gate.util.persistence.MapPersistence">
+ <mapType>gate.util.SimpleFeatureMapImpl</mapType>
+ <localMap>
+ <entry>
+ <string>modelFile</string>
+ <gate.util.persistence.PersistenceManager-URLHolder>
+ <urlString>$relpath$pos/gate-EN-twitter.model</urlString>
+ </gate.util.persistence.PersistenceManager-URLHolder>
+ </entry>
+ </localMap>
+ </initParams>
+ <features class="gate.util.persistence.MapPersistence">
+ <mapType>gate.util.SimpleFeatureMapImpl</mapType>
+ <localMap/>
+ </features>
+ </pr>
+ </gate.util.persistence.AnalyserRunningStrategyPersistence>
+ </localList>
+ <collectionType>java.util.ArrayList</collectionType>
+ </strategiesList>
+ <prList class="gate.util.persistence.CollectionPersistence">
+ <localList>
+ <gate.util.persistence.LanguageAnalyserPersistence
reference="../../../strategiesList/localList/gate.util.persistence.AnalyserRunningStrategyPersistence/pr"/>
+ </localList>
+ <collectionType>java.util.ArrayList</collectionType>
+ </prList>
+
<resourceType>gate.creole.ConditionalSerialAnalyserController</resourceType>
+ <resourceName>TwitIE PoS Tagger pipeline</resourceName>
+ <initParams class="gate.util.persistence.MapPersistence">
+ <mapType>gate.util.SimpleFeatureMapImpl</mapType>
+ <localMap/>
+ </initParams>
+ <features class="gate.util.persistence.MapPersistence">
+ <mapType>gate.util.SimpleFeatureMapImpl</mapType>
+ <localMap/>
+ </features>
+ </application>
+</gate.util.persistence.GateApplication>
Modified:
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/Normaliser.java
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/Normaliser.java
2016-11-24 14:40:06 UTC (rev 19789)
+++ gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/Normaliser.java
2016-11-24 15:03:23 UTC (rev 19790)
@@ -20,6 +20,7 @@
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
+import java.util.zip.GZIPInputStream;
import gate.*;
import gate.creole.*;
@@ -27,6 +28,8 @@
import gate.util.Files;
import gate.util.GateRuntimeException;
+import com.opencsv.CSVReader;
+
import pt.tumba.spell.*;
@CreoleResource(name = "Tweet Normaliser", comment = "Normalise texts in
tweets (convert into standard English spelling mistakes, colloquialisms, typing
variations and so on)", helpURL =
"http://gate.ac.uk/userguide/sec:social:twitter:prs";)
@@ -44,44 +47,36 @@
throw new ResourceInstantiationException("dict file not set");
checker = new SpellChecker();
- BufferedReader dict_reader;
- BufferedReader orth_reader;
wordlist = new HashSet<String>();
orthmappings = new HashMap<String, String>();
- // start up dict reader & checker
- try {
- String filepath = Files.fileFromURL(dictURL).getAbsolutePath();
- dict_reader = new BufferedReader(new FileReader(filepath));
- checker.initialize(filepath);
+ // initialize spell checker
+ try(InputStream in = openPossiblyGzip(dictURL);
+ InputStreamReader inReader = new InputStreamReader(in, dictEncoding);
+ BufferedReader dictReader = new BufferedReader(inReader)) {
+ checker.initialize(dictReader);
} catch(Exception e) {
- throw new ResourceInstantiationException(e);
+ throw new ResourceInstantiationException("Error initializing
spellchecker", e);
}
- // populate the wordlist
- String entry;
- try {
- while((entry = dict_reader.readLine()) != null) {
+ // read the wordlist
+ try(InputStream in = openPossiblyGzip(dictURL);
+ InputStreamReader inReader = new InputStreamReader(in, dictEncoding);
+ BufferedReader dictReader = new BufferedReader(inReader)) {
+ String entry;
+ while((entry = dictReader.readLine()) != null) {
String[] tokens = entry.split(" : ");
wordlist.add(tokens[0]);
}
- dict_reader.close();
- } catch(IOException e) {
+ } catch(Exception e) {
throw new ResourceInstantiationException(e);
}
// populate the common norm. lookup list
try {
- String filepath = Files.fileFromURL(orthURL).getAbsolutePath();
- orth_reader = new BufferedReader(new FileReader(filepath));
-
- while((entry = orth_reader.readLine()) != null) {
- String[] tokens = entry.split(",");
- orthmappings.put(tokens[0], tokens[1]);
- }
- orth_reader.close();
- } catch(Exception e) {
- throw new ResourceInstantiationException(e);
+ readOrthMappings(orthURL);
+ } catch(IOException e) {
+ throw new ResourceInstantiationException("Error loading orth mappings",
e);
}
dist = new LevenshteinDistance();
@@ -89,6 +84,44 @@
return this;
}
+ /**
+ * Read an orth mapping file. This file may be a "definition" file giving a
+ * list of other files to read, in which case each of those files will be
+ * read recursively.
+ */
+ protected void readOrthMappings(URL url) throws IOException {
+ try(InputStream in = openPossiblyGzip(url);
+ InputStreamReader reader = new InputStreamReader(in, orthEncoding);
+ CSVReader csvReader = new CSVReader(reader)) {
+ String[] line = csvReader.readNext();
+ if(line == null) return;
+
+ if(line.length < 2) {
+ // definition file pointing to other lists
+ do {
+ String relpath = line[0].trim();
+ if(!"".equals(relpath)) {
+ readOrthMappings(new URL(url, relpath));
+ }
+ } while((line = csvReader.readNext()) != null);
+ } else {
+ // single list
+ do {
+ orthmappings.put(line[0], line[1]);
+ } while((line = csvReader.readNext()) != null);
+ }
+ }
+ }
+
+ protected InputStream openPossiblyGzip(URL url) throws IOException {
+ InputStream in = url.openStream();
+ // if the URL ends ".gz" assume it's compressed
+ if(url.getPath().endsWith(".gz")) {
+ in = new GZIPInputStream(in);
+ }
+ return in;
+ }
+
@Override
public void execute() throws ExecutionException {
@@ -338,7 +371,19 @@
return this.dictURL;
}
- @CreoleParameter(comment = "Path to common normalisation terms list (for
orthographic mappings, e.g. 'b4' to 'before')", defaultValue =
"resources/normaliser/orth.en.csv")
+ @CreoleParameter(comment = "Character encoding used to read the dictionary
file", defaultValue = "UTF-8")
+ public void setDictEncoding(String encoding) {
+ this.dictEncoding = encoding;
+ }
+
+ public String getDictEncoding() {
+ return this.dictEncoding;
+ }
+
+ @CreoleParameter(comment = "Path to common normalisation terms list (for
orthographic mappings, e.g. 'b4' to 'before'). "
+ + "This can either be a single two-column CSV file where the first
column is the term to be mapped and the second "
+ + "column is the target, or a single column file listing relative paths
to other lists (which is useful if you want "
+ + "to include more than one list of terms)", defaultValue =
"resources/normaliser/orth.en.csv")
public void setOrthURL(URL orthURL) {
this.orthURL = orthURL;
}
@@ -347,6 +392,15 @@
return this.orthURL;
}
+ @CreoleParameter(comment = "Character encoding used to read the orth files",
defaultValue = "UTF-8")
+ public void setOrthEncoding(String encoding) {
+ this.orthEncoding = encoding;
+ }
+
+ public String getOrthEncoding() {
+ return this.orthEncoding;
+ }
+
@RunTime
@Optional
@CreoleParameter(comment = "Maximum distance to consider (this determines
OOV/IV threshold).\nBased on Levenshtein edit dist (with a case change
downweighted to 0.5) and double-metaphone.", defaultValue = "2.0")
@@ -380,6 +434,10 @@
private URL orthURL;
+ private String dictEncoding;
+
+ private String orthEncoding;
+
private double maxDistance;
}
Copied:
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/apps/TwitIEPOS.java
(from rev 19789,
gate/trunk/plugins/Twitter/src/gate/twitter/apps/TwitIEPOS.java)
===================================================================
--- gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/apps/TwitIEPOS.java
(rev 0)
+++ gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/apps/TwitIEPOS.java
2016-11-24 15:03:23 UTC (rev 19790)
@@ -0,0 +1,31 @@
+/*
+ * TwitIEPOS.java
+ *
+ * Copyright (c) 2013, The University of Sheffield. See the file COPYRIGHT.txt
+ * in the software or at http://gate.ac.uk/gate/COPYRIGHT.txt
+ *
+ * This file is part of GATE (see http://gate.ac.uk/), and is free software,
+ * licenced under the GNU Library General Public License, Version 2, June 1991
+ * (in the distribution as file licence.html, and also available at
+ * http://gate.ac.uk/gate/licence.html).
+ *
+ * Mark A. Greenwood, 06/09/2013
+ */
+
+package gate.twitter.apps;
+
+import gate.creole.PackagedController;
+import gate.creole.metadata.AutoInstance;
+import gate.creole.metadata.AutoInstanceParam;
+import gate.creole.metadata.CreoleResource;
+
+@CreoleResource(name = "TwitIE PoS Tagger",
+ comment = "English TwitIE part-of-speech tagger",
+ autoinstances = @AutoInstance(parameters = {
+ @AutoInstanceParam(name="pipelineURL",
value="resources/twitie-pos-tagger.xgapp"),
+ @AutoInstanceParam(name="menu", value="TwitIE")}))
+public class TwitIEPOS extends PackagedController {
+
+ private static final long serialVersionUID = 3419027514263819064L;
+
+}
Modified:
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/pos/POSTaggerEN.java
===================================================================
---
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/pos/POSTaggerEN.java
2016-11-24 14:40:06 UTC (rev 19789)
+++
gate/branches/sawdust2/plugins/Twitter/src/gate/twitter/pos/POSTaggerEN.java
2016-11-24 15:03:23 UTC (rev 19790)
@@ -1,7 +1,7 @@
/*
* POSTaggerEN.java
*
- * Copyright (c) 2013, The University of Sheffield. See the file COPYRIGHT.txt
+ * Copyright (c) 2016, The University of Sheffield. See the file COPYRIGHT.txt
* in the software or at http://gate.ac.uk/gate/COPYRIGHT.txt
*
* This file is part of GATE (see http://gate.ac.uk/), and is free software,
@@ -9,29 +9,265 @@
* (in the distribution as file licence.html, and also available at
* http://gate.ac.uk/gate/licence.html).
*
- * Mark A. Greenwood, 06/09/2013
+ * Ian Roberts, 24/05/2016
*/
package gate.twitter.pos;
import java.net.URL;
+import gate.Corpus;
+import gate.Document;
+import gate.Factory;
+import gate.Factory.DuplicationContext;
+import gate.FeatureMap;
+import gate.Gate;
+import gate.LanguageAnalyser;
+import gate.Resource;
+import gate.creole.AbstractLanguageAnalyser;
+import gate.creole.CustomDuplication;
+import gate.creole.ExecutionException;
+import gate.creole.ResourceInstantiationException;
import gate.creole.metadata.CreoleParameter;
-import gate.creole.metadata.CreoleResource;
+import gate.creole.metadata.CreoleResource;
+import gate.creole.metadata.Optional;
+import gate.creole.metadata.RunTime;
+import gate.event.ProgressListener;
+import gate.event.StatusListener;
-import gate.stanford.Tagger;
+/**
+ * Transitional compatibility PR to allow saved applications created
+ * with the GATE 8.0 version of TwitIE to continue operating in this
+ * version. New applications should not use this class, they should
+ * instantiate a Stanford Tagger PR directly with the appropriate model.
+ */
+@CreoleResource(name = "Twitter POS Tagger (deprecated)", comment =
"Transitional compatibility PR for Twitter POS tagger - new applications should
use the Stanford tagger directly", isPrivate = true)
+public class POSTaggerEN extends AbstractLanguageAnalyser implements
+ CustomDuplication {
-@CreoleResource(name = "Twitter POS Tagger (EN)",
- comment = "Stanford POS tagger trained on Tweets",
- helpURL = "http://gate.ac.uk/userguide/sec:social:twitter:prs";)
-public class POSTaggerEN extends Tagger {
+ private FeatureMap initParams;
- private static final long serialVersionUID = 5657607888874448666L;
+ private FeatureMap runtimeParams;
- @Override
- @CreoleParameter(comment = "Path to the tagger's model file",
- defaultValue = "resources/pos/gate-EN-twitter.model")
+ private AbstractLanguageAnalyser delegate;
+
+ public Resource init() throws ResourceInstantiationException {
+ if(delegate == null) {
+ FeatureMap features = Factory.newFeatureMap();
+ Gate.setHiddenAttribute(features, true);
+ delegate =
+ (AbstractLanguageAnalyser)Factory.createResource(
+ "gate.stanford.Tagger", getInitParams(), features,
+ getName());
+ }
+ return this;
+ }
+
+ public void reInit() throws ResourceInstantiationException {
+ delegate.setParameterValues(getInitParams());
+ delegate.reInit();
+ }
+
+ public void execute() throws ExecutionException {
+ try {
+ delegate.setParameterValues(getRuntimeParams());
+ } catch(ResourceInstantiationException e) {
+ // TODO Auto-generated catch block
+ throw new ExecutionException(e);
+ }
+ delegate.execute();
+ }
+
+ /**
+ * Only for use by duplication mechanism.
+ */
+ public void setDelegate(AbstractLanguageAnalyser delegate) {
+ this.delegate = delegate;
+ }
+
+ public AbstractLanguageAnalyser getDelegate() {
+ return delegate;
+ }
+
+ /**
+ * Only for use by duplication mechanism.
+ */
+ public void setInitParams(FeatureMap initParams) {
+ this.initParams = initParams;
+ }
+
+ public FeatureMap getInitParams() {
+ if(initParams == null) {
+ initParams = Factory.newFeatureMap();
+ }
+ return initParams;
+ }
+
+ /**
+ * Only for use by duplication mechanism.
+ */
+ public void setRuntimeParams(FeatureMap runtimeParams) {
+ this.runtimeParams = runtimeParams;
+ }
+
+ public FeatureMap getRuntimeParams() {
+ if(runtimeParams == null) {
+ runtimeParams = Factory.newFeatureMap();
+ }
+ return runtimeParams;
+ }
+
+ // init parameters
+
+ @CreoleParameter(comment = "Path to the tagger's model file", defaultValue =
"resources/pos/gate-EN-twitter.model")
public void setModelFile(URL modelFile) {
- super.setModelFile(modelFile);
+ getInitParams().put("modelFile", modelFile);
}
+
+ public URL getModelFile() {
+ return (URL)getInitParams().get("modelFile");
+ }
+
+ // runtime parameters
+
+ @Optional
+ @RunTime
+ @CreoleParameter
+ public void setInputASName(String inputASName) {
+ getRuntimeParams().put("inputASName", inputASName);
+ }
+
+ public String getInputASName() {
+ return (String)getRuntimeParams().get("inputASName");
+ }
+
+ @Optional
+ @RunTime
+ @CreoleParameter
+ public void setOutputASName(String outputASName) {
+ getRuntimeParams().put("outputASName", outputASName);
+ }
+
+ public String getOutputASName() {
+ return (String)getRuntimeParams().get("outputASName");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "true")
+ public void setFailOnMissingInputAnnotations(Boolean fail) {
+ getRuntimeParams().put("failOnMissingInputAnnotations", fail);
+ }
+
+ public Boolean getFailOnMissingInputAnnotations() {
+ return (Boolean)getRuntimeParams().get("failOnMissingInputAnnotations");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "true")
+ public void setPosTagAllTokens(Boolean tag) {
+ getRuntimeParams().put("posTagAllTokens", tag);
+ }
+
+ public Boolean getPosTagAllTokens() {
+ return (Boolean)getRuntimeParams().get("posTagAllTokens");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "Token")
+ public void setBaseTokenAnnotationType(String type) {
+ getRuntimeParams().put("baseTokenAnnotationType", type);
+ }
+
+ public String getBaseTokenAnnotationType() {
+ return (String)getRuntimeParams().get("baseTokenAnnotationType");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "Sentence")
+ public void setBaseSentenceAnnotationType(String type) {
+ getRuntimeParams().put("baseSentenceAnnotationType", type);
+ }
+
+ public String getBaseSentenceAnnotationType() {
+ return (String)getRuntimeParams().get("baseSentenceAnnotationType");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "Token")
+ public void setOutputAnnotationType(String type) {
+ getRuntimeParams().put("outputAnnotationType", type);
+ }
+
+ public String getOutputAnnotationType() {
+ return (String)getRuntimeParams().get("outputAnnotationType");
+ }
+
+ @RunTime
+ @CreoleParameter(defaultValue = "true")
+ public void setUseExistingTags(Boolean tag) {
+ getRuntimeParams().put("useExistingTags", tag);
+ }
+
+ public Boolean getUseExistingTags() {
+ return (Boolean)getRuntimeParams().get("useExistingTags");
+ }
+
+ public void setCorpus(Corpus c) {
+ delegate.setCorpus(c);
+ }
+
+ public Corpus getCorpus() {
+ return delegate.getCorpus();
+ }
+
+ public void setDocument(Document doc) {
+ delegate.setDocument(doc);
+ }
+
+ public Document getDocument() {
+ return delegate.getDocument();
+ }
+
+ // listener registration
+
+ @Override
+ public synchronized void removeStatusListener(StatusListener l) {
+ if(delegate != null) {
+ delegate.removeStatusListener(l);
+ }
+ }
+
+ @Override
+ public synchronized void addStatusListener(StatusListener l) {
+ if(delegate != null) {
+ delegate.addStatusListener(l);
+ }
+ }
+
+ @Override
+ public synchronized void addProgressListener(ProgressListener l) {
+ if(delegate != null) {
+ delegate.addProgressListener(l);
+ }
+ }
+
+ @Override
+ public synchronized void removeProgressListener(ProgressListener l) {
+ if(delegate != null) {
+ delegate.removeProgressListener(l);
+ }
+ }
+
+ @Override
+ public Resource duplicate(DuplicationContext ctx)
+ throws ResourceInstantiationException {
+ FeatureMap newParams = Factory.newFeatureMap();
+ newParams.put("initParams", Factory.duplicate(getInitParams(), ctx));
+ newParams.put("runtimeParams", Factory.duplicate(getRuntimeParams(), ctx));
+ newParams.put("delegate", Factory.duplicate(delegate, ctx));
+ FeatureMap newFeatures = Factory.duplicate(getFeatures(), ctx);
+ return Factory.createResource(POSTaggerEN.class.getName(), newParams,
+ newFeatures, getName());
+ }
+
}
This was sent by the SourceForge.net collaborative development platform, the
world's largest Open Source development site.
------------------------------------------------------------------------------
_______________________________________________
GATE-cvs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gate-cvs
