Unsupervised learning of semantic orientation from a hundred-billion-word corpus

Download
  1. (PDF, 315 KB)
DOIResolve DOI: http://doi.org/10.4224/8914027
AuthorSearch for: ; Search for:
TypeTechnical Report
Series titleERB; no. 1094
Physical description9 leaves
AbstractThe evaluative character of a word is called its semantic orientation. A positive semantic orientation implies desirability (e.g., "honest", "intrepid") and a negative semantic orientation implies undesirability (e.g., "disturbing", "superfluous"). This paper introduces a simple algorithm for unsupervised learning of semantic orientation from extremely large corpora. The method involves issuing queries to a Web search engine and using pointwise mutual information to analyse the results. The algorithm is empirically evaluated using a training corpus of approximately one hundred billion words - the subset of the Web that is indexed by the chosen search engine. Tested with 3,596 words (1,614 positive and 1,982 negative), the algorithm attains an accuracy of 80 percent. The 3,596 test words include adjectives, adverbs, nouns, and verbs. The accuracy is comparable with the results achieved by Hatzivassiloglou and McKeown (1997), using a complex four-stage supervised learning algorithm that is restricted to determining the semantic orientation of adjectives.
Publication date
PublisherNational Research Council Canada
LanguageEnglish
AffiliationNRC Institute for Information Technology; National Research Council Canada
Peer reviewedNo
NRC number44929
NPARC number8914027
Export citationExport as RIS
Report a correctionReport a correction
Record identifier3d270c0f-73ce-4c1f-9641-05e85aff3620
Record created2009-04-22
Record modified2016-09-29
Bookmark and share
  • Share this page with Facebook (Opens in a new window)
  • Share this page with Twitter (Opens in a new window)
  • Share this page with Google+ (Opens in a new window)
  • Share this page with Delicious (Opens in a new window)