Word Representation with Salient Features

Ming Zhang, Vasile Palade, Yan Wang, Zhicheng Ji

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
10 Downloads (Pure)


Inspired from the idea that the contexts in which a word occurs are of different significance, this paper proposes a novel method, called word representation with Salient Features (SaFe), to represent words using salient features selected from the context words. The SaFe method employs the point-wise mutual information (PMI) method with scaled context window to measure word association between a target word and its context. Then, contexts having word associations will be selected as salient features, where the number of salient features for a given word is decided by the ratio between the number of unique contexts and the total counts of occurrences in the whole corpus. The SaFe method can be used with the positive PMI matrix (PPMI), with each row representing a word, hence the name SaFe-PPMI. Moreover, the SaFe-PPMI model can be further decomposed by using the truncated singular vector decomposition technique to obtain dense vectors. In addition to efficient computation, the new models can achieve remarkable improvements in seven semantic relatedness tasks, and they show superior performance when compared with the state-of-the-art models.

Original languageEnglish
Article number8611085
Pages (from-to)30157-30173
Number of pages17
JournalIEEE Access
Publication statusPublished - 14 Jan 2019

Bibliographical note

Open Access journal published under a CC BY 4.0 license


  • Point-wise Mutual Information, Salient Features, Singular Vector Decomposition,
  • Word Representation
  • Singular Vector Decomposition
  • Salient Features
  • word representation
  • salient features
  • singular vector decomposition
  • Point-wise mutual information

ASJC Scopus subject areas

  • Engineering(all)
  • Materials Science(all)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Word Representation with Salient Features'. Together they form a unique fingerprint.

Cite this