Inspired from the idea that the contexts in which a word occurs are of different significance, this paper proposes a novel method, called word representation with Salient Features (SaFe), to represent words using salient features selected from the context words. The SaFe method employs the point-wise mutual information (PMI) method with scaled context window to measure word association between a target word and its context. Then, contexts having word associations will be selected as salient features, where the number of salient features for a given word is decided by the ratio between the number of unique contexts and the total counts of occurrences in the whole corpus. The SaFe method can be used with the positive PMI matrix (PPMI), with each row representing a word, hence the name SaFe-PPMI. Moreover, the SaFe-PPMI model can be further decomposed by using the truncated singular vector decomposition technique to obtain dense vectors. In addition to efficient computation, the new models can achieve remarkable improvements in seven semantic relatedness tasks, and they show superior performance when compared with the state-of-the-art models.
Bibliographical noteOpen Access journal published under a CC BY 4.0 license
- Point-wise Mutual Information, Salient Features, Singular Vector Decomposition,
- Word Representation
- Singular Vector Decomposition
- Salient Features
- word representation
- salient features
- singular vector decomposition
- Point-wise mutual information
ASJC Scopus subject areas
- Materials Science(all)
- Computer Science(all)
FingerprintDive into the research topics of 'Word Representation with Salient Features'. Together they form a unique fingerprint.
- Research Centre for Computational Science and Mathematical Modelling - Professor in Artificial Intelligence and Data Science
Person: Teaching and Research