Word Representation with Salient Features

Ming Zhang, Vasile Palade, Yan Wang, Zhicheng Ji

    Research output: Contribution to journalArticlepeer-review

    6 Citations (Scopus)
    15 Downloads (Pure)

    Abstract

    Inspired from the idea that the contexts in which a word occurs are of different significance, this paper proposes a novel method, called word representation with Salient Features (SaFe), to represent words using salient features selected from the context words. The SaFe method employs the point-wise mutual information (PMI) method with scaled context window to measure word association between a target word and its context. Then, contexts having word associations will be selected as salient features, where the number of salient features for a given word is decided by the ratio between the number of unique contexts and the total counts of occurrences in the whole corpus. The SaFe method can be used with the positive PMI matrix (PPMI), with each row representing a word, hence the name SaFe-PPMI. Moreover, the SaFe-PPMI model can be further decomposed by using the truncated singular vector decomposition technique to obtain dense vectors. In addition to efficient computation, the new models can achieve remarkable improvements in seven semantic relatedness tasks, and they show superior performance when compared with the state-of-the-art models.

    Original languageEnglish
    Article number8611085
    Pages (from-to)30157-30173
    Number of pages17
    JournalIEEE Access
    Volume7
    DOIs
    Publication statusPublished - 14 Jan 2019

    Bibliographical note

    Open Access journal published under a CC BY 4.0 license

    Keywords

    • Point-wise Mutual Information, Salient Features, Singular Vector Decomposition,
    • Word Representation
    • Singular Vector Decomposition
    • Salient Features
    • word representation
    • salient features
    • singular vector decomposition
    • Point-wise mutual information

    ASJC Scopus subject areas

    • Engineering(all)
    • Materials Science(all)
    • Computer Science(all)

    Fingerprint

    Dive into the research topics of 'Word Representation with Salient Features'. Together they form a unique fingerprint.

    Cite this