Attention-based word embeddings using Artificial Bee Colony algorithm for aspect-level sentiment classification

Ming Zhang, Vasile Palade, Yan Wang, Zhicheng Ji

    Research output: Contribution to journalArticlepeer-review

    22 Citations (Scopus)
    143 Downloads (Pure)


    Considering that most popular models solving aspect-level sentiment classification problems mainly focus on designing complicated neural networks to scale the importance of each word in the sentence, this paper addresses this problem from the view of semantic space. Motivated by the fact that the senses of a word can be sophisticatedly embedded into the semantic space using a distributed representation, this paper hypothesizes that each sense of a word can be represented by one or more specific dimensions, and thus the target of aspect-level sentiment classification can be simplified into searching the related dimensions for the aspects and sentiments concerned. Particularly, an Attention Vector (ATV) based on attention mechanism is designed for each aspect in terms of a specific task, which involves two sub-vectors, i.e., a Dimension Attention Vector (DATV) and a Sentiment Attention Vector (SATV). The DATV determines the significances of different dimensions based on their correlations with an aspect; and the SATV allocates weights for the attributes of words, which are decided by sentiment polarities and part-of-speech (PoS) tagging. Given a sub-dataset related to an aspect, the ATV will be optimized by an Artificial Bee Colony (ABC) algorithm with a Support Vector Machine (SVM) classifier, the objective of which is to maximize classification accuracy. Intrinsically, the DATV can reduce the ambiguity existing in polysemy, meanwhile, the SATV is an auxiliary means for the optimization of the DATV, which can help eliminate the misunderstandings caused by antonyms. Then, the optimized DATV will be applied on a Convolutional Neural Network (CNN) model via simply scaling the pretrained word embeddings as inputs (named as ATV-CNN model). Experimental results show that the ATV-CNN model can have substantial advantages when compared with state-of-the-art models.

    Original languageEnglish
    Pages (from-to)713-738
    Number of pages26
    JournalInformation Sciences
    Early online date28 Sept 2020
    Publication statusPublished - 4 Feb 2021

    Bibliographical note

    NOTICE: this is the author’s version of a work that was accepted for publication in Information Sciences. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Information Sciences [[545] (2021)] DOI: 10.1016/j.ins.2020.09.038

    © 2021, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International


    • artificial Bee Colony algorithm
    • Aspect-level sentiment classification
    • Attention mechanism
    • Support Vector Machine
    • Word embeddings

    ASJC Scopus subject areas

    • Software
    • Control and Systems Engineering
    • Theoretical Computer Science
    • Computer Science Applications
    • Information Systems and Management
    • Artificial Intelligence


    Dive into the research topics of 'Attention-based word embeddings using Artificial Bee Colony algorithm for aspect-level sentiment classification'. Together they form a unique fingerprint.

    Cite this