Combining Fine-Tuning with a Feature-Based Approach for Aspect Extraction on Reviews

Abstract

One key task of fine-grained sentiment analysis on reviews is to extract aspects or features that users have expressed opinions on. Generally, fine-tuning BERT with sophisticated task-specific layers can achieve better performance than only extend one extra task-specific layer (e.g., a fully-connected + softmax layer) since not all tasks can easily be represented by Transformer encoder architecture and special task-specific layer can capture task-specific features. However, BERT fine-tuning may be unstable on a small-scale dataset. Besides, in our experiments, directly fine-tuning BERT on extending sophisticated task-specific layers did not take advantage of the features of task-specific layers and even restrict the performance of BERT module. To address the above consideration, this paper combines Fine-tuning with a feature-based approach to extract aspect. To the best of our knowledge, this is the first paper to combine fine-tuning with a feature-based approach for aspect extraction.