Towards Comparable Ratings: Quantifying Evaluative Phrases in Physician Reviews

We present a concept for quantifying evaluative phrases to later compare rating texts numerically instead of just relying on stars or grades. We achievethis by combining deep learning models in an aspect-based sentiment analysis pipeline along with sentiment weighting, polarity, and correlation analyses that combine deep learning results with metadata. The results provide new insights for the medical field. Our application domain, physician reviews, shows that there are millions of review texts on the Internet that cannot yet be comprehensively analyzed because previous studies have focused on explicit aspects from other domains (e.g., products). We identify, extract, and classify implicit and explicit aspect phrases equally from German-language review texts. To do so, we annotated aspect phrases representing reviews on numerous aspects of a physician, medical practice, or practice staff. We apply the best performing transformer model, XLM-RoBERTa, to a large physician review dataset and correlate the results with existing metadata. As a result, we can show different correlations between the sentiment polarity of certain aspect classes (e.g., friendliness, practice equipment) and physicians’ professions (e.g., surgeon, ophthalmologist). As a result, we have individual numerical scores that contain a variety of information based on deep learning algorithms that extract textual (evaluative) information and metadata from the Web.