Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : V. D. Viellieber
- sl:arxiv_num : 2012.02558
- sl:arxiv_published : 2020-12-04T12:49:47Z
- sl:arxiv_summary : Recently it has been shown that large pre-trained language models like BERT
(Devlin et al., 2018) are able to store commonsense factual knowledge captured
in its pre-training corpus (Petroni et al., 2019). In our work we further
evaluate this ability with respect to an application from industry creating a
set of probes specifically designed to reveal technical quality issues captured
as described incidents out of unstructured customer feedback in the automotive
industry. After probing the out-of-the-box versions of the pre-trained models
with fill-in-the-mask tasks we dynamically provide it with more knowledge via
continual pre-training on the Office of Defects Investigation (ODI) Complaints
data set. In our experiments the models exhibit performance regarding queries
on domain-specific topics compared to when queried on factual knowledge itself,
as Petroni et al. (2019) have done. For most of the evaluated architectures the
correct token is predicted with a $Precision@1$ ($P@1$) of above 60\%, while
for $P@5$ and $P@10$ even values of well above 80\% and up to 90\% respectively
are reached. These results show the potential of using language models as a
knowledge base for structured analysis of customer feedback.@en
- sl:arxiv_title : Pre-trained language models as knowledge bases for Automotive Complaint Analysis@en
- sl:arxiv_updated : 2020-12-04T12:49:47Z
- sl:bookmarkOf : https://arxiv.org/abs/2012.02558
- sl:creationDate : 2021-04-11
- sl:creationTime : 2021-04-11T09:30:04Z
Documents with similar tags (experimental)
Préférences...