Research on Automatic Extraction of Scientific Terminology from Texts Based on Self-Attention

doi:10.3969/j.issn.1673-8578.2021.02.003

Abstract

Abstract:

Scientific terminology uses specific words to represent certain scientific concepts. The extraction of scientific terminology is an important part of the automatic processing of scientific terminology, and it is of great significance for the following tasks such as machine translation, information retrieval, and questions and answers. The traditional extraction of scientific terminology consumes a lot of manpower cost, and an automatic method for extracting scientific terminology is transforming terminology extraction into tagging problem and training out the tagging model through supervised learning methods, while the lack of annotated large-scale scientific terminology corpus is the problem. This paper introduces the method of distant supervision to generate large-scale annotated training corpus, and proposes Bi-LSTM model architecture based on Self-attention mechanism for the purpose of improving the extraction results of scientific terminology. We found that the ability of discovering new scientific terminology about our new model is far superior to the traditional machine learning model (CRF).

Key words: the extraction of scientific terminology, distant supervision, self-attention

CLC Number:

ZHAO Songge, ZHANG Hao, CHANG Baobao. Research on Automatic Extraction of Scientific Terminology from Texts Based on Self-Attention[J]. China Terminology, 2021, 23(2): 20-26.

Figures/Tables 7

	数量/个
训练集	60 000
测试集	100
科技术语词典术语	64 748

CRF模板	Unigram	Bigram	模板特征数	$F term$
模板1	17	1	15190338	0.505
模板2	23	1	15452469	0.508
模板3	41	1	30641751	0.506
模板4	59	1	42838950	0.511
模板5	95	1	155853543	0.501

CRF模板	Unigram	Bigram	模板特征数	$F term$
模板1	17	1	15190338	0.505
模板2	23	1	15452469	0.508
模板3	41	1	30641751	0.506
模板4	59	1	42838950	0.511
模板5	95	1	155853543	0.501

模型	$F term$
CRF模板4	0.511
1Layer_BiLSTM	0.586
2Layer_BiLSTM	0.626
3Layer_BiLSTM	0.633
S_att 1Layer_BiLSTM	0.601
S_att 2Layer_BiLSTM	0.639
S_att 3Layer_BiLSTM	0.637

模型	$F term$
CRF模板4	0.511
1Layer_BiLSTM	0.586
2Layer_BiLSTM	0.626
3Layer_BiLSTM	0.633
S_att 1Layer_BiLSTM	0.601
S_att 2Layer_BiLSTM	0.639
S_att 3Layer_BiLSTM	0.637

References 13

[1]	KAGEURA K, UMINO B. Methods of Automatic Term Recognition[J]. Terminology , 1996,3(2):29-35.
[2]	PANTEL P, LIN D. A Statistical Corpus-Based Term Extractor[M]//STUMPTNER M, CORBETT D, BROOKS M. Advances in Artificial Intelligence. Berlin Heidelberg: Springer-Verlag, 2001: 36-46.
[3]	HISAMITSU T, NIWA Y, TSUJII J. A method of measuring term representativeness baseline method using co-occurrence distribution[C]. COLING, 2000: 320-326.
[4]	CHANG J S. Domain specific word extraction from hierarchical web documents: a first step toward building lexicon trees from web corpora[C]// Proceedings of the 4th SIGHAN Workshop on Chinese Language Learning: 64-71.
[5]	FRANTZI K, ANANIADOU S, MIMA H. Automatic recognition of multi-word terms: the C value/NC-value method[J]. International Journal on Digital Libraries, 2000,3(2):115-130. doi: 10.1007/s007999900023 URL
[6]	NAKAGAWA H, MORI T. A simple but powerful automatic term extraction method[C]//Proceedings of 2nd International Workshop on Computational Terminology. COLING-2002 WORKSHOP, 2002,109(4):229-30.
[7]	WERMTER , JOACHIM , HAHN , et al. Paradigmatic modifiability statistics for the extraction of complex multi-word terms[J]. Proceedings of HLT-EMNLP’05, 2005: 843-850.
[8]	JU Z, ZHOU M, ZHU F. Identifying biological terms from text by support vector machine[J]. Industrial Electronics and Applications, 2011: 455-458.
[9]	ZHANG X, SONG Y, FANG A C. Term recognition using Conditional Random fields[J]. International Conference on Natural Language Processing and Knowledge Engineering, 2010: 1-6.
[10]	LI S, LI J, SONG T, et al. A novel topic model for automatic term extraction[C]// Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval (SIGIR’13). 2013: 885-888.
[11]	MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]. Meeting of the association for computational linguistics, 2009: 1003-1011.
[12]	word2vec project.[EB/OL].(2013-07-30)[2020-04-07]. https://code.google.com/p/word2vec.
[13]	CRF++: Yet Another CRF toolkit.[EB/OL].(2013-02-13)[2020-04-08]. http://taku910.github.io/crfpp.

[1]	CHEN Ke, CHAI Qidong. The Results Analysis of Machine Translation on Petroleum Engineering Terms Based on Vector Space Model [J]. China Terminology, 2022, 24(2): 21-25.
[2]	XIANG Lu, ZHOU Yu, ZONG Chengqing. Bilingual Terminology Alignment Based on Chinese-English Monolingual Terminological Bank [J]. China Terminology, 2022, 24(1): 14-25.
[3]	CHANG Baobao. Techniques of Automatic Term Extraction:Current Sate and Reflections [J]. China Terminology, 2022, 24(1): 3-13.
[4]	WANG Huashu, LIU Shijie. Evaluation Framework of Terminology Extraction Software [J]. China Terminology, 2022, 24(1): 45-54.
[5]	LU Xiaolei, WANG Fanke. Word Embedding: Concepts and Applications [J]. China Terminology, 2020, 22(3): 24-32.
[6]	QIU Bihua. A Frame-based Version of NATO Glossaries [J]. China Terminology, 2020, 22(3): 33-39.
[7]	MUHEYAT Niyazbek, KUENSSAULE Talp. Design and Implementation of a Terminology Recognition System in the IT Field [J]. China Terminology, 2020, 22(2): 29-32.
[8]	LEI Shujie, XING Fukun. Types and Patterns of the English Names of Military Weapons and Equipments [J]. China Terminology, 2019, 21(1): 14-20.
[9]	WANG Jianliang. Introduction of Intelligent Toothbrush [J]. China Terminology, 2014, 16(zk1): 22-23.
[10]	QIAO Yi. Affective Computing [J]. China Terminology, 2014, 16(zk1): 80-82.
[11]	WEI Yanyan. Development of the Internet of Vehicles [J]. China Terminology, 2014, 16(zk1): 146-147.
[12]	ZHANG Hui. Practices and Thoughts on Scientific Neologism Work [J]. China Terminology, 2013, 15(6): 5-9.
[13]	WANG Huashu. Terminology Management in Practice [J]. China Terminology, 2013, 15(2): 11-14.
[14]	LUO Jimei. Translation Errors of Terms in Machine Translation [J]. China Terminology, 2013, 15(1): 41-45.