| language: en | |
| license: mit | |
| ### SpaceSciBERT | |
| This is one of the 3 further pre-trained models from the SpaceTransformers family presented in [SpaceTransformers: Language Modeling for Space Systems](https://ieeexplore.ieee.org/document/9548078). The original Git repo is [strath-ace/smart-nlp](https://github.com/strath-ace/smart-nlp). | |
| The further pre-training corpus includes publications abstracts, books, and Wikipedia pages related to space systems. Corpus size is 14.3 GB. SpaceSciBERT was further pre-trained on this domain-specific corpus from [SciBERT-SciVocab (uncased)](https://huggingface.co/allenai/scibert_scivocab_uncased). In our paper, it is then fine-tuned for a Concept Recognition task. | |
| ### BibTeX entry and citation info | |
| ``` | |
| @ARTICLE{ | |
| 9548078, | |
| author={Berquand, Audrey and Darm, Paul and Riccardi, Annalisa}, | |
| journal={IEEE Access}, | |
| title={SpaceTransformers: Language Modeling for Space Systems}, | |
| year={2021}, | |
| volume={9}, | |
| number={}, | |
| pages={133111-133122}, | |
| doi={10.1109/ACCESS.2021.3115659} | |
| } | |
| ``` |