SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

Record

Authors
Kudo, T.; Richardson, J.
Year
2018
Title
SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
Published in
Proceedings of EMNLP 2018: System Demonstrations
Pages
66–71
Type
Conference paper
Our note
none yet
Verification
matched-by-title: found by title in an open index (same title, year and first author) (checked against OpenAlex, 2026-11-14).
Identifiers found: OpenAlex W2885185669 · DOI 10.18653/v1/d18-2012
Id
kudo-2018-sentencepiece-simple-language-independent-subword-tokenizer

Cited on

As cited

The pages cite this work in one form.

  • Kudo, T., and Richardson, J. (2018). SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of EMNLP 2018: System Demonstrations, 66–71.

BibLaTeX

@inproceedings{kudo-2018-sentencepiece-simple-language-independent-subword-tokenizer,
  author = {Kudo, T. and Richardson, J.},
  title = {{SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing}},
  booktitle = {Proceedings of EMNLP 2018: System Demonstrations},
  date = {2018},
  pages = {66--71}
}

Download · JSON with CSL-JSON · the whole bibliography