Linux Mint
My Places
Sign in
Community
Software
Users
Moderation
Beta Testing
ISO Images
Teams
sentencepiece
Unsupervised text tokenizer and detokenizer
https://github.com/google/sentencepiece
0
no reviews
Install
SentencePiece is an unsupervised text tokenizer/detokenizer mainly designed for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.