2 writes to UnkPiece
Microsoft.ML.Tokenizers (2)
SentencepieceModel.cs (2)
2357UnkPiece = other.UnkPiece; 2741UnkPiece = input.ReadString();
7 references to UnkPiece
Microsoft.ML.Tokenizers (7)
Model\SentencePieceBpeTokenizer.cs (1)
67UnknownToken = modelProto.TrainerSpec.UnkPiece ?? "<unk>";
SentencepieceModel.cs (6)
1648if (UnkPiece != other.UnkPiece) return false; 1704if (HasUnkPiece) hash ^= UnkPiece.GetHashCode(); 2051output.WriteString(UnkPiece); 2212size += 2 + pb::CodedOutputStream.ComputeStringSize(UnkPiece); 2357UnkPiece = other.UnkPiece;