2 writes to EosPiece
Microsoft.ML.Tokenizers (2)
SentencepieceModel.cs (2)
2363EosPiece = other.EosPiece; 2749EosPiece = input.ReadString();
7 references to EosPiece
Microsoft.ML.Tokenizers (7)
Model\SentencePieceTokenizer.cs (1)
65EndOfSentenceToken = modelProto.TrainerSpec.EosPiece ?? "</s>";
SentencepieceModel.cs (6)
1650if (EosPiece != other.EosPiece) return false; 1706if (HasEosPiece) hash ^= EosPiece.GetHashCode(); 2059output.WriteString(EosPiece); 2218size += 2 + pb::CodedOutputStream.ComputeStringSize(EosPiece); 2363EosPiece = other.EosPiece;