Skip to content

Latest commit

 

History

History
21 lines (16 loc) · 978 Bytes

Tokens.md

File metadata and controls

21 lines (16 loc) · 978 Bytes
!sed -i "113s/.*/                                   byte_fallback=False,/" ./llama2.c/tinystories.py
!cd llama2.c && python tinystories.py train_vocab --vocab_size=100
LOG(INFO) Adding meta_piece: <unk>
LOG(INFO) Adding meta_piece: <s>
LOG(INFO) Adding meta_piece: </s>
LOG(INFO) Alphabet size=102
Vocabulary size is smaller than required_chars. 100 vs 105.


!cd ./llama2.c && ./run out/model.bin -z data/tok105.bin -t 0.8 -n 256 -i "Once upon a time "
Once upon a time 0nrZAg5jjjXYYY>>kw™kkk“r  ]*g:KZZHHO]`w 7
achieved tok/s: 951.612903


!cd ./llama2.c && ./run out/model.bin -z data/tok32000.bin -t 0.8 -n 32 -i "Once upon a time "
Once upon a time anced storekeeper heelsuously MohammadBakerBeetles kettle transl nudgebrilliant hesitation rounderdge clinic pilotsWhispyarewelluddenly Naya unlechol bluedogprovide honey barks flag
achieved tok/s: 87.570621