!cd llama2.c && python train.py --vocab_source=custom --vocab_size=4096 --compile=False \
--dim=128 --n_layers=5 --n_heads=8 --n_kv_heads=4 --batch_size=32 \
--always_save_checkpoint=True --eval_interval=100 --max_iters=100 --init_from='resume'
--max_iters=1 val loss 8.3547
Once upon a time lifted pract and popcorn tap Her Sam Bunadow prize anymoreff envel white thanked glass uncle wagged closet trip colorful apologized ey pocket shorephant fox sandimal forever es sheepcanateral Comebow swimming ticket sadWhat cookrom rollonest el tal differe examvent soellyised restless journ refved inside hurt ey imagineves changed would. movingageil zipitediv Kittyisk lead eye scr mop Sam caulifere seek/ planetundith factor Br balance no syLook turt Fl sn ext cu clot create secret throw farm." yell eggs0 rot unexpected gener doing from ones yachtucky fastitchenF ready havingful ink soft now viol tegginky cross Mum colors cars buy Timuggledins fearfulggy puzz/ songs modern sho My sandwich became forestwhere horse dough shout ju magicaline zoom gettingued afterragcked track mask floor ind threw mad disapp tastyillyouse0 laug independent special trust trustessert Ros stre cand amazingool puddleikeitt creat smilingandy toucdom singingisy wool polishub wave streasingpes ingredientsfter gave d One helped Peteth shake ele our dreaming ornament rolled suddenly twig every super cage team sy does smiles firees scootro cold empty flowerurous splashed cooking bel nuts crayonsila pro driver createM ne ext
achieved tok/s: 498.046875
--max_iters=10 val loss 8.3436
Once upon a time can Mark callingfish hugs Frhatoin flow tripQu villffpectedC arms stranger peach ta cauliflower lift dinosaurinoc Mum gent Finally swingingoo e temuce pasta Tiny boss staygetable goodbye cheer groundoof blue thin slowly smell brokenation gettingbr talkingceed lakeearvedatch mushroometh maskKilaield wat always One eggs bird mop coaloolshine myrpar look Finornornseum harder comfort give Fluffy treious maze bottom boring carrot thu dolphin sett counted prothost ourventually ter buckonely thinking per underst plants wordsame sorryrel disappoin Joe int rec fightingockedisk into classoun exciting finished Buzz One bumpkinses leaves batter Ella hug pebble field R doing strong farm nowhere stayed But toys liked glassella firstra ingredients hard vase guitar popularbyeZloo anyormous desk baseball unpack fu tight motherumberllairp cries been shoe fore rubbumpet generous moms ru lunchHelp putting explower glass yours triedief magical dance weekbo carried O Tweet acc smiling fight mazenot L lookisyang envel enc saladoom biggest floor grateful day ride interesting Sammyery wide round Brown nurse came book dull stronger paws mad blebr hunt; shootupid watched tries delicious delicious idea bug like store O courvouserpoop decor cre distance queen away R getG boat
achieved tok/s: 518.292683
--max_iters=100 val loss 7.6735
Once upon a time let ears right each Sarah others childrenem Tom playever bite char basketball music Chirpy trueight by stret Soon other downHelp tripped make scoot Bobo' watched?" firework boring herb en wr eag fallen shake girl Don! fine cheerful cleaned dif much jelly basket tooth playful fore!” ju Emily dri woman invited laughingm carrot stuTimmyLookout Spot cere shrimp cost The around engelvoroubby tem dugmb does snake suitting learning sure as bat else move club wear run Polly balance teachow pri fall returnedsses own send mom factory apples Grandma first Polly imag arriowlually lots man alongunch line played dolphin create fallen mademb deer smo met shinyugg5 fire different nice butterflies berries surpr Timmy sign yogurt promise queearator rep bushes law kindcer stri shiny steakWhysc today flower dif fireworkiches squee soccer talT perfric fluffyM pun funnyull showedall watch supp pig gro kiss post So-aur forgot res allract treasureumsy if cop Ducky tie once dessertcreamaking m fish smile bandageorable prot ag barked smil barked tree jumping thick Mrstri syselves likesccoli turns terri cact coat kite toyckwhere Emma have hoped sold bunny pipe des mother fromicyister imag nature meal chess bandage turt but spinF Snowappy sail chocolJ
achieved tok/s: 497.076023
--max_iters=500 val loss 3.8695
Once upon a time
The Her friends, his mom saw a big he came back. Lily's mom smiled and started to get to play. It was a big, but she loved to me. He flew on the kitchen, but the cute the fake park.
S gu was so happy that he went. He could have better to go." She said, Lily.
Lily came to her news in the hole the sky.
achieved tok/s: 619.047619
--max_iters=1000 val loss 2.7603
Once upon a time
The picture was very surprised and couldn't keep making it make it feel better. He would make his little friend be careful with his friends and play with his friends.
One day, a boy named Tim went to the park to play with his friends. They would play together and take a moment and always put on his fire and carry the ball. One day, Tim's mommy said, "Max, you are too heavy for me to find a big box!" Tim said, "Of course, Tom. It's very pretty."
Tim looked at his mom and said, "No, I like the ball. I will help you find it!" They all wanted to play with the ball, but they knew that he needed to be kind and not work.
achieved tok/s: 643.137255