This repository has been archived by the owner on May 17, 2023. It is now read-only.

implement full decoder API #1

Open

dfalbel opened this issue Feb 1, 2021 · 0 comments

Member

dfalbel commented Feb 1, 2021

See https://github.com/huggingface/tokenizers/blob/master/bindings/python/src/decoders.rs

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.