Michael Yang 333e360422 model: handle multiple eos tokens (#10577) ...

* get eos_token_id from generation_config.json

* refactor

* include both ids and strings in trace

* comments

* remove special case for gemma3 special vocab (#10743)

2025-05-16 13:40:23 -07:00

4.6 KiB

Raw Permalink Blame History

View Raw