U ?f@sUddlmZddlmZmZddlmZddlmZerHddl m Z ndZ dddd Z da d ed <d d dddZd dddZd dddZdS)) annotations) TYPE_CHECKINGcast)Path TokenizerNr)returncCsttjdS)Nztokenizer.json)r__file__parentr r X/home/ankuromar296_gmail_com/.local/lib/python3.8/site-packages/anthropic/_tokenizers.py_get_tokenizer_cache_pathsr zTokenizerType | None _tokenizerstr TokenizerType)rawrcCs ddlm}tt||atS)Nrr) tokenizersrrrZfrom_strr)rrr r r _load_tokenizers rcCs&tdk r tSt}|jdd}t|SNzutf-8)encoding)rr read_textrZtokenizer_pathtextr r r sync_get_tokenizer#s  rcs0tdk r tStt}|jddIdH}t|Sr)r AsyncPathr rrrr r r async_get_tokenizer,s  r) __future__rtypingrrpathlibrZanyiorrrrr r__annotations__rrrr r r r s