B f@sUddlmZddlmZmZddlmZddlmZerHddl m Z ndZ dddd Z da d ed <d d dddZd dddZd dddZdS)) annotations) TYPE_CHECKINGcast)Path) TokenizerNr)returncCsttjdS)Nztokenizer.json)r__file__parentr r X/home/ankuromar296_gmail_com/.local/lib/python3.7/site-packages/anthropic/_tokenizers.py_get_tokenizer_cache_pathsr zTokenizerType | None _tokenizerstr TokenizerType)rawrcCs ddlm}tt||atS)Nr)r) tokenizersrrrZfrom_strr )rrr r r _load_tokenizers rcCs&tdk r tSt}|jdd}t|S)Nzutf-8)encoding)r r read_textr)tokenizer_pathtextr r r sync_get_tokenizer#s  rcs0tdk r tStt}|jddIdH}t|S)Nzutf-8)r)r AsyncPathr rr)rrr r r async_get_tokenizer,s  r) __future__rtypingrrpathlibrZanyiorrrrr r __annotations__rrrr r r r s