GPTCache#
core#
- class gptcache.core.Cache[source]#
Bases:
object
GPTCache core object.
Example
from gptcache import cache from gptcache.adapter import openai cache.init() cache.set_openai_key()
- init(cache_enable_func=<function cache_all>, pre_embedding_func=<function last_content>, pre_func=None, embedding_func=<function to_embeddings>, data_manager: gptcache.manager.data_manager.DataManager = <gptcache.manager.data_manager.MapDataManager object>, similarity_evaluation=<gptcache.similarity_evaluation.exact_match.ExactMatchEvaluation object>, post_process_messages_func=<function temperature_softmax>, post_func=None, config=<gptcache.config.Config object>, next_cache=None)[source]#
Pass parameters to initialize GPTCache.
- Parameters
cache_enable_func β a function to enable cache, defaults to
cache_all
pre_embedding_func β a function to preprocess embedding, defaults to
last_content
pre_func β a function to preprocess embedding, same as
pre_embedding_func
embedding_func β a function to extract embeddings from requests for similarity search, defaults to
string_embedding
data_manager β a
DataManager
module, defaults toget_data_manager()
similarity_evaluation β a module to calculate embedding similarity, defaults to
ExactMatchEvaluation()
post_process_messages_func β a function to post-process messages, defaults to
temperature_softmax
with a default temperature of 0.0post_func β a function to post-process messages, same as
post_process_messages_func
config β a module to pass configurations, defaults to
Config()
next_cache β customized method for next cache
- import_data(questions: List[Any], answers: List[Any], session_ids: Optional[List[Optional[str]]] = None) None [source]#
Import data to GPTCache
- Parameters
questions β preprocessed question Data
answers β list of answers to questions
session_ids β list of the session id.
- Returns
None
config#
- class gptcache.config.Config(log_time_func: Optional[Callable[[str, float], None]] = None, similarity_threshold: float = 0.8, prompts: Optional[List[str]] = None, template: Optional[str] = None, auto_flush: int = 20, enable_token_counter: bool = True, input_summary_len: Optional[int] = None, context_len: Optional[int] = None, skip_list: Optional[List[str]] = None, data_check: bool = False)[source]#
Bases:
object
Pass configuration.
- Parameters
log_time_func (Optional[Callable[[str, float], None]]) β optional, customized log time function
similarity_threshold (float) β a threshold ranged from 0 to 1 to filter search results with similarity score higher than the threshold. When it is 0, there is no hits. When it is 1, all search results will be returned as hits.
prompts (Optional[List[str]]) β optional, if the request content will remove the prompt string when the request contains the prompt list
template (Optional[str]) β optional, if the request content will remove the template string and only keep the parameter value in the template
auto_flush (int) β it will be automatically flushed every time xx pieces of data are added, default to 20
enable_token_counter (bool) β enable token counter, default to False
input_summary_len (Optional[int]) β optional, summarize input to specified length.
skip_list (Optional[List[str]]) β for sequence preprocessing, skip those sentences in skip_list.
context_len (Optional[int]) β optional, the length of context.
Example
from gptcache import Config configs = Config(similarity_threshold=0.6)
session#
- class gptcache.session.Session(name: Optional[str] = None, data_manager: Optional[gptcache.manager.data_manager.DataManager] = None, check_hit_func: Optional[Callable] = None)[source]#
Bases:
object
Session for gptcache. Session can isolate the context of each connection, and can also filter the results after recall, and if not satisfied will re-request rather than return the cache results directly.
- Parameters
name (str) β the name of the session, defaults to uuid.uuid4().hex.
data_manager (DataManager) β the DataManager of the session, defaults to cache.data_manager with the initialized cache.
check_hit_func (Callable) β a Callable to check the hit, defaults to processor.check_hit.check_hit_sessionοΌwhich will not return cached data if you ask the same or similar question in the same session.
Example
from gptcache import cache from gptcache.session import Session # init gptcache cache.init() cache.set_openai_key() session = Session() from gptcache.adapter import openai # run ChatCompletion model with gptcache on session response = openai.ChatCompletion.create( model='gpt-3.5-turbo', messages=[ { 'role': 'user', 'content': "what's github" }], session=session ) response_content = response['choices'][0]['message']['content']
- property name#
report#
- class gptcache.report.Report[source]#
Bases:
object
Get GPTCache report including time and counts for different operations.
- embedding(delta_time)[source]#
Embedding counts and time.
- Parameters
delta_time β additional runtime.
- evaluation(delta_time)[source]#
Evaluation counts and time.
- Parameters
delta_time β additional runtime.
client#
- class gptcache.client.Client(uri: str = 'http://localhost:8000')[source]#
Bases:
object
GPTCache client to send requests to GPTCache server.
- Parameters
uri (str) β the uri leads to the server, defaults to βhttp://localhost:8000β.
Example
from gptcache import client client = Client(uri="http://localhost:8000") client.put("Hi", "Hi back") ans = client.get("Hi")