• Support the openai chat completion normal and stream request

  • Get top_k similar search results, it can be set when creating the data manager

  • Support the cache chain, see: Cache#next_cache

bak_cache = Cache()
  • Whether to completely skip the current cache, that is, do not search the cache or save the Chat GPT results, see: Cache#cache_enable_func

  • In the cache initialization phase, no cache search is performed, but save the result returned by the chat gpt to cache, see: cache_skip=True in create request

  • Like Lego bricks, custom assemble all modules, including:

    • Adapter: The user interface to adapt different LLM model requests to the GPTCache protocol

    • Pre-processor: Extracts the key information from the request and preprocess

    • Context Buffer: Maintains session context

    • Encoder: Embed the text into a dense vector for similarity search

    • Cache manager: which includes searching, saving, or evicting data

    • Ranker: Evaluate similarity by judging the quality of cached answers

    • Post-processor: Determine which cached answers to the user, and generate the response