本地大模型知识库问答

1 min read

本地部署

下载大模型

大模型下载

ChatGLM3

git clone https://github.com/THUDM/ChatGLM3
cd ChatGLM3
pip install -r requirements.txt
python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("/home/wubigo/model/ZhipuAI/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("/home/wubigo/model/ZhipuAI/chatglm3-6b", trust_remote_code=True, device='cuda')
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])

初始化知识库

git clone --recursive https://github.com/chatchat-space/Langchain-Chatchat.git
cd Langchain-Chatchat
pip install -r requirements.txt
python copy_config_example.py
python init_database.py --recreate-vs

启动服务

python startup.py -a
LLM GPT

Related