小白也能玩转:最简单的win下本地部署大语言模型方法

寂夜清风

通过LM STUDIO在本地部署大语言模型  

https://lmstudio.ai/

LM Studio - Discover, download, and run local LLMs

下载一个LM STUDIO，只有一个文件，运行，搜索大模型名字，就能看到模型名称，右侧列出了本地兼容可以运行的版本。一般都是量化够后的模型，可以根据自己的电脑配置选择下载。（因为国内屏蔽了huggingface等等大模型网站，系统需要科学上网)
下载完之后，点左侧对话图标，可以选择加载已经下载的大模型了。右侧有选项，有GPU的同学可以选择加载到显存的层数，具体设置可以试，放不下会报错。设置为-1就是全部放进显存。我的P40显存有24g，32g大小的模型层数可设置到16。
实际运行这几天最火的dolphin mixtral 7b模型非常流畅，出字后每秒可以8个英文字符左右。

For Windows 链接：https://pan.baidu.com/s/1c35ICKyhfHCocHr5qmJ68A?pwd=we2n
提取码：we2n

For MAC M1/M2/M3  链接：https://pan.baidu.com/s/1pAIfl7M7JSMt4fzUCsbZVg?pwd=u7b4
提取码：u7b4

For Linux 链接：https://pan.baidu.com/s/1CYDziO5Lw9TpGSCUTRd29A?pwd=gilc
提取码：gilc





With LM Studio, you can ...
🤖 - Run LLMs on your laptop, entirely offline
👾 - Use models through the in-app Chat UI or an OpenAI compatible local server
📂 - Download any compatible model files from HuggingFace
🤗 repositories
🔭 - Discover new & noteworthy LLMs in the app's home page

LM Studio supports any ggml

Llama, MPT, and StarCoder model on Hugging Face (Llama 2, Orca, Vicuna, Nous Hermes, WizardCoder, MPT, etc.)

Minimum requirements: M1/M2/M3 Mac, or a Windows PC with a processor that supports AVX2. Linux is available in beta.

Made possible thanks to thellama.cpp project.

We are expanding our team. See our careers page.

寂夜清风

LM Studio Beta Releases

Here you may find from time to time beta releases of LM Studio, provided under the terms of use.

Join the community on Discord to discuss these beta releases.

---

🐧 Linux Beta (Based on 0.2.8)

Includes

• 0.2.8 build for Linux (AppImage)
• Does not support GGML; only GGUF
• Known issues on Ubuntu 20
• If you are getting errors: run the app from the terminal and watch for any error logs
• Does NOT support auto-update; please keep an eye on this page for updates or join the LM Studio Discord server for announcements
• [Bug] Missing icons on some Linux distributions
  

Downloads

• LM+Studio-0.2.8-beta-v1.AppImage
  

---

AVX Support (Based on 0.2.8)

Includes

• For older PCs without AVX2 instruction set
  

Downloads

Windows
• LM-Studio-0.2.8-Setup-avx-beta-2.exe
  

寂夜清风

Frequently Asked Questions

1. TLDR: The app does not collect data nor monitor your actions. Your data stays local on your machine. It's free for personal use. For business use, please get in touch.

复制代码

Does LM Studio collect any data?

No. One of the main reasons for using a local LLM is privacy, and LM Studio is designed for that. Your data remains private and local to your machine.

Can I use LM Studio at work?

Business use will be possible and is planned! As we get started, we ask you to reach out to the team at team@lmstudio.ai. Please allow us some time to respond.

What are the minimum hardware / software requirements?
Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer

Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
NVIDIA/AMD GPUs supported

Are you hiring?
Yes! See our careers page for open positions. We are a small team located in Brooklyn, New York, USA.


Yes! See our careers page for open positions. We are a small team located in Brooklyn, New York, USA.

寂夜清风

https://cloud.tencent.com/developer/article/2372607

使用LM Studio在本地运行LLM完整教程