找回密码
 加入我们

扫一扫,访问微社区

QQ登录

只需一步,快速开始

搜索
查看: 12|回复: 3

[智能] 小白也能玩转:最简单的win下本地部署大语言模型方法

[复制链接]
发表于 2023-12-28 22:38 | 显示全部楼层 |阅读模式
通过LM STUDIO在本地部署大语言模型  

https://lmstudio.ai/

LM Studio - Discover, download, and run local LLMs


下载一个LM STUDIO,只有一个文件,运行,搜索大模型名字,就能看到模型名称,右侧列出了本地兼容可以运行的版本。一般都是量化够后的模型,可以根据自己的电脑配置选择下载。(因为国内屏蔽了huggingface等等大模型网站,系统需要科学上网)
下载完之后,点左侧对话图标,可以选择加载已经下载的大模型了。右侧有选项,有GPU的同学可以选择加载到显存的层数,具体设置可以试,放不下会报错。设置为-1就是全部放进显存。我的P40显存有24g,32g大小的模型层数可设置到16。
实际运行这几天最火的dolphin mixtral 7b模型非常流畅,出字后每秒可以8个英文字符左右。


23122819335510843c23c82c72.jpg



For Windows 链接:https://pan.baidu.com/s/1c35ICKyhfHCocHr5qmJ68A?pwd=we2n
提取码:we2n

For MAC M1/M2/M3  链接:https://pan.baidu.com/s/1pAIfl7M7JSMt4fzUCsbZVg?pwd=u7b4
提取码:u7b4

For Linux 链接:https://pan.baidu.com/s/1CYDziO5Lw9TpGSCUTRd29A?pwd=gilc
提取码:gilc


demo2.9df5a0e5a9f1d72715e0.gif


With LM Studio, you can ...
🤖 - Run LLMs on your laptop, entirely offline
👾 - Use models through the in-app Chat UI or an OpenAI compatible local server
📂 - Download any compatible model files from HuggingFace
🤗 repositories
🔭 - Discover new & noteworthy LLMs in the app's home page


LM Studio supports any ggml
Llama, MPT, and StarCoder model on Hugging Face (Llama 2, Orca, Vicuna, Nous Hermes, WizardCoder, MPT, etc.)

Minimum requirements: M1/M2/M3 Mac, or a Windows PC with a processor that supports AVX2. Linux is available in beta.

Made possible thanks to thellama.cpp project.

We are expanding our team. See our careers page.

回复

使用道具 举报

 楼主| 发表于 2023-12-28 22:38 | 显示全部楼层
LM Studio Beta Releases
Here you may find from time to time beta releases of LM Studio, provided under the terms of use.
Join the community on Discord to discuss these beta releases.

🐧 Linux Beta (Based on 0.2.8)
Includes
  • 0.2.8 build for Linux (AppImage)
  • Does not support GGML; only GGUF
  • Known issues on Ubuntu 20
  • If you are getting errors: run the app from the terminal and watch for any error logs
  • Does NOT support auto-update; please keep an eye on this page for updates or join the LM Studio Discord server for announcements
  • [Bug] Missing icons on some Linux distributions
Downloads

AVX Support (Based on 0.2.8)
Includes
  • For older PCs without AVX2 instruction set
Downloads

回复 支持 反对

使用道具 举报

 楼主| 发表于 2023-12-28 22:41 | 显示全部楼层

Frequently Asked Questions
  1. TLDR: The app does not collect data nor monitor your actions. Your data stays local on your machine. It's free for personal use. For business use, please get in touch.
复制代码


Does LM Studio collect any data?

No. One of the main reasons for using a local LLM is privacy, and LM Studio is designed for that. Your data remains private and local to your machine.

Can I use LM Studio at work?

Business use will be possible and is planned! As we get started, we ask you to reach out to the team at team@lmstudio.ai. Please allow us some time to respond.

What are the minimum hardware / software requirements?
Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer

Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
NVIDIA/AMD GPUs supported

Are you hiring?
Yes! See our careers page for open positions. We are a small team located in Brooklyn, New York, USA.


Yes! See our careers page for open positions. We are a small team located in Brooklyn, New York, USA.
回复 支持 反对

使用道具 举报

 楼主| 发表于 2024-1-3 00:16 | 显示全部楼层
https://cloud.tencent.com/developer/article/2372607

使用LM Studio在本地运行LLM完整教程
回复 支持 反对

使用道具 举报

您需要登录后才可以回帖 登录 | 加入我们

本版积分规则

QQ|Archiver|手机版|小黑屋|吹友吧 ( 京ICP备05078561号 )

GMT+8, 2024-3-4 19:45 , Processed in 0.223182 second(s), 21 queries , Gzip On.

Powered by Discuz! X3.5 Licensed

© 2001-2023 Discuz! Team.

快速回复 返回顶部 返回列表