English
English Indonesia Türkçe Deutsch Español Português Pусский Français Italiano Nederlands Polski العربية ไทย Tiếng Việt 繁體中文 简体中文 日本語 한국어

#exllamav2 search results

turboderp_'s profile picture.
turboderp
 @turboderp_
Nov 23, 2024

Fun with grounding in Qwen2-VL. Finding the things. #wherearethethings #exllamav2 #cat

1
0
11
5
515
RichardGinsberg's profile picture. SVP, Head of Engineering @guidepoint
Richard Ginsberg
 @RichardGinsberg
Apr 8, 2024

In the top menu, to the right of "Select a model" there is a gear icon. It will bring up the Settings modal. Select Connections and it will have a OpenAI API section. Add the http://ip:port/v1 of your tabbyAPI and your API key. That's it. #exllamav2 #exl2 #llm #localLlama

1
0
0
0
40
GithubtrendingG's profile picture. Daily github trending , You can find it more on https://t.co/u13ED52r9b , Telegram Channel: https://t.co/eoC4iMVAK1
GitHubTrending｜ GitHub每日热点项目追踪
 @GithubtrendingG
Sep 14, 2023

#exllamav2 #Python A fast inference library for running LLMs locally on modern consumer-class GPUs gtrending.top/content/3391/

0
0
0
0
52
extractum_io's profile picture. Solutions for Business with a Touch of AI, LLM and NLP.
EXTRACTUM.IO
 @extractum_io
Dec 26, 2023

#EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs

extractum_io's tweet image. #EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs
0
1
0
0
65
xhinker's profile picture. Author: "Using Stable Diffusion with Python" Love AI, Love GPUs | ex-Data Scientist @ Microsoft | AGI and Python
Andrew Zhu
 @xhinker
Jan 6, 2024

#ExllamaV2 is currently the fastest inference framework for Mixtral 8x7 MoE. It is so good. Can run Mixtral 4bit GPTQ in a 24G + 8G GPU, 3 bit in just one 24G GPU. Its auto VRAM split loading is amazing. github.com/turboderp/exll…

xhinker's tweet card. A fast inference library for running LLMs locally on modern consumer-class GPUs - turboderp-org/exllamav2
GitHub - turboderp-org/exllamav2: A fast inference library for running LLMs locally on modern...
Source: github.com
0
0
0
0
208
turboderp_'s profile picture.
turboderp
 @turboderp_
Nov 23, 2024

Fun with grounding in Qwen2-VL. Finding the things. #wherearethethings #exllamav2 #cat

1
0
11
5
515
xhinker's profile picture. Author: "Using Stable Diffusion with Python" Love AI, Love GPUs | ex-Data Scientist @ Microsoft | AGI and Python
Andrew Zhu
 @xhinker
Jan 6, 2024

#ExllamaV2 is currently the fastest inference framework for Mixtral 8x7 MoE. It is so good. Can run Mixtral 4bit GPTQ in a 24G + 8G GPU, 3 bit in just one 24G GPU. Its auto VRAM split loading is amazing. github.com/turboderp/exll…

xhinker's tweet card. A fast inference library for running LLMs locally on modern consumer-class GPUs - turboderp-org/exllamav2
GitHub - turboderp-org/exllamav2: A fast inference library for running LLMs locally on modern...
Source: github.com
0
0
0
0
208
extractum_io's profile picture. Solutions for Business with a Touch of AI, LLM and NLP.
EXTRACTUM.IO
 @extractum_io
Dec 26, 2023

#EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs

extractum_io's tweet image. #EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs
0
1
0
0
65
GithubtrendingG's profile picture. Daily github trending , You can find it more on https://t.co/u13ED52r9b , Telegram Channel: https://t.co/eoC4iMVAK1
GitHubTrending｜ GitHub每日热点项目追踪
 @GithubtrendingG
Sep 14, 2023

#exllamav2 #Python A fast inference library for running LLMs locally on modern consumer-class GPUs gtrending.top/content/3391/

0
0
0
0
52
No results for "#exllamav2"
extractum_io's profile picture. Solutions for Business with a Touch of AI, LLM and NLP.
EXTRACTUM.IO
 @extractum_io
Dec 26, 2023

#EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs

extractum_io's tweet image. #EXL2 #quantization format introduced in #ExLlamaV2 supports 2 to 8-bit precision. High performance on consumer GPUs. Mixed precision, smaller model size, and lower perplexity while maintaining accuracy. Find EXL2 models at llm.extractum.io/list/?exl2 #MachineLearning #EXL2 #LLMs
0
1
0
0
65

Something went wrong.


Something went wrong.


United States Trends