Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: streamingllm

DefTruth/Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

Size: 114 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1,183 - Forks: 87

intel/intel-extension-for-transformers

âš¡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platformsâš¡

Language: Python - Size: 543 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1,909 - Forks: 180

Related Keywords
streamingllm 2 chatpdf 1 cpu 1 gaudi2 1 gpu 1 habana 1 intel-optimized-llamacpp 1 large-language-model 1 llm-cpu 1 llm-inference 1 neural-chat 1 neural-chat-7b 1 neurips2023 1 rag 1 retrieval 1 speculative-decoding 1 xeon 1 chatbot 1 4-bits 1 vllm 1 tensorrt-llm 1 streaming-llm 1 sora 1 paged-attention 1 mamba 1 kv-quant 1 inferflow 1 flash-decoding 1 flash-attention-2 1 flash-attention 1 awq 1 awesome-llm 1