intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Language: Python - Size: 409 MB - Last synced: 7 days ago - Pushed: 7 days ago - Stars: 2,009 - Forks: 239

stdlib-js/constants-int8

8-bit signed integer mathematical constants.

Language: JavaScript - Size: 417 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 2 - Forks: 0

stdlib-js/constants-int8-num-bytes

Size (in bytes) of an 8-bit signed integer.

Language: JavaScript - Size: 277 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 1 - Forks: 0

stdlib-js/napi-argv-strided-int8array

Convert a Node-API value representing a strided array to a signed 8-bit integer array.

Language: C - Size: 146 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 1 - Forks: 0

stdlib-js/napi-argv-int8array

Convert a Node-API value to a signed 8-bit integer array.

Language: C - Size: 145 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 1 - Forks: 0

stdlib-js/assert-is-int8array

Test if a value is an Int8Array.

Language: JavaScript - Size: 563 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 1 - Forks: 0

stdlib-js/constants-int8-max

Maximum signed 8-bit integer.

Language: JavaScript - Size: 281 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 1 - Forks: 0

stdlib-js/constants-int8-min

Minimum signed 8-bit integer.

Language: JavaScript - Size: 281 KB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 2 - Forks: 0

intel/neural-speed

An innovative library for efficient LLM inference via low-bit quantization

Language: C++ - Size: 14.8 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 179 - Forks: 20

psychose-club/IBO

IBO stands for "Internal binary operations" and it is a library for Java to read, write, and handle binary files and data types that aren't available in Java.

Language: Java - Size: 546 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 3 - Forks: 0

cbalint13/rvv-kernels

RISCV Vector Kernel C/LLVM-IR generator

Language: Python - Size: 13.5 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 1 - Forks: 1

Wulingtian/yolov5_tensorrt_int8

TensorRT int8 量化部署 yolov5s 模型，实测3.3ms一帧！

Language: C++ - Size: 6.66 MB - Last synced: 3 months ago - Pushed: about 3 years ago - Stars: 158 - Forks: 25

xuanandsix/Tensorrt-int8-quantization-pipline

a simple pipline of int8 quantization based on tensorrt.

Language: Python - Size: 836 KB - Last synced: 3 months ago - Pushed: over 1 year ago - Stars: 27 - Forks: 2

Wulingtian/yolov5_tensorrt_int8_tools

tensorrt int8 量化yolov5 onnx模型

Language: Python - Size: 7.51 MB - Last synced: 3 months ago - Pushed: about 3 years ago - Stars: 165 - Forks: 39

aahouzi/llama2-chatbot-cpu

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

Language: Python - Size: 30.3 MB - Last synced: 3 months ago - Pushed: 3 months ago - Stars: 6 - Forks: 0

clancylian/retinaface

Reimplement RetinaFace use C++ and TensorRT

Language: C++ - Size: 5.71 MB - Last synced: 7 months ago - Pushed: over 4 years ago - Stars: 288 - Forks: 89

MrFMach/Practice-C-types

Practicing C data types using the sizeof function

Language: C - Size: 2.93 KB - Last synced: 9 months ago - Pushed: over 3 years ago - Stars: 0 - Forks: 0

yester31/Quantization_EX

quantization example for pqt & qat

Language: Python - Size: 94.7 KB - Last synced: 10 months ago - Pushed: 10 months ago - Stars: 0 - Forks: 0

dasdristanta13/LLM-Lora-PEFT_accumulate

LLM-Lora-PEFT_accumulate explores optimizations for Large Language Models (LLMs) using PEFT, LORA, and QLORA. Contribute experiments and implementations to enhance LLM efficiency. Join discussions and push the boundaries of LLM optimization. Let's make LLMs more efficient together!

Language: Jupyter Notebook - Size: 138 KB - Last synced: 12 months ago - Pushed: 12 months ago - Stars: 3 - Forks: 1

Wulingtian/RepVGG_TensorRT_int8

RepVGG TensorRT int8 量化，实测推理不到1ms一帧！

Language: Python - Size: 469 KB - Last synced: about 1 year ago - Pushed: about 3 years ago - Stars: 57 - Forks: 14

ppogg/ncnn-yolov4-int8

NCNN+Int8+YOLOv4 quantitative modeling and real-time inference

Language: C++ - Size: 14 MB - Last synced: about 1 year ago - Pushed: almost 3 years ago - Stars: 20 - Forks: 4

whitelok/tensorrt-int8-python-sample

TensorRT Int8 Python version sample. TensorRT Int8 Python 实现例子。TensorRT Int8 Pythonの例です

Language: Python - Size: 1.5 MB - Last synced: about 1 year ago - Pushed: over 5 years ago - Stars: 14 - Forks: 1

stdlib-js/array-int8

Int8Array.

Language: JavaScript - Size: 788 KB - Last synced: 17 days ago - Pushed: about 2 months ago - Stars: 2 - Forks: 0

egbertYeah/mt-yolov6_tensorrt

MT-Yolov6 TensorRT Inference with Python.

Language: Python - Size: 55.6 MB - Last synced: about 1 year ago - Pushed: almost 2 years ago - Stars: 4 - Forks: 0

Wulingtian/nanodet_tensorrt_int8

nanodet int8 量化，实测推理2ms一帧！

Language: C++ - Size: 6.24 MB - Last synced: about 1 year ago - Pushed: about 3 years ago - Stars: 37 - Forks: 6

lbin/gie_int8_sample

Language: C++ - Size: 24.4 KB - Last synced: about 1 year ago - Pushed: about 7 years ago - Stars: 1 - Forks: 0