prompt-injection | Topic | Ecosyste.ms: Repos

Topic: "prompt-injection"

asgeirtj/system_prompts_leaks

Collection of extracted System Prompts from popular chatbots like ChatGPT, Claude & Gemini

Language: JavaScript - Size: 245 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 5,916 - Forks: 1,256

CyberAlbSecOP/Awesome_GPT_Super_Prompting

ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.

Size: 6.32 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 2,474 - Forks: 313

protectai/llm-guard

The Security Toolkit for LLM Interactions

Language: Python - Size: 4.01 MB - Last synced at: 4 days ago - Pushed at: 6 days ago - Stars: 1,748 - Forks: 227

protectai/rebuff 📦

LLM Prompt Injection Detector

Language: TypeScript - Size: 6.99 MB - Last synced at: 4 days ago - Pushed at: 10 months ago - Stars: 1,292 - Forks: 104

microsoft/AI-Red-Teaming-Playground-Labs

AI Red Teaming playground labs to run AI Red Teaming trainings including infrastructure.

Language: TypeScript - Size: 975 KB - Last synced at: 2 days ago - Pushed at: 5 days ago - Stars: 1,240 - Forks: 168

abilzerian/LLM-Prompt-Library

A playground of highly experimental prompts, tools & scripts for machine intelligence models from DeepSeek, OpenAI, Anthropic, Meta, Mistral, Google, xAI & others.

Language: Python - Size: 143 KB - Last synced at: about 2 months ago - Pushed at: 2 months ago - Stars: 1,116 - Forks: 116

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀

Language: Jupyter Notebook - Size: 4.38 MB - Last synced at: 6 days ago - Pushed at: 7 months ago - Stars: 915 - Forks: 69

utkusen/promptmap

a prompt injection scanner for custom LLM applications

Language: Python - Size: 239 KB - Last synced at: 24 days ago - Pushed at: 3 months ago - Stars: 785 - Forks: 80

yunanwg/brilliant-CV

💼 another CV template for your job application, yet powered by Typst and more

Language: Typst - Size: 7.73 MB - Last synced at: 6 days ago - Pushed at: 27 days ago - Stars: 576 - Forks: 45

zacfrulloni/Prompt-Engineering-Holy-Grail

Join our online community: https://www.skool.com/lovable-vibe-coding-4202

Language: HTML - Size: 888 KB - Last synced at: 30 days ago - Pushed at: 30 days ago - Stars: 460 - Forks: 50

tldrsec/prompt-injection-defenses

Every practical and proposed defense against prompt injection.

Size: 130 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 389 - Forks: 27

deadbits/vigil-llm

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

Language: Python - Size: 548 KB - Last synced at: 20 days ago - Pushed at: over 1 year ago - Stars: 385 - Forks: 39

automorphic-ai/aegis

Self-hardening firewall for large language models

Language: Python - Size: 21.5 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 264 - Forks: 6

langgptai/Awesome-Multimodal-Prompts

Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.

Size: 87.3 MB - Last synced at: 4 days ago - Pushed at: over 1 year ago - Stars: 253 - Forks: 18

dropbox/llm-security

Dropbox LLM Security research code and results

Language: Python - Size: 1.61 MB - Last synced at: 5 days ago - Pushed at: about 1 year ago - Stars: 227 - Forks: 29

liu00222/Open-Prompt-Injection

This repository provides a benchmark for prompt Injection attacks and defenses

Language: Python - Size: 1.49 MB - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 212 - Forks: 34

shell-nlp/gpt_server

gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。

Language: Python - Size: 4.18 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 191 - Forks: 16

yunwei37/prompt-hacker-collections

prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记

Size: 138 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 162 - Forks: 20

lakeraai/pint-benchmark

A benchmark for prompt injection detection systems.

Language: Jupyter Notebook - Size: 2.29 MB - Last synced at: 20 days ago - Pushed at: about 1 month ago - Stars: 113 - Forks: 12

pasquini-dario/project_mantis

Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks

Language: Python - Size: 1.21 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 67 - Forks: 6

HumanCompatibleAI/tensor-trust

A prompt injection game to collect data for robust ML research

Language: Python - Size: 8.34 MB - Last synced at: 2 days ago - Pushed at: 5 months ago - Stars: 61 - Forks: 5

TrustAI-laboratory/Learn-Prompt-Hacking

This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.

Language: Jupyter Notebook - Size: 6.7 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 51 - Forks: 4

NullTrace-Security/Exploiting-AI

This class is a broad overview and dive into Exploiting AI and the different attacks that exist, and best practice strategies.

Language: Python - Size: 19.6 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 50 - Forks: 14

CX330Blake/Spell-Whisperer

Language: TypeScript - Size: 4.48 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 44 - Forks: 1

GPTSafe/PromptGuard

Build production ready apps for GPT using Node.js & TypeScript

Language: TypeScript - Size: 848 KB - Last synced at: 3 days ago - Pushed at: about 2 years ago - Stars: 43 - Forks: 1

LostOxygen/llm-confidentiality

Whispers in the Machine: Confidentiality in Agentic Systems

Language: Python - Size: 1.39 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 37 - Forks: 4

jailbreakme-xyz/jailbreak

jailbreakme.xyz is an open-source decentralized app (dApp) where users are challenged to try and jailbreak pre-existing LLMs in order to find weaknesses and be rewarded. 🏆

Language: JavaScript - Size: 313 MB - Last synced at: 27 days ago - Pushed at: 27 days ago - Stars: 35 - Forks: 19

sinanw/llm-security-prompt-injection

This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.

Language: Jupyter Notebook - Size: 2.75 MB - Last synced at: 7 months ago - Pushed at: over 1 year ago - Stars: 34 - Forks: 7

kereva-dev/kereva-scanner

Code scanner to check for issues in prompts and LLM calls

Language: Python - Size: 7.12 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 29 - Forks: 2

wearetyomsmnv/Awesome-LLMSecOps

LLM | Security | Operations in one github repo with good links and pictures.

Language: HTML - Size: 511 KB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 29 - Forks: 3

microsoft/gandalf_vs_gandalf

Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.

Language: Jupyter Notebook - Size: 2.61 MB - Last synced at: 2 days ago - Pushed at: over 1 year ago - Stars: 29 - Forks: 1

MaxMLang/pytector

Easy to use LLM Prompt Injection Detection / Detector Python Package

Language: Python - Size: 49.8 KB - Last synced at: 8 days ago - Pushed at: about 1 month ago - Stars: 27 - Forks: 21

lakeraai/chainguard

Guard your LangChain applications against prompt injection with Lakera ChainGuard.

Language: Python - Size: 1.34 MB - Last synced at: 24 days ago - Pushed at: 3 months ago - Stars: 22 - Forks: 5

SemanticBrainCorp/SemanticShield

The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).

Language: Python - Size: 284 KB - Last synced at: 3 days ago - Pushed at: 10 months ago - Stars: 21 - Forks: 3

grepstrength/WideOpenAI

Short list of indirect prompt injection attacks for OpenAI-based models.

Size: 68.4 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 20 - Forks: 2

Safetorun/PromptDefender

A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.

Language: Go - Size: 6.55 MB - Last synced at: about 17 hours ago - Pushed at: 8 months ago - Stars: 17 - Forks: 0

shaialon/ai-security-demos

🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:

Language: JavaScript - Size: 363 KB - Last synced at: 5 months ago - Pushed at: 11 months ago - Stars: 16 - Forks: 3

presidio-oss/hai-guardrails

A TypeScript library providing a set of guards for LLM (Large Language Model) applications

Language: TypeScript - Size: 1.08 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 14 - Forks: 2

proventra/proventra-core

Secure your AI Agents against prompt injection attacks

Language: Python - Size: 1.22 MB - Last synced at: 20 days ago - Pushed at: about 2 months ago - Stars: 14 - Forks: 0

user1342/Folly

Open-source LLM Prompt-Injection and Jailbreaking Playground

Language: Python - Size: 3.96 MB - Last synced at: 5 days ago - Pushed at: 3 months ago - Stars: 14 - Forks: 1

levitation-opensource/Manipulative-Expression-Recognition

MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

Language: HTML - Size: 8.54 MB - Last synced at: 14 days ago - Pushed at: 11 months ago - Stars: 13 - Forks: 3

genia-dev/vibraniumdome

LLM Security Platform.

Language: Python - Size: 2.87 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 10 - Forks: 2

CyberAlbSecOP/MINOTAUR_Impossible_GPT_Security_Challenge

MINOTAUR: The STRONGEST Secure Prompt EVER! Prompt Security Challenge, Impossible GPT Security, Prompts Cybersecurity, Prompting Vulnerabilities, FlowGPT, Secure Prompting, Secure LLMs, Prompt Hacker, Cutting-edge Ai Security, Unbreakable GPT Agent, Anti GPT Leak, System Prompt Security.

Size: 23.4 KB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 2

AnthenaMatrix/Prompt-Injection-Testing-Tool

The Prompt Injection Testing Tool is a Python script designed to assess the security of your AI system's prompt handling against a predefined list of user prompts commonly used for injection attacks. This tool utilizes the OpenAI GPT-3.5 model to generate responses to system-user prompt pairs and outputs the results to a CSV file for analysis.

Language: Python - Size: 7.81 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 1

LLMPID/LLMPID-AS

LLM Prompt Injection Detection API Service PoC.

Language: Go - Size: 419 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 9 - Forks: 0

CursedPrograms/Image-Generator

This project leverages the SDXL-Turbo model for versatile image processing tasks. Offering a simple command-line interface, it facilitates both Text-to-image and Image-to-image operations. Users select an operation, input prompts, and the script dynamically generates and executes code snippets.

Language: Python - Size: 7.72 MB - Last synced at: about 2 months ago - Pushed at: 8 months ago - Stars: 9 - Forks: 0

CyberAlbSecOP/KEVLAR-Anti-Leak-System-Prompts

Bullet-proof your custom GPT system prompt security with KEVLAR, the ultimate prompt protector against rules extraction, prompt injections, and leaks of AI agent secret instructions.

Size: 16.6 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 9 - Forks: 2

jalvarezz13/prompt.fail

prompt.fail explores prompt injection techniques in large language models (LLMs), providing examples to improve LLM security and robustness.

Language: TypeScript - Size: 729 KB - Last synced at: 2 days ago - Pushed at: about 1 month ago - Stars: 8 - Forks: 0

peluche/deck-of-many-prompts

Manual Prompt Injection / Red Teaming Tool

Language: Python - Size: 40.6 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 8 - Forks: 0

HebeHH/llm-prompt-experiments

Experiment with multifactor analysis of different prompting strategies.

Language: TypeScript - Size: 1.33 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 6 - Forks: 1

GitsSaikat/Guardian-Agent

Improving AI Systems with Self-Defense Mechanisms

Language: Python - Size: 332 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 6 - Forks: 4

AdirD/prompt-security-node

🚀 Unofficial Node.js SDK for Prompt Security's Protection API.

Language: TypeScript - Size: 394 KB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 6 - Forks: 0

velocitatem/raccoon

Let Raccoon sample the unknown, safeguarding your AI's home.

Language: Python - Size: 5.99 MB - Last synced at: 5 days ago - Pushed at: about 2 years ago - Stars: 6 - Forks: 1

AINTRUST-AI/aixploit

Engineered to help red teams and penetration testers exploit large language model AI solutions vulnerabilities.

Language: Python - Size: 6.91 MB - Last synced at: 2 months ago - Pushed at: 3 months ago - Stars: 5 - Forks: 1

amk9978/Guardian

The LLM guardian kernel

Language: Go - Size: 295 KB - Last synced at: 2 months ago - Pushed at: 4 months ago - Stars: 5 - Forks: 1

kaistAI/GenPI

This repository is the official implementation of Generative Context Distillation.

Language: Python - Size: 2.57 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 4 - Forks: 0

Cutwell/canary

LLM prompt injection detection

Language: Python - Size: 5 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 3 - Forks: 1

s0lw1n/Un5hAx0rl3d

GPT4o, GPT4o-mini, and GPT4 Turbo jailbreak prompt for Research/IoC Development Purposes

Size: 105 KB - Last synced at: 5 months ago - Pushed at: 6 months ago - Stars: 3 - Forks: 0

M507/HackMeGPT

Vulnerable LLM Application

Language: Python - Size: 152 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 1

alphasecio/prompt-guard

A Streamlit app for testing Prompt Guard, a classifier model by Meta for detecting prompt attacks.

Language: Python - Size: 403 KB - Last synced at: 11 days ago - Pushed at: 12 days ago - Stars: 2 - Forks: 4

SamuelGong/PromptKeeper

A comprehensive framework designed to safeguard system prompts for Large Language Models (LLMs) against various types of attacks.

Language: Python - Size: 27.3 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 2 - Forks: 0

CharlesCreativeContent/copyHelper

Application to make it easy to create my ai prompts. May include some LLM endpoints directly integrated

Language: HTML - Size: 4.88 KB - Last synced at: 6 days ago - Pushed at: 8 months ago - Stars: 2 - Forks: 0

liuyaojialiuyaojia/Awesome-LLM-Security-Paper

Your best llm security paper library

Size: 637 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 2 - Forks: 1

311-code/Magic-Prompt-Injection Fork of DataCTE/prompt_injection

Magic Prompt Injection SDXL and SD 1.5 node for ComfyUI. Direct-to-block prompt injection, bypass "nightmare limb" layers, and improving AI video by prompting .weight and .bias layers directly. (video support coming soon)

Language: Python - Size: 79.1 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 2 - Forks: 0

genia-dev/vibraniumdome-docs

LLM Security Platform Docs

Language: MDX - Size: 635 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0

genia-dev/vibraniumdome-sdk

Client SDK to send LLM interactions to Vibranium Dome

Language: Python - Size: 113 KB - Last synced at: 21 days ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0

trylonai/gateway

The Open Source Firewall for LLMs. A self-hosted gateway to secure and control AI applications with powerful guardrails.

Language: Python - Size: 249 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 1 - Forks: 1

TechCeo/SecureLLMChatbot-GuardingWithLLMGuard

Prompt Injection Detection in LLaMA-based Chatbots using LLM Guard

Language: Jupyter Notebook - Size: 2.99 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 1 - Forks: 0

0x6f677548/copilot-instructions-unicode-injection

Proof of Concept (PoC) demonstrating prompt injection vulnerability in AI code assistants (like Copilot) using hidden Unicode characters within instruction files (copilot-instructions.md). Highlights risks of using untrusted instruction templates. For educational/research purposes only.

Size: 1.48 MB - Last synced at: 5 days ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0

GuardionAI/guardion

Lightning-fast AI Firewall, integrated with leading agent frameworks

Language: Python - Size: 183 KB - Last synced at: 8 days ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0

2alf/prmptinj

Curated + custom prompt injections.

Size: 237 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 1 - Forks: 1

AdityaBhatt3010/Hacking-Lakera-Gandalf-AI-via-Prompt-Injection

Lakera Gandalf AI challenge's step by step walkthrough, showcasing real-world prompt injection techniques and LLM security insights.

Size: 4.88 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 1 - Forks: 0

rishabh-panda/AleXPy

AleXPy is a tool designed to parse Alteryx workflow (.yxmd) files and convert them into Python scripts using LangChain and OpenAI's GPT model.

Language: Python - Size: 0 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

AmanPriyanshu/FRACTURED-SORRY-Bench-Automated-Multishot-Jailbreaking

FRACTURED-SORRY-Bench: This repository contains the code and data for the creating an Automated Multi-shot Jailbreak framework, as described in our paper.

Language: Python - Size: 2.38 MB - Last synced at: 1 day ago - Pushed at: 7 months ago - Stars: 1 - Forks: 0

AhsanAyub/malicious-prompt-detection

Detection of malicious prompts used to exploit large language models (LLMs) by leveraging supervised machine learning classifiers.

Language: Python - Size: 2.32 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 0

JohnNixon6972/Python-Multi-Agent

A Streamlit-based fitness app that leverages Langflow to provide personalized macro-nutritional recommendations and goal tracking. This tool enables users to manage profiles, set fitness goals, ask fitness-related questions, and store notes, all backed by an Astra database for secure data management.

Language: Python - Size: 442 KB - Last synced at: 2 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 1

Kartik-Katkar/Malicious-Prompt-Filter-for-RAG-Database

GitHub repository for a tool that detects and filters malicious prompts before they are entered into a Retrieval-Augmented Generation (RAG) database, ensuring data integrity and security.

Language: Jupyter Notebook - Size: 34.8 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 1 - Forks: 0

vladutdinu/prompty-api

PromptyAPI, people's LLM-based applications security layer

Language: Python - Size: 766 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

ZapDos7/lakera-gandalf

My solutions for Lakera's Gandalf

Size: 46.9 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 1

nodite/llm-guard-ts

The Security Toolkit for LLM Interactions (TS version)

Size: 0 Bytes - Last synced at: 15 days ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

rabbidave/Denzel-Crocker-Hunting-For-Fairly-Odd-Prompts

A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts

Language: Python - Size: 71.3 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 0

viniViado/LLMSecOps

LLMSecOps focuses on integrating security practices within the lifecycle of machine learning models. It ensures that models are robust against threats while maintaining compliance and performance standards.

Size: 461 KB - Last synced at: about 22 hours ago - Pushed at: about 23 hours ago - Stars: 0 - Forks: 0

abkareem/GitHub-Autofiller

Save prompts given to copilot -> Easily reuse if they match expectation

Size: 1000 Bytes - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

roynaldo1234/meta-ai-bug-bounty

Meta AI Bug Bounty focuses on vulnerabilities in Instagram's Group Chat feature, revealing critical security risks. Join us in improving AI safety and explore our findings on GitHub! 🐙✨

Size: 141 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

BaskLash/PromptIn

Effortlessly save prompts for your AI agent and quickly reuse them whenever the output meets your expectations.

Language: JavaScript - Size: 3.9 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

mark5420/Prompt-Assistant

<div align="center"># 🤖 Prompt Assistant - AI Prompt Engineering Tool[![Chrome Web Store](https://img.shields.io/badge/Chrome%20Web%20Store-Available-brightgreen?style=for-the-badge&logo=google-chrome)](https://chromewebstore.google.com/detail/prompt-assistant-ai-promp/khjjnklmccfmbidflahakjameliioidm)[![Version](https://img.shields.io/badge/V

Size: 42 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 0 - Forks: 0