Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: prompt-injection

liu00222/Open-Prompt-Injection

Prompt injection attacks and defenses in LLM-integrated applications

Language: Python - Size: 1.41 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 91 - Forks: 13

tldrsec/prompt-injection-defenses

Every practical and proposed defense against prompt injection.

Size: 130 KB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 230 - Forks: 18

AhsanAyub/malicious-prompt-detection

Detecting malicious prompts used to exploit large language models (LLMs) by leveraging supervised machine learning classifiers

Language: Python - Size: 4.88 KB - Last synced: 4 days ago - Pushed: 4 days ago - Stars: 0 - Forks: 0

CyberAlbSecOP/Awesome_GPT_Super_Prompting

ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.

Size: 239 KB - Last synced: 5 days ago - Pushed: 6 days ago - Stars: 156 - Forks: 16

lakeraai/pint-benchmark

A benchmark for prompt injection detection systems.

Language: Jupyter Notebook - Size: 2.23 MB - Last synced: 5 days ago - Pushed: 5 days ago - Stars: 59 - Forks: 7

whylabs/langkit

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀

Language: Jupyter Notebook - Size: 4.03 MB - Last synced: 18 days ago - Pushed: 18 days ago - Stars: 727 - Forks: 61

shaialon/ai-security-demos

🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:

Language: JavaScript - Size: 273 KB - Last synced: 6 days ago - Pushed: 6 days ago - Stars: 4 - Forks: 0

protectai/llm-guard

The Security Toolkit for LLM Interactions

Language: Python - Size: 5.42 MB - Last synced: 11 days ago - Pushed: 14 days ago - Stars: 893 - Forks: 102

dropbox/llm-security

Dropbox LLM Security research code and results

Language: Python - Size: 1.61 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 186 - Forks: 17

2alf/prmptinj

Curated + custom prompt injections.

Size: 186 KB - Last synced: 13 days ago - Pushed: 14 days ago - Stars: 0 - Forks: 1

langgptai/Awesome-Multimodal-Prompts

Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.

Size: 87.3 MB - Last synced: 13 days ago - Pushed: 7 months ago - Stars: 190 - Forks: 15

protectai/rebuff

LLM Prompt Injection Detector

Language: TypeScript - Size: 7.15 MB - Last synced: 18 days ago - Pushed: 2 months ago - Stars: 948 - Forks: 61

ZapDos7/lakera-gandalf

My solutions for Lakera's Gandalf

Size: 46.9 KB - Last synced: 19 days ago - Pushed: 20 days ago - Stars: 1 - Forks: 1

Safetorun/PromptDefender

A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.

Language: Go - Size: 5.89 MB - Last synced: about 5 hours ago - Pushed: about 6 hours ago - Stars: 8 - Forks: 0

MaxMLang/pytector

A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.

Language: Python - Size: 39.1 KB - Last synced: 25 days ago - Pushed: 25 days ago - Stars: 20 - Forks: 19

CursedPrograms/Image-Generator

This project leverages the SDXL-Turbo model for versatile image processing tasks. Offering a simple command-line interface, it facilitates both Text-to-image and Image-to-image operations. Users select an operation, input prompts, and the script dynamically generates and executes code snippets.

Language: Python - Size: 7.72 MB - Last synced: 23 days ago - Pushed: 4 months ago - Stars: 6 - Forks: 0

lakeraai/chainguard

Guard your LangChain applications against prompt injection with Lakera ChainGuard.

Language: Python - Size: 1.17 MB - Last synced: 27 days ago - Pushed: about 2 months ago - Stars: 13 - Forks: 1

utkusen/promptmap

automatically tests prompt injection attacks on ChatGPT instances

Language: Python - Size: 55.7 KB - Last synced: about 1 month ago - Pushed: 6 months ago - Stars: 519 - Forks: 50

SemanticBrainCorp/SemanticShield

The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).

Language: Python - Size: 273 KB - Last synced: about 1 month ago - Pushed: 4 months ago - Stars: 11 - Forks: 2

LostOxygen/llm-confidentiality

Whispers in the Machine: Confidentiality in LLM-integrated Systems

Language: Python - Size: 606 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 2

CyberAlbSecOP/KEVLAR-Anti-Leak-System-Prompts

Bullet-proof your custom GPT system prompt security with KEVLAR, the ultimate prompt protector against rules extraction, prompt injections, and leaks of AI agent secret instructions.

Size: 16.6 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

GPTSafe/PromptGuard

Build production ready apps for GPT using Node.js & TypeScript

Language: TypeScript - Size: 848 KB - Last synced: 5 days ago - Pushed: about 1 year ago - Stars: 30 - Forks: 1

sergeyklay/promptly

A customizable ChatGPT API interface for OpenAPI models.

Language: Python - Size: 7.19 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 0

CyberAlbSecOP/MINOTAUR_Impossible_GPT_Security_Challenge

MINOTAUR: The STRONGEST Secure Prompt EVER! Prompt Security Challenge, Impossible GPT Security, Prompts Cybersecurity, Prompting Vulnerabilities, FlowGPT, Secure Prompting, Secure LLMs, Prompt Hacker, Cutting-edge Ai Security, Unbreakable GPT Agent, Anti GPT Leak, System Prompt Security.

Size: 23.4 KB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 0 - Forks: 0

deadbits/vigil-llm

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

Language: Python - Size: 548 KB - Last synced: 2 months ago - Pushed: 4 months ago - Stars: 231 - Forks: 27

AnthenaMatrix/Prompt-Injection-Testing-Tool

The Prompt Injection Testing Tool is a Python script designed to assess the security of your AI system's prompt handling against a predefined list of user prompts commonly used for injection attacks. This tool utilizes the OpenAI GPT-3.5 model to generate responses to system-user prompt pairs and outputs the results to a CSV file for analysis.

Language: Python - Size: 7.81 KB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 10 - Forks: 1

automorphic-ai/aegis

Self-hardening firewall for large language models

Language: Python - Size: 21.5 KB - Last synced: 3 months ago - Pushed: 3 months ago - Stars: 236 - Forks: 6

HumanCompatibleAI/tensor-trust

A prompt injection game to collect data for robust ML research

Language: Python - Size: 8.31 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 32 - Forks: 6

juicete4/llm-prompt-injection

Repo hosting the data and results of my research on LLM prompt injection resistance.

Language: Python - Size: 10.1 MB - Last synced: 3 months ago - Pushed: 3 months ago - Stars: 0 - Forks: 0

microsoft/gandalf_vs_gandalf

Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.

Language: Jupyter Notebook - Size: 2.61 MB - Last synced: about 2 months ago - Pushed: 8 months ago - Stars: 15 - Forks: 0

levitation-opensource/Manipulative-Expression-Recognition

MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

Language: HTML - Size: 8.15 MB - Last synced: 4 months ago - Pushed: 4 months ago - Stars: 7 - Forks: 2

nodite/llm-guard-ts

The Security Toolkit for LLM Interactions (TS version)

Size: 0 Bytes - Last synced: 5 months ago - Pushed: 5 months ago - Stars: 0 - Forks: 0

M507/HackMeGPT

Vulnerable LLM Application

Language: Python - Size: 152 KB - Last synced: 5 months ago - Pushed: 5 months ago - Stars: 3 - Forks: 1

milosilo/Happy-Prompt

Happy Prompt is a unique tool designed to interject positive emotions into text prompts, allowing users to communicate joyful, uplifting, and enthusiastic expressions. It utilizes a series of cheerful emojis, symbols, and text representations to infuse the text with a sense of happiness, love, dancing, partying, and other upbeat themes.

Language: PHP - Size: 12.7 KB - Last synced: 6 months ago - Pushed: 9 months ago - Stars: 0 - Forks: 0

sinanw/llm-security-prompt-injection

This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.

Language: Jupyter Notebook - Size: 2.75 MB - Last synced: 6 months ago - Pushed: 6 months ago - Stars: 0 - Forks: 0

genia-dev/vibraniumdome-sdk

Client SDK to send LLM interactions to Vibranium Dome

Language: Python - Size: 113 KB - Last synced: about 1 hour ago - Pushed: 2 months ago - Stars: 2 - Forks: 0

milosilo/RateMyAI

Prompt Engineering Tool for AI Models with cli prompt or api usage

Language: Python - Size: 43.9 KB - Last synced: 6 months ago - Pushed: 9 months ago - Stars: 0 - Forks: 0

Cutwell/canary

LLM prompt injection detection

Language: Python - Size: 5 MB - Last synced: 27 days ago - Pushed: 7 months ago - Stars: 3 - Forks: 1

yunwei37/prompt-hacker-collections

prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记

Size: 125 KB - Last synced: 7 months ago - Pushed: 7 months ago - Stars: 38 - Forks: 5

rohilrg/CatchPromptInjection

This repo focus on how to deal with prompt injection problem faced by LLMs

Language: Python - Size: 10.9 MB - Last synced: 8 months ago - Pushed: 8 months ago - Stars: 0 - Forks: 0

ateja5/ReadyPlayer50

Prompt Engineering

Size: 6.91 MB - Last synced: 8 months ago - Pushed: 8 months ago - Stars: 0 - Forks: 0

Vivswan/ChatGPTAdversarialAttack2023

ChatGPT Adversarial Attack for The Pitt Challenge 2023

Language: TypeScript - Size: 232 KB - Last synced: 9 months ago - Pushed: 10 months ago - Stars: 0 - Forks: 0

costly-ai/costly

A new kind of MLOps platform purpose built for production generative ai apps

Size: 1000 Bytes - Last synced: about 1 month ago - Pushed: 9 months ago - Stars: 0 - Forks: 0

rabbidave/Denzel-Crocker-Hunting-For-Fairly-Odd-Prompts

A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts

Language: Python - Size: 71.3 KB - Last synced: 11 months ago - Pushed: 11 months ago - Stars: 1 - Forks: 0

gdalmau/lakera-gandalf-solutions

My inputs for the LLM Gandalf made by Lakera

Size: 1000 Bytes - Last synced: 12 months ago - Pushed: 12 months ago - Stars: 0 - Forks: 0

monmonmonmon/prompt-leaking-of-bing-in-Chinese

Prompt Leaking of Microsoft Bing Chat

Size: 3.59 MB - Last synced: 12 months ago - Pushed: 12 months ago - Stars: 0 - Forks: 0

Related Keywords
prompt-injection 46 prompt-engineering 23 llm 20 llm-security 12 chatgpt 12 ai 11 security 9 large-language-models 8 llmops 8 openai 6 gpt 6 security-tools 6 llms 6 adversarial-machine-learning 5 adversarial-attacks 5 machine-learning 4 generative-ai 4 ai-security 4 prompt-injection-tool 4 prompt 3 artificial-intelligence 3 transformers 3 openai-api 3 gpt-3 3 jailbreak 3 prompts 3 prompt-security 3 gpt-4 3 transformers-models 2 prompt-toolkit 2 gandalf 2 jailbreak-prompt 2 lakera 2 ai-prompts 2 prompt-hacking 2 prompt-leaking 2 sentiment-analysis 2 cyber-security 2 ai-cyber-security 2 ctf 2 aisecurity 2 prompting 2 cybersecurity 2 langchain 2 system-prompt 2 fraud-prevention 1 fraud-detection 1 rouge-l 1 human-computer-interaction 1 human-robot-interaction 1 llm-test 1 llm-training 1 manipulation 1 expression-recognition 1 conversation-analytics 1 conversation-analysis 1 benchmarking 1 bing-chat 1 privacy 1 jailbreaks 1 htmx 1 game 1 django 1 prompt-learning 1 yara-scanner 1 super-prompt 1 image-generation-ai 1 mlops 1 human-in-the-loop 1 embeddings 1 caching 1 hackathon 1 attack-defense 1 fastapi 1 vector-search 1 redteam-tools 1 red-team-tools 1 prompt-tuning 1 prompt-manipulation 1 pentesting 1 pentest-tool 1 large-language-model 1 adverarial-attacks 1 llm-prompting 1 cosine-similarity 1 sentiment-attack 1 redteam 1 pentest 1 machinelearning 1 vulnerable-llm-application 1 damn-vulnerable 1 typescript 1 transparency 1 sentiment-classification 1 psychometrics 1 misinformation 1 image-generation 1 art-generation 1 art 1 ai-art 1