Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / DLI1996 / vpc-llm-copilot

In this project, we developed an RAG-based copilot with Q&A capabilities, specifically designed for Amazon Virtual Private Cloud (VPC). We did comparion between GPT 3.5-Turbo as the cloud solutions vs. LLaMa II 7B with LoRA adapter as the easily deployable, lightweight, on-premises solution.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DLI1996%2Fvpc-llm-copilot

Stars: 0
Forks: 0
Open Issues: 0

License: None
Language: Jupyter Notebook
Repo Size: 8.51 MB
Dependencies: 455

Created: 5 months ago
Updated: 5 months ago
Last pushed: 5 months ago
Last synced: 5 months ago

Files

Readme

Dependencies

07_Docker/Dockerfile docker

python 3.10.7 build

03_Data_Ingestion_Pipelines/VPC_URL_Loader_and_Chunker/requirements.txt pypi

Babel ==2.13.1
Jinja2 ==3.1.2
MarkupSafe ==2.1.3
PyPDF2 ==3.0.1
PyPika ==0.48.9
PySocks ==1.7.1
PyYAML ==6.0.1
Pygments ==2.16.1
QtPy ==2.4.1
SQLAlchemy ==2.0.22
Send2Trash ==1.8.2
aiohttp ==3.8.6
aiosignal ==1.3.1
annotated-types ==0.6.0
anyio ==3.7.1
argon2-cffi ==23.1.0
argon2-cffi-bindings ==21.2.0
arrow ==1.3.0
asttokens ==2.4.0
async-lru ==2.0.4
async-timeout ==4.0.3
attrs ==23.1.0
backcall ==0.2.0
backoff ==2.2.1
bcrypt ==4.0.1
beautifulsoup4 ==4.12.2
bleach ==6.1.0
blis ==0.7.11
catalogue ==2.0.10
certifi ==2023.7.22
cffi ==1.16.0
chardet ==5.2.0
charset-normalizer ==3.3.0
chroma-hnswlib ==0.7.3
chromadb ==0.4.14
click ==8.1.7
cloudpathlib ==0.16.0
coloredlogs ==15.0.1
comm ==0.1.4
confection ==0.1.3
cymem ==2.0.8
dataclasses-json ==0.6.1
debugpy ==1.8.0
decorator ==5.1.1
defusedxml ==0.7.1
emoji ==2.8.0
exceptiongroup ==1.1.3
executing ==2.0.0
fastapi ==0.103.2
fastjsonschema ==2.18.1
filelock ==3.12.4
filetype ==1.2.0
flatbuffers ==23.5.26
fqdn ==1.5.1
frozenlist ==1.4.0
fsspec ==2023.9.2
greenlet ==3.0.0
grpcio ==1.59.0
h11 ==0.14.0
httptools ==0.6.0
huggingface-hub ==0.17.3
humanfriendly ==10.0
idna ==3.4
importlib-resources ==6.1.0
ipykernel ==6.25.2
ipython ==8.16.1
ipython-genutils ==0.2.0
ipywidgets ==8.1.1
isoduration ==20.11.0
jedi ==0.19.1
joblib ==1.3.2
json5 ==0.9.14
jsonpatch ==1.33
jsonpointer ==2.4
jsonschema ==4.19.2
jsonschema-specifications ==2023.7.1
jupyter ==1.0.0
jupyter-console ==6.6.3
jupyter-events ==0.8.0
jupyter-lsp ==2.2.0
jupyter_client ==8.4.0
jupyter_core ==5.4.0
jupyter_server ==2.9.1
jupyter_server_terminals ==0.4.4
jupyterlab ==4.0.7
jupyterlab-pygments ==0.2.2
jupyterlab-widgets ==3.0.9
jupyterlab_server ==2.25.0
langchain ==0.0.314
langchainhub ==0.1.13
langcodes ==3.3.0
langdetect ==1.0.9
langsmith ==0.0.43
lxml ==4.9.3
marshmallow ==3.20.1
matplotlib-inline ==0.1.6
mistune ==3.0.2
monotonic ==1.6
mpmath ==1.3.0
multidict ==6.0.4
murmurhash ==1.0.10
mypy-extensions ==1.0.0
nbclient ==0.8.0
nbconvert ==7.10.0
nbformat ==5.9.2
nest-asyncio ==1.5.8
nltk ==3.8.1
notebook ==7.0.6
notebook_shim ==0.2.3
numpy ==1.26.1
onnxruntime ==1.16.1
openai ==0.28.1
outcome ==1.3.0.post0
overrides ==7.4.0
packaging ==23.2
pandas ==2.1.2
pandocfilters ==1.5.0
parso ==0.8.3
pdfkit ==1.0.0
pexpect ==4.8.0
pickleshare ==0.7.5
platformdirs ==3.11.0
posthog ==3.0.2
preshed ==3.0.9
prometheus-client ==0.18.0
prompt-toolkit ==3.0.39
protobuf ==4.24.4
psutil ==5.9.6
ptyprocess ==0.7.0
pulsar-client ==3.3.0
pure-eval ==0.2.2
pycparser ==2.21
pydantic ==2.4.2
pydantic_core ==2.10.1
pyee ==11.0.1
python-dateutil ==2.8.2
python-dotenv ==1.0.0
python-iso639 ==2023.6.15
python-json-logger ==2.0.7
python-magic ==0.4.27
pytube ==15.0.0
pytz ==2023.3.post1
pyzmq ==25.1.1
qtconsole ==5.4.4
rapidfuzz ==3.4.0
referencing ==0.30.2
regex ==2023.10.3
requests ==2.31.0
rfc3339-validator ==0.1.4
rfc3986-validator ==0.1.1
rpds-py ==0.10.6
six ==1.16.0
smart-open ==6.4.0
sniffio ==1.3.0
sortedcontainers ==2.4.0
soupsieve ==2.5
spacy ==3.7.2
spacy-legacy ==3.0.12
spacy-loggers ==1.0.5
srsly ==2.4.8
stack-data ==0.6.3
starlette ==0.27.0
sympy ==1.12
tabulate ==0.9.0
tenacity ==8.2.3
terminado ==0.17.1
thinc ==8.2.1
tiktoken ==0.5.1
tinycss2 ==1.2.1
tokenizers ==0.14.1
tomli ==2.0.1
tornado ==6.3.3
tqdm ==4.66.1
traitlets ==5.11.2
trio ==0.22.2
trio-websocket ==0.11.1
typer ==0.9.0
types-python-dateutil ==2.8.19.14
types-requests ==2.31.0.9
typing-inspect ==0.9.0
typing_extensions ==4.8.0
tzdata ==2023.3
unstructured ==0.10.26
uri-template ==1.3.0
urllib3 ==2.0.6
uvicorn ==0.23.2
uvloop ==0.18.0
wasabi ==1.1.2
watchfiles ==0.21.0
wcwidth ==0.2.8
weasel ==0.3.3
webcolors ==1.13
webencodings ==0.5.1
websocket-client ==1.6.4
websockets ==11.0.3
widgetsnbextension ==4.0.9
wsproto ==1.2.0
yarl ==1.9.2
youtube-transcript-api ==0.6.1

06_Data/Capstone_Data/chunks/ network-access-analyzer security_iam_required-API-permissions_0.txt pypi

AWSDocumentationAmazonVirtualPrivateCloudNetworkAccessAnalyzerAdditionalinformationRequiredAPIpermissionsforNetworkAccessAnalyzerNetworkAccessAnalyzerreliesondatafromotherAWSservices.Itusespermissionsfromthefollowingservices *

06_Data/Capstone_Data/chunks/ network-access-analyzer security_iam_required-API-permissions_1.txt pypi

ec2 *

06_Data/Capstone_Data/chunks/ network-access-analyzer security_iam_required-API-permissions_2.txt pypi

Ifyouspecifyaresourcegroupinaresourcestatement ,NetworkAccessAnalyzerusesresource

06_Data/Capstone_Data/chunks/ network-access-analyzer security_iam_required-API-permissions_3.txt pypi

IfyoumonitorAPIcalls ,youmightseecallstoTirosAPIs.TirosisaservicethatisonlyaccessiblebyAWSservicesandthatsurfacesnetworkfindingstoNetworkAccessAnalyzer.CallstotheTirosendpointarerequiredforNetworkAccessAnalyzertofunction.ToaccessNetworkAccessAnalyzer,usersmustalsohavethesameAPIpermissions.Javascriptisdisabledorisunavailableinyourbrowser.TousetheAmazonWebServicesDocumentation,Javascriptmustbeenabled.Pleaserefertoyourbrowser

06_Data/Capstone_Data/chunks/ network-access-analyzer security_iam_required-API-permissions_4.txt pypi

Ifyou *

06_Data/Capstone_Data/chunks/ reachability security_iam_required-API-permissions_0.txt pypi

AWSDocumentationAmazonVirtualPrivateCloudReachabilityAnalyzerAdditionalinformationRequiredAPIpermissionsforReachabilityAnalyzerReachabilityAnalyzerreliesondatafromotherAWSservices.Itusespermissionsfromthefollowingservices *

06_Data/Capstone_Data/chunks/ reachability security_iam_required-API-permissions_1.txt pypi

ec2 *

06_Data/Capstone_Data/chunks/ reachability security_iam_required-API-permissions_2.txt pypi

iam *

06_Data/Capstone_Data/chunks/ reachability security_iam_required-API-permissions_3.txt pypi

TousetheAmazonWebServicesDocumentation ,Javascriptmustbeenabled.Pleaserefertoyourbrowser

06_Data/Capstone_Data/summaries/ network-access-analyzer security_iam_required-API-permissions.txt pypi

TheAWSNetworkAccessAnalyzerreliesondatafromvariousAWSservicesandrequiresspecificAPIpermissionstofunctionproperly.TheseservicesincludeAWSDirectConnect ,AmazonEC2,ElasticLoadBalancing,AWSGlobalAccelerator,AWSNetworkFirewall,AWSResourceGroups,AWSResourceGroupsTaggingAPI,andAWSTiros.UsersneedpermissionstocalltheNetworkAccessAnalyzerAPIsforcreating,analyzing,viewing,anddeletingnetworkaccessscopes.Additionally,usersmusthavepermissionsfordescribeAPIcallsfornetworking

06_Data/Capstone_Data/summaries/ reachability security_iam_required-API-permissions.txt pypi

TheReachabilityAnalyzerinAmazonVirtualPrivateCloud *

07_Docker/requirements.txt pypi

Flask ==3.0.0
Jinja2 ==3.1.2
MarkupSafe ==2.1.3
PyYAML ==6.0.1
Pygments ==2.16.1
SQLAlchemy ==2.0.21
Werkzeug ==3.0.0
aiohttp ==3.8.6
aiosignal ==1.3.1
annotated-types ==0.6.0
anyio ==3.7.1
asttokens ==2.4.0
async-timeout ==4.0.3
attrs ==23.1.0
backcall ==0.2.0
beautifulsoup4 ==4.12.2
blinker ==1.6.3
certifi ==2023.7.22
charset-normalizer ==3.3.0
click ==8.1.7
colorama ==0.4.6
comm ==0.1.4
dataclasses-json ==0.6.1
debugpy ==1.8.0
decorator ==5.1.1
dnspython ==2.4.2
exceptiongroup ==1.1.3
executing ==2.0.0
frozenlist ==1.4.0
greenlet ==3.0.0
html5lib ==1.1
idna ==3.4
ipykernel ==6.25.2
ipython ==8.16.1
itsdangerous ==2.1.2
jedi ==0.19.1
joblib ==1.3.2
jsonify ==0.5
jsonpatch ==1.33
jsonpointer ==2.4
jupyter_client ==8.3.1
jupyter_core ==5.3.2
langchain ==0.0.311
langsmith ==0.0.43
loguru ==0.7.2
lxml ==4.9.3
marshmallow ==3.20.1
matplotlib-inline ==0.1.6
multidict ==6.0.4
mypy-extensions ==1.0.0
nest-asyncio ==1.5.8
networkx ==3.1
nltk ==3.8.1
numpy ==1.26.0
openai ==0.28.1
packaging ==23.2
pandas ==2.1.1
parso ==0.8.3
pickleshare ==0.7.5
pinecone-client ==2.2.4
platformdirs ==3.11.0
prompt-toolkit ==3.0.39
psutil ==5.9.5
pure-eval ==0.2.2
pydantic ==2.4.2
pydantic_core ==2.10.1
python-dateutil ==2.8.2
python-dotenv ==1.0.0
pytz ==2023.3.post1
pyzmq ==25.1.1
regex ==2023.10.3
requests ==2.31.0
scikit-learn ==1.3.1
scipy ==1.11.3
six ==1.16.0
sniffio ==1.3.0
soupsieve ==2.5
stack-data ==0.6.3
tenacity ==8.2.3
threadpoolctl ==3.2.0
tiktoken ==0.5.1
tornado ==6.3.3
tqdm ==4.66.1
traitlets ==5.11.2
typing-inspect ==0.9.0
typing_extensions ==4.8.0
tzdata ==2023.3
urllib3 ==2.0.6
wcwidth ==0.2.8
webencodings ==0.5.1
win32-setctime ==1.1.0
yarl ==1.9.2

10_Test_and_Score/scoring/requirements.txt pypi

Babel ==2.13.1
Jinja2 ==3.1.2
MarkupSafe ==2.1.3
Pillow ==10.1.0
PyYAML ==6.0.1
Pygments ==2.17.2
QtPy ==2.4.1
Send2Trash ==1.8.2
absl-py ==2.0.0
aiohttp ==3.9.0
aiosignal ==1.3.1
anyio ==4.1.0
argon2-cffi ==23.1.0
argon2-cffi-bindings ==21.2.0
arrow ==1.3.0
asttokens ==2.4.1
async-lru ==2.0.4
async-timeout ==4.0.3
attrs ==23.1.0
beautifulsoup4 ==4.12.2
bert-score ==0.3.13
bleach ==6.1.0
certifi ==2023.11.17
cffi ==1.16.0
charset-normalizer ==3.3.2
click ==8.1.7
comm ==0.2.0
contourpy ==1.2.0
cycler ==0.12.1
datasets ==2.15.0
debugpy ==1.8.0
decorator ==5.1.1
defusedxml ==0.7.1
dill ==0.3.7
evaluate ==0.4.1
exceptiongroup ==1.2.0
executing ==2.0.1
fastjsonschema ==2.19.0
filelock ==3.13.1
fonttools ==4.45.1
fqdn ==1.5.1
frozenlist ==1.4.0
fsspec ==2023.10.0
huggingface-hub ==0.19.4
idna ==3.5
ipykernel ==6.26.0
ipython ==8.18.0
ipywidgets ==8.1.1
isoduration ==20.11.0
jedi ==0.19.1
joblib ==1.3.2
json5 ==0.9.14
jsonpointer ==2.4
jsonschema ==4.20.0
jsonschema-specifications ==2023.11.1
jupyter ==1.0.0
jupyter-console ==6.6.3
jupyter-events ==0.9.0
jupyter-lsp ==2.2.0
jupyter_client ==8.6.0
jupyter_core ==5.5.0
jupyter_server ==2.10.1
jupyter_server_terminals ==0.4.4
jupyterlab ==4.0.9
jupyterlab-widgets ==3.0.9
jupyterlab_pygments ==0.3.0
jupyterlab_server ==2.25.2
kiwisolver ==1.4.5
matplotlib ==3.8.2
matplotlib-inline ==0.1.6
mistune ==3.0.2
mpmath ==1.3.0
multidict ==6.0.4
multiprocess ==0.70.15
nbclient ==0.9.0
nbconvert ==7.11.0
nbformat ==5.9.2
nest-asyncio ==1.5.8
networkx ==3.2.1
nltk ==3.8.1
notebook ==7.0.6
notebook_shim ==0.2.3
numpy ==1.26.2
nvidia-cublas-cu12 ==12.1.3.1
nvidia-cuda-cupti-cu12 ==12.1.105
nvidia-cuda-nvrtc-cu12 ==12.1.105
nvidia-cuda-runtime-cu12 ==12.1.105
nvidia-cudnn-cu12 ==8.9.2.26
nvidia-cufft-cu12 ==11.0.2.54
nvidia-curand-cu12 ==10.3.2.106
nvidia-cusolver-cu12 ==11.4.5.107
nvidia-cusparse-cu12 ==12.1.0.106
nvidia-nccl-cu12 ==2.18.1
nvidia-nvjitlink-cu12 ==12.3.101
nvidia-nvtx-cu12 ==12.1.105
overrides ==7.4.0
packaging ==23.2
pandas ==2.1.3
pandocfilters ==1.5.0
parso ==0.8.3
pexpect ==4.8.0
platformdirs ==4.0.0
prometheus-client ==0.19.0
prompt-toolkit ==3.0.41
psutil ==5.9.6
ptyprocess ==0.7.0
pure-eval ==0.2.2
pyarrow ==14.0.1
pyarrow-hotfix ==0.6
pycparser ==2.21
pyparsing ==3.1.1
python-dateutil ==2.8.2
python-json-logger ==2.0.7
pytz ==2023.3.post1
pyzmq ==25.1.1
qtconsole ==5.5.1
referencing ==0.31.0
regex ==2023.10.3
requests ==2.31.0
responses ==0.18.0
rfc3339-validator ==0.1.4
rfc3986-validator ==0.1.1
rouge-score ==0.1.2
rpds-py ==0.13.1
safetensors ==0.4.0
six ==1.16.0
sniffio ==1.3.0
soupsieve ==2.5
stack-data ==0.6.3
sympy ==1.12
terminado ==0.18.0
tinycss2 ==1.2.1
tokenizers ==0.15.0
tomli ==2.0.1
torch ==2.1.1
tornado ==6.3.3
tqdm ==4.66.1
traitlets ==5.13.0
transformers ==4.35.2
triton ==2.1.0
types-python-dateutil ==2.8.19.14
typing_extensions ==4.8.0
tzdata ==2023.3
uri-template ==1.3.0
urllib3 ==2.1.0
wcwidth ==0.2.12
webcolors ==1.13
webencodings ==0.5.1
websocket-client ==1.6.4
widgetsnbextension ==4.0.9
xxhash ==3.4.1
yarl ==1.9.3