Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / temmyzeus / Start-Data-Eng-Projects

AWS ETL pipeline with Apache Airflow as orchestration tool.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/temmyzeus%2FStart-Data-Eng-Projects

Stars: 0
Forks: 0
Open Issues: 2

License: mit
Language: HCL
Repo Size: 62.5 KB
Dependencies: 315

Created: almost 2 years ago
Updated: about 1 year ago
Last pushed: about 1 year ago
Last synced: about 1 year ago

Files

Readme

Dependencies

begineer_batch_etl/requirements.txt pypi

Authlib ==1.1.0
Babel ==2.10.3
ConfigUpdater ==3.1.1
Deprecated ==1.2.13
Flask ==2.2.2
Flask-AppBuilder ==4.1.4
Flask-Babel ==2.0.0
Flask-Caching ==2.0.1
Flask-JWT-Extended ==4.4.4
Flask-Login ==0.6.2
Flask-SQLAlchemy ==2.5.1
Flask-Session ==0.4.0
Flask-WTF ==1.0.1
HeapDict ==1.0.1
Jinja2 ==3.1.2
Mako ==1.2.3
Markdown ==3.4.1
MarkupSafe ==2.1.1
PyJWT ==2.6.0
PyNaCl ==1.5.0
PyYAML ==6.0
Pygments ==2.13.0
SQLAlchemy ==1.4.27
SQLAlchemy-JSONField ==1.0.0
SQLAlchemy-Utils ==0.38.3
WTForms ==3.0.1
Werkzeug ==2.2.2
adal ==1.2.7
aiofiles ==0.8.0
aiohttp ==3.8.3
aiosignal ==1.2.0
alembic ==1.8.1
amqp ==5.1.1
anyio ==3.6.2
apache-airflow ==2.4.2
apache-airflow-providers-amazon ==6.0.0
apache-airflow-providers-celery ==3.0.0
apache-airflow-providers-cncf-kubernetes ==4.4.0
apache-airflow-providers-common-sql ==1.2.0
apache-airflow-providers-docker ==3.2.0
apache-airflow-providers-elasticsearch ==4.2.1
apache-airflow-providers-ftp ==3.1.0
apache-airflow-providers-google ==8.4.0
apache-airflow-providers-grpc ==3.0.0
apache-airflow-providers-hashicorp ==3.1.0
apache-airflow-providers-http ==4.0.0
apache-airflow-providers-imap ==3.0.0
apache-airflow-providers-microsoft-azure ==4.3.0
apache-airflow-providers-mysql ==3.2.1
apache-airflow-providers-odbc ==3.1.2
apache-airflow-providers-postgres ==5.2.2
apache-airflow-providers-redis ==3.0.0
apache-airflow-providers-sendgrid ==3.0.0
apache-airflow-providers-sftp ==4.1.0
apache-airflow-providers-slack ==6.0.0
apache-airflow-providers-sqlite ==3.2.1
apache-airflow-providers-ssh ==3.2.0
apispec ==3.3.2
argcomplete ==2.0.0
asgiref ==3.5.2
asn1crypto ==1.5.1
async-timeout ==4.0.2
attrs ==22.1.0
azure-batch ==12.0.0
azure-common ==1.1.28
azure-core ==1.26.0
azure-cosmos ==4.3.0
azure-datalake-store ==0.0.52
azure-identity ==1.11.0
azure-keyvault-secrets ==4.6.0
azure-kusto-data ==0.0.45
azure-mgmt-containerinstance ==1.5.0
azure-mgmt-core ==1.3.2
azure-mgmt-datafactory ==1.1.0
azure-mgmt-datalake-nspkg ==3.0.1
azure-mgmt-datalake-store ==0.5.0
azure-mgmt-nspkg ==3.0.2
azure-mgmt-resource ==21.2.1
azure-nspkg ==3.0.2
azure-servicebus ==7.8.1
azure-storage-blob ==12.8.1
azure-storage-common ==2.1.0
azure-storage-file ==2.1.0
azure-synapse-spark ==0.7.0
backoff ==1.10.0
bcrypt ==4.0.1
beautifulsoup4 ==4.11.1
billiard ==3.6.4.0
blinker ==1.5
boto3 ==1.24.94
botocore ==1.27.94
cachelib ==0.9.0
cachetools ==4.2.2
cattrs ==22.2.0
celery ==5.2.7
certifi ==2022.9.24
cffi ==1.15.1
chardet ==4.0.0
charset-normalizer ==2.1.1
click ==8.1.3
click-didyoumean ==0.3.0
click-plugins ==1.1.1
click-repl ==0.2.0
clickclick ==20.10.2
cloudpickle ==2.1.0
colorama ==0.4.5
colorlog ==4.8.0
commonmark ==0.9.1
connexion ==2.14.1
cron-descriptor ==1.2.31
croniter ==1.3.7
cryptography ==36.0.2
dask ==2022.10.0
db-dtypes ==1.0.4
decorator ==5.1.1
dill ==0.3.1.1
distlib ==0.3.6
distributed ==2022.10.0
dnspython ==2.2.1
docker ==6.0.0
docutils ==0.19
elasticsearch ==7.13.4
elasticsearch-dbapi ==0.2.9
elasticsearch-dsl ==7.4.0
email-validator ==1.3.0
eventlet ==0.33.1
exceptiongroup ==1.0.0rc9
filelock ==3.8.0
flower ==1.2.0
frozenlist ==1.3.1
fsspec ==2022.10.0
future ==0.18.2
gcloud-aio-auth ==4.0.1
gcloud-aio-bigquery ==6.1.2
gcloud-aio-storage ==7.0.1
gevent ==22.10.1
google-ads ==18.0.0
google-api-core ==2.8.2
google-api-python-client ==1.12.11
google-auth ==2.13.0
google-auth-httplib2 ==0.1.0
google-auth-oauthlib ==0.5.3
google-cloud-aiplatform ==1.16.1
google-cloud-appengine-logging ==1.1.3
google-cloud-audit-log ==0.2.4
google-cloud-automl ==2.8.0
google-cloud-bigquery ==2.34.4
google-cloud-bigquery-datatransfer ==3.7.0
google-cloud-bigquery-storage ==2.14.1
google-cloud-bigtable ==1.7.2
google-cloud-build ==3.9.0
google-cloud-container ==2.11.1
google-cloud-core ==2.3.2
google-cloud-datacatalog ==3.9.0
google-cloud-dataform ==0.2.0
google-cloud-dataplex ==1.1.0
google-cloud-dataproc ==5.0.0
google-cloud-dataproc-metastore ==1.6.0
google-cloud-dlp ==1.0.2
google-cloud-kms ==2.12.0
google-cloud-language ==1.3.2
google-cloud-logging ==3.2.1
google-cloud-memcache ==1.4.1
google-cloud-monitoring ==2.11.0
google-cloud-orchestration-airflow ==1.4.1
google-cloud-os-login ==2.7.1
google-cloud-pubsub ==2.13.5
google-cloud-redis ==2.9.0
google-cloud-resource-manager ==1.6.0
google-cloud-secret-manager ==1.0.2
google-cloud-spanner ==1.19.3
google-cloud-speech ==1.3.4
google-cloud-storage ==1.44.0
google-cloud-tasks ==2.10.1
google-cloud-texttospeech ==1.0.3
google-cloud-translate ==1.7.2
google-cloud-videointelligence ==1.16.3
google-cloud-vision ==1.0.2
google-cloud-workflows ==1.7.1
google-crc32c ==1.5.0
google-resumable-media ==2.4.0
googleapis-common-protos ==1.56.4
graphviz ==0.20.1
greenlet ==1.1.3.post0
grpc-google-iam-v1 ==0.12.4
grpcio ==1.50.0
grpcio-gcp ==0.2.2
grpcio-status ==1.48.2
gunicorn ==20.1.0
h11 ==0.12.0
httpcore ==0.15.0
httplib2 ==0.20.4
httpx ==0.23.0
humanize ==4.4.0
hvac ==1.0.2
idna ==3.4
inflection ==0.5.1
isodate ==0.6.1
itsdangerous ==2.1.2
jmespath ==0.10.0
json-merge-patch ==0.2
jsonpath-ng ==1.5.3
jsonschema ==4.16.0
kombu ==5.2.4
kubernetes ==23.6.0
lazy-object-proxy ==1.7.1
ldap3 ==2.9.1
linkify-it-py ==2.0.0
locket ==1.0.0
lockfile ==0.12.2
looker-sdk ==22.18.0
lxml ==4.9.1
markdown-it-py ==2.1.0
marshmallow ==3.18.0
marshmallow-enum ==1.5.1
marshmallow-oneofschema ==3.0.1
marshmallow-sqlalchemy ==0.26.1
mdit-py-plugins ==0.3.1
mdurl ==0.1.2
msal ==1.20.0
msal-extensions ==1.0.0
msgpack ==1.0.4
msrest ==0.7.1
msrestazure ==0.6.4
multidict ==6.0.2
mypy-boto3-appflow ==1.24.90
mypy-boto3-rds ==1.24.76
mypy-boto3-redshift-data ==1.24.36.post1
mysql-connector-python ==8.0.31
mysqlclient ==2.1.1
numpy ==1.22.4
oauthlib ==3.2.2
packaging ==21.3
pandas ==1.5.1
pandas-gbq ==0.17.9
paramiko ==2.11.0
partd ==1.3.0
pathspec ==0.9.0
pendulum ==2.1.2
platformdirs ==2.5.2
pluggy ==1.0.0
ply ==3.11
portalocker ==2.6.0
prison ==0.2.1
prometheus-client ==0.15.0
prompt-toolkit ==3.0.31
proto-plus ==1.19.6
protobuf ==3.20.0
psutil ==5.9.3
psycopg2 ==2.9.4
pyOpenSSL ==22.0.0
pyarrow ==6.0.1
pyasn1 ==0.4.8
pyasn1-modules ==0.2.8
pycparser ==2.21
pydata-google-auth ==1.4.0
pyhcl ==0.4.4
pyodbc ==4.0.34
pyparsing ==3.0.9
pyrsistent ==0.18.1
python-daemon ==2.3.1
python-dateutil ==2.8.2
python-http-client ==3.3.7
python-ldap ==3.4.3
python-nvd3 ==0.15.0
python-slugify ==6.1.2
pytz ==2022.5
pytzdata ==2020.1
redis ==3.5.3
redshift-connector ==2.0.909
requests ==2.28.1
requests-oauthlib ==1.3.1
requests-toolbelt ==0.10.0
rfc3986 ==1.5.0
rich ==12.6.0
rsa ==4.9
s3transfer ==0.6.0
scramp ==1.4.1
sendgrid ==6.9.7
setproctitle ==1.3.2
six ==1.16.0
slack-sdk ==3.19.1
sniffio ==1.3.0
sortedcontainers ==2.4.0
soupsieve ==2.3.2.post1
sqlalchemy-bigquery ==1.4.4
sqlalchemy-redshift ==0.8.11
sqlparse ==0.4.3
sshtunnel ==0.4.0
starkbank-ecdsa ==2.1.0
statsd ==3.3.0
swagger-ui-bundle ==0.0.9
tabulate ==0.9.0
tblib ==1.7.0
tenacity ==8.1.0
termcolor ==2.0.1
text-unidecode ==1.3
toolz ==0.12.0
tornado ==6.1
typing_extensions ==4.4.0
uamqp ==1.6.1
uc-micro-py ==1.0.1
unicodecsv ==0.14.1
uritemplate ==3.0.1
urllib3 ==1.26.12
vine ==5.0.0
virtualenv ==20.16.5
watchtower ==2.0.1
wcwidth ==0.2.5
websocket-client ==1.4.1
wrapt ==1.14.1
yarl ==1.8.1
zict ==2.2.0
zope.event ==4.5.0
zope.interface ==5.5.0