Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / Charbeneau / api_to_s3_etl
Let's hit an API, transforms the results, and put them in S3!
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charbeneau%2Fapi_to_s3_etl
Stars: 3
Forks: 0
Open Issues: 29
License: apache-2.0
Language: Python
Repo Size: 90.8 KB
Dependencies:
230
Created: almost 4 years ago
Updated: over 3 years ago
Last pushed: over 1 year ago
Last synced: about 1 year ago
Files
Loading...
Readme
Loading...
Dependencies
- Babel ==2.8.0
- Flask ==1.1.2
- Flask-Admin ==1.5.4
- Flask-AppBuilder ==2.3.4
- Flask-Babel ==1.0.0
- Flask-Caching ==1.3.3
- Flask-JWT-Extended ==3.24.1
- Flask-Login ==0.4.1
- Flask-OpenID ==1.2.5
- Flask-SQLAlchemy ==2.4.3
- Flask-WTF ==0.14.3
- JPype1 ==0.7.5
- JayDeBeApi ==1.2.1
- Jinja2 ==2.10.3
- Mako ==1.1.2
- Markdown ==2.6.11
- MarkupSafe ==1.1.1
- PyHive ==0.6.2
- PyJWT ==1.7.1
- PyNaCl ==1.4.0
- PyYAML ==5.3.1
- Pygments ==2.6.1
- SQLAlchemy ==1.3.15
- SQLAlchemy-JSONField ==0.9.0
- SQLAlchemy-Utils ==0.36.6
- WTForms ==2.3.1
- Werkzeug ==0.16.1
- alembic ==1.4.2
- amqp ==2.5.2
- apache-airflow ==1.10.11
- apispec ==1.3.3
- argcomplete ==1.11.1
- attrs ==19.3.0
- bcrypt ==3.1.7
- billiard ==3.6.3.0
- boto3 ==1.13.9
- botocore ==1.16.9
- cached-property ==1.5.1
- cattrs ==1.0.0
- celery ==4.4.2
- certifi ==2020.4.5.1
- cffi ==1.14.0
- chardet ==3.0.4
- click ==7.1.2
- colorama ==0.4.3
- colorlog ==4.0.2
- configparser ==3.5.3
- croniter ==0.3.32
- cryptography ==2.9.2
- defusedxml ==0.6.0
- dill ==0.3.1.1
- dnspython ==1.16.0
- docutils ==0.15.2
- email-validator ==1.1.1
- flask-swagger ==0.2.13
- flower ==0.9.4
- funcsigs ==1.0.2
- future ==0.16.0
- graphviz ==0.14
- gunicorn ==19.10.0
- hmsclient ==0.1.1
- humanize ==0.5.1
- idna ==2.9
- importlib-metadata ==1.6.0
- iso8601 ==0.1.12
- itsdangerous ==1.1.0
- jmespath ==0.10.0
- json-merge-patch ==0.2
- jsonschema ==3.2.0
- kombu ==4.6.8
- lazy-object-proxy ==1.4.3
- lockfile ==0.12.2
- marshmallow ==2.21.0
- marshmallow-enum ==1.5.1
- marshmallow-sqlalchemy ==0.23.0
- more-itertools ==8.3.0
- mysqlclient ==1.3.14
- natsort ==7.0.1
- ndg-httpsclient ==0.5.1
- newsapi-python ==0.2.6
- numpy ==1.18.4
- packaging ==20.3
- pandas ==0.25.3
- paramiko ==2.7.1
- pendulum ==1.4.4
- pluggy ==0.13.1
- prison ==0.1.3
- psutil ==5.7.0
- psycopg2-binary ==2.8.5
- py ==1.8.1
- pyOpenSSL ==19.1.0
- pyasn1 ==0.4.8
- pycparser ==2.20
- pyparsing ==2.4.7
- pyrsistent ==0.16.0
- pysftp ==0.2.9
- pytest ==5.4.2
- python-daemon ==2.1.2
- python-dateutil ==2.8.1
- python-editor ==1.0.4
- python-nvd3 ==0.15.0
- python-slugify ==4.0.1
- python3-openid ==3.1.0
- pytz ==2020.1
- pytzdata ==2019.3
- redis ==3.2.0
- requests ==2.23.0
- s3transfer ==0.3.3
- setproctitle ==1.1.10
- six ==1.14.0
- sshtunnel ==0.1.5
- tabulate ==0.8.7
- tenacity ==4.12.0
- termcolor ==1.1.0
- text-unidecode ==1.3
- thrift ==0.13.0
- tornado ==5.1.1
- typing-extensions ==3.7.4.2
- tzlocal ==1.5.1
- unicodecsv ==0.14.1
- urllib3 ==1.25.9
- vine ==1.3.0
- wcwidth ==0.1.9
- zipp ==3.1.0
- zope.deprecation ==4.4.0
- HeapDict ==1.0.1
- PyYAML ==5.3.1
- boto3 ==1.13.9
- botocore ==1.16.26
- certifi ==2020.6.20
- chardet ==3.0.4
- click ==7.1.2
- cloudpickle ==1.6.0
- croniter ==0.3.34
- dask ==2.24.0
- distributed ==2.24.0
- docker ==4.3.1
- docutils ==0.15.2
- idna ==2.10
- jmespath ==0.10.0
- marshmallow ==3.7.1
- marshmallow-oneofschema ==2.0.1
- msgpack ==1.0.0
- mypy-extensions ==0.4.3
- natsort ==7.0.1
- newsapi-python ==0.2.6
- numpy ==1.19.1
- pandas ==0.25.3
- pendulum ==2.1.2
- prefect ==0.13.4
- psutil ==5.7.2
- python-box ==4.2.3
- python-dateutil ==2.8.1
- python-slugify ==4.0.1
- pytz ==2020.1
- pytzdata ==2020.1
- requests ==2.24.0
- ruamel.yaml ==0.16.10
- ruamel.yaml.clib ==0.2.0
- s3transfer ==0.3.3
- six ==1.15.0
- sortedcontainers ==2.2.2
- tabulate ==0.8.7
- tblib ==1.7.0
- text-unidecode ==1.3
- toml ==0.10.1
- toolz ==0.10.0
- tornado ==6.0.4
- urllib3 ==1.25.10
- websocket-client ==0.57.0
- zict ==2.0.0
- HeapDict ==1.0.1
- PyYAML ==5.3.1
- attrs ==20.1.0
- boto3 ==1.13.9
- botocore ==1.16.26
- certifi ==2020.6.20
- chardet ==3.0.4
- click ==7.1.2
- cloudpickle ==1.5.0
- croniter ==0.3.34
- dask ==2.23.0
- distributed ==2.23.0
- docker ==4.3.1
- docutils ==0.15.2
- idna ==2.10
- importlib-metadata ==1.7.0
- jmespath ==0.10.0
- marshmallow ==3.7.1
- marshmallow-oneofschema ==2.0.1
- more-itertools ==8.4.0
- msgpack ==1.0.0
- mypy-extensions ==0.4.3
- natsort ==7.0.1
- newsapi-python ==0.2.6
- numpy ==1.19.1
- packaging ==20.4
- pandas ==0.25.3
- pendulum ==2.1.2
- pluggy ==0.13.1
- prefect ==0.13.3
- psutil ==5.7.2
- py ==1.9.0
- pyparsing ==2.4.7
- pytest ==5.4.2
- python-box ==4.2.3
- python-dateutil ==2.8.1
- python-slugify ==4.0.1
- pytz ==2020.1
- pytzdata ==2020.1
- requests ==2.24.0
- ruamel.yaml ==0.16.10
- ruamel.yaml.clib ==0.2.0
- s3transfer ==0.3.3
- six ==1.15.0
- sortedcontainers ==2.2.2
- tabulate ==0.8.7
- tblib ==1.7.0
- text-unidecode ==1.3
- toml ==0.10.1
- toolz ==0.10.0
- tornado ==6.0.4
- urllib3 ==1.25.10
- wcwidth ==0.2.5
- websocket-client ==0.57.0
- zict ==2.0.0
- zipp ==3.1.0
airflow/Dockerfile
docker
- python 3.7-slim-buster build
airflow/docker-compose.yml
docker