Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / tingpengJ / real-time-data-pipeline2

Built a complete Real-time data pipeline to analysis stock data from Google finance. Got the specified stock information from Google Finance and transfer them with Kafka.Used Cassandra as storage layer. Used Spark to process stock data such as calculating average.Used Node.js to implement a real-time dynamic data display.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tingpengJ%2Freal-time-data-pipeline2

Stars: 0
Forks: 1
Open Issues: 7

License: None
Language: Python
Repo Size: 11.5 MB
Dependencies: 57

Created: almost 6 years ago
Updated: almost 4 years ago
Last pushed: over 1 year ago
Last synced: about 1 year ago

Files

Readme

Dependencies

nodejs/package.json npm

bootstrap ^3.3.7
d3 ^3.4.4
express ^4.14.0
jquery ^3.1.0
minimist ^1.2.0
nvd3 ^1.8.4
redis ^2.6.2
smoothie ^1.27.0
socket.io ^1.4.8

cassandra/requirements.txt pypi

cassandra-driver ==3.6.0
futures ==3.0.5
googlefinance ==0.7
kafka-python ==1.3.1
schedule ==0.3.2
six ==1.10.0
wheel ==0.24.0

kafka/requirements.txt pypi

APScheduler ==3.2.0
Flask ==0.11.1
Jinja2 ==2.8
MarkupSafe ==0.23
Werkzeug ==0.11.11
click ==6.6
confluent-kafka ==0.9.1.2
funcsigs ==1.0.2
futures ==3.0.5
googlefinance ==0.7
itsdangerous ==0.24
kafka-python ==1.3.1
pytz ==2016.7
schedule ==0.3.2
six ==1.10.0
tzlocal ==1.3
wheel ==0.24.0

redis/requirements.txt pypi

futures ==3.0.5
kafka-python ==1.3.1
redis ==2.10.5
six ==1.10.0
wheel ==0.24.0

zipkin/requirements.txt pypi

appdirs ==1.4.3
cassandra-driver ==3.6.0
certifi ==2017.4.17
chardet ==3.0.4
futures ==3.1.1
googlefinance ==0.7
idna ==2.5
kafka-python ==1.3.3
packaging ==16.8
ply ==3.10
py-zipkin ==0.7.1
pyparsing ==2.2.0
requests ==2.13.0
schedule ==0.4.2
six ==1.10.0
thriftpy ==0.3.9
urllib3 ==1.21.1

kafka/Dockerfile docker

python 2.7.12-alpine build

nodejs/Dockerfile docker

mhart/alpine-node base build