Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / manitgupta / DataflowTemplates

Google-provided Cloud Dataflow template pipelines for solving simple in-Cloud data tasks

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manitgupta%2FDataflowTemplates

Fork of GoogleCloudPlatform/DataflowTemplates
Stars: 0
Forks: 0
Open Issues: 1

License: apache-2.0
Language: Java
Repo Size: 15.4 MB
Dependencies: 297

Created: almost 2 years ago
Updated: 1 day ago
Last pushed: 2 days ago
Last synced: 1 day ago

Files

Loading...

Readme

Loading...

Dependencies

.github/actions/setup-env/action.yml actions

./.github/actions/setup-java-env * composite
actions/setup-go 44e221478fc6847752e5c574fc7a7b3247b00fbf composite
tj-actions/changed-files 61ee456a9d0f512e7ecfdf28863634c97dae2d16 composite

.github/actions/setup-java-env/action.yml actions

actions/cache 72d1e4fdff0ff7b1b6e86b415f2d4f5941e5c006 composite
actions/setup-java a12e082d834968c1847f782019214fadd20719f6 composite

.github/workflows/go-pr.yml actions

actions/checkout 230611dbd0eb52da1e1f4f7bc8bb0c3a339fc8b7 composite
actions/setup-go 44e221478fc6847752e5c574fc7a7b3247b00fbf composite

.github/workflows/java-pr.yml actions

./.github/actions/cleanup-java-env * composite
./.github/actions/setup-env * composite
actions/checkout 230611dbd0eb52da1e1f4f7bc8bb0c3a339fc8b7 composite
actions/checkout v2.6.0 composite
actions/setup-java v3 composite

.github/workflows/prepare-java-cache.yml actions

./.github/actions/cleanup-java-env * composite
./.github/actions/setup-java-env * composite
actions/checkout 230611dbd0eb52da1e1f4f7bc8bb0c3a339fc8b7 composite

.github/workflows/release.yml actions

actions/checkout v3 composite

.github/workflows/scorecards-analysis.yml actions

actions/checkout ec3a7ce113134d7a93b817d10a8272cb61118579 composite
actions/upload-artifact 82c141cc518b40d92cc801eee768e7aafc9c2fa2 composite
github/codeql-action/upload-sarif 5f532563584d71fdef14ee64d17bafb34f751ce5 composite
ossf/scorecard-action e3e75cf2ffbf9364bbff86cdbdf52b23176fe492 composite

v2/flex-wordcount-python/Dockerfile docker

gcr.io/dataflow-templates-base/python2-template-launcher-base latest build

it/pom.xml maven

com.google.auto.value:auto-value ${autovalue.version} provided
com.google.apis:google-api-services-dataflow v1b3-rev20220920-2.0.0
com.google.auto.value:auto-value-annotations ${autovalue.version}
com.google.cloud.teleport.metadata:metadata 1.0-SNAPSHOT
com.google.cloud:google-cloud-bigquery
com.google.cloud:google-cloud-bigtable
com.google.cloud:google-cloud-monitoring
com.google.cloud:google-cloud-pubsub
com.google.cloud:google-cloud-spanner
com.google.cloud:google-cloud-storage
com.google.guava:guava 31.0.1-jre
com.google.re2j:re2j ${re2j.version}
com.google.truth:truth ${truth.version}
junit:junit ${junit.version}
org.apache.beam:beam-sdks-java-core
org.apache.kafka:kafka-clients 2.4.1
org.elasticsearch.client:elasticsearch-rest-high-level-client 7.9.2
org.mongodb:mongodb-driver-sync 3.12.10
org.slf4j:slf4j-simple ${slf4j.version}
org.testcontainers:elasticsearch 1.17.5
org.testcontainers:kafka 1.17.5
org.testcontainers:mongodb 1.17.5
org.testcontainers:testcontainers 1.17.5
org.mockito:mockito-core 3.7.7 test
org.mockito:mockito-inline 4.5.1 test

metadata/pom.xml maven

junit:junit 4.13.1
com.google.truth:truth 1.0.1 test

plugins/core-plugin/pom.xml maven

com.google.cloud.teleport.metadata:metadata 1.0-SNAPSHOT
com.google.code.gson:gson 2.9.1
org.apache.beam:beam-sdks-java-core
org.apache.commons:commons-lang3 3.11
org.reflections:reflections 0.9.11
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test

plugins/templates-maven-plugin/pom.xml maven

org.apache.maven.plugin-tools:maven-plugin-annotations 3.6.0 provided
org.apache.maven:maven-artifact 3.3.9 provided
org.apache.maven:maven-core 3.3.9 provided
org.apache.maven:maven-plugin-api 3.3.9 provided
com.google.cloud.teleport:core-plugin 1.0-SNAPSHOT
com.google.cloud:google-cloud-dataflow 0.7.6
org.twdata.maven:mojo-executor 2.4.0
junit:junit 4.12 test
org.apache.maven.plugin-testing:maven-plugin-testing-harness 3.3.0 test
org.apache.maven:maven-compat 3.3.9 test

pom.xml maven

com.fasterxml.jackson:jackson-bom 2.14.1 import
com.google.cloud:libraries-bom 26.1.3 import
org.apache.beam:beam-sdks-java-bom 2.43.0 import
commons-codec:commons-codec 1.15
commons-io:commons-io 2.11.0
junit:junit 4.13.1
org.apache.logging.log4j:log4j 2.17.2
org.apache.logging.log4j:log4j-api 2.17.2
org.apache.logging.log4j:log4j-core 2.17.2

structured-logging/pom.xml maven

ch.qos.logback:logback-classic 1.2.10
com.google.code.gson:gson 2.9.1
org.slf4j:slf4j-api ${slf4j.version}
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test

syndeo-template/pom.xml maven

com.google.cloud:libraries-bom 26.1.3 import
org.apache.beam:beam-sdks-java-bom 2.43.0 import
com.google.api-client:google-api-client 1.35.2
com.google.apis:google-api-services-bigquery v2-rev459-1.25.0
com.google.auto.service:auto-service-annotations 1.0.1
com.google.auto.value:auto-value 1.9
com.google.auto.value:auto-value-annotations 1.9
com.google.cloud:google-cloud-bigtable 2.10.3
com.google.protobuf:protobuf-java 3.21.5
com.google.protobuf:protoc 3.21.5
org.apache.arrow:arrow-memory-unsafe 9.0.0
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-vendor-guava-26_0-jre 0.1
org.apache.kafka:kafka-clients 2.4.1
com.google.cloud.teleport:integration-testing-lib 1.0-SNAPSHOT test
com.google.truth:truth 1.0.1 test
org.mockito:mockito-core 3.7.7 test
org.testcontainers:gcloud 1.17.3 test
org.testcontainers:kafka 1.17.3 test

v1/pom.xml maven

org.apache.beam:beam-sdks-java-io-parquet compile
org.apache.beam:beam-sdks-java-bom ${beam.version} import
com.google.auto.value:auto-value ${autovalue.version} provided
org.apache.hbase:hbase-common 1.4.5 provided
com.datastax.cassandra:cassandra-driver-core 3.6.0
com.google.api.grpc:proto-google-cloud-kms-v1 0.87.0
com.google.auto.service:auto-service 1.0-rc6
com.google.auto.value:auto-value-annotations ${autovalue.version}
com.google.cloud.bigtable:bigtable-beam-import 1.27.1
com.google.cloud.teleport.metadata:metadata 1.0-SNAPSHOT
com.google.cloud:google-cloud-bigquery 1.108.0
com.google.cloud:google-cloud-dlp 2.1.0
com.google.cloud:google-cloud-kms 1.40.0
com.google.cloud:google-cloud-secretmanager
com.google.http-client:google-http-client
com.google.http-client:google-http-client-apache-v2
com.google.re2j:re2j ${re2j.version}
commons-codec:commons-codec
commons-io:commons-io
io.opencensus:opencensus-api 0.24.0
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-sdks-java-io-cassandra
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.commons:commons-csv 1.8
org.apache.commons:commons-text 1.10.0
org.apache.hadoop:hadoop-common ${hadoop.version}
org.apache.hadoop:hadoop-mapreduce-client-core ${hadoop.version}
org.apache.kafka:kafka-clients 1.0.0
org.codehaus.jackson:jackson-core-asl 1.9.13
org.codehaus.jackson:jackson-mapper-asl 1.9.13
org.conscrypt:conscrypt-openjdk-uber ${conscrypt.version}
org.json:json ${json.version}
org.openjdk.nashorn:nashorn-core 15.4
org.springframework:spring-expression 5.2.22.RELEASE
org.tensorflow:proto 1.15.0
org.threeten:threetenbp 1.4.4
com.google.cloud.teleport:integration-testing-lib 1.0-SNAPSHOT test
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test
org.apache.derby:derby 10.14.2.0 test
org.apache.derby:derbyclient 10.14.2.0 test
org.apache.derby:derbynet 10.14.2.0 test
org.hamcrest:hamcrest-core ${hamcrest.version} test
org.hamcrest:hamcrest-library ${hamcrest.version} test
org.junit.jupiter:junit-jupiter-api 5.5.2 test
org.mock-server:mockserver-junit-rule 5.14.0 test
org.mock-server:mockserver-netty 5.14.0 test
org.mockito:mockito-core ${mockito-core.version} test
org.scassandra:cql-antlr 1.1.2 test
org.scassandra:java-client 1.1.2 test

v2/bigquery-to-bigtable/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}

v2/bigquery-to-parquet/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}

v2/cdc-parent/cdc-agg/pom.xml maven

com.google.cloud.dataflow.cdc:cdc-change-applier ${project.version}
com.google.cloud.dataflow.cdc:cdc-common ${project.version}
com.google.cloud.dataflow.cdc:cdc-embedded-connector ${project.version}

v2/cdc-parent/cdc-change-applier/pom.xml maven

com.google.auto.value:auto-value-annotations 1.6.2 compile
org.slf4j:slf4j-api ${slf4j.version} compile
com.google.cloud.dataflow.cdc:cdc-common ${project.version}
com.google.cloud:google-cloud-bigquery

v2/cdc-parent/cdc-common/pom.xml maven

com.google.api.grpc:grpc-google-cloud-datacatalog-v1beta1 ${data-catalog.version}
com.google.api.grpc:proto-google-cloud-datacatalog-v1beta1 ${data-catalog.version}
com.google.cloud:google-cloud-datacatalog ${data-catalog.version}
org.apache.beam:beam-sdks-java-extensions-sql
org.slf4j:slf4j-jdk14 ${slf4j.version}
org.slf4j:slf4j-nop ${slf4j.version}

v2/cdc-parent/cdc-embedded-connector/pom.xml maven

commons-beanutils:commons-beanutils 1.9.4 compile
io.debezium:debezium-connector-mysql ${debezium.version} compile
io.debezium:debezium-embedded ${debezium.version} compile
org.apache.commons:commons-configuration2 2.8.0 compile
com.google.cloud.dataflow.cdc:cdc-common ${project.version}
org.mockito:mockito-all 1.10.19 test

v2/cdc-parent/pom.xml maven

org.slf4j:slf4j-api ${slf4j.version} compile
com.google.api.grpc:grpc-google-cloud-datacatalog-v1beta1 0.35.0
com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
com.google.guava:guava 25.1-jre
org.junit.jupiter:junit-jupiter-api 5.5.1 test
org.junit.jupiter:junit-jupiter-engine 5.5.1 test

v2/common/pom.xml maven

org.mongodb:mongo-java-driver 3.12.11 compile
com.google.cloud.bigtable:bigtable-hbase-beam 1.15.0
com.google.cloud:google-cloud-bigquery 2.9.0
com.google.cloud:google-cloud-kms
com.google.cloud:google-cloud-secretmanager 2.1.2
org.apache.avro:avro 1.8.2
org.apache.beam:beam-sdks-java-io-parquet
org.apache.commons:commons-csv 1.8
org.apache.commons:commons-text 1.10.0
org.apache.hadoop:hadoop-client ${hadoop.version}
org.openjdk.nashorn:nashorn-core 15.4
com.google.truth.extensions:truth-proto-extension 1.0.1 test
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test

v2/datastream-to-bigquery/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}

v2/datastream-to-mongodb/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-mongodb

v2/datastream-to-postgres/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.postgresql:postgresql 42.2.27

v2/datastream-to-spanner/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.json:json 20160810

v2/datastream-to-sql/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
mysql:mysql-connector-java 8.0.13
org.postgresql:postgresql 42.2.27
com.google.truth:truth ${truth.version} test

v2/elasticsearch-common/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-elasticsearch

v2/file-format-conversion/pom.xml maven

com.google.cloud.teleport.v2:common 1.0-SNAPSHOT

v2/googlecloud-to-elasticsearch/pom.xml maven

com.google.cloud.teleport.v2:elasticsearch-common ${project.version}

v2/googlecloud-to-googlecloud/pom.xml maven

com.google.api.grpc:proto-google-cloud-kms-v1 0.87.0
com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
com.google.cloud:google-cloud-core
com.google.cloud:google-cloud-dlp 2.1.0
com.google.cloud:google-cloud-pubsub
com.google.guava:guava ${guava.version}
org.apache.beam:beam-sdks-java-io-hadoop-common
org.apache.hadoop:hadoop-common ${hadoop.version}
org.apache.hadoop:hadoop-mapreduce-client-core ${hadoop.version}
com.google.cloud:google-cloud-spanner test
com.google.truth:truth ${truth.version} test
org.apache.derby:derby 10.14.2.0 test
org.apache.derby:derbyclient 10.14.2.0 test
org.apache.derby:derbynet 10.14.2.0 test
org.mockito:mockito-core ${mockito-core.version} test

v2/googlecloud-to-mongodb/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-mongodb

v2/googlecloud-to-neo4j/pom.xml maven

org.projectlombok:lombok 1.18.24 provided
com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-io-neo4j
org.neo4j:neo4j-cypher-dsl 2022.4.0
com.google.truth:truth ${truth.version} test

v2/googlecloud-to-splunk/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-splunk ${beam.version}
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test

v2/kafka-common/pom.xml maven

com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
org.apache.beam:beam-sdks-java-io-kafka
org.apache.kafka:kafka-clients 2.3.0

v2/kafka-to-bigquery/pom.xml maven

com.google.cloud.teleport.v2:kafka-common ${project.version}

v2/kafka-to-gcs/pom.xml maven

com.google.cloud.teleport.v2:kafka-common ${project.version}

v2/kafka-to-pubsub/pom.xml maven

com.google.cloud.teleport.v2:kafka-common ${project.version}

v2/kudu-to-bigquery/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-kudu

v2/mongodb-to-googlecloud/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-mongodb
com.google.truth:truth ${truth.version} test

v2/pom.xml maven

com.google.auto.value:auto-value ${autovalue.version} provided
com.google.auto.service:auto-service 1.0-rc6
com.google.auto.value:auto-value-annotations ${autovalue.version}
com.google.cloud.teleport.metadata:metadata 1.0-SNAPSHOT
com.google.cloud.teleport:structured-logging 1.0-SNAPSHOT
com.google.re2j:re2j ${re2j.version}
io.opencensus:opencensus-api 0.31.0
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-io-jdbc
org.conscrypt:conscrypt-openjdk-uber ${conscrypt.version}
org.json:json ${json.version}
com.google.cloud.teleport:integration-testing-lib 1.0-SNAPSHOT test
com.google.cloud.teleport:plaintext-logging 1.0-SNAPSHOT test
junit:junit ${junit.version} test
junit:junit-dep 4.10 test
org.hamcrest:hamcrest ${hamcrest.version} test
org.hamcrest:hamcrest-core ${hamcrest.version} test
org.hamcrest:hamcrest-library ${hamcrest.version} test
org.mockito:mockito-core ${mockito-core.version} test

v2/pubsub-binary-to-bigquery/pom.xml maven

com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
com.google.guava:guava ${guava.version}
com.google.truth:truth ${truth.version} test
junit:junit ${junit.version} test

v2/pubsub-cdc-to-bigquery/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}

v2/pubsub-to-mongodb/pom.xml maven

com.google.cloud.teleport.v2:common ${project.version}
org.apache.beam:beam-sdks-java-io-mongodb

v2/streaming-data-generator/pom.xml maven

com.github.vincentrussell:json-data-generator 1.10
com.google.cloud.teleport.v2:common 1.0-SNAPSHOT
org.yaml:snakeyaml 1.33
com.google.truth:truth ${truth.version} test

v1/src/main/python/requirements.txt pypi

Copyright2020GoogleInc.AllRightsReserved. *
LicensedundertheApacheLicense ,Version2.0
SeetheLicenseforthespecificlanguagegoverningpermissionsand *
Unlessrequiredbyapplicablelaworagreedtoinwriting ,software
WITHOUTWARRANTIESORCONDITIONSOFANYKIND ,eitherexpressorimplied.
YoumayobtainacopyoftheLicenseat *
distributedundertheLicenseisdistributedonan *
google-cloud *
google-cloud-pubsub *
limitationsundertheLicense. *
youmaynotusethisfileexceptincompliancewiththeLicense. *