dataflow | Topic | Ecosyste.ms: Repos

Topic: "dataflow"

pathwaycom/pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Language: Python - Size: 133 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 28,765 - Forks: 758

marimo-team/marimo

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. All in a modern, AI-native editor.

Language: Python - Size: 149 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 14,717 - Forks: 605

jerosoler/Drawflow

Simple flow library 🖥️🖱️

Language: JavaScript - Size: 7.09 MB - Last synced at: 19 days ago - Pushed at: 9 months ago - Stars: 5,339 - Forks: 823

nextflow-io/nextflow

A DSL for data-driven computational pipelines

Language: Groovy - Size: 60.1 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 3,075 - Forks: 714

spotify/scio

A Scala API for Apache Beam and Google Cloud Dataflow.

Language: Scala - Size: 81.6 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 2,606 - Forks: 520

joernio/joern

Open-source code analysis platform for C/C++/Java/Binary/Javascript/Python/Kotlin based on code property graphs. Discord https://discord.gg/vv4MH284Hc

Language: Scala - Size: 160 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2,509 - Forks: 336

DORA (Dataflow-Oriented Robotic Architecture) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.

Language: Rust - Size: 12.3 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 2,365 - Forks: 200

Bearer/bearer

Code security scanning tool (SAST) to discover, filter and prioritize security and privacy risks.

Language: Go - Size: 23.9 MB - Last synced at: 9 days ago - Pushed at: about 1 month ago - Stars: 2,343 - Forks: 127

cocoindex-io/cocoindex

Data transformation framework for AI. Ultra performant, with incremental processing.

Language: Rust - Size: 11.1 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 2,246 - Forks: 154

python-security/pyt

A Static Analysis Tool for Detecting Security Vulnerabilities in Python Web Applications

Language: Python - Size: 3.2 MB - Last synced at: 5 days ago - Pushed at: over 4 years ago - Stars: 2,188 - Forks: 248

willcrichton/flowistry

Flowistry is an IDE plugin for Rust that helps you focus on relevant code.

Language: Rust - Size: 80.8 MB - Last synced at: 5 days ago - Pushed at: 7 months ago - Stars: 2,001 - Forks: 50

newcat/baklavajs

Graph / node editor in the browser using VueJS

Language: TypeScript - Size: 10.5 MB - Last synced at: 22 days ago - Pushed at: about 2 months ago - Stars: 1,793 - Forks: 129

bytewax/bytewax

Python Stream Processing

Language: Python - Size: 12 MB - Last synced at: 9 days ago - Pushed at: 4 months ago - Stars: 1,780 - Forks: 86

serverless/event-gateway 📦

React to any event with serverless functions across clouds

Language: Go - Size: 1.13 MB - Last synced at: 4 months ago - Pushed at: 8 months ago - Stars: 1,647 - Forks: 97

trustmaster/goflow

Flow-based and dataflow programming library for Go (golang)

Language: Go - Size: 289 KB - Last synced at: 2 months ago - Pushed at: almost 4 years ago - Stars: 1,635 - Forks: 127

ipyflow/ipyflow

A reactive Python kernel for Jupyter notebooks.

Language: Python - Size: 23 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 1,233 - Forks: 23

scipipe/scipipe

Robust, flexible and resource-efficient pipelines using Go and the commandline

Language: Go - Size: 1.06 MB - Last synced at: 2 months ago - Pushed at: 12 months ago - Stars: 1,096 - Forks: 72

shaiwz/data-platform-open

🔥🔥🔥可视化拖拽式大数据集成平台、大数据平台、大数据，包含数据流、数据源、数据对齐、查询模板、完善的监控等。像画流程图一样且无代码方式同步、清洗数据。

Language: Java - Size: 4.84 MB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 1,049 - Forks: 93

OWASP/pytm

A Pythonic framework for threat modeling

Language: Python - Size: 1.64 MB - Last synced at: 5 days ago - Pushed at: about 2 months ago - Stars: 1,017 - Forks: 196

RaftLib/RaftLib

The RaftLib C++ library, streaming/dataflow concurrency via C++ iostream-like operators

Language: C++ - Size: 23.6 MB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 970 - Forks: 127

composewell/streamly

High performance, concurrent functional programming abstractions

Language: Haskell - Size: 17.4 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 896 - Forks: 68

Xilinx/finn

Dataflow compiler for QNN inference on FPGAs

Language: Python - Size: 85.9 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 845 - Forks: 268

nipy/nipype

Workflows and interfaces for neuroimaging packages

Language: Python - Size: 75.6 MB - Last synced at: about 8 hours ago - Pushed at: about 1 month ago - Stars: 791 - Forks: 535

lw-lin/streaming-readings

Streaming System 相关的论文读物

Size: 6.84 KB - Last synced at: 2 months ago - Pushed at: over 3 years ago - Stars: 733 - Forks: 154

jaystack/repatch

Dispatch reducers

Language: TypeScript - Size: 133 KB - Last synced at: about 12 hours ago - Pushed at: over 7 years ago - Stars: 502 - Forks: 10

google/shenzhen-go 📦

Experimental visual Go environment

Language: JavaScript - Size: 87.6 MB - Last synced at: 3 months ago - Pushed at: about 4 years ago - Stars: 464 - Forks: 43

wotbrew/relic

Functional relational programming for Clojure(Script).

Language: Clojure - Size: 1.68 MB - Last synced at: 17 days ago - Pushed at: 8 months ago - Stars: 420 - Forks: 7

chigraph/chigraph

A visual systems language for beginners compiled using LLVM

Language: C++ - Size: 63.5 MB - Last synced at: 8 days ago - Pushed at: almost 2 years ago - Stars: 420 - Forks: 76

asg017/dataflow

An experimental self-hosted Observable notebook editor, with support for FileAttachments, Secrets, custom standard libraries, and more!

Language: JavaScript - Size: 704 KB - Last synced at: 5 days ago - Pushed at: almost 4 years ago - Stars: 418 - Forks: 26

pothosware/PothosCore

The Pothos data-flow framework

Language: C++ - Size: 63 MB - Last synced at: 2 months ago - Pushed at: about 2 years ago - Stars: 314 - Forks: 50

spotify/pythonflow

:snake: Dataflow programming for python.

Language: Python - Size: 222 KB - Last synced at: 28 days ago - Pushed at: about 2 years ago - Stars: 292 - Forks: 48

ghostiam/vue-blocks 📦

Vue2 dataflow graph editor

Language: Vue - Size: 1.66 MB - Last synced at: 9 days ago - Pushed at: over 2 years ago - Stars: 281 - Forks: 61

Chaffelson/nipyapi

A convenient Python wrapper for Apache NiFi

Language: Python - Size: 4.38 MB - Last synced at: 27 days ago - Pushed at: about 1 month ago - Stars: 260 - Forks: 76

meemoo/meemooapp

Creative apps to use, build, share, and hack in the browser.

Language: JavaScript - Size: 3.63 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 254 - Forks: 52

GoogleCloudPlatform/data-analytics-golden-demo

An end to end demo of Google's Cloud data and analytic stack.

Language: Jupyter Notebook - Size: 14.8 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 253 - Forks: 86

cross-platform/dspatch

The Refreshingly Simple Cross-Platform C++ Dataflow / Patching / Pipelining / Graph Processing / Stream Processing / Reactive Programming Framework

Language: C++ - Size: 6.21 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 238 - Forks: 43

stanford-mast/nn_dataflow

Explore the energy-efficient dataflow scheduling for neural networks.

Language: Python - Size: 1.05 MB - Last synced at: 23 days ago - Pushed at: almost 5 years ago - Stars: 225 - Forks: 84

GoogleCloudPlatform/fraudfinder

Fraudfinder: A comprehensive lab series on how to build a real-time fraud detection system on Google Cloud

Language: Jupyter Notebook - Size: 13 MB - Last synced at: 2 months ago - Pushed at: 4 months ago - Stars: 220 - Forks: 84

nnmer/azure-services-map

A visual representation and reference to Azure services

Language: JavaScript - Size: 6.76 MB - Last synced at: 5 days ago - Pushed at: over 2 years ago - Stars: 219 - Forks: 33

microflo/microflo

Live dataflow programming for microcontrollers and embedded

Language: CoffeeScript - Size: 10.5 MB - Last synced at: 29 days ago - Pushed at: about 5 years ago - Stars: 218 - Forks: 27

Gregwar/blocks.js

JavaScript dataflow graph editor

Language: JavaScript - Size: 412 KB - Last synced at: 19 days ago - Pushed at: over 6 years ago - Stars: 217 - Forks: 37

maestro-project/maestro

An analytical cost model evaluating DNN mappings (dataflows and tiling).

Language: MATLAB - Size: 743 KB - Last synced at: 4 months ago - Pushed at: over 1 year ago - Stars: 211 - Forks: 64

erdos-project/erdos

Dataflow system for building self-driving car and robotics applications.

Language: Rust - Size: 10.2 MB - Last synced at: 6 days ago - Pushed at: almost 3 years ago - Stars: 205 - Forks: 45

siemens/embb

Embedded Multicore Building Blocks (EMB²): Library for parallel programming of embedded systems. Star us on GitHub? +1

Language: C++ - Size: 18.9 MB - Last synced at: 10 days ago - Pushed at: over 1 year ago - Stars: 187 - Forks: 41

skydoves/Chamber

⚖️ A lightweight Android lifecycle-aware and thread-safe pipeline for communicating between components with custom scopes.

Language: Kotlin - Size: 255 KB - Last synced at: 4 months ago - Pushed at: about 4 years ago - Stars: 187 - Forks: 14

GoogleCloudPlatform/df-ml-anomaly-detection

Streaming Anomaly Detection Solution by using Pub/Sub, Dataflow, BQML & Cloud DLP

Language: Java - Size: 12.3 MB - Last synced at: 2 months ago - Pushed at: 6 months ago - Stars: 181 - Forks: 49

cda-group/arcon

State-first Streaming Applications in Rust

Language: Rust - Size: 3.27 MB - Last synced at: about 14 hours ago - Pushed at: almost 3 years ago - Stars: 175 - Forks: 17

flowbase/flowbase

A Flow-based Programming inspired micro-framework / un-framework for Go (Golang)

Language: Go - Size: 99.6 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 166 - Forks: 4

Blocks-Editor/blocks

Blocks. An online drag-and-drop smart contract builder.

Language: JavaScript - Size: 25.4 MB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 161 - Forks: 35

GoogleCloudPlatform/bigquery-data-lineage 📦

Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.

Language: Java - Size: 356 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 143 - Forks: 41

msgflo/msgflo

Distributed Flow-Based Programming via message queues

Language: CoffeeScript - Size: 5.2 MB - Last synced at: 3 days ago - Pushed at: over 4 years ago - Stars: 142 - Forks: 13

googlegenomics/gcp-variant-transforms

GCP Variant Transforms

Language: Python - Size: 20.5 MB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 139 - Forks: 55

google/megalista 📦

First Party data integration solution built for marketing teams to enable audience and conversion onboarding into Google Marketing products (Google Ads, Campaign Manager, Google Analytics).

Language: Python - Size: 1.34 MB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 137 - Forks: 55

Nicolas-Constanty/Dnai.Editor

Dnai Editor - Visual Scripting (Node Editor)

Language: C++ - Size: 205 MB - Last synced at: over 1 year ago - Pushed at: over 5 years ago - Stars: 134 - Forks: 30

yubowenok/visflow

Web-based Dataflow Framework for Visual Data Exploration

Language: TypeScript - Size: 11.9 MB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 123 - Forks: 16

allegro/bigflow

A Python framework for data processing on GCP.

Language: Python - Size: 107 MB - Last synced at: 2 days ago - Pushed at: 4 months ago - Stars: 119 - Forks: 23

IgorBuchelnikov/ObservableComputations

Cross-platform .NET library for computations whose arguments and results are objects that implement INotifyPropertyChanged and INotifyCollectionChanged (ObservableCollection) interfaces.

Language: C# - Size: 39.5 MB - Last synced at: 1 day ago - Pushed at: 2 months ago - Stars: 116 - Forks: 8

asyncvlsi/act

ACT hardware description language and core tools.

Language: C++ - Size: 5.17 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 115 - Forks: 27

anna-geller/dataflow-ops

Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate

Language: Python - Size: 1.32 MB - Last synced at: 4 months ago - Pushed at: about 2 years ago - Stars: 113 - Forks: 24

martinklepsch/derivatives

🌱 Your companion to create derived values from a single source (atom)

Language: Clojure - Size: 126 KB - Last synced at: 15 days ago - Pushed at: about 6 years ago - Stars: 112 - Forks: 13

anna-geller/prefect-deployment-patterns

Code examples showing flow deployment to various types of infrastructure

Language: Python - Size: 249 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 107 - Forks: 10

kestra-io/docs

Documentation for Kestra — an event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

Language: Vue - Size: 1.67 GB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 106 - Forks: 116

certtools/intelmq-manager

IntelMQ Manager is a graphical interface to manage configurations for IntelMQ framework.

Language: JavaScript - Size: 6.43 MB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 105 - Forks: 56

radiantone/entangle

A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs.

Language: Python - Size: 2.33 MB - Last synced at: 8 days ago - Pushed at: almost 3 years ago - Stars: 104 - Forks: 7

UCLA-VAST/RapidStream

[FPGA 2022, Best Paper Award] Parallel placement and routing of Vivado HLS dataflow designs.

Language: Python - Size: 142 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 103 - Forks: 18

Refefer/Dampr

Python Data Processing library

Language: Python - Size: 178 KB - Last synced at: 8 days ago - Pushed at: over 1 year ago - Stars: 102 - Forks: 6

JasonThon/lightflus

A Lightweight, Cloud-Native Stateful Distributed Dataflow Engine

Language: Rust - Size: 1.86 MB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 100 - Forks: 10

anna-geller/prefect-dataplatform

Example repository showing how to build a data platform with Prefect, dbt and Snowflake

Language: Python - Size: 1.23 MB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 100 - Forks: 17

GoogleCloudPlatform/dlp-dataflow-deidentification

Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP

Language: Java - Size: 47.5 MB - Last synced at: 3 months ago - Pushed at: 12 months ago - Stars: 92 - Forks: 51

lilactown/flex

flex is a reactive signal library for Clojure(Script)

Language: Clojure - Size: 1.14 MB - Last synced at: about 24 hours ago - Pushed at: 9 days ago - Stars: 89 - Forks: 3

DFiantHDL/DFHDL

DFiant HDL (DFHDL): A Dataflow Hardware Descripition Language

Language: Scala - Size: 50.7 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 88 - Forks: 11

GoogleCloudPlatform/google-cloud-eclipse

Google Cloud Platform plugin for Eclipse

Language: Java - Size: 13.5 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 86 - Forks: 48

hasielhassan/PlumberManager

A helper tool to design CG Pipeline interactive diagrams and data flow documentation

Language: Python - Size: 3.57 MB - Last synced at: 2 months ago - Pushed at: 4 months ago - Stars: 81 - Forks: 10

deib-polimi/renoir

Reactive Network of Operators In Rust. Framework for Parallel and distributed computation inspired from the DataFlow model

Language: Rust - Size: 6.3 MB - Last synced at: 19 days ago - Pushed at: about 1 month ago - Stars: 80 - Forks: 9

krthr/vue-dataflow-editor 📦

Vue 2 dataflow graph editor

Language: Vue - Size: 847 KB - Last synced at: 2 days ago - Pushed at: almost 5 years ago - Stars: 80 - Forks: 15