dask/old-dask-yarn 📦

Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead

Language: Python - Size: 32.2 KB - Last synced: about 4 hours ago - Pushed: almost 7 years ago - Stars: 7 - Forks: 2

ibis-project/ibis

the portable Python dataframe library

Language: Python - Size: 71.8 MB - Last synced: about 3 hours ago - Pushed: about 5 hours ago - Stars: 4,237 - Forks: 525

bioio-devs/bioio-ome-zarr

A BioIO reader plugin for reading Zarr files in the OME format.

Language: Python - Size: 68.4 KB - Last synced: about 4 hours ago - Pushed: 2 days ago - Stars: 0 - Forks: 1

bioio-devs/bioio

Image reading, metadata management, and image writing for Microscopy images in Python

Language: Python - Size: 5.26 MB - Last synced: about 4 hours ago - Pushed: 2 days ago - Stars: 16 - Forks: 1

Vizzuality/cog_worker

Scalable arbitrary analysis on COGs

Language: Jupyter Notebook - Size: 33.1 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 26 - Forks: 1

xarray-contrib/flox

Fast & furious GroupBy operations for dask.array

Language: Python - Size: 1.65 MB - Last synced: about 10 hours ago - Pushed: about 10 hours ago - Stars: 116 - Forks: 15

pytroll/satpy

Python package for earth-observing satellite data processing

Language: Python - Size: 20.8 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 1,009 - Forks: 283

nebari-dev/nebari-docs

📖 Documentation for Nebari

Size: 42.9 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 14 - Forks: 21

chrimerss/RainfallCamera

Rainfall Camera

Language: Python - Size: 113 MB - Last synced: about 24 hours ago - Pushed: over 4 years ago - Stars: 4 - Forks: 1

donkomura/fsspec-chfs

fsspec implementations for CHFS

Language: Python - Size: 19.5 KB - Last synced: 1 day ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

jgrss/geowombat

GeoWombat: Utilities for geospatial data

Language: Jupyter Notebook - Size: 240 MB - Last synced: about 8 hours ago - Pushed: 1 day ago - Stars: 173 - Forks: 10

discovery-unicamp/dasf-core

Framework for computing Machine Learning algorithms in Python using Dask and RAPIDS AI.

Language: Python - Size: 16.7 MB - Last synced: about 13 hours ago - Pushed: 1 day ago - Stars: 9 - Forks: 1

capitalone/datacompy

Pandas and Spark DataFrame comparison for humans and more!

Language: Python - Size: 9.11 MB - Last synced: about 7 hours ago - Pushed: 1 day ago - Stars: 386 - Forks: 122

mohgavin/code-repository

Code Repository

Language: Jupyter Notebook - Size: 125 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

casangi/graphviper

Dask Based MapReduce for Multi Xarray Datasets.

Language: Python - Size: 2.24 MB - Last synced: about 4 hours ago - Pushed: 2 days ago - Stars: 1 - Forks: 0

basnijholt/adaptive-scheduler

Run many functions (adaptively) on many cores (>10k-100k) using mpi4py.futures, ipyparallel, loky, or dask-mpi. :tada:

Language: Python - Size: 931 KB - Last synced: about 5 hours ago - Pushed: 2 days ago - Stars: 26 - Forks: 9

nebari-dev/nebari

🪴 Nebari - your open source data science platform

Language: Python - Size: 15.1 MB - Last synced: about 7 hours ago - Pushed: about 13 hours ago - Stars: 257 - Forks: 86

fschuch/xcompact3d_toolbox

A set of tools for pre and postprocessing prepared for the high-order Navier-Stokes solver XCompact3d

Language: Python - Size: 17.8 MB - Last synced: 2 days ago - Pushed: 3 days ago - Stars: 8 - Forks: 5

geoxarray/geoxarray

Geolocation utilities for xarray

Language: Python - Size: 363 KB - Last synced: about 13 hours ago - Pushed: 3 days ago - Stars: 95 - Forks: 7

dask-contrib/dask-sql

Distributed SQL Engine in Python using Dask

Language: Python - Size: 3.33 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 366 - Forks: 70

umr-lops/xsarsea

scientific functions to compute radar or geophysical parameters from satellite images over ocean

Language: Python - Size: 1.67 MB - Last synced: 3 days ago - Pushed: about 1 month ago - Stars: 9 - Forks: 6

TimeEval/TimeEval

Evaluation Tool for Anomaly Detection Algorithms on Time Series

Language: Jupyter Notebook - Size: 24.8 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 73 - Forks: 13

ESDS-Leipzig/cubo

On-Demand Earth System Data Cubes (ESDCs) in Python

Language: Python - Size: 1.64 MB - Last synced: about 19 hours ago - Pushed: 9 days ago - Stars: 151 - Forks: 9

polyaxon/traceml

Engine for ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

Language: Python - Size: 118 MB - Last synced: 17 days ago - Pushed: 21 days ago - Stars: 492 - Forks: 43

jmcarpenter2/swifter

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner

Language: Python - Size: 2.15 MB - Last synced: 3 days ago - Pushed: about 1 month ago - Stars: 2,466 - Forks: 101

Nixtla/mlforecast

Scalable machine 🤖 learning for time series forecasting.

Language: Python - Size: 27 MB - Last synced: 3 days ago - Pushed: 17 days ago - Stars: 720 - Forks: 68

raw-lab/mercat2

MerCat2: python code for versatile k-mer counting and diversity estimation for database independent property analysis for metaome data

Language: HTML - Size: 105 MB - Last synced: 4 days ago - Pushed: 5 days ago - Stars: 7 - Forks: 1

RichardScottOZ/xarray-notes

Notes on working with xarray

Size: 67.4 KB - Last synced: 4 days ago - Pushed: 5 days ago - Stars: 0 - Forks: 0

DataCanvasIO/HyperGBM

A full pipeline AutoML tool for tabular data

Language: Python - Size: 11 MB - Last synced: 3 days ago - Pushed: 2 months ago - Stars: 323 - Forks: 45

NVIDIA-Merlin/models

Merlin Models is a collection of deep learning recommender system model reference implementations

Language: Python - Size: 113 MB - Last synced: 4 days ago - Pushed: 5 days ago - Stars: 241 - Forks: 48

itamarst/eliot

Eliot: the logging system that tells you *why* it happened

Language: Python - Size: 1.91 MB - Last synced: about 9 hours ago - Pushed: 2 months ago - Stars: 1,087 - Forks: 65

shauryashaurya/learn-data-munging

Notes on Data Engineering with Pandas, PySpark, Dask, Ray, Arrow DataFusion, Polars etc.

Language: Jupyter Notebook - Size: 582 MB - Last synced: 6 days ago - Pushed: 6 days ago - Stars: 41 - Forks: 21

aurelienmorgan/french_text_sentiment

Sentiment Analysis in texts written in French language using Tensorflow/Keras (and using XGBoost for hyperparameters optimization)

Language: Python - Size: 21.9 MB - Last synced: 6 days ago - Pushed: over 3 years ago - Stars: 1 - Forks: 0

elcorto/psweep

Loop like a pro, make parameter studies fun.

Language: Python - Size: 5.81 MB - Last synced: 7 days ago - Pushed: 8 days ago - Stars: 12 - Forks: 2

TDAmeritrade/stumpy

STUMPY is a powerful and scalable Python library for modern time series analysis

Language: Python - Size: 129 MB - Last synced: 7 days ago - Pushed: about 1 month ago - Stars: 2,986 - Forks: 282

umr-lops/xsar

Synthetic Aperture Radar (SAR) Level-1 GRD python mapper for efficient xarray/dask based processing

Language: Python - Size: 20 MB - Last synced: 8 days ago - Pushed: 8 days ago - Stars: 24 - Forks: 8

milesgranger/cluster-clyde

Python tool for launching EC2 clusters with AWS within your script/notebook

Language: Python - Size: 264 KB - Last synced: 8 days ago - Pushed: about 7 years ago - Stars: 0 - Forks: 1

miniufo/xgrads

Parse and read ctl and associated binary file commonly used by GrADS into xarray

Language: Jupyter Notebook - Size: 16.5 MB - Last synced: 1 day ago - Pushed: 8 months ago - Stars: 69 - Forks: 25

bioio-devs/bioio-base

Typing, base classes, and more for BioIO projects.

Language: Python - Size: 1.18 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

mapsacosta/htcdaskgateway

A Dask Gateway client extension for heterogeneous cluster mode combining the Kubernetes backend for pain-free scheduler networking, with COFFEA-powered HTCondor workers

Language: Python - Size: 67.4 KB - Last synced: about 2 hours ago - Pushed: about 19 hours ago - Stars: 2 - Forks: 2

nilsoncunha/nilsoncunha

Repositório para criação do readme personalizado e listagem dos meus projetos

Size: 29.3 KB - Last synced: 26 days ago - Pushed: 26 days ago - Stars: 0 - Forks: 0

dask-contrib/dask-awkward

Native Dask collection for awkward arrays, and the library to use it.

Language: Python - Size: 1.29 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 56 - Forks: 15

climate-service-center/index_calculator

Calculate climate indicators based on xclim

Language: Jupyter Notebook - Size: 113 MB - Last synced: 11 days ago - Pushed: about 2 months ago - Stars: 1 - Forks: 4

godaai/distributed-python

Python 分布式编程

Language: Jupyter Notebook - Size: 17.5 MB - Last synced: 11 days ago - Pushed: 11 days ago - Stars: 9 - Forks: 5

ranaroussi/pystore

Fast data store for Pandas time-series data

Language: Python - Size: 138 KB - Last synced: 10 days ago - Pushed: about 1 month ago - Stars: 539 - Forks: 97

Quansight/qhub-ops 📦

A tool for initialising and maintaining the state of QHub deployments on Digital Ocean, Amazon Web Services, and Google Cloud Platform

Language: Python - Size: 27.6 MB - Last synced: 12 days ago - Pushed: over 3 years ago - Stars: 8 - Forks: 4

Quansight/ibis-posts

Programs about the ibis sql productivity framework.

Language: Jupyter Notebook - Size: 25.7 MB - Last synced: 12 days ago - Pushed: about 4 years ago - Stars: 0 - Forks: 2

dask-contrib/dask-deltatable

A Delta Lake reader for Dask

Language: Python - Size: 249 KB - Last synced: 8 days ago - Pushed: 29 days ago - Stars: 42 - Forks: 13

fugue-project/fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Language: Python - Size: 6.3 MB - Last synced: 18 days ago - Pushed: 19 days ago - Stars: 1,866 - Forks: 92

JulianWgs/dask-log-server

Preserve all necessary runtime data of a Dask client in order to "replay" and analyze the performance and behavior of the client after the fact

Language: Python - Size: 289 KB - Last synced: 12 days ago - Pushed: over 3 years ago - Stars: 3 - Forks: 0

quantori/scip-dockingfactory

Docking Factory is a tool to automate molecular docking runs on an HPC cluster using the Dask framework. DockingFactory provides unified way of running molecular docking with different software backends: AutoDock Vina, Smina, Qvina2, and rDock.

Language: MATLAB - Size: 1.69 MB - Last synced: 13 days ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

casangi/astrohack

Antenna panel and position corrections.

Language: Python - Size: 77.4 MB - Last synced: about 4 hours ago - Pushed: about 19 hours ago - Stars: 9 - Forks: 3

mpes-kit/mpes

Distributed data processing routines for multidimensional photoemission spectroscopy (MPES)

Language: Python - Size: 27.5 MB - Last synced: 7 days ago - Pushed: over 1 year ago - Stars: 27 - Forks: 6

CoffeaTeam/coffea-casa

Repository with configuration setup of a prototype of analysis facility - "coffea-casa"

Language: Python - Size: 11.3 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 16 - Forks: 17

ratt-ru/dask-ms

Implementation of a dask/xarray dataset backed by a CASA MS

Language: Python - Size: 6.67 MB - Last synced: about 19 hours ago - Pushed: 16 days ago - Stars: 18 - Forks: 6

ORNL/flowcept Fork of renan-souza/flowcept

Runtime data integration system that empowers any data processing system to capture and query workflow provenance using data observability.

Language: Python - Size: 53.1 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 1 - Forks: 2

MITgcm/xmitgcm

Read MITgcm mds binary files into xarray

Language: Python - Size: 117 MB - Last synced: 2 days ago - Pushed: 3 months ago - Stars: 54 - Forks: 64

thewtex/ngff-zarr

A lean and kind Open Microscopy Environment (OME) Next Generation File Format (NGFF) Zarr implementation.

Language: Python - Size: 223 KB - Last synced: 13 days ago - Pushed: 14 days ago - Stars: 18 - Forks: 3

dask/dask

Parallel computing with task scheduling

Language: Python - Size: 66.7 MB - Last synced: 17 days ago - Pushed: 17 days ago - Stars: 11,971 - Forks: 1,665

beartell/anymlops

🐻‍❄️ Anymlops: A data science platform that literally works !

Language: HCL - Size: 13.2 MB - Last synced: 15 days ago - Pushed: about 1 year ago - Stars: 0 - Forks: 0

coiled/dask-snowflake

Dask integration for Snowflake

Language: Python - Size: 57.6 KB - Last synced: 13 days ago - Pushed: about 1 month ago - Stars: 28 - Forks: 7

dask/distributed

A distributed task scheduler for Dask

Language: Python - Size: 191 MB - Last synced: 18 days ago - Pushed: 18 days ago - Stars: 1,539 - Forks: 703

NCAR/ncar-python-tutorial 📦

Numerical & Scientific Computing with Python Tutorial

Language: Jupyter Notebook - Size: 49.4 MB - Last synced: 16 days ago - Pushed: about 4 years ago - Stars: 63 - Forks: 32

BrunoKreiner/cic-mc2

Testing SaturnCloud's GPU Cluster using Dask and PyTorch Parallelized training methods. Test model is a simple convolutional neural network.

Language: Jupyter Notebook - Size: 215 KB - Last synced: 17 days ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

saturncloud/workshop-dask-pytorch 📦

Introduction to Dask for PyTorch Workflows

Language: Jupyter Notebook - Size: 7.12 MB - Last synced: 17 days ago - Pushed: about 3 years ago - Stars: 13 - Forks: 1

saturncloud/workshop-lightgbm-dask

Saturn Cloud workshop on using LightGBM with Dask

Language: Jupyter Notebook - Size: 114 KB - Last synced: 17 days ago - Pushed: over 2 years ago - Stars: 0 - Forks: 0

pydata/xarray

N-D labeled arrays and datasets in Python

Language: Python - Size: 41.3 MB - Last synced: 18 days ago - Pushed: 18 days ago - Stars: 3,396 - Forks: 1,016

rapidsai/cudf

cuDF - GPU DataFrame Library

Language: C++ - Size: 134 MB - Last synced: 19 days ago - Pushed: 19 days ago - Stars: 7,236 - Forks: 829

Ouranosinc/xclim

Library of derived climate variables, ie climate indicators, based on xarray.

Language: Python - Size: 57.1 MB - Last synced: 18 days ago - Pushed: 20 days ago - Stars: 296 - Forks: 49

ratt-ru/codex-africanus

Radio Astronomy Algorithms Library

Language: Python - Size: 1.42 MB - Last synced: 23 days ago - Pushed: 23 days ago - Stars: 16 - Forks: 10

TGSAI/mdio-python

Cloud native, scalable storage engine for various types of energy data.

Language: Python - Size: 3.48 MB - Last synced: about 7 hours ago - Pushed: 1 day ago - Stars: 28 - Forks: 10

scipp/sciline

Build scientific pipelines for your data

Language: Python - Size: 2.16 MB - Last synced: 20 days ago - Pushed: 20 days ago - Stars: 7 - Forks: 1

xarray-contrib/xeofs

Comprehensive EOF analysis in Python with xarray: A versatile, multidimensional, and scalable tool for advanced climate data analysis

Language: Python - Size: 33.3 MB - Last synced: 19 days ago - Pushed: 20 days ago - Stars: 82 - Forks: 16

gjbex/Python-for-HPC

Repository for participants of the "Python for HPC" training

Language: Jupyter Notebook - Size: 6.63 MB - Last synced: 17 days ago - Pushed: 20 days ago - Stars: 30 - Forks: 18

carpentries-incubator/lesson-parallel-python

Parallel Programming in Python

Language: Python - Size: 5.12 MB - Last synced: 8 days ago - Pushed: about 1 year ago - Stars: 10 - Forks: 14

hi-primus/optimus

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Language: Python - Size: 110 MB - Last synced: 17 days ago - Pushed: about 1 month ago - Stars: 1,441 - Forks: 233

NCAR/ncar-jobqueue

Utilities for configuring dask-jobqueue with appropriate settings for NCAR clusters

Language: Python - Size: 140 KB - Last synced: 16 days ago - Pushed: about 1 month ago - Stars: 12 - Forks: 2

aws-samples/amazon-sagemaker-local-mode

Amazon SageMaker Local Mode Examples

Language: Python - Size: 5.98 MB - Last synced: 21 days ago - Pushed: 21 days ago - Stars: 228 - Forks: 55

msoechting/lexcube

Lexcube: 3D Data Cube Visualization in Jupyter Notebooks

Language: TypeScript - Size: 4.6 MB - Last synced: 18 days ago - Pushed: about 1 month ago - Stars: 89 - Forks: 3

LDO-CERT/orochi

The Volatility Collaborative GUI

Language: JavaScript - Size: 35.7 MB - Last synced: 14 days ago - Pushed: 14 days ago - Stars: 189 - Forks: 17

treebeardtech/kubeflow-bootstrap

🪐 1-click Kubeflow using ArgoCD

Language: Shell - Size: 2.67 MB - Last synced: 24 days ago - Pushed: 24 days ago - Stars: 18 - Forks: 5

anovv/svoe

A scalable, declarative, low-code framework for real-time and batch feature calculation/management (quant finance, anomaly/fraud detection, etc.), predictive ML training/inference and simulation. Built on top of Ray

Language: Python - Size: 78.6 MB - Last synced: 15 days ago - Pushed: 3 months ago - Stars: 15 - Forks: 10

octoenergy/dask-remote 📦

Procurement: Dask Cluster as a Process.

Language: Python - Size: 389 KB - Last synced: 27 days ago - Pushed: almost 2 years ago - Stars: 5 - Forks: 0

aertslab/arboreto

A scalable python-based framework for gene regulatory network inference using tree-based ensemble regressors.

Language: Jupyter Notebook - Size: 63.9 MB - Last synced: 27 days ago - Pushed: 27 days ago - Stars: 45 - Forks: 24

bioio-devs/bioio-czi

A BioIO reader plugin for reading CZI files.

Language: Python - Size: 305 KB - Last synced: about 1 month ago - Pushed: 2 months ago - Stars: 0 - Forks: 0

pangeo-data/climpred

:earth_americas: Verification of weather and climate forecasts :earth_africa:

Language: Python - Size: 58.1 MB - Last synced: 18 days ago - Pushed: 30 days ago - Stars: 217 - Forks: 48

pytroll/pyresample

Geospatial image resampling in Python

Language: Python - Size: 16.4 MB - Last synced: about 1 month ago - Pushed: 2 months ago - Stars: 324 - Forks: 94

facultyai/lens

Summarise and explore Pandas DataFrames

Language: Python - Size: 229 KB - Last synced: about 1 month ago - Pushed: almost 4 years ago - Stars: 102 - Forks: 9

dymaxionlabs/dask-rasterio

Read and write rasters in parallel using Rasterio and Dask

Language: Python - Size: 813 KB - Last synced: 8 days ago - Pushed: over 3 years ago - Stars: 94 - Forks: 8

dask/dask-jobqueue

Deploy Dask on job schedulers like PBS, SLURM, and SGE

Language: Python - Size: 667 KB - Last synced: about 4 hours ago - Pushed: about 1 month ago - Stars: 230 - Forks: 137

CNES/zcollection

Python library allowing to manipulate data splited into a collection of groups stored in Zarr format.

Language: Python - Size: 640 KB - Last synced: 21 days ago - Pushed: about 2 months ago - Stars: 12 - Forks: 3

IsaacCheng9/machine-learning-in-chess

A final year project for the University of Exeter, using machine learning to study patterns in millions of chess games (~350 GB). Ranked 1st in the cohort for undergraduate projects (85%).

Language: Jupyter Notebook - Size: 1.27 GB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 1 - Forks: 1

Becksteinlab/Parallel-analysis-in-the-MDAnalysis-Library

Benchmarking MDAnalysis with Dask (and MPI). Supplementary Information for SciPy 2017 paper.

Language: Python - Size: 78.1 KB - Last synced: about 1 month ago - Pushed: over 6 years ago - Stars: 3 - Forks: 4

timkpaine/paperboy

A web frontend for scheduling Jupyter notebook reports

Language: Python - Size: 12.5 MB - Last synced: 19 days ago - Pushed: about 2 years ago - Stars: 248 - Forks: 26

splunk/deep-learning-toolkit

Deep Learning Toolkit for Splunk

Language: Python - Size: 15.4 MB - Last synced: 18 days ago - Pushed: about 1 month ago - Stars: 15 - Forks: 5

mariusvniekerk/dask-hivemetastore

Language: Python - Size: 122 KB - Last synced: about 1 month ago - Pushed: over 6 years ago - Stars: 1 - Forks: 1

ivanbgd/dask_demo_reins

A Dask library for Big Data processing in Python demo

Language: Python - Size: 11.7 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

NucciTheBoss/DS330_final_project 📦

Visualizing some Spotify data!

Language: Jupyter Notebook - Size: 28.3 KB - Last synced: about 1 month ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 0

AllenCellModeling/aicsimageio

Image Reading, Metadata Conversion, and Image Writing for Microscopy Images in Python

Language: Python - Size: 173 MB - Last synced: 23 days ago - Pushed: about 2 months ago - Stars: 190 - Forks: 49

vbprojects/ukraine_war_sentiment

Measuring how events shape discourse on twitter surrounding the Ukraine War in 2022 using piecewise exponential decay models.

Language: Jupyter Notebook - Size: 230 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0