GitHub topics: aws-batch
gps31320779/insightflow-retail-economic-pipeline
A data engineering portfolio project using AWS cloud services to analyze correlations between Malaysian retail performance and fuel prices. Features Terraform IaC, ETL/ELT with AWS S3, Glue, SQL analytics via Athena coupled with data transformation via dbt, and workflow orchestration with Kestra.
Size: 8.79 KB - Last synced at: about 2 hours ago - Pushed at: about 4 hours ago - Stars: 0 - Forks: 0
NIGMS/Transcriptome-Assembly-Refinement-and-Applications
In this module, you will learn how to use a Nextflow pipeline to assemble and annotate a novel transcriptome using RNA-seq data
Language: Jupyter Notebook - Size: 6.16 MB - Last synced at: about 23 hours ago - Pushed at: about 24 hours ago - Stars: 0 - Forks: 4
wlandau/crew.aws.batch
A {crew} launcher plugin for AWS Batch
Language: R - Size: 6.01 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 16 - Forks: 1
aws-samples/aws-batch-operational-dashboards
Language: Python - Size: 2.46 MB - Last synced at: 2 days ago - Pushed at: 10 days ago - Stars: 13 - Forks: 2
ohsu-comp-bio/funnel
Funnel is a toolkit for distributed task execution via a simple, standard API.
Language: Go - Size: 57.6 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 129 - Forks: 36
pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Language: Python - Size: 34.1 MB - Last synced at: 3 days ago - Pushed at: 16 days ago - Stars: 366 - Forks: 132
aws-samples/quant-research-sample-using-amazon-ecs-and-aws-batch
This project deploys an AWS Batch infrastructure for Quant Research.
Language: Python - Size: 38.1 KB - Last synced at: 2 days ago - Pushed at: about 2 months ago - Stars: 3 - Forks: 2
NIGMS/Metagenomics-Analysis-of-Biofilm-Microbiome
In this module, you will learn how to use a Docker container to analyze amplicon sequencing metagenomics data with common tools such as qiime2 and PICRUSt2
Language: HTML - Size: 50 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 8 - Forks: 4
aws-samples/amazon-redshift-serverless-rsql-etl-framework
Amazon Redshift Serverless RSQL ETL Framework
Language: TypeScript - Size: 1.13 MB - Last synced at: 2 days ago - Pushed at: 2 months ago - Stars: 10 - Forks: 2
Ditectrev/Amazon-Web-Services-Certified-AWS-Certified-Machine-Learning-MLS-C01-Practice-Tests-Exams-Question
⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.
Size: 6.88 MB - Last synced at: 19 days ago - Pushed at: 8 months ago - Stars: 63 - Forks: 38
lynnlangit/aws-for-bioinformatics
AWS for Bioinformatics Researchers
Language: Jupyter Notebook - Size: 118 MB - Last synced at: 6 days ago - Pushed at: 10 months ago - Stars: 148 - Forks: 35
terraform-aws-modules/terraform-aws-batch
Terraform module to create AWS Batch resources 🇺🇦
Language: HCL - Size: 49.8 KB - Last synced at: 2 days ago - Pushed at: 8 months ago - Stars: 37 - Forks: 41
pizofreude/insightflow-retail-economic-pipeline
A data engineering portfolio project using AWS cloud services to analyze correlations between Malaysian retail performance and fuel prices. Features Terraform IaC, ETL/ELT with AWS S3, Glue, SQL analytics via Athena coupled with data transformation via dbt, and workflow orchestration with Kestra.
Language: HCL - Size: 672 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0
fulcrumgenomics/pyfgaws
Python tools and APIs for working with AWS
Language: Python - Size: 88.9 KB - Last synced at: 25 days ago - Pushed at: about 2 months ago - Stars: 9 - Forks: 0
ryfeus/stepfunctions2processing
Configuration with AWS step functions and lambdas which initiates processing from activity state
Language: Python - Size: 23.2 MB - Last synced at: 6 days ago - Pushed at: 8 days ago - Stars: 119 - Forks: 30
aws-samples/aws-step-function-batch
This project is an example of using AWS Step functions to manage and track a series of AWS Batch jobs in N_TO_N mode.
Language: TypeScript - Size: 496 KB - Last synced at: 2 days ago - Pushed at: 4 months ago - Stars: 13 - Forks: 5
aws-cloudformation/aws-cloudformation-resource-providers-batch
The CloudFormation Resource Provider Package For AWS Batch.
Language: Java - Size: 54.7 KB - Last synced at: 6 days ago - Pushed at: over 3 years ago - Stars: 2 - Forks: 0
kislyuk/aegea
Amazon Web Services Operator Interface
Language: Python - Size: 2.09 MB - Last synced at: 19 days ago - Pushed at: 3 months ago - Stars: 68 - Forks: 16
aksh-patel1/parallel-web-scraper-on-cloud
This project demonstrates an event-driven architecture for parallel web scraping and processing tasks using AWS services. The scraper job, running on AWS Batch, collects data from multiple web pages simultaneously and stores it in S3. The processing job, triggered by AWS EventBridge, efficiently processes the scraped data and updates Google-Sheet.
Language: Python - Size: 9.77 KB - Last synced at: 4 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0
aws-samples/aws-batch-runtime-monitoring
Serverless application to monitor an AWS Batch architecture through dashboards.
Language: Python - Size: 631 KB - Last synced at: 2 days ago - Pushed at: 7 months ago - Stars: 61 - Forks: 16
ales-t/batchman
AWS Batch job explorer and manager
Language: Python - Size: 64.5 KB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0
randyridgley/awesome-aws-research
A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources for Academic Researchers new to AWS
Size: 58.6 KB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 42 - Forks: 11
aws-samples/aws-backup-actions-framework
AWS Backup Actions is a framework for automating actions triggered by AWS Backup events.
Language: TypeScript - Size: 31.3 KB - Last synced at: 2 days ago - Pushed at: over 2 years ago - Stars: 5 - Forks: 1
petergtz/g2s3
G2S3 is a solution to regularly back up your Google data by copying it over to S3.
Language: Rust - Size: 737 KB - Last synced at: about 2 months ago - Pushed at: 9 months ago - Stars: 24 - Forks: 5
JBris/aws-botorch-torchx-nas
Testing submission of AWS batch jobs using BoTorch (via Ax) and TorchX for neural architecture search
Language: HTML - Size: 2.92 MB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0
aws-samples/aws-cdk-deep-learning-image-vector-embeddings-at-scale-using-aws-batch
AWS Blog post code for running feature-extraction on images using AWS Batch and Cloud Development Kit (CDK).
Language: Python - Size: 12.7 KB - Last synced at: 2 days ago - Pushed at: over 2 years ago - Stars: 20 - Forks: 12
JBris/aws-crew-r
Testing submission of AWS batch jobs using the crew package in R
Language: R - Size: 678 KB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0
dmrhimali/terraform
Tutorial on how to create and run terraform scripts for providers aws and newrelic
Language: HCL - Size: 20.6 MB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0
Brender-Studio/brender-snippets
Brender Snippets is a collection of custom scripts that enhance rendering in Brender Studio.
Size: 249 MB - Last synced at: about 2 months ago - Pushed at: 10 months ago - Stars: 3 - Forks: 0
aws-samples/batch-mpi-examples
Examples for running multi-node parallel (MNP) MPI jobs on AWS Batch.
Language: Shell - Size: 73.2 KB - Last synced at: 2 days ago - Pushed at: almost 2 years ago - Stars: 10 - Forks: 0
Brender-Studio/brender-studio-devcontainer
Docker-based local development environment for Brender Studio. Mirrors AWS Batch setup for testing Blender rendering workflows before cloud deployment.
Language: Python - Size: 1.92 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 0
lukewaite/laravel-queue-aws-batch
A Laravel queue connector for AWS Batch
Language: PHP - Size: 64.5 KB - Last synced at: 3 days ago - Pushed at: over 2 years ago - Stars: 9 - Forks: 17
vt-digital-libraries-platform/aws-batch-iiif-generator
A project that demonstrates the use of AWS Batch to create IIIF tiles and manifests. It is used for VTDLP Derivative Service
Language: Python - Size: 14.7 MB - Last synced at: 30 days ago - Pushed at: 9 months ago - Stars: 9 - Forks: 2
NikiforovAll/aws-batch-dotnet
Learn how to leverage AWS Batch to efficiently process files in S3 using .NET
Language: C# - Size: 176 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0
aws-samples/serverless-batch-cost-guardian
Code for Serverless Batch Cost Guardian AWS HPC Blog Post
Language: Python - Size: 432 KB - Last synced at: 2 days ago - Pushed at: almost 2 years ago - Stars: 6 - Forks: 0
parlaynu/studio-aws-batch
Create an AWS Batch cluster with supporting infrastructure and tooling ready to do real work.
Language: HCL - Size: 335 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 1
Andresmup/aws-batch-fargate
AWS Batch data processing showcase using Docker on ECS Fargate with GitHub Actions CI/CD!
Language: Python - Size: 2.76 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
ImerM/sag_aws
Repo that runs a SAG assembly pipeline on AWS, just run the bash script and you are good to go.
Language: Python - Size: 55.7 KB - Last synced at: about 1 year ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0
skyFrankie/Canada-Supermarket-Groceries-Data-Web-Scrapping
This is a project demonstrating a scheduled web scrapping system in AWS to store the supermarket data into AWS hosted database for further analysis
Language: Python - Size: 69.7 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
perfectbase/typescript-cli
A Dockerized TypeScript CLI built with Commander to be used for running batch jobs on AWS.
Language: TypeScript - Size: 35.2 KB - Last synced at: 2 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0
vijayanandrp/terraform-example-aws-batch-lambda-demo
Create a container using Docker and create Batch job and trigger from Python with environmental variables or runtime values
Language: HCL - Size: 178 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
aws-samples/aws-rosettafold
Infrastructure template and Jupyter notebooks for running RoseTTAFold on AWS Batch.
Language: Python - Size: 26 MB - Last synced at: 2 days ago - Pushed at: over 3 years ago - Stars: 22 - Forks: 2
yu-croco/sample_aws_batch_terraform
This is a sample Terraform to build AWS Batch resource
Language: HCL - Size: 131 KB - Last synced at: 3 months ago - Pushed at: about 4 years ago - Stars: 2 - Forks: 1
AgnostiqHQ/covalent-awsbatch-plugin
Executor plugin interfacing Covalent with AWS Batch
Language: Python - Size: 479 KB - Last synced at: 8 days ago - Pushed at: 2 months ago - Stars: 11 - Forks: 2
AgnostiqHQ/covalent-aws-plugins
Executor plugins interfacing Covalent with various AWS compute platforms
Language: Python - Size: 499 KB - Last synced at: 10 days ago - Pushed at: over 1 year ago - Stars: 17 - Forks: 0
OElesin/project-matt
Project Matt: Scan your AWS S3 Buckets for PII Data to Guard against GDPR
Language: Scala - Size: 890 KB - Last synced at: about 1 year ago - Pushed at: about 7 years ago - Stars: 14 - Forks: 5
knowsuchagency/aws-container-demo
Building containers that run on AWS Lambda and AWS Batch
Language: Python - Size: 26.4 KB - Last synced at: 4 days ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
jer-nc/cdk-efs-batch-s3
An AWS CDK project enabling file transfer and management between Amazon EFS, AWS Batch, and S3.
Language: TypeScript - Size: 306 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
webmapp/gdal-docker
GDAL Docker
Size: 32.2 KB - Last synced at: over 1 year ago - Pushed at: almost 7 years ago - Stars: 9 - Forks: 3
jer-nc/cdk-aws-batch-typescript
AWS Batch - Docker Hub Public Image - CDK Typescript
Language: TypeScript - Size: 40 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
thinklumo/batch-cli
Command line wrapper for AWS Batch
Language: Python - Size: 8.79 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
ivansabik/aws-step-functions-batch-memory-auto-scaling-example
Sample code for running an AWS Batch job that automatically scales up required resources (VCPUs and Memory)
Language: Python - Size: 63.5 KB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
MediaMath/lambda-cron
LambdaCron - serverless cron tool
Language: Python - Size: 302 KB - Last synced at: 17 days ago - Pushed at: over 7 years ago - Stars: 25 - Forks: 4
CityOfPhiladelphia/taskflow 📦
An advanced yet simple system to run your background tasks and workflows
Language: Python - Size: 96.7 KB - Last synced at: about 1 year ago - Pushed at: over 7 years ago - Stars: 19 - Forks: 1
aws-samples/automated-language-analysis
This repository contains a CDK project that deploys a fully automated language analysis pipeline.
Language: Python - Size: 2.36 MB - Last synced at: 2 days ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 5
csiro-crop-informatics/repset
Reproducible evaluation of short read mappers
Language: Nextflow - Size: 3.37 MB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 2 - Forks: 3
nfultz/aws-batch 📦
A jenkins plugin for submitting jobs to AWS batch // use the official one now
Language: Java - Size: 97.7 KB - Last synced at: about 1 year ago - Pushed at: about 7 years ago - Stars: 7 - Forks: 13
jl-n/aws-cdk-batch-ts
Typescript client with Lambda backend and S3 Bucket for running jobs on AWS Batch with Fargate.
Language: TypeScript - Size: 1.15 MB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0
botisan-ai/cuda-python
CUDA-powered Python Docker Image for AWS Batch
Language: Dockerfile - Size: 8.79 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 1
RafalWilinski/deploy-with-serverless
Deploy Serverless Framework architectures with a single click, no terminal required
Language: JavaScript - Size: 726 KB - Last synced at: 10 days ago - Pushed at: over 7 years ago - Stars: 39 - Forks: 3
mattarderne/meltano-on-aws-batch
Running Meltano ELT on AWS Batch, infra with Terraform
Language: HCL - Size: 50.8 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 0
dawngerpony/aws-batch-poc 📦
A proof of concept demonstrating the use of AWS Batch
Language: HCL - Size: 15.6 KB - Last synced at: about 4 hours ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0
rbreslow/aws-batch-gpu-ephemeral-storage 📦
Example project exercising AWS Batch on Terraform 0.12, launch templates, and the ECS GPU-optimized AMI.
Language: HCL - Size: 5.86 KB - Last synced at: 12 months ago - Pushed at: almost 6 years ago - Stars: 0 - Forks: 0
dejonghe/aws-batch-example
Example use of AWS batch
Language: Python - Size: 81.1 KB - Last synced at: over 2 years ago - Pushed at: about 4 years ago - Stars: 96 - Forks: 37
OpenKBC/multiple_sclerosis_AWSmodule
This repository contains AWS module for MS project in OpenKBC
Language: Shell - Size: 1.05 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0
jkahn117/aws-batch-image-processor
Simple sample of AWS Batch
Language: HCL - Size: 137 KB - Last synced at: over 2 years ago - Pushed at: about 7 years ago - Stars: 26 - Forks: 20
coding-chris-kao/nestjs-aws-batch-cdk
Language: TypeScript - Size: 161 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
ardacetinkaya/some-aws-journey
Language: HCL - Size: 4.74 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
nhatthaiquang-agilityio/kedro-aws-batch
Run a Kedro project on AWS Batch
Language: Python - Size: 1.59 MB - Last synced at: about 2 months ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 1
petedannemann/aws-batch-demo
A simple demo of using Airflow to submit AWS Batch jobs
Language: Python - Size: 122 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 5 - Forks: 2
yukihira1992/aws-batch-template
Terraform code template for AWS Batch.
Language: HCL - Size: 5.86 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 5 - Forks: 2
CQYaDoKaRi/aws.batch
AWS Batch
Language: Shell - Size: 13.7 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
OpenKBC/deg-pipeline-batch-image
Language: Python - Size: 11.7 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
OpenKBC/feature-extraction-batch-image
Language: Python - Size: 39.1 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
hisplan/cromwell-gwf-log
Language: Python - Size: 19.5 KB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 0
tosh2230/practice-aws-batch
AWS Batch on AWS Fargate, calling from AWS Step Functions.
Language: Shell - Size: 29.3 KB - Last synced at: 6 days ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 1
fmdlc/terraform-module-aws-batch
:alien: Terraform module for AWS Batch.
Language: HCL - Size: 31.3 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 2 - Forks: 7
graveyard/batchcli
Utility to manage inputs and outputs of batch worker executions
Language: Go - Size: 5.96 MB - Last synced at: 12 months ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0
crosenth/aws_batch
Moves data to and from AWS Batch system and monitors logs
Language: Python - Size: 48.8 KB - Last synced at: 4 days ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
karvounis/terraform-aws-batch-container-properties
Terraform module that generates the container properties object to use in AWS Batch.
Language: HCL - Size: 23.4 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 5 - Forks: 4
FanWangEcon/pyfan
A research support tools package.
Language: Python - Size: 18.6 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 2
nfultz/AWR.Batch
R package for submitting jobs to AWS Batch
Language: R - Size: 2.93 KB - Last synced at: 3 months ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0
iamlmn/terraform-aws-batch
High Throughput Batch Computing in AWS - Terraform module
Language: HCL - Size: 209 KB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 4
zowens/aws_batch_csv_processor
Sample AWS Batch project to read CSV files
Language: HCL - Size: 6.84 KB - Last synced at: about 2 years ago - Pushed at: over 7 years ago - Stars: 10 - Forks: 5
cmdoret/vapormap
Cloud based pipeline for parallelized iterative alignment of Hi-C data
Language: Nextflow - Size: 74.4 MB - Last synced at: 22 days ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
Christianivh/aws-batch-testing
Guideline para AWS Batch
Language: Python - Size: 12.7 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0
srinivasreddych/awsbatchjob-failuretracker
This solution is to alert on every AWS Batch job failure
Size: 1.95 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0
kuhnuri/kuhnuri-cdk
Kuhnuri server environment for AWS
Language: TypeScript - Size: 1.61 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
anddimario/tuttofare
Get metrics, run commands, parse logs and do checks on servers using aws serverless features
Language: Python - Size: 109 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
FredHutch/batch-pi
Calculate PI using Monte Carlo simulation and AWS Batch Processing
Language: Python - Size: 16.6 KB - Last synced at: 2 months ago - Pushed at: about 8 years ago - Stars: 1 - Forks: 0
ResearchKernel/aws-scripts
Size: 42 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0
adamstruck/ebsmount
Mount and unmount EBS volumes on an EC2 instance
Language: Go - Size: 28.3 KB - Last synced at: 12 months ago - Pushed at: about 7 years ago - Stars: 0 - Forks: 0
rgfindl/aws-genomics-workflow
Move your genomic sequencing analysis into the cloud. Cost effective, pay for what you use. Unlimited scaling for high throughput.
Language: Shell - Size: 44.9 KB - Last synced at: about 2 years ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0
supinf/aws-batch-refarch
A reference architecture for deploying AWS Batch with CloudFormation
Size: 6.84 KB - Last synced at: about 2 years ago - Pushed at: over 8 years ago - Stars: 6 - Forks: 0
pottava/mxnet-char-lstm
A multilayer LSTM network with MXNet.
Language: Python - Size: 36.1 KB - Last synced at: about 2 months ago - Pushed at: about 8 years ago - Stars: 2 - Forks: 0
