An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: multimodal-fusion

mmu-dermatology-research/multimodal-hardnet

Source code for the paper: "Multimodal HarDNet for Enhanced Weakly Supervised Chronic Wound Segmentation"

Size: 0 Bytes - Last synced at: 2 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

icey-zhang/SuperYOLO

SuperYOLO is accepted by TGRS

Language: Python - Size: 16.5 MB - Last synced at: 5 days ago - Pushed at: 8 months ago - Stars: 403 - Forks: 63

mmu-dermatology-research/multimodal-grf

Source code for the paper: "Gaussian Random Fields as an Abstract Representation of Patient Metadata for Multimodal Medical Image Segmentation"

Size: 8.79 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

Sean2CS/RTGD-MVC

This repository provides the official MATLAB implementation for the paper "RTGD-MVC: Robust Tensor Learning with Graph Diffusion for Scalable Multi-view Graph Clustering".

Language: MATLAB - Size: 2.92 MB - Last synced at: 13 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

declare-lab/Multimodal-Infomax

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

Language: Python - Size: 145 KB - Last synced at: 18 days ago - Pushed at: about 2 years ago - Stars: 179 - Forks: 34

mahmoodlab/MCAT

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images - ICCV 2021

Language: Jupyter Notebook - Size: 540 MB - Last synced at: 17 days ago - Pushed at: about 3 years ago - Stars: 195 - Forks: 39

pacocp/Med-CrossViT

Med-CrossViT: A Transformer-based architecture for WSI and RNA-Seq data fusion

Language: Python - Size: 57.6 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

EesunMoon/On-device_Multimodal_ER

[Research - MINES Lab] Multimodal Emotion Recognition for On-device AI

Language: Python - Size: 56.6 KB - Last synced at: 14 days ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

AlfredsLapkovskis/MultimodalPlantClassifier

Source code for the paper "Automatic Fused Multimodal Deep Learning for Plant Identification" (Alfreds Lapkovskis, Natalia Nefedova & Ali Beikmohammadi, 2024)

Language: Jupyter Notebook - Size: 572 MB - Last synced at: about 2 months ago - Pushed at: 3 months ago - Stars: 5 - Forks: 1

fatemafaria142/BanglaCalamityMMD-A-Comprehensive-Benchmark-Dataset-for-Multimodal-Disaster-Identification Fork of Mukaffi28/BanglaCalamityMMD-A-Comprehensive-Benchmark-Dataset-for-Multimodal-Disaster-Identification

This study presents a novel multimodal fusion technique for disaster identification in Bangla, combining text and image data using the "BanglaCalamityMMD" dataset. Employing DisasterTextNet, DisasterImageNet, and DisasterMultFusionNet, the approach addresses a key gap in Bangla disaster research.

Language: Jupyter Notebook - Size: 290 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

icey-zhang/E2E-MFD-HOD

E2E-MFD-HOD

Language: Python - Size: 1.71 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 13 - Forks: 1

icey-zhang/E2E-MFD

E2E-MFD-OOD

Language: Jupyter Notebook - Size: 21.8 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 54 - Forks: 3

ai-forever/fusion_brain_aij2021

Creating multimodal multitask models

Language: Jupyter Notebook - Size: 4.29 MB - Last synced at: 6 days ago - Pushed at: over 2 years ago - Stars: 50 - Forks: 15

v-iashin/BMT

Source code for "Bi-modal Transformer for Dense Video Captioning" (BMVC 2020)

Language: Jupyter Notebook - Size: 11.1 MB - Last synced at: 12 days ago - Pushed at: about 2 years ago - Stars: 227 - Forks: 57

AlfredsLapkovskis/MultimodalPlantClassifier-iOS

Source code of a sample iOS app for the paper "Automatic Fused Multimodal Deep Learning for Plant Identification" (Alfreds Lapkovskis, Natalia Nefedova & Ali Beikmohammadi, 2024)

Language: Swift - Size: 21.9 MB - Last synced at: about 2 months ago - Pushed at: 5 months ago - Stars: 3 - Forks: 1

shengyangsun/MSBT

Official implementation of "Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection"

Language: Python - Size: 54.4 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 11 - Forks: 2

declare-lab/hfusion

Multimodal sentiment analysis using hierarchical fusion with context modeling

Language: Python - Size: 1.55 MB - Last synced at: 12 days ago - Pushed at: about 2 years ago - Stars: 44 - Forks: 22

declare-lab/M2H2-dataset

This repository contains the dataset and baselines explained in the paper: M2H2: A Multimodal Multiparty Hindi Dataset For HumorRecognition in Conversations

Language: Python - Size: 2.21 GB - Last synced at: 12 days ago - Pushed at: about 2 years ago - Stars: 18 - Forks: 12

zzbn12345/Climate_Stance_Multimodal

The code and data for the Paper 'Inferring Climate Change Stances from Multimodal Tweets' accepted by the Short Paper track of SIGIR 2024

Language: Jupyter Notebook - Size: 245 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 4 - Forks: 0

ivanovsdesign/information_retrieval

Web scraper for Wildberries + simple vectorization/multimodal embedding workflow

Language: Jupyter Notebook - Size: 3.88 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

usc-sail/mica-context-emotion-recognition

Repository for context based emotion recognition

Language: Python - Size: 45.9 KB - Last synced at: 12 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

anisha0325/MM-CliConSummation Fork of NLP-RL/MM-CliConSummation

The codebase for our paper on Multi-modal Medical Dialogue Summarization

Size: 1.6 MB - Last synced at: 12 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

gholste/breast_mri_fusion

[CVAMD 2021] "End-to-End Learning of Fused Image and Non-Image Feature for Improved Breast Cancer Classification from MRI"

Language: Python - Size: 366 MB - Last synced at: 11 months ago - Pushed at: over 1 year ago - Stars: 27 - Forks: 7

kaykobad/MMSFormer Fork of CSIPlab/MMSFormer

We propose Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates a novel fusion strategy to perform multimodal material segmentation.

Language: Python - Size: 720 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

Clealiya/Multimodal-model

[FR|EN - Trio] 2023 - 2024 Centrale Méditerranée AI Master | Multimodal retranscription with text, audio and video

Language: Python - Size: 15.5 MB - Last synced at: 12 months ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 0

imadhou/multimodal-sentiment-analysis

Multimodal sentiment analysis

Language: Jupyter Notebook - Size: 1.34 MB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 8 - Forks: 2

brian-zZZ/Guided-PLI

A Transferability-guided Protein-Ligand Interaction Prediction Method

Language: Python - Size: 28.3 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

ai-forever/fbc2_aij2022

FusionBrain Challenge 2.0: creating multimodal multitask model

Language: Python - Size: 26.4 MB - Last synced at: 6 days ago - Pushed at: over 2 years ago - Stars: 16 - Forks: 1

sustainable-computing/Centaur

Repo for "Centaur: Robust Multimodal Fusion for Human Activity Recognition"

Language: Jupyter Notebook - Size: 637 MB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 1

thuiar/MIntRec

MIntRec: A New Dataset for Multimodal Intent Recognition (ACM MM 2022)

Language: Python - Size: 1.49 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 53 - Forks: 8

akashe/Multimodal-action-recognition

Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.

Language: Python - Size: 64.7 MB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 69 - Forks: 11

marcomoldovan/multimodal-self-distillation

A generalized self-supervised training paradigm for unimodal and multimodal alignment and fusion.

Language: Python - Size: 526 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 2

marcomoldovan/kg-augmented-language-modeling

Leveraging knowledge graphs to learn a more factually grounded language model for retrieval and question answering downstream tasks.

Language: Python - Size: 456 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 2 - Forks: 0

Asichurter/MalFusionFSL

Few-Shot malware classification using fused features of static analysis and dynamic analysis (基于静态+动态分析的混合特征的小样本恶意代码分类框架)

Language: Python - Size: 2.11 MB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 10 - Forks: 0

sverma88/Deep-HOSeq--ICDM-2020

Deep-HOSeq: Deep Higher-Order Sequence Fusion for Multimodal Sentiment Analysis.

Language: Python - Size: 383 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 7 - Forks: 4

Related Keywords
multimodal-fusion 35 deep-learning 7 multimodal-deep-learning 7 multimodal-learning 5 transformer 3 pytorch 3 emotion-recognition 3 multimodal 3 object-detection 3 multimodal-sentiment-analysis 3 artificial-intelligence 2 architecture-search 2 fusion-automation 2 neural-architecture-search 2 plant-classification 2 plant-identification 2 dataset 2 multimodality 2 machine-learning 2 computer-vision 2 sentiment-analysis 2 chronic-wounds 2 medical-image-segmentation 2 gaussian-random-field 2 feature-fusion-network 2 nlp 1 twitter-sentiment-analysis 1 sentiment-classification 1 multimodal-classification 1 ai 1 segmentation 1 breast-cancer 1 summarization 1 medical-image-processing 1 context-understanding 1 vectorization 1 siglip 1 selenium 1 scraper 1 embeddings 1 stance-detection 1 stance-dataset 1 tensor-factorization 1 stance-classification 1 sealevelrise 1 dutch-language 1 climate-change 1 classification 1 humor-detection 1 emotion-recognition-in-conversation 1 multimodal-interactions 1 knowledge-graph 1 nlu 1 question-answering 1 information-retrieval 1 self-supervised-learning 1 few-shot-learning 1 self-distillation 1 multimodal-retrieval 1 multimodal-alignment 1 multimodal-data 1 multimodal-action-recognition 1 cross-attention 1 speaker-recognition 1 multimodal-intent-analysis 1 few-shot-malware-classification 1 fused-features 1 malware 1 malware-classification 1 simple 1 convolutional-neural-networks 1 acm-mm-22 1 acm-mm 1 sensor-faults 1 human-activity-recognition 1 tensor 1 multitask-learning 1 transferability 1 representation-learning 1 protein-ligand-interactions 1 fusion 1 mbert 1 disaster-identification 1 benchmarking 1 benchmark 1 banglacalamitymmd 1 bangla-dataset 1 wearable-devices 1 tensorflow 1 speech-recognition 1 speech-processing 1 python 1 on-device 1 npu 1 heart-rate-analysis 1 embedded-systems 1 data-analysis 1 rna-seq 1 multimodal-deeplearning 1 digital-pathology 1