KerasFormers is a collection of models with pretrained weights, built entirely with Keras 3. It supports a range of tasks, including classification, object detection (DETR, RT-DETR, RT-DETRv2, RF-DETR, D-FINE, OWL-ViT, OWLv2, Grounding DINO), segmentation (SAM, SAM2, SAM3, SegFormer, DeepLabV3, EoMT, MaskFormer, Mask2Former, OneFormer, MobileViT-DeepLabV3, RF-DETR), monocular depth estimation (Depth Anything V1, Depth Anything V2), feature extraction (DINO, DINOv2, DINOv3), vision-language modeling (CLIP, SigLIP, SigLIP2, MetaCLIP 2), speech recognition (Whisper, Speech2Text, Moonshine), speech-aware language modeling (Granite Speech, Granite Speech Plus), text encoding and masked language modeling (BERT, RoBERTa, XLM-RoBERTa, DeBERTa, DeBERTa-v2, DeBERTa-v3), text generation with large language models (GPT, GPT-2, Qwen2, Qwen2-MoE, Qwen3, Qwen3-MoE, Qwen3.5, Qwen3.5-MoE, GPT-OSS, Llama 2, Llama 3, Llama 4, Mistral, Mixtral, Gemma, Gemma 2, Gemma 4, MiniMax-Text-01, MiniMax-M2, DeepSeek-V2, DeepSeek-V3, DeepSeek-V4, Cohere/Command-R, Cohere2, Cohere2-MoE, GLM-4, GLM-4-0414, GLM-4.5/GLM-4.6, GLM-5/GLM-5.1/GLM-5.2), multimodal vision-language generation (Qwen2-VL, Qwen2.5-VL, Qwen3-VL, InternVL3, Gemma 3, Mistral 3, DeepSeek-VL, Janus-Pro, MiniMax-M3-VL, Cohere2-Vision, GLM-4V, GLM-4.5V, Kimi K2.5, Kimi K2.6, Kimi K2.7-Code), vision-language grounding across object detection, OCR, pointing, and referring (LocateAnything), and more. It includes hybrid architectures like MaxViT alongside traditional CNNs and pure transformers. kerasformers includes custom layers and backbone support, providing flexibility and efficiency across various applications. For backbones, there are various weight variants like in1k, in21k, fb_dist_in1k, ms_in22k, fb_in22k_ft_in1k, ns_jft_in1k, aa_in1k, cvnets_in1k, augreg_in21k_ft_in1k, augreg_in21k, and many more.
From PyPI (recommended)
pip install -U kerasformersFrom Source
pip install -U git+https://github.com/IMvision12/KerasFormersPer-model guides - with architecture notes, usage examples, and available pretrained weights, live in the docs/ folder, one page per model across every supported task (classification, object detection, segmentation, depth estimation, feature extraction, vision-language, speech recognition, text encoding, and language modeling). Classification backbones share a single page since they all follow the same XModel / XImageClassify two-class structure; each other model has its own. Browse docs/ for the complete, always-up-to-date list.
-
Text Encoders (text → embeddings, masked LM, classification)
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights BERT BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding transformersRoBERTa RoBERTa: A Robustly Optimized BERT Pretraining Approach transformersXLM-RoBERTa Unsupervised Cross-lingual Representation Learning at Scale transformersDeBERTa DeBERTa: Decoding-enhanced BERT with Disentangled Attention transformersDeBERTa-v2 DeBERTa: Decoding-enhanced BERT with Disentangled Attention transformersDeBERTa-v3 DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing transformers -
Text LLMs (text → text)
-
Backbones
-
Object Detection
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights D-FINE D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement transformersDETR End-to-End Object Detection with Transformers transformersRT-DETR DETRs Beat YOLOs on Real-time Object Detection transformersRT-DETRv2 RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformers transformersRF-DETR RF-DETR: Neural Architecture Search for Real-Time Detection Transformers transformersOWL-ViT Simple Open-Vocabulary Object Detection with Vision Transformers transformersOWLv2 Scaling Open-Vocabulary Object Detection transformersGrounding DINO Marrying DINO with Grounded Pre-Training for Open-Set Object Detection transformers
-
Segmentation
-
Feature Extraction
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights DINO Emerging Properties in Self-Supervised Vision Transformers torch.hubDINOv2 DINOv2: Learning Robust Visual Features without Supervision transformersDINOv3 DINOv3: Self-Supervised Visual Representation Learning at Scale transformers(gated)
-
Depth Estimation
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights Depth Anything V1 Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data transformersDepth Anything V2 Depth Anything V2 transformers
-
Vision-Language Encoders
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights CLIP Learning Transferable Visual Models From Natural Language Supervision transformersMetaCLIP 2 MetaCLIP 2: A Worldwide Scaling Recipe transformersSigLIP Sigmoid Loss for Language Image Pre-Training transformersSigLIP2 SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features transformers
-
Multimodal LLMs (image + text → text)
-
Vision-Language Grounding (object detection, OCR, pointing, referring)
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights LocateAnything LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding transformers
-
Speech (speech → text)
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights Whisper Robust Speech Recognition via Large-Scale Weak Supervision transformersSpeech2Text fairseq S2T: Fast Speech-to-Text Modeling with fairseq transformersMoonshine Moonshine: Speech Recognition for Live Transcription and Voice Commands transformers
-
Speech LLMs (audio + text → text)
🏷️ Model Name 📜 Reference Paper 📦 Source of Weights Granite Speech Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities transformersGranite Speech Plus ibm-granite/granite-speech-4.1-2b-plus transformers
This project leverages timm and transformers for converting pretrained weights from PyTorch to Keras. For licensing details, please refer to the respective repositories.
- 🔖 kerasformers Code: This repository is licensed under the Apache 2.0 License.
- The Keras team for their powerful and user-friendly deep learning framework
- The Transformers library for its robust tools for loading and adapting pretrained models
- The pytorch-image-models (timm) project for pioneering many computer vision model implementations
- All contributors to the original papers and architectures implemented in this library
@misc{gc2025kerasformers,
author = {Gitesh Chawda},
title = {KerasFormers},
year = {2025},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/IMvision12/KerasFormers}}