Fast ML inference & training for ONNX models in Rust
- Rust 98.1%
- Python 0.7%
- TypeScript 0.6%
- JavaScript 0.6%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Reynard User
52c9ec9559
Some checks failed
🧪 Cargo Tests / Run tests-3 (push) Has been cancelled
🧩 Backends / Candle (push) Has been cancelled
🧩 Backends / Candle-1 (push) Has been cancelled
🧩 Backends / Tract (push) Has been cancelled
🧩 Backends / Tract-1 (push) Has been cancelled
🧩 Backends / Tract-2 (push) Has been cancelled
🧩 Backends / Web (push) Has been cancelled
📋 Code Quality / Lint & format (push) Has been cancelled
📋 Code Quality / Code coverage (push) Has been cancelled
📋 Code Quality / docs.rs build (push) Has been cancelled
🔗 Static Linking / Linux (x64) (push) Has been cancelled
🚩 Misc Checks / --no-default-features (push) Has been cancelled
🚩 Misc Checks / MSRV (1.88) (push) Has been cancelled
🚩 Misc Checks / Multiversion (push) Has been cancelled
🧪 Cargo Tests / Run tests (push) Has been cancelled
🧪 Cargo Tests / Run tests-1 (push) Has been cancelled
🧪 Cargo Tests / Run tests-2 (push) Has been cancelled
Signed-off-by: Reynard User <kade@sly.so> |
||
| .cargo | ||
| .github | ||
| .vscode | ||
| backends | ||
| docs | ||
| examples | ||
| ort-sys | ||
| src | ||
| tests | ||
| tools | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| AGENTS.md | ||
| Cargo.toml | ||
| clippy.toml | ||
| codecov.yml | ||
| CONTRIBUTING.md | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| README.md | ||
| rustfmt.toml | ||
| SECURITY.md | ||
| 💖 Sponsored by | |
|---|---|
 |
Authentic AI voice models for enterprise. |
ort is a Rust interface for performing hardware-accelerated inference & training on machine learning models in the Open Neural Network Exchange (ONNX) format.
Based on the now-inactive onnxruntime-rs crate, ort is primarily a wrapper for Microsoft's ONNX Runtime library, but offers support for other pure-Rust runtimes.
ort with ONNX Runtime is super quick - and it supports almost any hardware accelerator you can think of. Even still, it's light enough to run on your users' devices.
When you need to deploy a PyTorch/TensorFlow/Keras/scikit-learn/PaddlePaddle model either on-device or in the datacenter, ort has you covered.
📖 Documentation
🤔 Support
🌠 Backers
💖 FOSS projects using ort
Open a PR to add your project here 🌟
- Text Embeddings Inference (TEI) uses
ortto deliver high-performance ONNX Runtime inference for text embedding models. - Magika uses
ortfor neural network-based file type detection. - retto uses
ortfor reliable, fast ONNX inference of PaddleOCR models on Desktop and WASM platforms. - edge-transformers uses
ortfor accelerated transformer model inference at the edge. sbv2-apiis a fast implementation of Style-BERT-VITS2 text-to-speech usingort.- BoquilaHUB uses
ortfor local AI deployment in biodiversity conservation efforts. - CamTrap Detector uses
ortto detect animals, humans and vehicles in trail camera imagery. - Ortex uses
ortfor safe ONNX Runtime bindings in Elixir. - oar-ocr A comprehensive OCR library, built in Rust with
ortfor efficient inference. FastEmbed-rsusesortfor generating vector embeddings, reranking locally.- Ahnlich uses
ortto power their AI proxy for semantic search applications. - Murmure uses
ortas its core engine, leveraging NVIDIA Parakeet to deliver fully local, free, private and cross‑platform Speech‑to‑Text enhanced with LLM post‑processing. - Valentinus uses
ortto provide embedding model inference inside LMDB. - SilentKeys uses
ortfor fast, on-device real-time dictation with NVIDIA Parakeet and Silero VAD.