license: mit
datasets:
- wikimedia/wikipedia
- bookcorpus/bookcorpus
- SetFit/mnli
- sentence-transformers/all-nli
language:
- en
new_version: v1.1
base_model:
- google-bert/bert-base-uncased
pipeline_tag: text-classification
tags:
- BERT
- MNLI
- NLI
- transformer
- pre-training
- nlp
- tiny-bert
- edge-ai
- transformers
- low-resource
- micro-nlp
- quantized
- iot
- wearable-ai
- offline-assistant
- intent-detection
- real-time
- smart-home
- embedded-systems
- command-classification
- toy-robotics
- voice-ai
- eco-ai
- english
- lightweight
- mobile-nlp
metrics:
- accuracy
- f1
- inference
- recall
library_name: transformers
🌟 bert-lite: A Lightweight BERT for Efficient NLP 🌟
🚀 Overview
Meet bert-lite—a streamlined marvel of NLP! 🎉 Designed with efficiency in mind, this model features a compact architecture tailored for tasks like MNLI and NLI, while excelling in low-resource environments. With a lightweight footprint, bert-lite is perfect for edge devices, IoT applications, and real-time NLP needs. 🌍
🌟 Why bert-lite? The Lightweight Edge
- 🔍 Compact Power: Optimized for speed and size
- ⚡ Fast Inference: Blazing quick on constrained hardware
- 💾 Small Footprint: Minimal storage demands
- 🌱 Eco-Friendly: Low energy consumption
- 🎯 Versatile: IoT, wearables, smart homes, and more!
🧠 Model Details
| Property | Value |
|---|---|
| 🧱 Layers | Custom lightweight design |
| 🧠 Hidden Size | Optimized for efficiency |
| 👁️ Attention Heads | Minimal yet effective |
| ⚙️ Parameters | Ultra-low parameter count |
| 💽 Size | Quantized for minimal storage |
| 🌐 Base Model | google-bert/bert-base-uncased |
| 🆙 Version | v1.1 (April 04, 2025) |
📜 License
MIT License — free to use, modify, and share.
🔤 Usage Example – Masked Language Modeling (MLM)
from transformers import pipeline
# 📢 Start demo
print("\n🔤 Masked Language Model (MLM) Demo")
# 🧠 Load masked language model
mlm_pipeline = pipeline("fill-mask", model="bert-base-uncased")
# ✍️ Masked sentences
masked_sentences = [
"The robot can [MASK] the room in minutes.",
"He decided to [MASK] the project early.",
"This device is [MASK] for small tasks.",
"The weather will [MASK] by tomorrow.",
"She loves to [MASK] in the garden.",
"Please [MASK] the door before leaving.",
]
# 🤖 Predict missing words
for sentence in masked_sentences:
print(f"\nInput: {sentence}")
predictions = mlm_pipeline(sentence)
for pred in predictions[:3]:
print(f"✨ → {pred['sequence']} (score: {pred['score']:.4f})")
🔤 Masked Language Model (MLM) Demo
Input: The robot can [MASK] the room in minutes.
✨ → The robot can clean the room in minutes. (score: 0.3124)
✨ → The robot can scan the room in minutes. (score: 0.1547)
✨ → The robot can paint the room in minutes. (score: 0.0983)
Input: He decided to [MASK] the project early.
✨ → He decided to finish the project early. (score: 0.3876)
✨ → He decided to start the project early. (score: 0.2109)
✨ → He decided to abandon the project early. (score: 0.0765)
Input: This device is [MASK] for small tasks.
✨ → This device is perfect for small tasks. (score: 0.2458)
✨ → This device is great for small tasks. (score: 0.1894)
✨ → This device is useful for small tasks. (score: 0.1321)
Input: The weather will [MASK] by tomorrow.
✨ → The weather will improve by tomorrow. (score: 0.2987)
✨ → The weather will change by tomorrow. (score: 0.1765)
✨ → The weather will clear by tomorrow. (score: 0.1034)
Input: She loves to [MASK] in the garden.
✨ → She loves to work in the garden. (score: 0.3542)
✨ → She loves to play in the garden. (score: 0.1986)
✨ → She loves to relax in the garden. (score: 0.0879)
Input: Please [MASK] the door before leaving.
✨ → Please close the door before leaving. (score: 0.4673)
✨ → Please lock the door before leaving. (score: 0.3215)
✨ → Please open the door before leaving. (score: 0.0652)