Update pipeline tag and add library name, results and installation instructions (#2)

- Update pipeline tag and add library name, results and installation instructions (cdced992acf416477c9e40b19f3b51ee742d245d)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

README.md

@@ -1,15 +1,17 @@
 ---
+datasets:
+- ILSVRC/imagenet-21k
 license: other
 license_name: nvclv1
 license_link: LICENSE
-datasets:
-- ILSVRC/imagenet-21k
-pipeline_tag: image-feature-extraction
+pipeline_tag: image-classification
+library_name: transformers
 ---
 
-
 [**MambaVision: A Hybrid Mamba-Transformer Vision Backbone**](https://arxiv.org/abs/2407.08083).
 
+[Project page](https://github.com/NVlabs/MambaVision)
+
 ## Model Overview
 
 We have developed the first hybrid model for computer vision which leverages the strengths of Mamba and Transformers. Specifically, our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features. In addition, we conducted a comprehensive ablation study on the feasibility of integrating Vision Transformers (ViT) with Mamba. Our results demonstrate that equipping the Mamba architecture with several self-attention blocks at the final layers greatly improves the modeling capacity to capture long-range spatial dependencies. Based on our findings, we introduce a family of MambaVision models with a hierarchical architecture to meet various design criteria.
@@ -37,7 +39,6 @@ MambaVision-L3-512-21K is pretrained on ImageNet-21K dataset and finetuned on Im
     <td>512x512</td>
 </tr>
 
-
 </table>
 
 In addition, the MambaVision models demonstrate a strong performance by achieving a new SOTA Pareto-front in
@@ -48,12 +49,10 @@ terms of Top-1 accuracy and throughput.
 class="center">
 </p>
 
-
 ## Model Usage
 
 It is highly recommended to install the requirements for MambaVision by running the following:
 
-
 ```Bash
 pip install mambavision
 ```
@@ -66,13 +65,11 @@ In the following example, we demonstrate how MambaVision can be used for image c
 
 Given the following image from [COCO dataset](https://cocodataset.org/#home)  val set as an input:
 
-
 <p align="center">
 <img src="https://cdn-uploads.huggingface.co/production/uploads/64414b62603214724ebd2636/4duSnqLf4lrNiAHczSmAN.jpeg" width=70% height=70% 
 class="center">
 </p>
 
-
 The following snippet can be used for image classification:
 
 ```Python
@@ -147,7 +144,176 @@ print("Size of extracted features in stage 1:", features[0].size()) # torch.Size
 print("Size of extracted features in stage 4:", features[3].size()) # torch.Size([1, 1568, 16, 16])
 ```
 
-
 ### License: 
 
-[NVIDIA Source Code License-NC](https://huggingface.co/nvidia/MambaVision-L3-512-21K/blob/main/LICENSE)
+[NVIDIA Source Code License-NC](https://huggingface.co/nvidia/MambaVision-L3-512-21K/blob/main/LICENSE)
+
+## Results + Pretrained Models
+
+### ImageNet-21K
+
+<table>
+  <tr>
+    <th>Name</th>
+    <th>Acc@1(%)</th>
+    <th>Acc@5(%)</th>
+    <th>#Params(M)</th>
+    <th>FLOPs(G)</th>
+    <th>Resolution</th>
+    <th>HF</th>
+    <th>Download</th>
+  </tr>
+
+<tr>
+    <td>MambaVision-B-21K</td>
+    <td>84.9</td>
+    <td>97.5</td>
+    <td>97.7</td>
+    <td>15.0</td>
+    <td>224x224</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-B-21K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-B-21K/resolve/main/mambavision_base_21k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L-21K</td>
+    <td>86.1</td>
+    <td>97.9</td>
+    <td>227.9</td>
+    <td>34.9</td>
+    <td>224x224</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L-21K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L-21K/resolve/main/mambavision_large_21k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L2-512-21K</td>
+    <td>87.3</td>
+    <td>98.4</td>
+    <td>241.5</td>
+    <td>196.3</td>
+    <td>512x512</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L2-512-21K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L2-512-21K/resolve/main/mambavision_L2_21k_240m_512.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L3-256-21K</td>
+    <td>87.3</td>
+    <td>98.3</td>
+    <td>739.6</td>
+    <td>122.3</td>
+    <td>256x256</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L3-256-21K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L3-256-21K/resolve/main/mambavision_L3_21k_740m_256.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L3-512-21K</td>
+    <td>88.1</td>
+    <td>98.6</td>
+    <td>739.6</td>
+    <td>489.1</td>
+    <td>512x512</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L3-512-21K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L3-512-21K/resolve/main/mambavision_L3_21k_740m_512.pth.tar">model</a></td>
+</tr>
+
+</table>
+
+### ImageNet-1K
+
+<table>
+  <tr>
+    <th>Name</th>
+    <th>Acc@1(%)</th>
+    <th>Acc@5(%)</th>
+    <th>Throughput(Img/Sec)</th>
+    <th>Resolution</th>
+    <th>#Params(M)</th>
+    <th>FLOPs(G)</th>
+    <th>HF</th>
+    <th>Download</th>
+  </tr>
+
+<tr>
+    <td>MambaVision-T</td>
+    <td>82.3</td>
+    <td>96.2</td>
+    <td>6298</td>
+    <td>224x224</td>
+    <td>31.8</td>
+    <td>4.4</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-T-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-T-1K/resolve/main/mambavision_tiny_1k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-T2</td>
+    <td>82.7</td>
+    <td>96.3</td>
+    <td>5990</td>
+    <td>224x224</td>
+    <td>35.1</td>
+    <td>5.1</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-T2-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-T2-1K/resolve/main/mambavision_tiny2_1k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-S</td>
+    <td>83.3</td>
+    <td>96.5</td>
+    <td>4700</td>
+    <td>224x224</td>
+    <td>50.1</td>
+    <td>7.5</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-S-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-S-1K/resolve/main/mambavision_small_1k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-B</td>
+    <td>84.2</td>
+    <td>96.9</td>
+    <td>3670</td>
+    <td>224x224</td>
+    <td>97.7</td>
+    <td>15.0</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-B-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-B-1K/resolve/main/mambavision_base_1k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L</td>
+    <td>85.0</td>
+    <td>97.1</td>
+    <td>2190</td>
+    <td>224x224</td>
+    <td>227.9</td>
+    <td>34.9</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L-1K/resolve/main/mambavision_large_1k.pth.tar">model</a></td>
+</tr>
+
+<tr>
+    <td>MambaVision-L2</td>
+    <td>85.3</td>
+    <td>97.2</td>
+    <td>1021</td>
+    <td>224x224</td>
+    <td>241.5</td>
+    <td>37.5</td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L2-1K">link</a></td>
+    <td><a href="https://huggingface.co/nvidia/MambaVision-L2-1K/resolve/main/mambavision_large2_1k.pth.tar">model</a></td>
+</tr>
+
+</table>
+
+## Installation
+
+We provide a [docker file](./Dockerfile). In addition, assuming that a recent [PyTorch](https://pytorch.org/get-started/locally/) package is installed, the dependencies can be installed by running:
+
+```bash
+pip install -r requirements.txt
+```