Fast SAM is a strong tool that employs attention processing and deep learning to segment images rapidly and reliably. This is accomplished by focusing on the most significant aspects of an image while disregarding irrelevant information. This makes it an effective and user-friendly picture segmentation solution.
Fast SAM has received a lot of attention due to its ease of use and high performance. It has the potential to completely transform the field of picture segmentation. In this article, we will look into Fast SAM’s capabilities and benefits. Then show how it can easily segment a variety of images.
Table of Contents
Speed of FastSAM
FastSAM, which was trained using only 2% of the SA-1B dataset published by the respected SAM authors, combines efficiency and precision like never before.
FastSAM has transformed the field of picture segmentation by outperforming the renowned SAM approach while operating at an incredible 50 times faster run-time speed. FastSAM swiftly solves the most complex segmentation problems in a quarter of the time, eliminating the need to wait indefinitely for results.
How to Install FastSAM?
Step 1: Clone the repository first.
Begin by copying the FastSAM repository to your own system with the command:
git clone https://github.com/CASIA-IVA-Lab/FastSAM.gitStep 2: Configure the Conda Environment
FastSAM requires Python 3.7 or higher, as well as PyTorch (1.7 or higher) and TorchVision (0.8 or higher). Conda environments can be created and activated using the following commands:
conda create -n FastSAM python=3.9
conda activate FastSAMStep 3: Install Dependencies
Navigate to the FastSAM directory and use pip to install the necessary packages:
cd FastSAM
pip install -r requirements.txtStep 4: Install CLIP
FastSAM makes use of the CLIP library as well. Install it with the following command:
pip install git+https://github.com/openai/CLIP.gitGetting Started with FastSAM
Follow these steps to get started with FastSAM:
Step 1: Get the Model Checkpoint.
Download a model checkpoint for FastSAM.
Step 2: Run the Inference Scripts
You can use FastSAM in various modes by running the inference scripts provided. Here are some examples:
Everything mode
python Inference.py --model_path ./weights/FastSAM.pt --img_path ./images/dogs.jpgText prompt
python Inference.py --model_path ./weights/FastSAM.pt --img_path ./images/dogs.jpg --text_prompt "the yellow dog"Box prompt (xywh)
python Inference.py --model_path ./weights/FastSAM.pt --img_path ./images/dogs.jpg --box_prompt "[570,200,230,400]"Points prompt
python Inference.py --model_path ./weights/FastSAM.pt --img_path ./images/dogs.jpg --point_prompt "[[520,360],[620,300]]" --point_label "[1,0]"Step 3: Create Masks and View the Results
Use the code below to produce masks, select masks depending on prompts, and visualize the results:
from fastsam import FastSAM, FastSAMPrompt
model = FastSAM('./weights/FastSAM.pt')
IMAGE_PATH = './images/dogs.jpg'
DEVICE = 'cpu'
everything_results = model(IMAGE_PATH, device=DEVICE, retina_masks=True, imgsz=1024, conf=0.4, iou=0.9,)
prompt_process = FastSAMPrompt(IMAGE_PATH, everything_results, device=DEVICE)
# everything prompt
ann = prompt_process.everything_prompt()
# bbox default shape [0,0,0,0] -> [x1,y1,x2,y2]
ann = prompt_process.box_prompt(bbox=[200, 200, 300, 300])
# text prompt
ann = prompt_process.text_prompt(text='a photo of a dog')
# point prompt
# points default [[0,0]] [[x1,y1],[x2,y2]]
# point_label default [0] [1,0] 0:background, 1:foreground
ann = prompt_process.point_prompt(points=[[620, 360]], pointlabel=[1])
prompt_process.plot(annotations=ann,output='./output/',)Results for Fastest Segmentation Model
Fast SAM is the fastest segmentation model available on a single NVIDIA GeForce RTX 3090.
Inference Times with Fast SAM
FastSAM can segment one point prompt image in 40 milliseconds, a 10 point prompt image in 40 milliseconds, and a 100 point prompt image in 40 milliseconds.
Method Params 1 10 100 E(16×16) E(32×32*) E(64×64) SAM-H 0.6G 446 464 627 852 2099 6972 SAM-B 136M 110 125 230 432 1383 5417 FastSAM 68M 40 40 40 40 40 40
Optimize Memory Usage with FastSAM
FastSAM excels not only in speed but also in memory efficiency, allowing you to fully utilise your GPU resources. Here’s a comparison of GPU memory utilisation (in megabytes) between FastSAM and two additional approaches, SAM-H and SAM-B, when applied to the COCO 2017 dataset:
Dataset Method GPU Memory (MB) COCO 2017 FastSAM 2608 COCO 2017 SAM-H 7060 COCO 2017 SAM-B 4670
Zero-shot Transfer Experiments
Edge Detection Performance: Fast SAM, a cutting-edge segmentation model, performs zero-shot transfer tests on the BSDB500 dataset for edge detection tasks. Let’s compare FastSAM’s performance to that of two other approaches, HED and SAM, using several assessment metrics:
Method Year ODS OIS AP R50 HED 2015 .788 .808 .840 .923 SAM 2023 .768 .786 .794 .928 FastSAM 2023 .750 .790 .793 .903
Visualizes FastSAM’s Power
Fast SAM, the cutting-edge segmentation model, achieves outstanding results across a wide range of pictures Using huggingface. Take a look at the following visualisations to see how effective Fast SAM is in segmenting various objects:
To try FastSAM to segment anything on hugging face.
Upload and Segment: Experience the power of Fast Segment Anything by uploading an image and letting it do its magic. In this sample, you can try the “Everything mode” for comprehensive segmentation.
Quick Segmentation: The segmentation process usually takes roughly 6 seconds to complete. Please be patient if the queue has a large concurrency_count, as this may cause minor delays.
Faster Results: If you want faster results, choose a lower input size and uncheck the high_visual_quality option. This optimisation has the potential to dramatically accelerate the process.
Convenient Colab Option: For added convenience, you can acquire segmentation results for any image by using the given Colab link. Open it in Colab for quick access to the results.
Benefits of FastSAM
| Benefit | Demonstration |
|---|---|
| Fast inference speed | Can segment an image with a single point prompt in 40 milliseconds. This makes it excellent for real-time applications like autonomous driving and robotics. |
| High accuracy | On the Cityscapes dataset, achieves a mean intersection over union (mIoU) of 80.3%, which is comparable to the accuracy of other segmentation models. This implies it can separate objects in photos precisely, even if they are little or overlapping. |
| Flexible input | Can be used to segment objects in photos using only a few points as input. This makes it suitable for a wide range of activities, including object detection, image manipulation, and scene comprehension. |
Also read: You might also find useful our guide on SAM and HQ-SAM: A New Generation of Image Segmentation Models
Conclusion
Fast SAM is a game-changing solution that simplifies the process of segmenting anything. Fast SAM simplifies the complicated work of segmentation with its advanced capabilities and user-friendly interface, allowing users to produce accurate and high-quality results with ease. Fast SAM is a dependable and efficient solution for any segmentation job, with lightning-fast inference speeds, efficient memory utilisation, and excellent performance across multiple datasets. Discover the power of Fast SAM and achieve a new level of convenience and efficiency when segmenting anything you want.
Please feel free to share your thoughts and feedback in the comment section below.
