Nvidia Unveils Futuristic Gaming Experience at Computex 2023
News

Nvidia Unveils Futuristic Gaming Experience at Computex 2023, Blending Gaming and AI

by Isabel
May 29, 2023
0

At Computex 2023, Nvidia displays a futuristic gaming experience that...

Read more
Adobe Introduces Powerful Generative AI Tools in Photoshop

Adobe Introduces Powerful Generative AI Tools in Photoshop Beta

May 29, 2023
Adobe Photoshop's Generative Fill Feature

Exploring the Power of Adobe Photoshop’s Generative Fill Feature

May 27, 2023
NVIDIA and Microsoft Partner to Accelerate AI

NVIDIA and Microsoft Partner to Accelerate AI

May 25, 2023
google photos security and privacy

Exploring the Top 5 Privacy and Security Risks of using Google Photos

May 24, 2023
Google help me write

Google’s New “Help Me Write” Feature Makes Emailing Faster and Easier

May 16, 2023
e2b: The Open-Source Platform for Building AI-Powered Virtual Software Developers

e2b: The Open-Source Platform for Building AI-Powered Virtual Software Developers

May 17, 2023
ChatGPT Error

ChatGPT Errors: How to Resolve Common Issues

May 11, 2023
10 Ways How ChatGPT Can Help Data Scientists Enhance Their Work

10 Ways How ChatGPT Can Help Data Scientists Enhance Their Work

February 27, 2023
PrivateGPT

PrivateGPT: The Most Powerful and Secure AI Language Model

May 12, 2023
How to disable ChatGPT history

How to Disable ChatGPT History and Keep Your Conversations Private

May 13, 2023
Cloudbooklet
  • News
  • Artificial Intelligence
  • Linux
  • Google Cloud
  • AWS
No Result
View All Result
Cloudbooklet
  • News
  • Artificial Intelligence
  • Linux
  • Google Cloud
  • AWS
No Result
View All Result
Cloudbooklet
No Result
View All Result
Home Artificial Intelligence

How to install MLC LLM

by Cloudbooklet
May 2, 2023
in Artificial Intelligence
Reading Time: 7 mins read
How-to-install-MLC-LLM
Share on FacebookShare on TwitterShare on WhatsAppShare on Telegram

Large language models(LLM) and generative artificial intelligence have transformed the area of machine learning, allowing developers to design strong and accurate language-based models. However, deploying and running LLMs can be time-consuming and difficult, necessitating powerful hardware and optimization techniques. In this essay, we will look into the MLC LLM idea, its benefits, and how to use it to construct scalable and efficient AI systems.

Table of Contents

  1. What is MLC LLM?
  2. How does MLC Enable Universal Native Deployment?
  3. How to install MLC LLM on Windows, Linux, and Mac

What is MLC LLM?

In recent years, generative artificial intelligence (AI) and large language models (LLMs) have made significant advances and are becoming more widely used. These models may now be used to construct personal AI helpers thanks to open-source projects. LLMs, on the other hand, are often resource-intensive and demand a large amount of computer power. To provide a scalable service, developers may need to rely on strong clusters and costly hardware for model inference. Furthermore, deploying LLMs presents several challenges, such as continuous model innovation, memory constraints, and the need for potential optimization techniques.

You might also like

ChatGPT app

The Easiest Way to Download ChatGPT App Free

May 31, 2023
LLM Connected with APIs

Gorilla: LLM Connected with APIs

May 31, 2023

The goal of this program is to make it easier to create, improve, and deploy AI models for inference on a variety of devices, including not only server-class hardware but also users’ browsers, laptops, and mobile apps. To achieve this aim, we must address the various features of computing equipment as well as deployment conditions. Among the key challenges are:

  • Assisting multiple CPU and GPU models and potentially other co-processors and accelerators.
  • Implementing on user devices’ native environment, which might not have Python or other required dependencies easily accessible.
  • Managing memory constraints by carefully allocating and compressing model parameters.

MLC LLM offers developers and AI system researchers with a flexible, systematic, and repeatable process that prioritizes efficiency and a Python-first approach for constructing models and optimizations. This technique enables quick experimentation with new models, ideas, and compiler passes, resulting in native deployment to the appropriate targets. Furthermore, we are constantly improving LLM acceleration by expanding TVM backends to improve model compilation transparency and efficiency.

How does MLC Enable Universal Native Deployment?

At the heart of our approach is machine learning compilation (MLC), which we use to automate the deployment of AI models. To do this, we depend on a variety of open-source ecosystems, including tokenizers supplied by Hugging Face and Google, as well as open-source LLMs like as Llama, Vicuna, Dolly, MOSS, and others. Our primary process revolves around Apache TVM Unity, an interesting ongoing initiative within the Apache TVM Community.

  • Dynamic shape: We bake a language model as a TVM IRModule with native dynamic shape support, which eliminates the requirement for additional padding to the maximum length and reduces both computation amount and memory use.
  • Composable ML compilation optimizations: Many model deployment optimizations, such as better compilation code transformation, fusion, memory planning, library offloading, and manual code optimization, can be easily incorporated as TVM’s IRModule transformations exposed as Python APIs.
  • Quantization: To compress the model weights, we use low-bit quantization’s and leverage TVM’s loop-level TensorIR to easily customize code generators for multiple compression encoding strategies.
  • Runtime: The final created libraries operate in the native environment, with a TVM runtime that has few dependencies and supports several GPU driver APIs as well as native language bindings (C, JavaScript, and so on).
MLC LLM

In addition, we include a lightweight C++-based example CLI programme that demonstrates how to wrap up the compiled things and appropriate pre/post-processing, which should help simplify the workflow for embedding them into native apps.

MLC LLM creates GPU shaders for CUDA, Vulkan, and Metal as a starting point. More support, such as OpenCL, sycl, and webgpu-native, can be added by improving the TVM compiler and runtime. MLC also supports a variety of CPU targets, including ARM and x86, through LLVM.

We rely heavily on the open-source ecosystem, specifically TVM Unity, an exciting new development in the TVM project that enables python-first interactive MLC LLM development experiences, allowing us to easily compose new optimizations all in Python and incrementally bring our app to the environment of interest. We also employed optimizations including fused quantization kernels, first-rate dynamic shape support, and a variety of GPU backends.

Also read Machine Learning Algorithms in Python.

How to install MLC LLM on Windows, Linux, and Mac

To install MLC LLM, we provide a CLI (command-line interface) app to chat with the bot in your terminal. Before installing the CLI app, we should install some dependencies first.

  1. We use Conda to manage our app, so we need to install a version of conda. We can install Miniconda or Miniforge.
  2. On Windows and Linux, the chatbot application runs on GPU via the Vulkan platform. For Windows and Linux users, please install the latest Vulkan driver. For NVIDIA GPU users, please make sure to install Vulkan driver, as the CUDA driver may not be good.

After installing all the dependencies, just follow the instructions below the install the CLI app:

1. Open your terminal or command prompt.

2. Type the following command to create a new conda environment named “mlc-chat”:

conda create -n mlc-chat

3. After creating the environment, activate it by typing:

conda activate mlc-chat

4. Next, install Git and Git-LFS using the following command:

conda install git git-lfs

5. Now, install the chat CLI app from Conda using this command:

conda install -c mlc-ai -c conda-forge mlc-chat-nightly

6. Create a directory named “dist” using the following command:

mkdir -p dist

7. Download the model weights from Hugging Face and binary libraries from GitHub using the following commands:

git lfs install

git clone https://huggingface.co/mlc-ai/demo-vicuna-v1-7b-int3 dist/vicuna-v1-7b

git clone https://github.com/mlc-ai/binary-mlc-llm-libs.git dist/lib

8. Finally, to start chatting with the bot running natively on your machine, enter the following command:

mlc_chat_cli
MLC LLM

You should now be able to enjoy chatting with the bot!

This article is to help you learn how to install MLC LLM on Windows, Linux, and Mac. We trust that it has been helpful to you. Please feel free to share your thoughts and feedback in the comment section below.

Share2Tweet1SendShare
Cloudbooklet

Cloudbooklet

Help us grow and support our blog! Your contribution can make a real difference in providing valuable content to our readers. Join us in our journey by supporting our blog today!
Buy me a Coffee

Related Posts

Soundstorm-Pytorch

Soundstorm-Pytorch: A Powerful Tool for Audio Generation

May 30, 2023
Midjourney vs Adobe Firefly

Midjourney vs Adobe Firefly: A Comparison of Two AI Image Generation Tools

May 30, 2023
ChatGPT

How to Use ChatGPT Code Interpreter

May 31, 2023
Leonardo AI Login

How to login and use Leonardo AI to generate high-quality image

May 30, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

I agree to the Terms & Conditions and Privacy Policy.

  • Trending
  • Comments
  • Latest
DragGAN The AI-Powered Image Editing Tool

DragGAN: The AI-Powered Image Editing Tool That Makes Editing Images Easy

May 30, 2023
DragGAN AI editing Tool Install and Use DragGAN Photo Editor

DragGAN AI editing Tool Install and Use DragGAN Photo Editor

May 27, 2023
Bard API key

Everything You Need to Know About Google’s Bard API Key

May 20, 2023
Install PHP 8.1 on Ubuntu

How to Install or Upgrade PHP 8.1 on Ubuntu 20.04

May 17, 2023
DragGAN The AI-Powered Image Editing Tool

DragGAN: The AI-Powered Image Editing Tool That Makes Editing Images Easy

75
Upgrade PHP version to PHP 7.4 on Ubuntu

Upgrade PHP version to PHP 7.4 on Ubuntu

28
Install Odoo 13 on Ubuntu 18.04 with Nginx - Google Cloud

Install Odoo 13 on Ubuntu 18.04 with Nginx – Google Cloud

25
Best Performance WordPress with Google Cloud CDN and Load Balancing

Best Performance WordPress with Google Cloud CDN and Load Balancing

23
How to Setup SSH Keys on Ubuntu

How to Setup SSH Keys on Ubuntu 20.04

May 31, 2023
ChatGPT app

The Easiest Way to Download ChatGPT App Free

May 31, 2023
LLM Connected with APIs

Gorilla: LLM Connected with APIs

May 31, 2023
Soundstorm-Pytorch

Soundstorm-Pytorch: A Powerful Tool for Audio Generation

May 30, 2023

Popular Articles

  • DragGAN The AI-Powered Image Editing Tool

    DragGAN: The AI-Powered Image Editing Tool That Makes Editing Images Easy

    1436 shares
    Share 574 Tweet 359
  • DragGAN AI editing Tool Install and Use DragGAN Photo Editor

    332 shares
    Share 133 Tweet 83
  • Auto-Photoshop-Stable Diffusion-Plugin: A New Way to Create AI-Generated Images in Photoshop

    70 shares
    Share 28 Tweet 18
  • InternGPT: A New Way to Interact with ChatGPT

    54 shares
    Share 22 Tweet 14
  • Midjourney vs Adobe Firefly: A Comparison of Two AI Image Generation Tools

    10 shares
    Share 4 Tweet 3
Cloudbooklet

Welcome to our technology blog, where we explore the latest advancements in the field of artificial intelligence (AI) and how they are revolutionizing cloud computing. In this blog, we dive into the powerful capabilities of cloud platforms like Google Cloud Platform (GCP), Amazon Web Services (AWS), and Microsoft Azure, and how they are accelerating the adoption and deployment of AI solutions across various industries. Join us on this exciting journey as we explore the endless possibilities of AI and cloud computing.

  • About
  • Contact
  • Disclaimer
  • Privacy Policy

Cloudbooklet © 2023 All rights reserved.

No Result
View All Result
  • News
  • Artificial Intelligence
  • Linux
  • Google Cloud
  • AWS

Cloudbooklet © 2023 All rights reserved.

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy and Cookie Policy.