I'm learning and sharing my Home Server Ai quest in real time. This video is a step-by-step guide to installing a complete, self-hosted Ollama on WebUI powerhouse. I also show you how to install the docker and nvidia container toolkit and how to download and run some tests on the new hotness 8b and 70b meta llama 3.1. These models are great! This Homelab mega Ai server was built to help me understand and deploy the most advanced self-hosted LLM Ai tools.

We'll start by installing ubuntu and docker with Nvidia Container Toolkit. Next we set up WebUI and Ollama. Then I'll show you how to download your first models and load them into the client. Finally, I show some of the speed of the lama3.1 70b model. This tutorial builds on our quad 3090 GPU setup we built in the last video.

Open WebUI-powered Ollama with LLAMA 3.1 70b in Docker on Ubuntu 22.04

Written piece and instructions for GPU rack modification
https://digitalspaceport.com/ollama-gpu-3090-home-server-quad-gpu-ai-training-rig/

GPU rack frame https://geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 motherboard https://geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
512GB DDR4 2400 RAM KIT https://geni.us/512GB_DDR4_RAM
PCIe4 risers (x4) https://geni.us/PCIe4_Riser_Cable
AMD EPYC 7702p https://geni.us/EPYC_7702p
iCUE H170i ELITE CAPELLIX https://geni.us/iCUE_H170i_Capellix
ELITE Retention Kit https://geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) https://geni.us/SFF8654_to_SFF8654
HDD rack screws for fans https://geni.us/HDD_RackScrews
Make sure you subscribe for more content like this!

Join this channel https://www.youtube.com/@digitalspaceport/join

Chapters
0:00 Ollama Server Installation Guide
0:30 Quad GPU Ai Rig updates
1:40 Installation Ubuntu 22.04 Desktop for Nvidia
10:05 Install Docker Nvidia Toolkit
11:33 Open WebUI and Ollama installation
14:24 How do I install LLAMA 3.1 70b
18:52 LAMA 3.1 405b
20:59 LAMA 3.1 8b
22:48 LAMA 3.1 Statistics
24:32 The ultimate Ollama installation guide for noobs

Share this video to spread the message and leave a comment below with your thoughts or questions. Thanks for looking!

Digital spaceport website
https://digitalspaceport.com

Shopping (Channel members get 3% or 5% discount)
Check out https://Shop.DigitalSpaceport.com for great deals on hardware and merchandise.

*****
As an Amazon Associate I earn from qualifying purchases.

When you click on links to various merchants on this site and make a purchase, this may result in this site earning a commission. Affiliate programs and relationships include, but are not limited to, the eBay Affiliate Network.

Other Merchant Affiliate Partners for this site include, but are not limited to, Newegg and Best Buy. I earn a commission if you click on links and make a purchase from the merchant.
*****

Please take the opportunity to connect and share this video with your friends and family if you find it helpful.