I recently tried to install Proxmox on my Dell Wyse 3040 thin client so it can act as a cluster member, since it’s arguably easier to setup a full Proxmox install on x64 based devices than it is to setup a QDevice. Proxmox has a great installer, you have the full GUI to manage the node (not that you need to do much with it as a QDevice), and you have the option of setting it up for other Proxmox services and containers even if it’s really not very capable. So, I’m committed to installing Proxmox on my 3040. But then the problems came. The Proxmox installer was unhappy with the eMMC storage, and wasn’t able to create a partition table on it. Well, I guess I’m diving down the more difficult path of installing Debian, and then installing Proxmox on top of it.
- Video⌗
- Networking⌗
- Add Proxmox Repository⌗
- APT Update, Upgrade, Install Proxmox⌗
- Introduction
- Section 1 — Install Debian 11 Bullseye on Intel NUC
- Add System User to Sudo Group
- Update Software Sources
- Install Bluetooth and Wireless Drivers
- Section 2 — Install Proxmox VE On Debian 11 in Intel NUC
- Add Proxmox VE Software Repositories
- Install Bluetooth drivers, again
- Create Network Bridge for Containers and VMs
- Setup WiFi Network Interface as bridge (Masquerading (NAT) with iptables)
- Conclusion
- What is Proxmox VE?
- Install Proxmox VE step by step
- Before the Installation
- Step 1 — Installation
- Step 1. 2 — Adjust the APT Sources
- Step 1. 3 — Install Proxmox VE
- Step 2 — Network Configuration
- Pros and Cons of LXC Option
- Pros and Cons of KVM Option
- Pros and Cons of Routed Network Option
- Pros and Cons of Bridged Network Option
- Example of the Host Configuration in Routed Setup
- Enable IP Forwarding on the Host
- Configure IP Addresses on the Host
- Add Routes on the Host
- Example of the Guest Configuration in Routed Setup
- Example of the Host Configuration in Bridged Setup
- Example of the Guest Configuration in Bridged Setup
- Step 3 — Security
- Introducing Proxmox
- What is Proxmox?
- Proxmox ISO vs Proxmox package
- Add the Proxmox repository
- Getting started with Proxmox on Debian
- Proxmox web interface
- Going further
- Additional Resources
- Table of Contents
- Updating repositories using the web interface
- Open the Proxmox Web Interface
- Disable the Enterprise Repository
- Add the No-Subscription Repository
- Review configured repositories
- Connecting using a public key
- Update the system
- Storage
- Check ZFS pool status
- Backup Drive
- Add to Backup drive to Proxmox as a directory
- Storage Content Types
- Configure Backups
- Download Guest Virtual Machine ISOs
- Example ISO download
- Other ISOs to Download
- Download Container Templates
- Using the command line (pveam)
- Network card bonding
- Create Proxmox Linux Bond
- Configure Network Switch
- Backup sources. list
- Create a new sources. list
- Proxmox NoSubscription Repo
- Proxmox CEPH Repo
- Trust the Repositories
- Disable Enterprise Repository
- Download ISO
Video⌗
First we need to install Debian. I started with the Debian 11 (Bullseye) netinst image, ran through it, deseleted the desktop, and selected SSH server. From there, it’s pretty much a standard Debian install. Of course, if you’re going through the trouble of installing Proxmox on Debian instead of ’natively’, you probably want to use a custom partition layout on the boot partition, and here is where you’d configure that.
Networking⌗
Before installing Proxmox we need to make sure we have a static IP address, and the hostname resolves back to our public static IP address by setting it in the hosts file.
First, let’s set that static address by editing /etc/network/interfaces:
Find the line for your adapter (mine is enp1s0) and modify it from dhcp to static and add the static IP. Here’s what mine looks like:
iface enp1s0 inet static
After that, we need to add this static address to the hosts file by editing /etc/hosts:
By default, Debian will map 127.0.1.1 to your hostname and 127.0.0.1 to localhost. We need to change the second entry to point to our network-accessible IP instead. This is what mine looks like:
127.0.0.1 localhost.localdomain localhost
172.27.1.154 pve3040.palnet.net pve3040
::1 localhost ip6-localhost ip6-loopback
You should only need to change the first two lines here.
Add Proxmox Repository⌗
Now download the Proxmox GPG key that the repository is signed with. The file naming convention has changed, so you can’t just go from proxmox-ve-release-6.x.gpg to proxmox-ve-release-7.x.gpg like the Proxmox guide would let you believe. I believe this is because the same key now signs the PBS releases as well as PVE.
wget http://download.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
APT Update, Upgrade, Install Proxmox⌗
Now do the usual apt update and apt full-upgrade to prepare. This should also download package lists from PVE’s repo.
Now we can finally install Proxmox:
apt install proxmox-ve postfix open-iscsi
Postfix will ask you some questions, choose Local Only unless you know you have a better answer.
Proxmox also suggests we remove os-prober:
sudo apt remove os-prober
Now reboot, and DONE!
Introduction
I recently bought an Intel NUC mini PC. The exact model of Intel NUC is NUC11PAHi3. The specification of my Intel NUC is Core i3 11th gen, with 32 GB RAM and 500 GB NVMe disk.
I decided to use my NUC PC to learn virtualization and deploy many VMs and containers for testing purpose. I already have experience with Proxmox VE, so I decided to install it. My plan was to install Debian 11 and then install Proxmox on top of Debian.
However, installing Proxmox VE on Debian bullseye in an latest Intel NUC wasn’t easy, but I managed to get it working after several attempts. The main issue with latest Intel NUC is the drivers.
Debian linux-firmware and proxmox-firmware packages conflicts with each other. I can’t install both. So I had to extract the firmware package and manually copy/paste the contents of the firmware package in order to install the Bluetooth and sound drivers on Debian after installing Proxmox. This is the tricky part.
I have split this tutorial in three sections. The first section explains how to install Debian 11 bullseye on Intel NUC11PAHi3 system. Then we will see how to update the Debian 11 system and install the required drivers.
The second section explains how to install Proxmox VE on Debian 11 in the Intel NUC PC and how to manually install the drivers. This is the main goal of this tutorial.
The third and final section includes the steps to access the Proxmox web dashboard, and how to create a network bridge to allow containers and virtual machines to interact with the host machine (i.e. Debian 11).
Let us get started.
Section 1 — Install Debian 11 Bullseye on Intel NUC
Installing Debian 11 bullseye is not a big deal. We already have documented the Debian 11 installation steps with screenshots in the link below.
Add System User to Sudo Group
$ su —
And then run:
Update Software Sources
$ sudo software-properties-gtk
This will open Software & Updates section where you can select the main, contrib and non-free repositories.
Software And Updates Settings
Click Close button. You will be prompted to update the repositories sources. Click yes to agree to update repositories list.
deb http://deb.debian.org/debian/ bullseye main contrib non-free
deb http://deb.debian.org/debian/ bullseye-updates main non-free contrib
deb http://security.debian.org/debian-security bullseye-security main contrib non-free
$ sudo apt update
$ sudo apt upgrade
$ sudo apt dist-upgrade
$ sudo apt full-upgrade
Reboot your system to use the latest version. At this stage, you will have an updated Debian 11 system.
Install Bluetooth and Wireless Drivers
Even though we have an up-to-date Debian version, the Wifi and Bluetooth devices will not work in the latest Intel NUC models.
To install Wifi and Bluetooth drivers on Debian 11 in Intel NUC PC, you should install the firmware-iwlwifi package. This package is available only in non-free repository, so make sure you have enabled the non-free repository as shown in the previous steps.
$ sudo apt install firmware-iwlwifi
Once again, reboot your system.
Congratulations! You now have an updated Debian 11 OS with all drivers on the Intel NUC PC. Now Bluetooth Wifi drivers will work. You can connect to the any available Wifi networks and disconnect the wired connection if you don’t need it.
Section 2 — Install Proxmox VE On Debian 11 in Intel NUC
The hostname of your system must be resolvable via the hosts file. To do so, open /etc/hosts file and add the IP address of your system.
$ sudo nano /etc/hosts
Update the IP address as shown below.
192.168.43.100 pvedebian.ostechnix.lan pvedebian
Update IP Address In hosts File
Replace the IP address and hostname with your own in the above file. Press CTRL+O, and hit ENTER to save the file and hit CTRL+X to close it.
$ hostname —ip-address
This should return the IP address of your Proxmox system.
Verify If Hostname Is Resolvable
Add Proxmox VE Software Repositories
$ sudo sh -c ‘wget https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg’
Update the software repository list and upgrade your system:
$ sudo apt install proxmox-ve postfix open-iscsi
If you have a mail server in your network, you should configure postfix as a satellite system. Your existing mail server will then be the relay host which will route the emails sent by Proxmox VE to their final recipient.
If you don’t have a mail server and If you don’t know what to enter here, choose local only and choose OK and hit ENTER.
Leave the system name as is and hit ENTER to continue.
System Mail Name
This will install Proxmox kernel and set it as a default kernel.
Finally, reboot your system.
$ sudo reboot
The system will now boot into Proxmox kernel by default.
Install Bluetooth drivers, again
As stated earlier, once you installed Proxmox packages, the Bluetooth device will not work anymore in Intel NUC. Because, linux-firmware and proxmox-firmware packages conflicts with each other. So linux-firmware package along with firmware-iwlwifi package will be uninstalled during Proxmox installation.
To make the Bluetooth work again, you should have linux-firmware package installed. But proxmox won’t allow it. You can install only either of the packages. So in order to fix Bluetooth issue, we need to manually install the drivers.
Download firmware-iwlwifi package from Debian packages site. The firmware-iwlwifi package contains the binary firmware for Intel Wireless cards supported by the iwl3945, iwl4965, and iwlwifi drivers.
Extract firmware-iwlwifi package anywhere in your local disk (just right click and choose extract). I extracted in ~/Downloads directory. Inside the firmware-iwlwifi folder, you will see a tar file named data.tar.xz.
Extract the data.tar.xz file too.
Extract ‘data.tar.xz’ File
You will get two more folders namely usr and lib.
The lib folder contains a directory named firmware, which contains all the driver files for Intel wireless and Bluetooth devices.
Go to the lib/firmware folder using command:
$ cd Downloads/firmware-iwlwifi_20210315-3_all/lib/firmware/
Copy everything from here to your system’s /lib/firmware/ directory.
$ sudo cp -r . /lib/firmware/
$ sudo update-initramfs -k all -u
This command will add all the drivers to Proxmox kernel and update it.
Install Bluetooth Drivers On debian In Intel NUC
Reboot your system:
If you see «Found device firmware: intel/ibt-xx-x-x.sfi» line in the output, your Bluetooth driver is loaded and it should be working already.
Open your web browser and navigate to https://IP-Address:8006. You will see an security warning message. Just click «Advanced» button and then click «Accept the risk and Continue» button to bypass the warning message.
Accept Security Warning
Login To Proxmox VE Web Dashboard
You will see «No Valid Subscription» message upon login.
Proxmox No Valid Subscription Message
This is normal. We haven’t added Proxmox license yet, so this window will appear whenever you login to Proxmox dashboard. Click OK to close this window.
- Create a network bridge in order to let the containers and VMs to interact with the host system.
- Masquerading (NAT) with iptables.
- If you installed Proxmox on top of Debian. If you installed Proxmox standalone version, just skip this step.
- If you want to use Wireless device as network bridge.
Create Network Bridge for Containers and VMs
As I already mentioned, If you have installed Proxmox standalone version, you don’t need to do this. A network bridge named vmbr0 will be automatically created and configured by default. However, if you installed Proxmox on Debian, you must create a network bridge manually.
Click the Proxmox host on the left pane. Click «Network» tab under «System» section. Click «Create» drop-down button on the right pane and click «Linux Bridge» option.
Create Linux Bridge In Proxmox
Enter the IP address and gateway for the network bridge and click Create button
Enter IP Address For Linux Bridge In Proxmox
Heads Up: Here, I left the field «Bridge ports» field blank. Because, I want to use my wireless device as a bridge. If you plan to use Ethernet card, you must mention the name of Ethernet card (E.g. enp89s0) in the Bridge ports field.
Click Apply Configuration button to apply the changes to take effect.
Apply Network Configuration
Done. we just created a network bridge for the containers and VMs.
Setup WiFi Network Interface as bridge (Masquerading (NAT) with iptables)
This step is also optional and needs to be done only when you want to use Wireless network device as a bridge instead of the Wired Ethernet network device.
Open the /etc/network/interfaces file in your favorite editor:
$ sudo nano /etc/network/interfaces
Masquerading (NAT) With Iptables
Replace 192.168.1.0/24 with your own subnet and wlo1. You can get the name of the Wireless card with ‘ip a’ command.
Heads Up: Please note that your subnet should match with your network bridge’s IP address. In this setup, my network bridge’s IP address is 192.168.1.101/24 and subnet is 192.168.1.0/24.
Press CTRL+O and ENTER to save the changes and press CTRL+X to close the file.
Restart NetworkManager service and then reboot your system.
$ sudo systemctl restart NetworkManager
Conclusion
In this detailed guide, we discussed how to install Debian 11 bullseye on an Intel NUC PC. Then we learned how to install Proxmox VE on Debian 11 with Wireless and Bluetooth driver support.
In our upcoming guides, we will see how to how to create containers and VMs and various other Proxmox topics in detail.
In this tutorial, we will discuss what is Proxmox VE (Virtual Environment) and how to install Proxmox VE to setup server virtualization environment to create and manage virtual machines and containers.
What is Proxmox VE?
Proxmox VE is full-featured, highly secure GNU/Linux system that can be used to deploy and manage virtual machines, and containers. It is based on Debian Linux distribution with modified RHEL kernel.
Proxmox fills the gap between high performance Linux virtualization and the missing parts — easy deployment and management.
It comes with a simple, easy to usable web interface that allows you to easily create, configure and manage VMs, containers, software-defined storage and networking, high-availability clustering and many out-of-the box tools on a single solution.
All VMs and containers can be managed from a central place. We can create and manage virtual machines by simply selecting the node where should it run.
With Proxmox VE, you can join multiple physical servers to one VE cluster. This flexible mult-master cluster eliminates a single point of failure.
We can backup and restore the running VMs. Even the running virtual servers can be moved from one physical host to another without downtime.
It includes RESTful web API which can be used to enable integration for third party management tools.
In a nutshell, Proxmox
- is an enterprise-grade virtualization solution,
- is capable of handling several hundred virtual machines,
- provides central management for all guests,
- provides clustering,
- offers online backup solution,
- supports live migration,
- supports 32 and 64 bit guests.
Install Proxmox VE step by step
Installing Proxmox VE is fairly simple and straight forward. You can Proxmox VE on a top of existing Debian or install a standalone version. In this guide, we will be installing the standalone version.
Download the latest Proxmox installer ISO from the link below.
Proxmox VE Download link
After downloading Proxmox VE ISO, create bootable USB with Proxmox ISO. There are many bootable USB creation tools available to create a bootable USB medium. We already have reviewed a few bootable USB creation applications. You can pick any one that suits you.
Command line bootable USB creation tools:
Graphical bootable USB creation tools:
After creating Proxmox bootable USB, boot your system with the newly created bootable usb drive.
You will be pleased with Proxmox installer welcome screen. Choose ‘Install Prxomox VE’ option and hit enter.
Click ‘I agree’ button to accept the license agreement and continue the Proxmox installation.
Accept license agreement
Next, choose target hard drive to install Proxmox and click Next. The Proxmox installer will automatically partition the selected hard disk, install all required packages and finally make the system bootble from the hard disk. Please note that all existing partitions and data will be lost.
Choose target drive
If you want to manually create partitions change filesystem (Default is EXT4), click ‘Options’ in the above screen and create the partitions of your choice and change the desired filesystem.
I go with automatic partition scheme.
Next, choose your country location and timezone.
Choose location and timezone
Enter hostname, ip address, netmask, gateway, and DNS. You must provide a valid network details to access proxmox configuration interface after installation.
You will see the summary of installation. If it is OK for you, click Next to start Proxmox installation. Or, simply click Previous button to go back and change the values accordingly.
Proxmox installation summary
Now, the Proxmox installation process will begin. It will take a few minutes to complete.
If you have checked the option that says «Automatically reboot after successful installation» in the previous screen, your system will automatically reboot upon successful installation. If you didn’t check that option, you will see the Installation successful message. Remove the installation medium and hit Reboot button.
Congratulations! Proxmox VE is installed and ready to use.
As mentioned earlier, Proxmox is based on Debian. So, you can install any additional applications or update it as the way you do in Debian. Say for example, to update Proxmox, simply run:
# apt update && apt upgrade
We have installed Proxmox. Next, we will see how to access the web console.
SSH is pre-configured in Proxmox. So you can access the Proxmox server from any system from LAN via SSH like below.
Replace the IP address with your Proxmox server’s IP.
You need a web browser to access Proxmox VE web UI. The vanilla Proxmox doesn’t have a GUI. You need to access the web dashboard either from a remote system’s browser or install any Desktop Environment on Proxmox VE with a web browser and start configuring it.
Login to Proxmox web console
Here is how Proxmox dashboard looks like.
Proxmox web interface
This tutorial shows how to install Proxmox VE 7 on Debian 11 and configure IP addresses on virtual machines.
Before the Installation
First, some suggestions and advice before starting to set up the new environment:
- Are only Linux machines going to be used? Then, under certain circumstances, LXC would be sufficient.
- Should LXC or KVM be used? Both have their advantages as well as disadvantages.
A thoughtful decision and good research can provide less work/trouble in the future.
A conversion of the virtual disks in formats such as VMDK is simple.
Step 1 — Installation
Boot the server into the Rescue System.
Configure the RAID level, hostname, and partitioning.
Save the configuration and after completion of the installation perform a restart.
Step 1. 2 — Adjust the APT Sources
The next step is to add a key and to adjust the APT sources:
Now update the packages:
apt update # Update package lists
apt full-upgrade # Update system
Step 1. 3 — Install Proxmox VE
The output should contain pve, for example: 5.15.74-1-pve
Step 2 — Network Configuration
First of all, it is important to decide which virtualization solution (LXC and/or KVM) and which variant (bridged/routed) will be used.
If you are not familiar with mentioned technologies, please find rough comparisons in the «Pros and Cons of xxx option» sections below.
Pros and Cons of LXC Option
- Lightweight, fast, lower RAM requirement.
- The kernel of the host system is used.Only Linux distributions can be used.
- The kernel of the host system is used.
- Only Linux distributions can be used.
Pros and Cons of KVM Option
- Almost any operating systems can be installed.
- Higher resource usage, since VMs are using their own kernel.
Pros and Cons of Routed Network Option
- Multiple single IP addresses and subnets can be used on one VM.
- Extra routes on the host are required.Point-to-point setup is required.
- Extra routes on the host are required.
- Point-to-point setup is required.
Pros and Cons of Bridged Network Option
- The host is transparent and not part of the routing.VMs can directly communicate with the gateway of the assigned IP.VMs can get their single IPv4 address from Hetzner’s DHCP server.
- The host is transparent and not part of the routing.
- VMs can directly communicate with the gateway of the assigned IP.
- VMs can get their single IPv4 address from Hetzner’s DHCP server.
- VMs may only communicate via the MAC address assigned to the respective IP address.That MAC address must be requested in Hetzner Robot.
IP addresses from additional subnets can only be used on the host system or on a single VM with a single IP (if the subnet is routed to it) (applies to both IPv4 and IPv6 subnets). - VMs may only communicate via the MAC address assigned to the respective IP address.
- IP addresses from additional subnets can only be used on the host system or on a single VM with a single IP (if the subnet is routed to it) (applies to both IPv4 and IPv6 subnets).
Example of the Host Configuration in Routed Setup
When using a routed setup, the IP of the bridge in the host system is always used as gateway.
- The main IPv4 is a gateway for additional single IPv4 addresses.
- The main IPv4 is a gateway for additional IPv4 subnet addresses.
- The IPv6 address from the subnet is a gateway for IPv6 subnet.
It is necessary to manually add the route to a virtual machine.
Additionally, existing separate MAC addresses should be removed from the respective IP addresses.
Since a host route is set, IP addresses from other subnets are easily possible.
Enable IP Forwarding on the Host
sysctl -w net.ipv4.ip_forward=1
Forwarding for IPv6 needs to be activated as well.
This is also available in the Hetzner standard installation and only needs to be activated:
sysctl -w net.ipv6.conf.all.forwarding=1
The commands above only change runtime values.
To make a persistent configuration, please adjust /etc/sysctl.conf file.
Configure IP Addresses on the Host
Change the prefix of the IPv6 address to /128:
After that, create a new Linux Bridge interface with the same IPv4 address and prefix /32, and an IPv6 address from the main IPv6 subnet and original IPv6 prefix.
Select the Apply Configuration button.
The /etc/network/interfaces file will look like that:
Add Routes on the Host
Apply the changes with the ifreload command:
Traffic for additional IPs (except for the addresses configured on the host) will now be forwarded to the bridge vmbr0.
Example of the Guest Configuration in Routed Setup
The guest’s configuration is much simpler than the host’s one.
The guest has to use the pointopoint option in order for IPv4 addresses to reach the host system.
Example of the Host Configuration in Bridged Setup
When using KVM in bridged mode, it is ABSOLUTELY necessary to apply for separate MAC addresses for the single IPs in advance.
Subnets have to be routed via the additional single IP, which has the separate MAC address.
For IPv6 subnet, you can change it directly via the Robot panel.
For IPv4 subnet, please send a support request in order to route it via the additional single IP.
You have to remove IP addresses from the main NIC and add them to the bridge interface.
Do not forget to add the main NIC name to the Bridge ports field in the bridge settings.
The result is the main NIC is a member of bridge interface vmbr0.
Example of the Guest Configuration in Bridged Setup
Once you have created a virtual machine for your bridged setup, please set the MAC address on the VM’s NIC to the separate MAC address provided by Hetzner.
Here is an example configuration for:
- Additional single IPv4 address.
- Additional IPv4 subnet.
- IPv6 subnet.
Step 3 — Security
The web interface is protected by two different authentication methods: Proxmox VE standard authentication (Proxmox proprietary authentication) and Linux PAM standard authentication.
Nevertheless, additional protection measures would be recommended to protect against the exploitation of any security vulnerabilities or various other attacks.
Here are several possibilities:
By now, you should have installed and configured Proxmox VE as a virtualization platform on your server.
Proxmox VE also support clustering.
Please find details in tutorial «Setting up your own public cloud with Proxmox on Hetzner bare metal».
To install Proxmox on Debian, a custom repository must be added to the sources.list. Once the sources are updated, the proxmox-ve package is available and can be installed with the Debian package manager.
Don’t worry, I’ll explain all the steps in detail below, but let’s start with a quick introduction about Proxmox.
Introducing Proxmox
If you are reading this, you may already be familiar with Proxmox, but I will still start with a brief introduction for those who randomly came across this article.
What is Proxmox?
Proxmox is an open-source server management and server virtualization platform. That’s the official definition, but to clarify, it’s a solution you can use to host virtual machines and containers with a simple web interface.
Proxmox ISO vs Proxmox package
If you open the Proxmox website, you’ll quickly find an ISO file to install Proxmox on your server. It’s the recommended installation method: starting from scratch and creating a virtualization server that will only run Proxmox.
It’s probably a good idea for a production server, but if you just want to try it quickly, it’s possible to install it on most distributions, including Debian (I’ll show you how in the next section) or Ubuntu (I have an article about this here).
The ISO file of Proxmox is based on Debian, so it won’t be that different. Just pick the solution that best fits your needs and your current environment.
Get My Cheat Sheet!
Grab your free PDF file with all the commands you need to know on Linux!
Here are the required steps to install Proxmox on Debian:
- Install Debian, connect to the Internet and do the system updates.
- Add the Proxmox repository to the APT sources.list file.
Let’s learn how to do this step-by-step.
The first step is to install Debian on your server. Download the latest image available from the official website and install it,
By the way, sudo isn’t available by default on a minimal installation, so you might need to log in as root, or switch to a root terminal with:su
Before going any further with the Proxmox installation procedure, you need to update your hosts file with your current IP address.
Setting up a static IP address for your Debian server is also a good idea if your DHCP server is not handling it automatically.
Help me help them <3
Help me raise funds for education by donating to this Pencils of Promise campaign. Together we could fund a school for them!
I forgot to edit this file during my tests, and had some issues because of it, so make sure to do it right now:
That’s it, you can now move on and add the Proxmox repository to the package manager.
Add the Proxmox repository
Proxmox can be installed with APT, we just need to add a new repository corresponding to the current Debian version. It’s the easiest way to install it, and it will keep it up-to-date with the system updates later.
That’s it, now that the repository is properly configured, installing Proxmox will be straightforward.
The Proxmox VE package can now be installed with APT directly by using this command:apt install proxmox-ve postfix open-iscsi
After running this command, all of the dependencies will be installed at the same time. Just press “Y” to confirm the installation.
As you can see on my screenshot, it’s about 500 MB to download and over 2 GB of disk space is required so it might take a while depending on your hardware and Internet connection.
Finally, reboot your system, and should see something like that on the next log-in:
It gives you the Proxmox web interface URL, which you can use from any computer on the same network.
Getting started with Proxmox on Debian
Once Proxmox is installed on your Debian server, using it will be the same as you would after a standard installation (with the ISO file).
Is it your first try? Let me guide you to access the web interface and get started with it.
Proxmox web interface
If everything has gone well until now, the Proxmox web interface should be available at https://IP:8006.The IP address is the one you have above the login prompt, so in my example it’s https://192.168.222.37:8006.
Then you’ll get a login form. The default credentials for Proxmox are:
Finally, you can access the usual interface of Proxmox. If you are familiar with it, you probably don’t need me anymore!
You’ll get an error message saying you don’t have a valid subscription. Subscriptions are for support only, you don’t need one for a simple test.
Going further
Everyone will have a different setup and different goals, so the steps will be different in each case. Remember that Proxmox has a ton of features, but you can start with something simple if you are just trying it for the first time.
Always start with a basic virtual machine before trying to build a cluster with high availability :-).
Additional Resources
Lost with the command line?You don’t need to learn and remember everything, just download my cheat sheet where I explain the most important commands on Linux, with a brief explanation and some examples.Download my free PDF cheat sheet.
Note: This is not intended to be a best practice guide. It is just what “works for me”. Any and all comments/improvements are very welcome.
Table of Contents
- Install Proxmox
- Updating repositories using the web interface
- Connecting using a public key
- Update the system
- StorageVM StorageBackup DriveStorage Content Types
- VM Storage
- Backup Drive
- Storage Content Types
- Configure Backups
- Download Guest Virtual Machine ISOs
- Download Container Templates
- Network card bondingCreate Proxmox Linux BondConfigure Network Switch
- Create Proxmox Linux Bond
- Configure Network Switch
Note: This section will be updated in the future to include information about BTRFS.
Updating repositories using the web interface
Proxmox 7.x now supports modifying repositories using the web interface. This method just automates the process of editing /etc/apt/sources.list; either method can be used.
Open the Proxmox Web Interface
Open a web browser and navigate to : http://xxx.xxx.xxx.xxx:8006 and go to Server / Updates / Repositories.
Disable the Enterprise Repository
If you do not have an Enterprise subscription, select the Enterprise repository and click the disable button.
Proxmox 7.x repository configuration
Add the No-Subscription Repository
Click add and from the drop-down select “No-Subscription” and finally click the Add button.
Note: Whilst the disclaimer says that this repo should not be used for production environments, I have never experienced any issues.
Add No-Subscription repository
Review configured repositories
Finally review the configured repositories.
Correctly configured Proxmox 7.x repositories (without subscription)
Connecting using a public key
Note: This article is still valid even though Proxmox 7.x is based on Debian 11.
Update the system
Now that we can easily SSH into our server, we now need to update the system –
apt update
apt -y upgrade
Storage
The ZFS pools that I created are optimized for speed and not reliability. If one disk fails, I loose the whole pool; but I do not really care about this because I take nightly backups to a separate drive – the 4tb SATA.
When creating the pools, it is always best to reference the disks by their ID because this does not change if they are re-connected differently for some reason.
Create the zfs-hot pool:
zpool create zfs-hot -f -o ashift=12 /dev/disk/by-id/ata-SanDisk_Ultra_II_480GB_161803800213 /dev/disk/by-id/ata-SanDisk_Ultra_II_480GB_161803800899 /dev/disk/by-id/ata-SanDisk_Ultra_II_480GB_161803801332 /dev/disk/by-id/ata-Samsung_SSD_860_EVO_1TB_S3Z6NB0K305604K
Create the zfs-warm pool:
zpool create zfs-warm -f -o ashift=12 /dev/disk/by-id/ata-WDC_WD10EZEX-60ZF5A0_WD-WCC1S3374777 /dev/disk/by-id/ata-WDC_WD10EZEX-60ZF5A0_WD-WMC1S3070843 /dev/disk/by-id/ata-WDC_WD10EZEX-60ZF5A0_WD-WMC1S3071084
Set compression for both pools:
zfs set compression=lz4 zfs-hot
zfs set compression=lz4 zfs-warm
Note: If you are re-installing and the pools have already been created, they can be imported into your system using:
zpool import -a -f
Check ZFS pool status
pvesm add zfspool zfs-hot -pool zfs-hot
pvesm add zfspool zfs-warm -pool zfs-warm
Backup Drive
First mount the drive to a folder on the system:
mkdir /backup
mount /dev/disk/by-id/ata-TOSHIBA_HDWE140_58D3Y018FBRG-part1 /backup
Now run use lsblk to determine the UUID of the drive we mounted:
Now use mtab to find the mount information:
Now modify /etc/fstab to make this mount permanent:
UUID=»04188fa7-87d1-4e25-b948-2ac51ea1d4ed» /backup ext4 rw,relatime 0 0
We can now unmount the drive and use mount -a to test that our fstab entries are correct without needing to reboot.
umount /backup
mount -a
Add to Backup drive to Proxmox as a directory
pvesm add dir backup —path /backup
Storage Content Types
Using the web interface, we can now define the content types for each of our storage pools. The table above defines the content types I uses each of my storage pools for.
Navigate to Datacenter/Storage:
Double click a pool (backup for example) and from the content drop down, select the storage types that you would like to use this pool for.
Storage pool content types
I typically store my ISOs and Container Templates on the Backup pool so that they persist when I decided to re-install the system volume.
The ZFS pools are only used for VM Disk Images and Containers.
I typically now remove the local-lvm pool to remove the risk of any content being stored on them accidentally (again to allow easy re-install). It does not seem possible to remove the local pool, so I just restrict this to snippets (which I currently do not use).
Note: I should probably use a custom partitioning scheme during installation so that I am not wasting disk space for the local-lvm pool.
Here is my final storage configuration:
Finalized storage configuration
Configure Backups
Now that we have all of our storage pools configure, we can configure automatic backups. Normally I set the backups to happen daily during off hours.
Navigate to Datacentre/Backup:
Backup job configuration
Click the Add button to configure your backup job –
- Storage Pool: Select your storage pool – in our case this is “backup”
- Day of Week: Multi-select which days you would like the back up to occur on.
- Start time: 3:00am
- Selection mode: Which VMs to back up, ALL
- Send email to: Enter the email address for notifications
- Email notification: Always or Failures only
- Compression: The default ZSTD (fast and good) works for me
- Mode: Snapshot / Suspend / Stop. I normally use snapshot.
Backup job – Multi-select days of week
Backup job – Mode
Click create and you have completed your automatic backup configuration.
Backup job – Configuration complete
Download Guest Virtual Machine ISOs
Proxmox 7.x includes a new feature that allows you to download ISOs for guest operating systems directly to the server. Simplifying the old process of having to download ISOs to a client machine and then upload them.
Navigate to the storage pool that you have configured to store ISO Images and select ISO Images.
Example ISO download
As an example we will start by downloading the Debian 11 Network install CD. Go to Debian — Network install from a minimal CD and copy the URL for amd64 – https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.0.0-amd64-netinst.iso
Now, click the download from URL button, paste the URL into the URL field and click the Query URL button. The dialog will be populated with information about the ISO you are about to download.
Download from URL (Debian 11 example)
You can also use this dialog to validate the Hash algorithm if you want to. Once you are happy, click the download button.
A task dialog will appear for you to monitor the status
ISO Image download task.
When it is complete and you close the dialog, the ISO will be listed in your available images.
ISO Images with Debian 11 downloaded.
Other ISOs to Download
You can repeat this process to download any other ISOs that you desire. Two other ISOs I always download are –
- Windows 10 direct from the Microsoft site – https://www.microsoft.com/en-ca/software-download/windows10
- Windows VirtIO drivers – https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/virtio-win.iso
- An explanation of why you need these can be found here – Windows VirtIO Drivers – Proxmox VE
However, to make installation of Windows into virtual machines easier, I suggest “slipstreaming” the drivers into the Windows source. I have created a walk though of this process, Slipstreaming Proxmox Virtio Drivers into Windows 10 – gareth.com.
Download Container Templates
Proxmox provides a variety of basic templates for most common Linux distributions. They can be downloaded using the web interface or by using the command line via ssh.
Navigate to the storage pool that you have configured to store Container Templates and select CT Templates.
Click the templates button to show a list of available templates:
Select the template you want and click the download button. A download task viewer is then displayed.
Template download task viewer
Sadly, you can not multiple select templates so you will need to repeat this process for each template you need. Downloading templates in bulk is easier using the command line.
Once you have downloaded all the templates you require they are displayed under the storage pool.
Using the command line (pveam)
The Proxmox VE Appliance Manager (pveam) can also be used to perform these functions.
The list of available templates is updated automatically on a daily basis, you can also update manually by exedcuting:
# pveam update
To list the templates that are available:
# pveam available
# pveam available —section system
Downloading the templates is as simple as passing the storage pool and the name of the template to download. These templates are reasonably small, so I download the current version for all the popular distributions:
# pveam download local archlinux-base_20200508-1_amd64.tar.gz
# pveam download local centos-8-default_20191016_amd64.tar.xz
# pveam download local debian-10.0-standard_10.0-1_amd64.tar.gz
# pveam download local fedora-32-default_20200430_amd64.tar.xz
# pveam download local gentoo-current-default_20200310_amd64.tar.xz
# pveam download local opensuse-15.1-default_20190719_amd64.tar.xz
# pveam download local ubuntu-20.04-standard_20.04-1_amd64.tar.gz
Note: Whilst the version numbers are correct at the time of writing, they will have probably changed by the time you are reading this!
Network card bonding
My Proxmox server motherboard has 4 network cards. To increase performance and reliability, I combine all 4 of these together using a Linux Bond. In order to achieve this you will need a switch that supports 802.3ad.
This is a two stage process, first we need to configure the Proxmox server, then we need to configure the switch. As both steps are required, the server gets temporarily disconnected whilst step 2 is being performed.
Note: It is important to perform these steps in the exact order they are described. If done incorrectly you could lose access to your server.
Create Proxmox Linux Bond
Navigate to the System/Network of the Proxmox server you are configuring.
Make a note of the network devices you are going to bond together, in this case they are enp5s0, enp6s0, enp10s0 and enp11s0.
Edit the existing Linux Bridge and remove the existing Bridge port, leave the IP address and Gateway configured.
Linux Bridge – No Bridge Port
Next, create a new Linux Bond by selecting it from the Create menu.
- Slaves: Enter the list of network devices we gathered earlier separated by spaces.
- Mode: Set to LACP (802.3ad)
Note: I am not sure if there is any major benefit to setting the Hash Policy. Leaving it blank seems to work.
Create Linux Bond
Modify Linux Bridge to use this bond by entering bond0 into the Bridge port field.
Linux Bridge configured to use Bond
The network configuration screen now tells us – Pending changes (Either reboot or use ‘Apply Configuration’)
Network – Pending Changes
IMPORTANT: When you click the Apply Configuration button, the server will be disconnected from the network and connectivity will not be resumed until the switch has been configured in the next step.
Configure Network Switch
Port aggregation needs to be turned on between the ports that are connected to the Proxmox server. I am not going into much detail here because this configuration will be dependent on your switch manufacturer.
Note: On the Ubiquiti switch these ports have to be next to each other because it uses port ranges.
Ubiquiti Port Aggregation
Note: I have experienced some slow down in the initial SSH connection to Linux virtual machines, I am currently unsure what causes this.
References:1. How to Set Up SSH Keys on Debian 10 – https://linuxize.com/post/how-to-set-up-ssh-keys-on-debian-10/2. Slipstreaming Proxmox Virtio Drivers into Windows 10 – https://gareth.com/index.php/2020/07/17/slipstreaming-proxmox-virtio-drivers-into-windows-10/
These are the additional steps that need to be completed to create a working Proxmox 6.x installation.
This post does not cover creation of optimal disk pools for virtual machine images. That will be covered in a later post.
Backup sources. list
We are going to create a new version of sources.list, first we should backup the default one in case we need to go back.
mv /etc/apt/sources.list /etc/apt/sources.list.old
Create a new sources. list
deb http://ftp.us.debian.org/debian buster main contrib non-free
deb-src http://ftp.debian.org/debian/ buster main contrib non-free
deb http://ftp.us.debian.org/debian buster-updates main contrib non-free
# security updates
deb http://security.debian.org buster/updates main contrib non-free
deb http://security.debian.org/debian-security buster/updates main contrib non-free
deb-src http://security.debian.org/debian-security buster/updates main contrib non-free
Proxmox NoSubscription Repo
Use an editor to create a new file called /etc/apt/sources.list.d/pve-nosubscription.list
deb http://download.proxmox.com/debian/pve buster pve-no-subscription
Proxmox CEPH Repo
Use an editor to create a new file called /etc/apt/sources.list.d/pve-ceph.list
deb http://download.proxmox.com/debian/ceph-nautilus buster main
Trust the Repositories
wet http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
chmod +r /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
Disable Enterprise Repository
If you do not have an enterprise subscription, disable the enterprise repository to remove authentication errors when running updates.
sed -i ‘s/deb/#deb/’ /etc/apt/sources.list.d/pve-enterprise.list
apt update
apt -y dist-upgrade
Note: I like to run a “dist-upgrade” at this point, which is probably not totally necessary; but what the heck.
We can now download the templates for each container that we require. To list the templates that are available:
and download the templates (at the time of writing, these are the templates that I normally install – version numbers will have probably changed by the time you are reading this!)
pveam download local archlinux-base_20200508-1_amd64.tar.gz
pveam download local centos-8-default_20191016_amd64.tar.xz
pveam download local debian-10.0-standard_10.0-1_amd64.tar.gz
pveam download local fedora-32-default_20200430_amd64.tar.xz
pveam download local gentoo-current-default_20200310_amd64.tar.xz
pveam download local opensuse-15.1-default_20190719_amd64.tar.xz
pveam download local ubuntu-20.04-standard_20.04-1_amd64.tar.gz
Download ISO
In addition to container templates, you can also download full ISOs, allowing you to create custom virtual machines. Lets start with the ISO for Arch Linux:
wget ca.us.mirror.archlinux-br.org/iso/2021.01.01/archlinux-2021.01.01-x86_64.iso
mv archlinux-2021.01.01-x86_64.iso /var/lib/vz/template/iso/