- Possible errors
- Repository does not contain a Release file
- Become a maintainer. Part Four
- Structure of the repository
- apt-ftparchive
- reprepro
- Configs with a list of repositories
- Debian release classes
- Branches main, contrib, non-free
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to telegram channel author
- Build src package
- Environment preparation
- Adding third-party repositories
- Repository public key
- Checking the added repository
- Mirror of the official repository yandex mirror
- Local repository
- Build binary package and check
- Theory
- Option to use official repositories
- Store changes in git
- Git interaction schema
- Добавить новый repository в debian
- Классы релизов в Debian
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author's telegram channel
- Build src package
- Environment preparation
- Добавление сторонних репозиториев
- Repository public key
- <span id="Checking the added repository
- Mirror of the official repository yandex mirror
- Local repository
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author's telegram channel
- Build src package
- Environment preparation
- Adding third party repositories
- Repository public key
- Проверка добавленного репозитория
- Зеркало официального репозитория yandex mirror
- Локальный репозиторий
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author's telegram channel
- Build src package
- Environment preparation
- Добавление сторонних репозиториев
- Repository public key
- <span id="Checking the added repository
- Mirror of the official repository yandex mirror
- Local repository
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author’s telegram channel
- Build src package
- Environment preparation
- Adding third party repositories
- Repository public key
- Проверка добавленного репозитория
- Зеркало официального репозитория yandex mirror
- Локальный репозиторий
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author’s telegram channel
- Build src package
- Environment preparation
- Adding third party repositories
- Repository public key
- Проверка добавленного репозитория
- Зеркало официального репозитория yandex mirror
- Локальный репозиторий
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author’s telegram channel
- Build src package
- Environment preparation
- Adding third party repositories
- Repository public key
- Проверка добавленного репозитория
- Зеркало официального репозитория yandex mirror
- Локальный репозиторий
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author’s telegram channel
- Build src package
- Environment preparation
- Добавление сторонних репозиториев
- Repository public key
- <span id="Checking the added repository
- Mirror of the official repository yandex mirror
- Local repository
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
- Debian release classes
- Main, contrib, non-free branches
- Sources List Generator
- Tips and tricks
- Kubernetes online course
- Did the article help? Subscribe to author’s telegram channel
- Build src package
- Environment preparation
- Adding third party repositories
- Repository public key
- Проверка добавленного репозитория
- Зеркало официального репозитория yandex mirror
- Локальный репозиторий
- Build binary package and check
- Theory
- Option to use the official repositories
- Store changes in git
- Git interaction schema
- Add new repository to debian
Possible errors
Let’s consider the most popular errors that occur when adding and updating repositories.
Repository does not contain a Release file
The text of the error, in theory, gives a ready answer. The repository does not have a required Release file. But the bottom line is that it probably is. The point here is most often that you have added a repository to yourself that does not contain the branch you specified. For example, you added a repository to the Buster distribution, but the repository does not support this distribution. The previous ones are there, but this one is not.
Exactly the same error you’ll get if you’re using an old deprecated version of Debian. At some point, the standard repositories will stop supporting your distribution version and you will get an error. You will need to either upgrade to a more recent version or use the archive repositories.
Become a maintainer. Part Four
While fans of exotics on Habré are actively drinking cups of Java, taking doses of F # and injecting themselves with other Haskell, you and I learned how to assemble their creations into deb packages. Over the time that has passed since the previous part, someone has probably already accumulated several ready-made packages, and we have not even tried to put them in the official Debian and Ubuntu repository yet. Therefore, it is time to think about how to organize all the accumulated wealth into one big beautiful repository, which you will not be ashamed to offer for use by other users.
(Parts 1
, 2
and 3
)
For these purposes, we will consider two programs (simply because they were enough for me to comprehend the Tao) — this is apt-ftparchive
from the standard apt-utils package and package reprepro
.
Structure of the repository
Let’s first see how a normal repository works. There are two types of repositories — ugly and oblique (or in the original — flat — flat, we will not consider them) and normal hierarchical. Here we will learn to create hierarchical ones. So, a normal repository consists of two directories: pool
and dists
.
pool/
This directory can contain packages in an arbitrary hierarchy. I say «arbitrary» because you can really arrange them however you like, but in practice the following hierarchy is usually implemented:
pool/main/q/qutim/
Here “main” is the name of the section to which we refer our package (read about sections in the first article), followed by the first letter of the src package name and then the directory named after the src package name. This directory will contain the src package and all binary packages built from it. For numerous libraries whose package names all begin with «lib», a separate «liba» is used instead of the letter «l», where «a» is the first letter of the name after «lib».
dists/
This directory is organized much more complicated. First of all, we must find a directory with the name of the repository in it — in Ubuntu these are various “intrepid”, “intrepid-proposed”, “jaunty”, etc., in Debian it is “etch”, “etch-proposed-updates” , «lenny» as well as their synonyms — «stable», «testing», etc. — they are usually symlinks to the canonical name. Thus, you can store packages for several repositories in one place at once.
- Contents-arch
— a file that contains a list of all files from all packages. «arch» is replaced with the name of the package architecture, and may also end with «.gz» and «.bz2» (if the file is properly compressed). It is optional and is used primarily by utilities like apt-file to find out which package contains the required file. Name example — Contents-amd64.gz - Release
is a file that contains a short description of the repository and hashes of all supporting files that are contained in dists/
. - Release.gpg
is the gpg signature of the Release file, certifying the creation of the file by a specific author. Both files are optional, but without them, APT is guaranteed to swear at an unauthenticated repository.
In addition, in the same directory we will find directories with names corresponding to the names of the repository sections — main, contrib, non-free or main, universe, multiverse, restricted, or those that you define yourself.
dists/unstable/main/
In each section (not only main, of course) you can see a set of directories like this:
- binary-arch
— a directory for describing packages of the corresponding architecture, for example, binary-amd64
— for amd64 architecture. Packages with architecture all (such as dev packages) will be listed in the directory for each architecture. - source
— this directory describes src packages - i18n
— this directory is rarely used and mostly only in official repositories — it contains files with translations of package descriptions into other languages. I won’t describe this directory as it’s not really that important if you want to use it — you can always look in the ubuntu repository
dists/unstable/main/binary-i386/
The file Packages is stored here
(and also Packages.gz
and Packages.bz2
) containing descriptions of binary packages of the corresponding architecture. In fact, it is a combination of control files from all binary packages of the repository for a given architecture. Often you can still find a file here Release
, which, unlike the same file in dists/unstable/, does not contain file hashes, but only stores descriptions of the repository for the given directory.
dists/unstable/main/source/
Everything here is the same as in binary-arch, except that the word Sources is used instead of the word Packages.
apt-ftparchive
Of course, all these files can be written with pens, but I would not advise you. Especially if you have at least 3-4 packages. Therefore, we will use the magic utility apt-ftparchive from the apt-utils package and dpkg-scanpackages from the dpkg-dev package.
So, let’s say we have a certain qutim src package created by us in the previous series, and the qutim and qutim-dev binary packages built from it. We collected them for two architectures, so now we have the following set of files:
qutim_0.1.99.138.orig.tar.gz
qutim_0.1.99.138-1.diff.gz
qutim_0.1.99.138-1.dsc
qutim_0.1.99.138-1_amd64.deb
qutim_0.1.99.138-1_i386.deb
qutim-dev_0.1.99.138-1_all.deb
The first three files, as we remember, are the src package, and the remaining ones are the compiled binary packages. We also have the qutim_0.1.99.138-1_i386.changes and qutim_0.1.99.138-1_amd64.changes files generated when the package was built.
Let’s build a minimally decent repository out of them.
So, let’s create a directory with the initial structure and copy the packages into it:
$ mkdir -p rep/dists$ mkdir -p rep/pool/main
$ cp qutim_0.1.99.138.orig.tar.gz qutim_0.1.99.138-1.diff.gz qutim_0.1.99.138-1.dsc qutim_0.1.99.138-1_amd64.deb qutim_0.1.99.138-1_i386. deb qutim-dev_0.1.99.138-1_all.deb rep/pool/main/
$ cd rep
As you can see, there is nothing difficult at all in placing the packages themselves in the repository. It will be slightly more complicated to generate a set of descriptions for these packages:
$ mkdir -p dists/unstable/main/binary-amd64$ mkdir dists/unstable/main/binary-i386
$ mkdir dists/unstable/main/source
$ dpkg-scanpackages -a amd64 pool/main >dists/unstable/main/binary-amd64/Packages 2>/dev/null
$ dpkg-scanpackages -a amd64 pool/main | gzip -c9 >dists/unstable/main/binary-amd64/Packages.gz 2>/dev/null
$ dpkg-scanpackages -a amd64 pool/main | bzip2 -c9 >dists/unstable/main/binary-amd64/Packages.bz2 2>/dev/null
$ dpkg-scanpackages -a i386 pool/main >dists/unstable/main/binary-i386/Packages 2>/dev/null
$ dpkg-scanpackages -a i386 pool/main | gzip -c9 > dists/unstable/main/binary-i386/Packages.gz 2>/dev/null
$ dpkg-scanpackages -a i386 pool/main | bzip2 -c9 > dists/unstable/main/binary-i386/Packages.bz2 2>/dev/null
$ apt-ftparchive sources pool > dists/unstable/main/source/Sources 2>/dev/null
$ apt-ftparchive sources pool | gzip -c9 > dists/unstable/main/source/Sources.gz 2>/dev/null
$ apt-ftparchive sources pool | bzip2 -c9 > dists/unstable/main/source/Sources.bz2 2>/dev/null
$ echo "Archive: unstable" > dists/unstable/Release$ echo "Suite: unstable" >> dists/unstable/Release
$ echo "Components: main" >> dists/unstable/Release
$ echo "Origin: qutim.org" >> dists/unstable/Release
$ echo "Label: qutim.org Debian repository" >> dists/unstable/Release
$ echo "Architectures: amd64 i386" >> dists/unstable/Release
$ echo "Description: Debian qutIM unstable" >> dists/unstable/Release
$ apt-ftparchive release dists/unstable >> dists/unstable/Release
Описание всё равно писать руками, apt-ftparchive делать это не умеет.
And last step — subscribe our repository:
$ gpg -abs -o dists/unstable/Release.gpg dists/unstable/Release
Voila, our repository can be used. Add to /etc/apt/sources.list lines:
deb file:///path/to/rep unstable main
deb-src file:///path/to/rep unstable main
Update the list of packages — and the repository can be used.
Разумеется, это не очень удобный — перегенерировать файли запасний продётся качей раз при обновлении набора пакетов. Of course, everything is automated by scripts, but it’s still so uninteresting. Therefore, we will move on to the second program used, which will do everything for us.
reprepro
This program is simply paradise for those who don’t want to bother with the structure of the repository at all, but want to configure and use it once. Therefore, we take down our homemade repository and put the reprepro package:
$ cd . && rm -rf rep/*$ sudo apt-get install reprepro
$ mkdir rep/conf
Now we create rep/conf file distributions in the catalog, in which we will describe our repository:
Codename: lenny
Suite: Unstable
Version: unstable
Origin: qutim.org
Label: qutim.org Debian Repository
Description: qutim.org Debian repository
Architectures: source i386 amd64
Components: main
SignWith: default
DebIndices: Packages Release gz bz2
DscIndices: Sources Release gz bz2
Contents: gz bz2
Codename should contain the name of the repository (lenny, hardy, etch, jaunty), and Suite — synonym for creating symbolic links (stable, unstable). Parameters Version, Origin, Label and Description do not carry a special semantic load and are simply copied to the Release file. Параметр SignWith indicates на необходим подписать репозиторий, создав файл Release.gpg. DebIndices, DscIndices and Contents — indicate the necessity of creating file Packages, Sources and Contents-arch — in simple and archived form.
Now we can add our packages:
$ reprepro -b rep/ createsymlinks$ reprepro -b rep/ --ask-passphrase -C main include unstable qutim_0.1.99.138-1_amd64.changes
$ reprepro -b rep/ --ask-passphrase -C main includedeb unstable qutim_0.1.99.138-1_i386.deb
ключ -b indicates reprepro, где лежит наш репозиторий, the command —ask-passphrase informs that наш gpg-ключ is zaparolen и при подписывании дистриктива недежда бы просирой пароль, ключ -C informs that мы включаем пакет в сексию main. One interesting moment should be noted here. The include command accepts as an argument the name of the repository to include the file and the name of the .changes file created when compiling the src-package. At the same time, the src package and the compiled binary packages for this architecture are added to the repository. And here we have a problem — the developers of reprepro flatly refuse to add packages to the repository of the version that already exists in it. Therefore, if we now try to specify a .changes file for a different architecture, we will get an error — we have already added not only the src package, but also the qutim-dev package compiled for all architectures. For this case, there is the includedeb command, which will allow you to specify only a deb file with a different architecture.
View the contents of the rep directory now. The conf subdirectory contains the repository configuration (more on this in ` man reprepro
`), the db directory is its internal database. But the pool and dists directories have become very different from what we created with apt-ftparchive and have become similar to the structure that I described at the beginning. Since we already have the repository address in sources.list, we type ` sudo apt-get update
`and use 🙂Configs with a list of repositories
Package managers that can install packages from repositories need to know the addresses of the repositories. And these addresses are written to the config — /etc/apt/sources.list.
And you can also create additional configs with the extension .list
in directory /etc/apt/sources.list.d/
. All this is true for Debian
and for Ubuntu
.If you remember, during the installation of systems, we chose a repository:
- for Debian — deb.debian.org
;- for Ubuntu — en.archive.ubuntu.com
.alex@ubu:~$ egrep -v '^#|^
This file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Looking at the Debian repository branches above, we saw the following:
- bullseye ;
- bullseye-updates ;
- bullseye security .
But, in addition to code names of system versions, special release classes can be used in branch names:
- stable - refers to the current Debian stable repository, currently bullseye . As soon as a new version of Debian is released, stable will link to the newer version;
- oldstable - refers to the previous stable repository;
- testing - refers to a special branch of the repository for the development of a new stable release;
- unstable - links to the latest, but not tested packages;
- experimental - packages that have just begun to be developed are stored here;
- backports - refers to testing and unstable , but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a high probability that the system will be damaged very soon due to untested updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainBranches main, contrib, non-free
Each official Debian repository has 3 branches:
- main
consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.- contrib
packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.- non-free
contains all other software that does not comply with the DFSG.DFSG - Debian Free Software Guidelines
, Debian Free Software Criteria
. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.Now, knowing all the theory about Debian repositories, we can analyze the sources.list file that we received after installation. It has 3 repositories with branches main.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is a stable repo of the current release. Next comes the security repository for installing the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use a ready-made one.
An example of such a generator that you can install yourself and configure for use - debgen
. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/
.I don't know who maintains such List Generators and whether they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror - Russia
- Enable Security and Updates repositories
- Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#--------------------------------------------- ---------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys for import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggestions.
In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it's faster
- Use this page
. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General "Developer corner" of Debian community
Kubernetes online course
Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.
If you answer "yes" to at least one question, then this is your course:
- Tired of wasting time on automation?
- want uniform environments?;
- want to develop and use modern tools?
- Do you care about infrastructure reliability?
- have to scale the infrastructure to meet the growing needs of the business?
- Do you want to free product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link
and join the new set!Did the article help? Subscribe to telegram channel
authorAnnouncements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgency=low * Initial release. -- Alexey Lukyanchuk <skif@skif-web.ru> Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ../ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat << EOF | tee -a ~/.bashrc 2>/dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat << EOF |tee ~/.quiltrc 2>/dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^set mouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don't forget to set up your git variables. Full explanation may be found here
, I will do it in fast way:git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third-party repositories
You can add repositories to the main config: /etc/apt/sources.list
or create separate configs in the directory /etc/apt/sources.list.d/
. I myself think that it is more correct to create separate configs for each third-party repository.For example, to connect the repository nginx
create the following config. For Debian
:alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu
:alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxSuppose we have registered an additional repository for nginx
, but how can the system understand from which repository to take the package for installation? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.To set the repository priority, create a file /etc/apt/preferences.d/XX
, where XX is the file number, the higher it is, the later it will be processed, that is, it will take precedence over other settings.According to our example for nginx, you need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package
: package name. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;- Pin
: Attachment options. There are many options, I'll just cover a few:
- origin
"name of author or supplier";- release o=nginx
- means that there is a provider (Origin = o) with the name nginx in the repository's Release file;- release l=Debian
- means that in the Release file of the repository there is a Label (l) with the name Debian;- Pin Priority
: a priority.That is Package
and Pin
these are the conditions for assigning a priority, and Pin-Priority
it is an action (assigning a priority). In our example, the following is obtained: if the package name is anything, but the owner of the repository is nginx.org
and in file Release
written " Origin: nginx
“, then for such packages we set priority 900.Priority can be in the following ranges:
- P >= 1000
- the package will be installed from this repository, even if it will downgrade an already installed package;- 990 <= P < 1000
- the package will be installed from this repository if no newer version is installed;- 500 <= P < 990
— package will be installed if there is no package belonging to target release
or a newer version is not installed;- 100 <= P < 500
- the package will be installed if there are no candidates from other repositories or an installed package of a newer version;- 0 < P < 100
- the package will be installed only if it is not installed yet (of any version) and if there are no candidates from other repositories;- P < 0
- the package will not be installed under any circumstances;- P = 0
- not used.Priorities 500 to 990 and 990 to 1000 are very similar. To distinguish them, you need to understand what is target release
. For Ubuntu or Debian, this is the version name of the distribution. For example for Ubuntu — jammy
, and for Debian - bullseye
. But this name still needs to be set in this way:alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let's try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secureman page for information about creating a repository and user settings.
And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! Utility gnupg2 for Ubuntu only available in repository universe , if you have commented these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget
):$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With cat
we read the key file, and pass what we read to the gpg utility. Utility gpg
translates the read into the required format and passes the output to the next tee command. Utility tee
(under sudo) saves the resulting text to a file. At the end add >/dev/null
so that there is no output to the terminal. Here is the command itself:$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Example for wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Example for curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxAnd try again to apply the changes:
alex@deb:~$ sudo apt update Ex:1 http://security.debian.org/debian-security bullseye-security InRelease Ex:2 http://deb.debian.org/debian bullseye InRelease Ex:3 http://deb.debian.org/debian bullseye-updates InRelease Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B] Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B] Received 7633 B in 1s (9420 B/s) Reading Package Lists... Done Building the dependency tree… Done Reading Status Information… Done All packages are up to date.Everything went well this time.
By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:
### Example ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxChecking the added repository
Well, to understand from which repository the package will be installed, run the command:
alex@deb:~$ apt-cache policy nginx nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesFrom the output it becomes clear that the package nginx
not yet installed on the system, and a package with version 1.20.2 is a candidate for installation
from repository http://nginx.org/packages/debian
. The priority of all packages, by the way, became equal to = 990
. This happened after we set target release = bullseye
. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned repositories for bullseye
such a priority.Mirror of the official repository yandex mirror
A Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru
. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror can also be used for network installation of systems.
Local repository
There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another simple option is to use reprepro
. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2We initialize the repository.
# cd /mnt/repo/debian # reprepro export #reprepro createsymlinksNow you can add packages to the local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build a binary package. Simple way to do it - use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
Debian generally lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src
. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:alex@deb:~$ egrep -v '^#|^Ubuntu checked out a lot more of its repositories during installation. But they can also be reduced to three. For example, I consider it necessary to disable universe , multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^To apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of a package is added to the repository, the system will only know about it after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way utility apt
- this is package manager
. We will consider it and other package managers in the following articles.Store changes in git
Now we come to the important thing - how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.
Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $ git tag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use such tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Добавить новый repository в debian
Теперь от теории перейдем к практике. Давайте вручную добавим новый репозиторий в Debian. К примеру, нам нужно установить на сервер стабильную версию MariaDB. Для этого добавим ее репозиторий. Это можно сделать либо в файле sources.list, но лучше создать отдельный в sources.list.d
. Назовем его MariaDB.list.deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainПосле подключения репозитория, надо добавить его gpg ключ.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Теперь обновим кэш пакетов. Это нужно делать каждый раз после подключения нового репозитория.
# apt update
Можно выполнить поиск пакета, чтобы убедиться, что новый репозиторий подключен.
# apt search mariadb-server
Как я уже говорил, для настройки нового репозитория, вы могли просто добавить эти же 2 строки с параметрами в sources.list напрямую. Разницы никакой нет.
/etc/apt/sources.list | cat -n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiversealex@deb:~$ egrep -v '^#|^
Этот файл состоит из строк, а строки состоят из следующих столбцов:
Классы релизов в Debian
Рассматривая выше ветки репозиториев Debian мы увидели следующее:
- bullseye;
- bullseye-updates;
- bullseye-security.
Но, помимо кодовых имён версий системы, в названиях веток, можно использовать специальные классы релизов:
- stable — ссылается на текущей стабильный репозиторий Debian, сейчас это bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing - refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports - Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .
I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror - Russia
- Enable the Security and Updates repositories
- Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it's faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General "Developer corner" of Debian community
Kubernetes online course
Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.
If you answer "yes" to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author's telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat << EOF | tee -a ~/.bashrc 2>/dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat << EOF |tee ~/.quiltrc 2>/dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^set mouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don't forget to setup your git variables. Full explanation may be found here, I will do it in fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Добавление сторонних репозиториев
Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.
Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxИли для Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxДопустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.
Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.
По нашему примеру для nginx нужно создать следующий файл:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin-Priority: 900
- Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
- Pin: опции прикрепления. There are many options, I'll just go over a few:
- origin "author or vendor name";
- release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository's Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let's try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Example for wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Example for curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxAnd try again to apply the changes:
alex@deb:~$ sudo apt update Ex:1 http://security.debian.org/debian-security bullseye-security InRelease Ex:2 http://deb.debian.org/debian bullseye InRelease Ex:3 http://deb.debian.org/debian bullseye-updates InRelease Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B] Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B] Received 7633 B in 1s (9420 B/s) Reading Package Lists... Done Building the dependency tree… Done Reading Status Information… Done All packages are up to date.This time everything went well.
By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:
### Example ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx<span id="Checking the added repository
Well, to understand from which repository the package will be installed, run the command:
alex@deb:~$ apt-cache policy nginx nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesFrom the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.
Mirror of the official repository yandex mirror
Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror can also be used for network installation of systems.
Local repository
There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it - use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to an important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let's update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/ bullseye-updates mainThis file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable - links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing - refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports - Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .
I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror - Russia
- Enable the Security and Updates repositories
- Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it's faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General "Developer corner" of Debian community
Kubernetes online course
Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.
If you answer "yes" to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author's telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat </dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat </dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^setmouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don't forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third party repositories
You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.
For example, to connect the nginx repository, create the following config. For Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxLet's say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.
To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.
According to our example, for nginx we need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
- Pin: Pin options. There are many options, I'll just go over a few:
- origin "author or vendor name";
- release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository's Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let's try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Пример для wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Пример для curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxИ пробуем ещё раз применить изменения:
alex@deb:~$ sudo apt update Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease Сущ:2 http://deb.debian.org/debian bullseye InRelease Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B] Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B] Получено 7 633 B за 1с (9 420 B/s) Чтение списков пакетов… Готово Построение дерева зависимостей… Готово Чтение информации о состоянии… Готово Все пакеты имеют последние версии.На этот раз всё прошло успешно.
Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:
### Пример ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxПроверка добавленного репозитория
Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:
alex@deb:~$ apt-cache policy nginx nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesИз вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.
Зеркало официального репозитория yandex mirror
В рунете популярен репозиторий Яндекса под названием andex. Mirror - https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror можно так же использовать для сетевой установки систем.
Локальный репозиторий
Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it - use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let's update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb http://security.debian.org/debian-security bullseye-security main
3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.
Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let's update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiversealex@deb:~$ egrep -v '^#|^
This file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable - links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing - refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports - Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .
I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror - Russia
- Enable the Security and Updates repositories
- Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it's faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General "Developer corner" of Debian community
Kubernetes online course
Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.
If you answer "yes" to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author's telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat << EOF | tee -a ~/.bashrc 2>/dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat << EOF |tee ~/.quiltrc 2>/dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^set mouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don't forget to setup your git variables. Full explanation may be found here, I will do it in fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Добавление сторонних репозиториев
Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.
Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxИли для Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxДопустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.
Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.
По нашему примеру для nginx нужно создать следующий файл:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin-Priority: 900
- Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
- Pin: опции прикрепления. There are many options, I'll just go over a few:
- origin "author or vendor name";
- release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository's Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let's try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Example for wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Example for curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxAnd try again to apply the changes:
alex@deb:~$ sudo apt update Ex:1 http://security.debian.org/debian-security bullseye-security InRelease Ex:2 http://deb.debian.org/debian bullseye InRelease Ex:3 http://deb.debian.org/debian bullseye-updates InRelease Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B] Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B] Received 7633 B in 1s (9420 B/s) Reading Package Lists... Done Building the dependency tree… Done Reading Status Information… Done All packages are up to date.This time everything went well.
By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:
### Example ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx<span id="Checking the added repository
Well, to understand from which repository the package will be installed, run the command:
alex@deb:~$ apt-cache policy nginx nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesFrom the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.
Mirror of the official repository yandex mirror
Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror can also be used for network installation of systems.
Local repository
There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it - use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let's update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates mainThis file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing — refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports — Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .
I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror — Russia
- Enable the Security and Updates repositories
- Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it’s faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General «Developer corner» of Debian community
Kubernetes online course
Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.
If you answer «yes» to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author’s telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat </dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat </dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^setmouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third party repositories
You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.
For example, to connect the nginx repository, create the following config. For Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxLet’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.
To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.
According to our example, for nginx we need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
- Pin: Pin options. There are many options, I’ll just go over a few:
- origin «author or vendor name»;
- release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository’s Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let’s try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Пример для wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Пример для curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxИ пробуем ещё раз применить изменения:
alex@deb:~$ sudo apt update Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease Сущ:2 http://deb.debian.org/debian bullseye InRelease Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B] Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B] Получено 7 633 B за 1с (9 420 B/s) Чтение списков пакетов… Готово Построение дерева зависимостей… Готово Чтение информации о состоянии… Готово Все пакеты имеют последние версии.На этот раз всё прошло успешно.
Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:
### Пример ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxПроверка добавленного репозитория
Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:
alex@deb:~$ apt-cache policy nginx nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesИз вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.
Зеркало официального репозитория yandex mirror
В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror можно так же использовать для сетевой установки систем.
Локальный репозиторий
Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it — use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.
Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiversealex@deb:~$ egrep -v ‘^#|^
This file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing — refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports — Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .
I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror — Russia
- Enable the Security and Updates repositories
- Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it’s faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General «Developer corner» of Debian community
Kubernetes online course
Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.
If you answer «yes» to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author’s telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat </dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat </dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^setmouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third party repositories
You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.
For example, to connect the nginx repository, create the following config. For Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxLet’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.
To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.
According to our example, for nginx we need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
- Pin: Pin options. There are many options, I’ll just go over a few:
- origin «author or vendor name»;
- release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository’s Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let’s try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Пример для wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Пример для curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxИ пробуем ещё раз применить изменения:
alex@deb:~$ sudo apt update Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease Сущ:2 http://deb.debian.org/debian bullseye InRelease Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B] Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B] Получено 7 633 B за 1с (9 420 B/s) Чтение списков пакетов… Готово Построение дерева зависимостей… Готово Чтение информации о состоянии… Готово Все пакеты имеют последние версии.На этот раз всё прошло успешно.
Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:
### Пример ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxПроверка добавленного репозитория
Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:
alex@deb:~$ apt-cache policy nginx nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesИз вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.
Зеркало официального репозитория yandex mirror
В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror можно так же использовать для сетевой установки систем.
Локальный репозиторий
Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it — use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates mainThis file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing — refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports — Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .
I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror — Russia
- Enable the Security and Updates repositories
- Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it’s faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General «Developer corner» of Debian community
Kubernetes online course
Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.
If you answer «yes» to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author’s telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat </dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat </dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^setmouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third party repositories
You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.
For example, to connect the nginx repository, create the following config. For Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxLet’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.
To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.
According to our example, for nginx we need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
- Pin: Pin options. There are many options, I’ll just go over a few:
- origin «author or vendor name»;
- release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository’s Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let’s try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Пример для wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Пример для curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxИ пробуем ещё раз применить изменения:
alex@deb:~$ sudo apt update Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease Сущ:2 http://deb.debian.org/debian bullseye InRelease Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B] Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B] Получено 7 633 B за 1с (9 420 B/s) Чтение списков пакетов… Готово Построение дерева зависимостей… Готово Чтение информации о состоянии… Готово Все пакеты имеют последние версии.На этот раз всё прошло успешно.
Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:
### Пример ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxПроверка добавленного репозитория
Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:
alex@deb:~$ apt-cache policy nginx nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesИз вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.
Зеркало официального репозитория yandex mirror
В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror можно так же использовать для сетевой установки систем.
Локальный репозиторий
Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it — use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb http://security.debian.org/debian-security bullseye-security main
3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.
Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiversealex@deb:~$ egrep -v ‘^#|^
This file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing — refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports — Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .
I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror — Russia
- Enable the Security and Updates repositories
- Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it’s faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General «Developer corner» of Debian community
Kubernetes online course
Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.
If you answer «yes» to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author’s telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat << EOF | tee -a ~/.bashrc 2>/dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat << EOF |tee ~/.quiltrc 2>/dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^set mouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don’t forget to setup your git variables. Full explanation may be found here, I will do it in fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Добавление сторонних репозиториев
Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.
Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxИли для Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxДопустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.
Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.
По нашему примеру для nginx нужно создать следующий файл:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin-Priority: 900
- Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
- Pin: опции прикрепления. There are many options, I’ll just go over a few:
- origin «author or vendor name»;
- release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository’s Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let’s try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Example for wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Example for curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxAnd try again to apply the changes:
alex@deb:~$ sudo apt update Ex:1 http://security.debian.org/debian-security bullseye-security InRelease Ex:2 http://deb.debian.org/debian bullseye InRelease Ex:3 http://deb.debian.org/debian bullseye-updates InRelease Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B] Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B] Received 7633 B in 1s (9420 B/s) Reading Package Lists... Done Building the dependency tree… Done Reading Status Information… Done All packages are up to date.This time everything went well.
By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:
### Example ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx<span id="Checking the added repository
Well, to understand from which repository the package will be installed, run the command:
alex@deb:~$ apt-cache policy nginx nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesFrom the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.
Mirror of the official repository yandex mirror
Yandex repository called andex is popular in RuNet. Mirror — https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror can also be used for network installation of systems.
Local repository
There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it — use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.
/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates mainThis file is made up of lines, and the lines are made up of the following columns:
Debian release classes
Browsing the Debian repository branches above, we saw the following:
- bullseye;
- bullseye-updates;
- bullseye-security .
But, in addition to the code names of system versions, special release classes can be used in branch names:
- stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
- oldstable — refers to the previous stable repository;
- testing — refers to a special branch of the repository for the development of a new stable release;
- unstable — links to the most recent, but not tested packages;
- experimental — packages that have just started development are stored here ;
- backports — Links to testing and unstable, but only for security updates.
That is, you can change your repositories to testing, and be at the forefront of progress:
### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ### deb http://deb.debian.org/debian/ testing main deb-src http://deb.debian.org/debian/ testing main deb http://security.debian.org/debian-security testing-security main deb-src http://security.debian.org/debian-security testing-security main deb http://deb.debian.org/debian/ testing-updates main deb-src http://deb.debian.org/debian/ testing-updates mainMain, contrib, non-free branches
Each official Debian repository has 3 branches:
- main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
- contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
- non-free contains all other software that does not comply with the DFSG.
DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.
Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.
deb http://deb.debian.org/debian bullseye main deb-src http://deb.debian.org/debian bullseye mainThis is the stable repo of the current release. Next comes the security repository to install the latest security updates.
deb http://deb.debian.org/debian-security/ bullseye-security main deb-src http://deb.debian.org/debian-security/ bullseye-security mainAnd finally, stable-updates to get stable updates to the next Point Release of the current distribution.
deb http://deb.debian.org/debian bullseye-updates main deb-src http://deb.debian.org/debian bullseye-updates mainTo be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.
Sources List Generator
There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.
An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .
I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.
For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:
- Stable repository
- main branch (just disabled contrib and non-free branches)
- Mirror — Russia
- Enable the Security and Updates repositories
- Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin
As a result, I got this sources.list
#---------------------------------------------- --------------------------------# # OFFICIAL DEBIAN REPOS #---------------------------------------------------------------- ------------------------------# ###### Debian Main Repos deb http://ftp.ru.debian.org/debian/ stable main deb http://ftp.ru.debian.org/debian/ stable-updates main deb http://security.debian.org/ stable-security main #---------------------------------------------------------------- ------------------------------# # UNOFFICIAL REPOS #---------------------------------------------------------------- ------------------------------# ###### 3rd Party Binary Repos ###DockerCE deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable ###MariaDB deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main ###nginx deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx ###NodeJs deb https://deb.nodesource.com/node_12.x bullseye main deb-src https://deb.nodesource.com/node_12.x bullseye main ###PHP deb https://packages.sury.org/php/ bullseye main ###webmin deb http://download.webmin.com/download/repository sarge contribIt also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.
In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.
Tips and tricks
- Make your own podman image, it’s faster
- Use this page. Here you can find a large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
- If you want to add new package to Debian, check this list if requests here
- General «Developer corner» of Debian community
Kubernetes online course
Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.
If you answer «yes» to at least one question, then this is your course:
- Tired of wasting time on automation?
- want consistent environments?;
- Do you want to develop and use modern tools?
- Do you care about infrastructure reliability?
- Do you have to scale your infrastructure to meet growing business needs?
- Do you want to free your product teams from some administration and automation tasks and focus them on product development?
Take the entrance test at link and join the new recruitment!.
Did the article help? Subscribe to author’s telegram channel
Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.
Build src package
Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.
dch -r --distribution testing ignored vim debian/changelog $ cat debian/changelog cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300We are ready to build source package. I prefer to use debuild tool:
debuild -us -uc -SAnd after some magic we will see ready files:
$ ls -1 ./ cri-o cri-o-1.26.0 cri-o-1.26.0.tar.gz cri-o-v1.26.0 cri-o-v1.26.0.tar.gz cri-o_1.26.0-1.debian.tar.xz cri-o_1.26.0-1.dsc cri-o_1.26.0-1_amd64.build cri-o_1.26.0-1_amd64.buildinfo cri-o_1.26.0-1_amd64.changes cri-o_1.26.0-1_amd64.deb cri-o_1.26.0-1_source.build cri-o_1.26.0-1_source.buildinfo cri-o_1.26.0-1_source.changes cri-o_1.26.0.orig.tar.gz cri-o_v1.26.0.orig.tar.gzEnvironment preparation
First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc
sed -i '/DEBEMAIL/d' ~/.bashrc sed -i '/DEBFULLNAME/d' ~/.bashrc cat </dev/null DEBEMAIL="skif@skif-web.ru" DEBFULLNAME="Alexey Lukyanchuk" export DEBEMAIL DEBFULLNAME EOF source ~/.bashrcSecond step is config for quilt:
cat </dev/null QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`. QUILT_PATCH_OPTS="--reject-format=unified" QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33" EOFAnd I detest visual mode in vim, so
touch ~/.vimrc sed -i '/^setmouse/d' ~/.vimrc echo "set mouse-=a" >> ~/.vimrcAnd don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:
git config --global user.email "skif@skif-web.ru" git config --global user.name "Alexey Lukyanchuk"If we talk about Debian, you need to install some dependencies:
- build-essential
- debmake
- quilt
- devscripts
I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.
Adding third party repositories
You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.
For example, to connect the nginx repository, create the following config. For Debian:
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/debian bullseye nginxOr for Ubutnu:
alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb http://nginx.org/packages/ubuntu jammy nginxLet’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.
To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.
According to our example, for nginx we need to create the following file:
$ sudo nano /etc/apt/preferences.d/99nginx Package: * Pin: origin nginx.org Pin: release o=nginx Pin Priority: 900
- Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
- Pin: Pin options. There are many options, I’ll just go over a few:
- origin «author or vendor name»;
- release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
- release l=Debian means that the repository’s Release file has a Label (l) named Debian;
- Pin-Priority: Priority.
That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.
Priority can be in the following ranges:
- P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
- 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
- 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
- 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
- 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
- P < 0 — the package will not be installed under any circumstances;
- P = 0 — not used.
Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:
alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "bullseye"; alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default APT::Default-Release "jammy";Repository public key
And so, we added the repository, set the priority. Let’s try to apply the changes:
alex@deb:~$ sudo apt update Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B] Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Ex:3 http://deb.debian.org/debian bullseye InRelease Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB] Reading Package Lists... Done W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62 E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed. N: This repository cannot be updated securely, so it is disabled by default. N: See the apt-secure(8) man page for information about creating a repository and user settings.And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.
To do this, first install the necessary tools:
### For Debian ### alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring ### For Ubuntu ### alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringAttention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines: deb http://ru.archive.ubuntu.com/ubuntu jammy universe deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe And don't forget to apply the changes by running: $ sudo apt updateAnd then download the public key (using wget):
$ wget https://nginx.org/keys/nginx_signing.keyNext, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:
$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null### Пример для wget ### $ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ### Пример для curl ### $ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/nullalex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxИ пробуем ещё раз применить изменения:
alex@deb:~$ sudo apt update Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease Сущ:2 http://deb.debian.org/debian bullseye InRelease Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B] Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B] Получено 7 633 B за 1с (9 420 B/s) Чтение списков пакетов… Готово Построение дерева зависимостей… Готово Чтение информации о состоянии… Готово Все пакеты имеют последние версии.На этот раз всё прошло успешно.
Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:
### Пример ### deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginxПроверка добавленного репозитория
Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:
alex@deb:~$ apt-cache policy nginx nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 PackagesИз вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.
Зеркало официального репозитория yandex mirror
В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.
deb http://mirror.yandex.ru/debian bullseye main deb-src http://mirror.yandex.ru/debian bullseye main deb http://mirror.yandex.ru/debian bullseye-updates main deb-src http://mirror.yandex.ru/debian bullseye-updates main deb https://mirror.yandex.ru/debian-security bullseye-security main deb-src https://mirror.yandex.ru/debian-security bullseye-security mainRepository yandex mirror можно так же использовать для сетевой установки систем.
Локальный репозиторий
Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:
File not found updates/main/i18n/Translation-en (2: No such file or directory)Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.
# apt install repreproNext, create a directory for the local repository and config.
# mkdir -p /mnt/repo/debian/conf # touch /mnt/repo/debian/conf/distributionsWe make the config approximately as follows.
Codename: bullseye Suite: stable Version: 11.x Origin: Debian Label: Debian 11.x Description: Debian Stable Updates Repository Architectures: amd64 source Components: main DebIndices: Packages Release . .gz .bz2 DscIndices: Sources Release . .gz .bz2 Contents: . .gz .bz2Initializing the repository.
# cd /mnt/repo/debian # reprepro export # reprepro createsymlinksYou can now add packages to your local repository with the following command.
# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.debIn order to connect a new repository locally, you need to add it to sources.list.
deb [trusted=yes] file:/mnt/repo/debian bullseye mainAfter that, update the package cache and you will see your local one in the list of repositories.
Build binary package and check
Ok, now we can build binary package. Simple way to do it — use debuild again:
debuild -us -ucAnd now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);
$ ls -1 /volume/*.deb /volume/cri-o_1.26.0-1_amd64.debTheory
Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.
I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:
- control: file with mandatory information like package name, version, source, checksums, other data
- rules: make-file with instructions how to build software
- patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.
Option to use the official repositories
In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.
In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:
alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates mainUbuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:
alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restrictedTo apply the changes, run the following command on both systems:
$ sudo apt updateThis command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.
And if you want to update the system, then run the command:
$ sudo apt upgradeThis command already downloads all updates and installs them.
By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.
Store changes in git
Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:
make -f debian/rules src-clean dh_clean
- version before release: v${version}-rc${number}
- release version: v$(version)-release
$ git tag -a v1.26.0-rc0 $gittag v1.26.0-rc0Git interaction schema
Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.
I prefer to use these tips:
- store ready src-deb and binary-deb in repo (everybody does it)
- make separate git for your debian-dir with files for source-package
- create and use additional target in rules:
- src-clean to clean «vanilla» source codes
- src-get to get «vanilla» source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)
Therefore there are 2 gits:
- original software git, called vanilla
- your git with debian dir for package creation
Add new repository to debian
Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.
deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye mainAfter connecting the repository, you need to add its gpg key.
# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc # chmod -c 644 mariadb_release_signing_key.asc # mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/Now let’s update the package cache. This must be done every time after connecting a new repository.
# apt update
You can search for the package to make sure the new repository is connected.
# apt search mariadb-server
As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.</p




