Deb src

Deb src Хостинг
Содержание
  1. Possible errors
  2. Repository does not contain a Release file
  3. Become a maintainer. Part Four
  4. Structure of the repository
  5. apt-ftparchive
  6. reprepro
  7. Configs with a list of repositories
  8. Debian release classes
  9. Branches main, contrib, non-free
  10. Sources List Generator
  11. Tips and tricks
  12. Kubernetes online course
  13. Did the article help? Subscribe to telegram channel author
  14. Build src package
  15. Environment preparation
  16. Adding third-party repositories
  17. Repository public key
  18. Checking the added repository
  19. Mirror of the official repository yandex mirror
  20. Local repository
  21. Build binary package and check
  22. Theory
  23. Option to use official repositories
  24. Store changes in git
  25. Git interaction schema
  26. Добавить новый repository в debian
  27. Классы релизов в Debian
  28. Main, contrib, non-free branches
  29. Sources List Generator
  30. Tips and tricks
  31. Kubernetes online course
  32. Did the article help? Subscribe to author's telegram channel
  33. Build src package
  34. Environment preparation
  35. Добавление сторонних репозиториев
  36. Repository public key
  37. <span id="Checking the added repository
  38. Mirror of the official repository yandex mirror
  39. Local repository
  40. Build binary package and check
  41. Theory
  42. Option to use the official repositories
  43. Store changes in git
  44. Git interaction schema
  45. Add new repository to debian
  46. Debian release classes
  47. Main, contrib, non-free branches
  48. Sources List Generator
  49. Tips and tricks
  50. Kubernetes online course
  51. Did the article help? Subscribe to author's telegram channel
  52. Build src package
  53. Environment preparation
  54. Adding third party repositories
  55. Repository public key
  56. Проверка добавленного репозитория
  57. Зеркало официального репозитория yandex mirror
  58. Локальный репозиторий
  59. Build binary package and check
  60. Theory
  61. Option to use the official repositories
  62. Store changes in git
  63. Git interaction schema
  64. Add new repository to debian
  65. Store changes in git
  66. Git interaction schema
  67. Add new repository to debian
  68. Debian release classes
  69. Main, contrib, non-free branches
  70. Sources List Generator
  71. Tips and tricks
  72. Kubernetes online course
  73. Did the article help? Subscribe to author's telegram channel
  74. Build src package
  75. Environment preparation
  76. Добавление сторонних репозиториев
  77. Repository public key
  78. <span id="Checking the added repository
  79. Mirror of the official repository yandex mirror
  80. Local repository
  81. Build binary package and check
  82. Theory
  83. Option to use the official repositories
  84. Store changes in git
  85. Git interaction schema
  86. Add new repository to debian
  87. Debian release classes
  88. Main, contrib, non-free branches
  89. Sources List Generator
  90. Tips and tricks
  91. Kubernetes online course
  92. Did the article help? Subscribe to author’s telegram channel
  93. Build src package
  94. Environment preparation
  95. Adding third party repositories
  96. Repository public key
  97. Проверка добавленного репозитория
  98. Зеркало официального репозитория yandex mirror
  99. Локальный репозиторий
  100. Build binary package and check
  101. Theory
  102. Option to use the official repositories
  103. Store changes in git
  104. Git interaction schema
  105. Add new repository to debian
  106. Store changes in git
  107. Git interaction schema
  108. Add new repository to debian
  109. Debian release classes
  110. Main, contrib, non-free branches
  111. Sources List Generator
  112. Tips and tricks
  113. Kubernetes online course
  114. Did the article help? Subscribe to author’s telegram channel
  115. Build src package
  116. Environment preparation
  117. Adding third party repositories
  118. Repository public key
  119. Проверка добавленного репозитория
  120. Зеркало официального репозитория yandex mirror
  121. Локальный репозиторий
  122. Build binary package and check
  123. Theory
  124. Option to use the official repositories
  125. Store changes in git
  126. Git interaction schema
  127. Add new repository to debian
  128. Debian release classes
  129. Main, contrib, non-free branches
  130. Sources List Generator
  131. Tips and tricks
  132. Kubernetes online course
  133. Did the article help? Subscribe to author’s telegram channel
  134. Build src package
  135. Environment preparation
  136. Adding third party repositories
  137. Repository public key
  138. Проверка добавленного репозитория
  139. Зеркало официального репозитория yandex mirror
  140. Локальный репозиторий
  141. Build binary package and check
  142. Theory
  143. Option to use the official repositories
  144. Store changes in git
  145. Git interaction schema
  146. Add new repository to debian
  147. Store changes in git
  148. Git interaction schema
  149. Add new repository to debian
  150. Debian release classes
  151. Main, contrib, non-free branches
  152. Sources List Generator
  153. Tips and tricks
  154. Kubernetes online course
  155. Did the article help? Subscribe to author’s telegram channel
  156. Build src package
  157. Environment preparation
  158. Добавление сторонних репозиториев
  159. Repository public key
  160. <span id="Checking the added repository
  161. Mirror of the official repository yandex mirror
  162. Local repository
  163. Build binary package and check
  164. Theory
  165. Option to use the official repositories
  166. Store changes in git
  167. Git interaction schema
  168. Add new repository to debian
  169. Debian release classes
  170. Main, contrib, non-free branches
  171. Sources List Generator
  172. Tips and tricks
  173. Kubernetes online course
  174. Did the article help? Subscribe to author’s telegram channel
  175. Build src package
  176. Environment preparation
  177. Adding third party repositories
  178. Repository public key
  179. Проверка добавленного репозитория
  180. Зеркало официального репозитория yandex mirror
  181. Локальный репозиторий
  182. Build binary package and check
  183. Theory
  184. Option to use the official repositories
  185. Store changes in git
  186. Git interaction schema
  187. Add new repository to debian

Possible errors

Let’s consider the most popular errors that occur when adding and updating repositories.

Repository does not contain a Release file

The text of the error, in theory, gives a ready answer. The repository does not have a required Release file. But the bottom line is that it probably is. The point here is most often that you have added a repository to yourself that does not contain the branch you specified. For example, you added a repository to the Buster distribution, but the repository does not support this distribution. The previous ones are there, but this one is not.

Exactly the same error you’ll get if you’re using an old deprecated version of Debian. At some point, the standard repositories will stop supporting your distribution version and you will get an error. You will need to either upgrade to a more recent version or use the archive repositories.

Become a maintainer. Part Four

Time to read

While fans of exotics on Habré are actively drinking cups of Java, taking doses of F # and injecting themselves with other Haskell, you and I learned how to assemble their creations into deb packages. Over the time that has passed since the previous part, someone has probably already accumulated several ready-made packages, and we have not even tried to put them in the official Debian and Ubuntu repository yet. Therefore, it is time to think about how to organize all the accumulated wealth into one big beautiful repository, which you will not be ashamed to offer for use by other users.

(Parts 1
, 2
and 3
)


For these purposes, we will consider two programs (simply because they were enough for me to comprehend the Tao) — this is apt-ftparchive
from the standard apt-utils package and package reprepro
.

Structure of the repository

Let’s first see how a normal repository works. There are two types of repositories — ugly and oblique (or in the original — flat — flat, we will not consider them) and normal hierarchical. Here we will learn to create hierarchical ones. So, a normal repository consists of two directories: pool
and dists
.

pool/

This directory can contain packages in an arbitrary hierarchy. I say «arbitrary» because you can really arrange them however you like, but in practice the following hierarchy is usually implemented:

pool/main/q/qutim/

Here “main” is the name of the section to which we refer our package (read about sections in the first article), followed by the first letter of the src package name and then the directory named after the src package name. This directory will contain the src package and all binary packages built from it. For numerous libraries whose package names all begin with «lib», a separate «liba» is used instead of the letter «l», where «a» is the first letter of the name after «lib».

dists/

This directory is organized much more complicated. First of all, we must find a directory with the name of the repository in it — in Ubuntu these are various “intrepid”, “intrepid-proposed”, “jaunty”, etc., in Debian it is “etch”, “etch-proposed-updates” , «lenny» as well as their synonyms — «stable», «testing», etc. — they are usually symlinks to the canonical name. Thus, you can store packages for several repositories in one place at once.

  • Contents-arch
    — a file that contains a list of all files from all packages. «arch» is replaced with the name of the package architecture, and may also end with «.gz» and «.bz2» (if the file is properly compressed). It is optional and is used primarily by utilities like apt-file to find out which package contains the required file. Name example — Contents-amd64.gz
  • Release
    is a file that contains a short description of the repository and hashes of all supporting files that are contained in dists/
    .
  • Release.gpg
    is the gpg signature of the Release file, certifying the creation of the file by a specific author. Both files are optional, but without them, APT is guaranteed to swear at an unauthenticated repository.

In addition, in the same directory we will find directories with names corresponding to the names of the repository sections — main, contrib, non-free or main, universe, multiverse, restricted, or those that you define yourself.

dists/unstable/main/

In each section (not only main, of course) you can see a set of directories like this:

  • binary-arch
    — a directory for describing packages of the corresponding architecture, for example, binary-amd64
    — for amd64 architecture. Packages with architecture all (such as dev packages) will be listed in the directory for each architecture.
  • source
    — this directory describes src packages
  • i18n
    — this directory is rarely used and mostly only in official repositories — it contains files with translations of package descriptions into other languages. I won’t describe this directory as it’s not really that important if you want to use it — you can always look in the ubuntu repository

dists/unstable/main/binary-i386/

The file Packages is stored here
(and also Packages.gz
and Packages.bz2
) containing descriptions of binary packages of the corresponding architecture. In fact, it is a combination of control files from all binary packages of the repository for a given architecture. Often you can still find a file here Release
, which, unlike the same file in dists/unstable/, does not contain file hashes, but only stores descriptions of the repository for the given directory.

dists/unstable/main/source/

Everything here is the same as in binary-arch, except that the word Sources is used instead of the word Packages.

apt-ftparchive

Of course, all these files can be written with pens, but I would not advise you. Especially if you have at least 3-4 packages. Therefore, we will use the magic utility apt-ftparchive from the apt-utils package and dpkg-scanpackages from the dpkg-dev package.

So, let’s say we have a certain qutim src package created by us in the previous series, and the qutim and qutim-dev binary packages built from it. We collected them for two architectures, so now we have the following set of files:

qutim_0.1.99.138.orig.tar.gz
qutim_0.1.99.138-1.diff.gz
qutim_0.1.99.138-1.dsc
qutim_0.1.99.138-1_amd64.deb
qutim_0.1.99.138-1_i386.deb
qutim-dev_0.1.99.138-1_all.deb

The first three files, as we remember, are the src package, and the remaining ones are the compiled binary packages. We also have the qutim_0.1.99.138-1_i386.changes and qutim_0.1.99.138-1_amd64.changes files generated when the package was built.

Let’s build a minimally decent repository out of them.

So, let’s create a directory with the initial structure and copy the packages into it:

$ mkdir -p rep/dists

$ mkdir -p rep/pool/main

$ cp qutim_0.1.99.138.orig.tar.gz qutim_0.1.99.138-1.diff.gz qutim_0.1.99.138-1.dsc qutim_0.1.99.138-1_amd64.deb qutim_0.1.99.138-1_i386. deb qutim-dev_0.1.99.138-1_all.deb rep/pool/main/

$ cd rep

As you can see, there is nothing difficult at all in placing the packages themselves in the repository. It will be slightly more complicated to generate a set of descriptions for these packages:

$ mkdir -p dists/unstable/main/binary-amd64

$ mkdir dists/unstable/main/binary-i386

$ mkdir dists/unstable/main/source

$ dpkg-scanpackages -a amd64 pool/main >dists/unstable/main/binary-amd64/Packages 2>/dev/null

$ dpkg-scanpackages -a amd64 pool/main | gzip -c9 >dists/unstable/main/binary-amd64/Packages.gz 2>/dev/null

$ dpkg-scanpackages -a amd64 pool/main | bzip2 -c9 >dists/unstable/main/binary-amd64/Packages.bz2 2>/dev/null

$ dpkg-scanpackages -a i386 pool/main >dists/unstable/main/binary-i386/Packages 2>/dev/null

$ dpkg-scanpackages -a i386 pool/main | gzip -c9 > dists/unstable/main/binary-i386/Packages.gz 2>/dev/null

$ dpkg-scanpackages -a i386 pool/main | bzip2 -c9 > dists/unstable/main/binary-i386/Packages.bz2 2>/dev/null

$ apt-ftparchive sources pool > dists/unstable/main/source/Sources 2>/dev/null

$ apt-ftparchive sources pool | gzip -c9 > dists/unstable/main/source/Sources.gz 2>/dev/null

$ apt-ftparchive sources pool | bzip2 -c9 > dists/unstable/main/source/Sources.bz2 2>/dev/null

$ echo "Archive: unstable" > dists/unstable/Release

$ echo "Suite: unstable" >> dists/unstable/Release

$ echo "Components: main" >> dists/unstable/Release

$ echo "Origin: qutim.org" >> dists/unstable/Release

$ echo "Label: qutim.org Debian repository" >> dists/unstable/Release

$ echo "Architectures: amd64 i386" >> dists/unstable/Release

$ echo "Description: Debian qutIM unstable" >> dists/unstable/Release

$ apt-ftparchive release dists/unstable >> dists/unstable/Release

Описание всё равно писать руками, apt-ftparchive делать это не умеет.

And last step — subscribe our repository:

$ gpg -abs -o dists/unstable/Release.gpg dists/unstable/Release

Voila, our repository can be used. Add to /etc/apt/sources.list lines:

deb file:///path/to/rep unstable main

deb-src file:///path/to/rep unstable main

Update the list of packages — and the repository can be used.

Разумеется, это не очень удобный — перегенерировать файли запасний продётся качей раз при обновлении набора пакетов. Of course, everything is automated by scripts, but it’s still so uninteresting. Therefore, we will move on to the second program used, which will do everything for us.

reprepro

This program is simply paradise for those who don’t want to bother with the structure of the repository at all, but want to configure and use it once. Therefore, we take down our homemade repository and put the reprepro package:

$ cd . && rm -rf rep/*

$ sudo apt-get install reprepro

$ mkdir rep/conf

Now we create rep/conf file distributions in the catalog, in which we will describe our repository:

Codename: lenny

Suite: Unstable

Version: unstable

Origin: qutim.org

Label: qutim.org Debian Repository

Description: qutim.org Debian repository

Architectures: source i386 amd64

Components: main

SignWith: default

DebIndices: Packages Release gz bz2

DscIndices: Sources Release gz bz2

Contents: gz bz2

Codename should contain the name of the repository (lenny, hardy, etch, jaunty), and Suite — synonym for creating symbolic links (stable, unstable). Parameters Version, Origin, Label and Description do not carry a special semantic load and are simply copied to the Release file. Параметр SignWith indicates на необходим подписать репозиторий, создав файл Release.gpg. DebIndices, DscIndices and Contents — indicate the necessity of creating file Packages, Sources and Contents-arch — in simple and archived form.

Now we can add our packages:

$ reprepro -b rep/ createsymlinks

$ reprepro -b rep/ --ask-passphrase -C main include unstable qutim_0.1.99.138-1_amd64.changes

$ reprepro -b rep/ --ask-passphrase -C main includedeb unstable qutim_0.1.99.138-1_i386.deb

ключ -b indicates reprepro, где лежит наш репозиторий, the command —ask-passphrase informs that наш gpg-ключ is zaparolen и при подписывании дистриктива недежда бы просирой пароль, ключ -C informs that мы включаем пакет в сексию main. One interesting moment should be noted here. The include command accepts as an argument the name of the repository to include the file and the name of the .changes file created when compiling the src-package. At the same time, the src package and the compiled binary packages for this architecture are added to the repository. And here we have a problem — the developers of reprepro flatly refuse to add packages to the repository of the version that already exists in it. Therefore, if we now try to specify a .changes file for a different architecture, we will get an error — we have already added not only the src package, but also the qutim-dev package compiled for all architectures. For this case, there is the includedeb command, which will allow you to specify only a deb file with a different architecture.

View the contents of the rep directory now. The conf subdirectory contains the repository configuration (more on this in ` man reprepro
`), the db directory is its internal database. But the pool and dists directories have become very different from what we created with apt-ftparchive and have become similar to the structure that I described at the beginning. Since we already have the repository address in sources.list, we type ` sudo apt-get update
`and use 🙂

Configs with a list of repositories

Package managers that can install packages from repositories need to know the addresses of the repositories. And these addresses are written to the config — /etc/apt/sources.list.
And you can also create additional configs with the extension .list
in directory /etc/apt/sources.list.d/
. All this is true for Debian
and for Ubuntu
.

If you remember, during the installation of systems, we chose a repository:

  • for Debian — deb.debian.org
    ;
  • for Ubuntu — en.archive.ubuntu.com
    .
 alex@ubu:~$ egrep -v '^#|^ 

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Looking at the Debian repository branches above, we saw the following:

  • bullseye ;
  • bullseye-updates ;
  • bullseye security .

But, in addition to code names of system versions, special release classes can be used in branch names:

  • stable - refers to the current Debian stable repository, currently bullseye . As soon as a new version of Debian is released, stable will link to the newer version;
  • oldstable - refers to the previous stable repository;
  • testing - refers to a special branch of the repository for the development of a new stable release;
  • unstable - links to the latest, but not tested packages;
  • experimental - packages that have just begun to be developed are stored here;
  • backports - refers to testing and unstable , but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

 ### This is just an example, there is a high probability that the system will be damaged very soon due to untested updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main 

Branches main, contrib, non-free

Each official Debian repository has 3 branches:

  1. main
    consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib
    packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free
    contains all other software that does not comply with the DFSG.

DFSG - Debian Free Software Guidelines
, Debian Free Software Criteria
. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now, knowing all the theory about Debian repositories, we can analyze the sources.list file that we received after installation. It has 3 repositories with branches main.

 deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main 

This is a stable repo of the current release. Next comes the security repository for installing the latest security updates.

 deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main 

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

 deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main 

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use a ready-made one.

An example of such a generator that you can install yourself and configure for use - debgen
. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/
.

I don't know who maintains such List Generators and whether they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror - Russia
  4. Enable Security and Updates repositories
  5. Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

 #--------------------------------------------- ---------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib 

It also has a list of gpg keys for import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggestions.

In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it's faster
  • Use this page
    . Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General "Developer corner" of Debian community

Kubernetes online course

Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.

If you answer "yes" to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want uniform environments?;
  • want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • have to scale the infrastructure to meet the growing needs of the business?
  • Do you want to free product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link
and join the new set!

Did the article help? Subscribe to telegram channel
author

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

 dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgency=low * Initial release. -- Alexey Lukyanchuk <skif@skif-web.ru> Wed, 04 Jan 2023 11:10:46 +0300 

We are ready to build source package. I prefer to use debuild tool:

 debuild -us -uc -S 

And after some magic we will see ready files:

 $ ls -1 ../
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz 

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

 sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat << EOF | tee -a ~/.bashrc 2>/dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc 

Second step is config for quilt:

 cat << EOF |tee ~/.quiltrc 2>/dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF 

And I detest visual mode in vim, so

 touch ~/.vimrc
sed -i '/^set mouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc 

And don't forget to set up your git variables. Full explanation may be found here
, I will do it in fast way:

 git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk" 

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third-party repositories

You can add repositories to the main config: /etc/apt/sources.list
or create separate configs in the directory /etc/apt/sources.list.d/
. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the repository nginx
create the following config. For Debian
:

 alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu
:

 alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx 

Suppose we have registered an additional repository for nginx
, but how can the system understand from which repository to take the package for installation? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, create a file /etc/apt/preferences.d/XX
, where XX is the file number, the higher it is, the later it will be processed, that is, it will take precedence over other settings.

According to our example for nginx, you need to create the following file:

 $ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900 
  • Package
    : package name. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin
    : Attachment options. There are many options, I'll just cover a few:

    • origin
      "name of author or supplier";
    • release o=nginx
      - means that there is a provider (Origin = o) with the name nginx in the repository's Release file;
    • release l=Debian
      - means that in the Release file of the repository there is a Label (l) with the name Debian;
  • Pin Priority
    : a priority.

That is Package
and Pin
these are the conditions for assigning a priority, and Pin-Priority
it is an action (assigning a priority). In our example, the following is obtained: if the package name is anything, but the owner of the repository is nginx.org
and in file Release
written " Origin: nginx
“, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000
    - the package will be installed from this repository, even if it will downgrade an already installed package;
  • 990 <= P < 1000
    - the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990
    — package will be installed if there is no package belonging to target release
    or a newer version is not installed;
  • 100 <= P < 500
    - the package will be installed if there are no candidates from other repositories or an installed package of a newer version;
  • 0 < P < 100
    - the package will be installed only if it is not installed yet (of any version) and if there are no candidates from other repositories;
  • P < 0
    - the package will not be installed under any circumstances;
  • P = 0
    - not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To distinguish them, you need to understand what is target release
. For Ubuntu or Debian, this is the version name of the distribution. For example for Ubuntu — jammy
, and for Debian - bullseye
. But this name still needs to be set in this way:

 alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy"; 

Repository public key

And so, we added the repository, set the priority. Let's try to apply the changes:

 alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure

man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

 ### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring 
  Attention!  Utility  gnupg2  for  Ubuntu  only available in repository  universe , if you have commented these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update 

And then download the public key (using wget
):

 $ wget https://nginx.org/keys/nginx_signing.key 

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With cat
we read the key file, and pass what we read to the gpg utility. Utility gpg
translates the read into the required format and passes the output to the next tee command. Utility tee
(under sudo) saves the resulting text to a file. At the end add >/dev/null
so that there is no output to the terminal. Here is the command itself:

 $ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null 
 ### Example for wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null 
 alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx 

And try again to apply the changes:

 alex@deb:~$ sudo apt update
Ex:1 http://security.debian.org/debian-security bullseye-security InRelease
Ex:2 http://deb.debian.org/debian bullseye InRelease
Ex:3 http://deb.debian.org/debian bullseye-updates InRelease
Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B]
Received 7633 B in 1s (9420 B/s)
Reading Package Lists... Done
Building the dependency tree… Done
Reading Status Information… Done
All packages are up to date. 

Everything went well this time.

By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:

 ### Example ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx 

Checking the added repository

Well, to understand from which repository the package will be installed, run the command:

 alex@deb:~$ apt-cache policy nginx
nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages 

From the output it becomes clear that the package nginx
not yet installed on the system, and a package with version 1.20.2 is a candidate for installation
from repository http://nginx.org/packages/debian
. The priority of all packages, by the way, became equal to = 990
. This happened after we set target release = bullseye
. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned repositories for bullseye
such a priority.

Mirror of the official repository yandex mirror

A Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru
. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.

 deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main 

Repository yandex mirror can also be used for network installation of systems.

Local repository

There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

 File not found updates/main/i18n/Translation-en (2: No such file or directory) 

Another simple option is to use reprepro
. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

 # apt install reprepro 

Next, create a directory for the local repository and config.

 # mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions 

We make the config approximately as follows.

 Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2 

We initialize the repository.

 # cd /mnt/repo/debian
# reprepro export
#reprepro createsymlinks 

Now you can add packages to the local repository with the following command.

 # reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb 

In order to connect a new repository locally, you need to add it to sources.list.

 deb [trusted=yes] file:/mnt/repo/debian bullseye main 

After that, update the package cache and you will see your local one in the list of repositories.

Локальный репозиторий Debian

Build binary package and check

Ok, now we can build a binary package. Simple way to do it - use debuild again:

 debuild -us -uc 

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

 $ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb 

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.

I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

Debian generally lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src
. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

 alex@deb:~$ egrep -v '^#|^ 

Ubuntu checked out a lot more of its repositories during installation. But they can also be reduced to three. For example, I consider it necessary to disable universe , multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

 alex@ubu:~$ egrep -v '^#|^ 

To apply the changes, run the following command on both systems:

 $ sudo apt update 

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of a package is added to the repository, the system will only know about it after the next execution of this command.

And if you want to update the system, then run the command:

 $ sudo apt upgrade 

This command already downloads all updates and installs them.

By the way utility apt
- this is package manager
. We will consider it and other package managers in the following articles.

Store changes in git

Now we come to the important thing - how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?

I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

 make -f debian/rules src-clean
dh_clean 

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

 $ git tag -a v1.26.0-rc0
$ git tag
v1.26.0-rc0 

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use such tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean "vanilla" source codes
    • src-get to get "vanilla" source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Добавить новый repository в debian

Теперь от теории перейдем к практике. Давайте вручную добавим новый репозиторий в Debian. К примеру, нам нужно установить на сервер стабильную версию MariaDB. Для этого добавим ее репозиторий. Это можно сделать либо в файле sources.list, но лучше создать отдельный в sources.list.d
. Назовем его MariaDB.list.

 deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main 

После подключения репозитория, надо добавить его gpg ключ.

 # curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/ 

Теперь обновим кэш пакетов. Это нужно делать каждый раз после подключения нового репозитория.

 # apt update 

Добавление репозитория в Debian

Можно выполнить поиск пакета, чтобы убедиться, что новый репозиторий подключен.

 # apt search mariadb-server 

Новый репозиторий

Как я уже говорил, для настройки нового репозитория, вы могли просто добавить эти же 2 строки с параметрами в sources.list напрямую. Разницы никакой нет.

/etc/apt/sources.list | cat -n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiverse

alex@deb:~$ egrep -v '^#|^

Этот файл состоит из строк, а строки состоят из следующих столбцов:

Классы релизов в Debian

Рассматривая выше ветки репозиториев Debian мы увидели следующее:

  • bullseye;
  • bullseye-updates;
  • bullseye-security.

Но, помимо кодовых имён версий системы, в названиях веток, можно использовать специальные классы релизов:

  • stable — ссылается на текущей стабильный репозиторий Debian, сейчас это bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing - refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports - Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .

I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror - Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it's faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General "Developer corner" of Debian community

Kubernetes online course

Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.

If you answer "yes" to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author's telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat << EOF | tee -a ~/.bashrc 2>/dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat << EOF |tee ~/.quiltrc 2>/dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^set mouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don't forget to setup your git variables. Full explanation may be found here, I will do it in fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Добавление сторонних репозиториев

Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.

Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Или для Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Допустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.

Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.

По нашему примеру для nginx нужно создать следующий файл:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin-Priority: 900
  • Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
  • Pin: опции прикрепления. There are many options, I'll just go over a few:
    • origin "author or vendor name";
    • release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository's Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let's try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

And try again to apply the changes:

alex@deb:~$ sudo apt update
Ex:1 http://security.debian.org/debian-security bullseye-security InRelease
Ex:2 http://deb.debian.org/debian bullseye InRelease
Ex:3 http://deb.debian.org/debian bullseye-updates InRelease
Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B]
Received 7633 B in 1s (9420 B/s)
Reading Package Lists... Done
Building the dependency tree… Done
Reading Status Information… Done
All packages are up to date.

This time everything went well.

By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:

### Example ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

<span id="Checking the added repository

Well, to understand from which repository the package will be installed, run the command:

alex@deb:~$ apt-cache policy nginx
nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

From the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.

Mirror of the official repository yandex mirror

Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror can also be used for network installation of systems.

Local repository

There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

 Debian local repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it - use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to an important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean "vanilla" source codes
    • src-get to get "vanilla" source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let's update the package cache. This must be done every time after connecting a new repository.

# apt update

 Adding a repository to Debian 02.png 909w, https://serveradmin.ru/wp-content/uploads/2021/08/debian-repository-configure-02-300x130.png 300w, https://serveradmin.ru/wp-content/uploads/ 2021/08/debian-repository-configure-02-768x333.png 768w

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

 New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/ bullseye-updates main

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable - links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing - refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports - Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .

I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror - Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it's faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General "Developer corner" of Debian community

Kubernetes online course

Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.

If you answer "yes" to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author's telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat </dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat </dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^setmouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don't forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third party repositories

You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the nginx repository, create the following config. For Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Let's say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.

According to our example, for nginx we need to create the following file:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900
  • Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin: Pin options. There are many options, I'll just go over a few:
    • origin "author or vendor name";
    • release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository's Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let's try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

И пробуем ещё раз применить изменения:

alex@deb:~$ sudo apt update
Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease
Сущ:2 http://deb.debian.org/debian bullseye InRelease
Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease
Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B]
Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B]
Получено 7 633 B за 1с (9 420 B/s)
Чтение списков пакетов… Готово
Построение дерева зависимостей… Готово
Чтение информации о состоянии… Готово
Все пакеты имеют последние версии.

На этот раз всё прошло успешно.

Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:

### Пример ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

Проверка добавленного репозитория

Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:

alex@deb:~$ apt-cache policy nginx
nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

Из вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.

Зеркало официального репозитория yandex mirror

В рунете популярен репозиторий Яндекса под названием andex. Mirror - https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror можно так же использовать для сетевой установки систем.

Локальный репозиторий

Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

Local Debian repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it - use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean "vanilla" source codes
    • src-get to get "vanilla" source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let's update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb http://security.debian.org/debian-security bullseye-security main
3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?

I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean "vanilla" source codes
    • src-get to get "vanilla" source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let's update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiverse

alex@deb:~$ egrep -v '^#|^

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable - links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing - refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports - Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG - Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content - https://debgen.simplylinux.ch/ .

I don't know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror - Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software - Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I'm not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don't see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it's faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General "Developer corner" of Debian community

Kubernetes online course

Kubernetes online course - for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners - you need to pass an entrance test.

If you answer "yes" to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author's telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat << EOF | tee -a ~/.bashrc 2>/dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat << EOF |tee ~/.quiltrc 2>/dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^set mouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don't forget to setup your git variables. Full explanation may be found here, I will do it in fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Добавление сторонних репозиториев

Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.

Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Или для Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Допустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.

Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.

По нашему примеру для nginx нужно создать следующий файл:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin-Priority: 900
  • Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
  • Pin: опции прикрепления. There are many options, I'll just go over a few:
    • origin "author or vendor name";
    • release o=nginx - means that the repository's Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository's Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says "Origin: nginx", then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 - the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 - the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 - the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 - the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu - jammy, and for Debian - bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let's try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I'll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

And try again to apply the changes:

alex@deb:~$ sudo apt update
Ex:1 http://security.debian.org/debian-security bullseye-security InRelease
Ex:2 http://deb.debian.org/debian bullseye InRelease
Ex:3 http://deb.debian.org/debian bullseye-updates InRelease
Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B]
Received 7633 B in 1s (9420 B/s)
Reading Package Lists... Done
Building the dependency tree… Done
Reading Status Information… Done
All packages are up to date.

This time everything went well.

By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:

### Example ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

<span id="Checking the added repository

Well, to understand from which repository the package will be installed, run the command:

alex@deb:~$ apt-cache policy nginx
nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

From the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.

Mirror of the official repository yandex mirror

Yandex repository called andex is popular in RuNet. Mirror - https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror can also be used for network installation of systems.

Local repository

There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won't go into detail on setting up a local repository for Debian, as that's a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

 Debian local repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it - use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches - directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean - how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean "vanilla" source codes
    • src-get to get "vanilla" source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let's move from theory to practice. Let's manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let's call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let's update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates main

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing — refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports — Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .

I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror — Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it’s faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General «Developer corner» of Debian community

Kubernetes online course

Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.

If you answer «yes» to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author’s telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat </dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat </dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^setmouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third party repositories

You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the nginx repository, create the following config. For Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Let’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.

According to our example, for nginx we need to create the following file:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900
  • Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin: Pin options. There are many options, I’ll just go over a few:
    • origin «author or vendor name»;
    • release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository’s Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let’s try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

И пробуем ещё раз применить изменения:

alex@deb:~$ sudo apt update
Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease
Сущ:2 http://deb.debian.org/debian bullseye InRelease
Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease
Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B]
Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B]
Получено 7 633 B за 1с (9 420 B/s)
Чтение списков пакетов… Готово
Построение дерева зависимостей… Готово
Чтение информации о состоянии… Готово
Все пакеты имеют последние версии.

На этот раз всё прошло успешно.

Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:

### Пример ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

Проверка добавленного репозитория

Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:

alex@deb:~$ apt-cache policy nginx
nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

Из вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.

Зеркало официального репозитория yandex mirror

В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror можно так же использовать для сетевой установки систем.

Локальный репозиторий

Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

Local Debian repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?

I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiverse

alex@deb:~$ egrep -v ‘^#|^

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing — refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports — Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .

I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror — Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it’s faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General «Developer corner» of Debian community

Kubernetes online course

Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.

If you answer «yes» to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author’s telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat </dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat </dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^setmouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third party repositories

You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the nginx repository, create the following config. For Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Let’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.

According to our example, for nginx we need to create the following file:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900
  • Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin: Pin options. There are many options, I’ll just go over a few:
    • origin «author or vendor name»;
    • release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository’s Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let’s try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

И пробуем ещё раз применить изменения:

alex@deb:~$ sudo apt update
Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease
Сущ:2 http://deb.debian.org/debian bullseye InRelease
Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease
Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B]
Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B]
Получено 7 633 B за 1с (9 420 B/s)
Чтение списков пакетов… Готово
Построение дерева зависимостей… Готово
Чтение информации о состоянии… Готово
Все пакеты имеют последние версии.

На этот раз всё прошло успешно.

Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:

### Пример ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

Проверка добавленного репозитория

Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:

alex@deb:~$ apt-cache policy nginx
nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

Из вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.

Зеркало официального репозитория yandex mirror

В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror можно так же использовать для сетевой установки систем.

Локальный репозиторий

Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

 Debian local repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates main

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing — refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports — Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .

I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror — Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it’s faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General «Developer corner» of Debian community

Kubernetes online course

Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.

If you answer «yes» to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author’s telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat </dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat </dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^setmouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third party repositories

You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the nginx repository, create the following config. For Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Let’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.

According to our example, for nginx we need to create the following file:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900
  • Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin: Pin options. There are many options, I’ll just go over a few:
    • origin «author or vendor name»;
    • release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository’s Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let’s try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

И пробуем ещё раз применить изменения:

alex@deb:~$ sudo apt update
Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease
Сущ:2 http://deb.debian.org/debian bullseye InRelease
Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease
Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B]
Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B]
Получено 7 633 B за 1с (9 420 B/s)
Чтение списков пакетов… Готово
Построение дерева зависимостей… Готово
Чтение информации о состоянии… Готово
Все пакеты имеют последние версии.

На этот раз всё прошло успешно.

Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:

### Пример ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

Проверка добавленного репозитория

Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:

alex@deb:~$ apt-cache policy nginx
nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

Из вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.

Зеркало официального репозитория yandex mirror

В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror можно так же использовать для сетевой установки систем.

Локальный репозиторий

Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

Local Debian repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb http://security.debian.org/debian-security bullseye-security main
3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?

I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted
2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted
3 deb http://ru.archive.ubuntu.com/ubuntu jammy universe
4 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
5 deb http://ru.archive.ubuntu.com/ubuntu jammy multiverse
6 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates multiverse
7 deb http://ru.archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse
8 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted
9 deb http://ru.archive.ubuntu.com/ubuntu jammy-security universe
10 deb http://ru.archive.ubuntu.com/ubuntu jammy-security multiverse

alex@deb:~$ egrep -v ‘^#|^

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing — refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports — Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .

I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror — Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it’s faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General «Developer corner» of Debian community

Kubernetes online course

Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.

If you answer «yes» to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author’s telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat << EOF | tee -a ~/.bashrc 2>/dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat << EOF |tee ~/.quiltrc 2>/dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^set mouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don’t forget to setup your git variables. Full explanation may be found here, I will do it in fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Добавление сторонних репозиториев

Добавлять репозитории можно в основной конфиг: /etc/apt/sources.list или создавать отдельные конфиги в каталоге /etc/apt/sources.list.d/. Сам я считаю что правильнее для каждого стороннего репозитория создавать отдельные конфиги.

Например, чтобы подключить репозиторий nginx создайте следующий конфиг. Для Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Или для Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Допустим, мы прописали дополнительный репозиторий для nginx, но как системе понять из какого репозитория брать пакет для установки? Ведь пакеты для nginx есть и в системном репозитории и в репозитории от самого nginx. Чтобы ответить на этот вопрос придумали приоритеты репозиториев.

Чтобы задать приоритет репозитория нужно создать файл /etc/apt/preferences.d/XX<имя_репозитория>, где XX это номер файла, чем он выше, тем обработается позднее, то есть будет иметь приоритет над другими настройками.

По нашему примеру для nginx нужно создать следующий файл:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin-Priority: 900
  • Package: имя пакета. Можно поставить знак * чтобы применить приоритет для всех пакетов из этого репозитория. Также можно указать несколько имён через пробел;
  • Pin: опции прикрепления. There are many options, I’ll just go over a few:
    • origin «author or vendor name»;
    • release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository’s Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let’s try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Here is the command itself:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Example for curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

And try again to apply the changes:

alex@deb:~$ sudo apt update
Ex:1 http://security.debian.org/debian-security bullseye-security InRelease
Ex:2 http://deb.debian.org/debian bullseye InRelease
Ex:3 http://deb.debian.org/debian bullseye-updates InRelease
Gender:4 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Gender:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7633 B]
Received 7633 B in 1s (9420 B/s)
Reading Package Lists... Done
Building the dependency tree… Done
Reading Status Information… Done
All packages are up to date.

This time everything went well.

By the way, if you want to specify the architecture and the public key in the package source, then this is done with a space:

### Example ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

<span id="Checking the added repository

Well, to understand from which repository the package will be installed, run the command:

alex@deb:~$ apt-cache policy nginx
nginx: Installed: (missing) Candidate: 1.20.2-1~bullseye Version table: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

From the output, it becomes clear that the nginx package has not yet been installed on the system, and the package with version 1.20.2 from the http:// repository is a candidate for installation nginx.org/packages/debian. The priority of all packages, by the way, became equal to = 990. This happened after we set target release=bullseye. Since all repositories belong to this release, the system stopped looking at the priority I assigned, and assigned this priority to the repositories for bullseye.

Mirror of the official repository yandex mirror

Yandex repository called andex is popular in RuNet. Mirror — https://mirror.yandex.ru. It is a mirror of popular Linux distributions, Freebsd and other projects, including Debian. Works with HTTP, FTP and rsync protocols.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror can also be used for network installation of systems.

Local repository

There are several ways to create a local Debian repository. From what I tried, apt-mirror seemed to me the simplest and most convenient, but it has one bug if it is used as a mirror of the official repositories. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

 Debian local repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.

/etc/apt/sources.list | cat-n
1 deb http://deb.debian.org/debian/ bullseye main
2 deb-src http://deb.debian.org/debian/ bullseye main
3 deb http://security.debian.org/debian-security bullseye-security main
4 deb-src http://security.debian.org/debian-security bullseye-security main
5 deb http://deb.debian.org/debian/bullseye-updates main
6 deb-src http://deb.debian.org/debian/bullseye-updates main

This file is made up of lines, and the lines are made up of the following columns:

Debian release classes

Browsing the Debian repository branches above, we saw the following:

  • bullseye;
  • bullseye-updates;
  • bullseye-security .

But, in addition to the code names of system versions, special release classes can be used in branch names:

  • stable — links to the current Debian stable repository, currently bullseye. As soon as a new version of Debian is released, stable will refer to the newer version;
  • oldstable — refers to the previous stable repository;
  • testing — refers to a special branch of the repository for the development of a new stable release;
  • unstable — links to the most recent, but not tested packages;
  • experimental — packages that have just started development are stored here ;
  • backports — Links to testing and unstable, but only for security updates.

That is, you can change your repositories to testing, and be at the forefront of progress:

### This is just an example, there is a strong possibility that the system will be damaged very soon due to unchecked updates ###
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
deb http://security.debian.org/debian-security testing-security main
deb-src http://security.debian.org/debian-security testing-security main
deb http://deb.debian.org/debian/ testing-updates main
deb-src http://deb.debian.org/debian/ testing-updates main

Main, contrib, non-free branches

Each official Debian repository has 3 branches:

  1. main consists of DFSG-compliant packages that do not require other software from other sources. These packages are considered part of the Debian distribution. They are completely free for any use.
  2. contrib packages also contain DFSG-compliant software, but their dependencies may require additional software, which may be in other sources, such as the non-free branch.
  3. non-free contains all other software that does not comply with the DFSG.

DFSG — Debian Free Software Guidelines, Debian criteria for defining free software. In any case, the packages from all three main, contrib and non-free branches are fully tested and prepared to work with the Debian distribution.

Now that we know all the theory about Debian repositories, we can analyze the sources.list file we got after installation. It has 3 repositories with main branches.

deb http://deb.debian.org/debian bullseye main
deb-src http://deb.debian.org/debian bullseye main

This is the stable repo of the current release. Next comes the security repository to install the latest security updates.

deb http://deb.debian.org/debian-security/ bullseye-security main
deb-src http://deb.debian.org/debian-security/ bullseye-security main

And finally, stable-updates to get stable updates to the next Point Release of the current distribution.

deb http://deb.debian.org/debian bullseye-updates main
deb-src http://deb.debian.org/debian bullseye-updates main

To be honest, I have been administering Debian servers for many years, but I did not fully understand what was written in my sources.list. I figured it out completely just now, when I wrote the article. Before that, I just copied configs with turnips out of habit. I guessed what it was about, but I didn’t know for sure. Now I filled the gap and shared information with you.

Sources List Generator

There are services on the Internet that automatically generate sources.list based on your needs. You can use your own generator or use the ready one.

An example of such a generator that you can install and configure for yourself is debgen. The source code used to be on github, but then disappeared. And here is its finished implementation with content — https://debgen.simplylinux.ch/ .

I don’t know who runs these List Generators or if they can be trusted. Always check the list of repositories that will be generated for you. In fact, this is a regular text file that you copy to yourself.

For example, I specified in the List Generator that I need to prepare a list of repositories with the following parameters:

  1. Stable repository
  2. main branch (just disabled contrib and non-free branches)
  3. Mirror — Russia
  4. Enable the Security and Updates repositories
  5. Add repo for software — Docker, MariaDB, Nginx, NodeJS, Php, Webmin

As a result, I got this sources.list

#---------------------------------------------- --------------------------------#
# OFFICIAL DEBIAN REPOS
#---------------------------------------------------------------- ------------------------------#
###### Debian Main Repos
deb http://ftp.ru.debian.org/debian/ stable main
deb http://ftp.ru.debian.org/debian/ stable-updates main
deb http://security.debian.org/ stable-security main
#---------------------------------------------------------------- ------------------------------#
# UNOFFICIAL REPOS
#---------------------------------------------------------------- ------------------------------#
###### 3rd Party Binary Repos
###DockerCE
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
###MariaDB
deb [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
deb-src [arch=i386,amd64] http://mirror.23media.de/mariadb/repo/10.6/debian bullseye main
###nginx
deb [arch=amd64,i386] http://nginx.org/packages/debian/bullseye nginx
deb-src [arch=amd64,i386] http://nginx.org/packages/debian/ bullseye nginx
###NodeJs
deb https://deb.nodesource.com/node_12.x bullseye main
deb-src https://deb.nodesource.com/node_12.x bullseye main
###PHP
deb https://packages.sury.org/php/ bullseye main
###webmin
deb http://download.webmin.com/download/repository sarge contrib

It also has a list of gpg keys to import. In principle, I have no complaints about the list. All to the point. You only need to check php and mariadb repository. I’m not familiar with the suggested ones.

In general, I would not recommend using such services for generating ready-made lists. I don’t see any problems to manually do everything and control the process.

Tips and tricks

  • Make your own podman image, it’s faster
  • Use this page. Here you can find a large amount of information:
    • does package exist in Debian
    • what is source package for this binary package
    • which package provides file
    • etc
  • If you want to add new package to Debian, check this list if requests here
  • General «Developer corner» of Debian community

Kubernetes online course

Kubernetes online course — for developers, administrators, technical leaders who want to learn the modern Kubernetes microservices platform. The most complete Russian-language course on highly demanded and well-paid skills. The course is not for beginners — you need to pass an entrance test.

If you answer «yes» to at least one question, then this is your course:

  • Tired of wasting time on automation?
  • want consistent environments?;
  • Do you want to develop and use modern tools?
  • Do you care about infrastructure reliability?
  • Do you have to scale your infrastructure to meet growing business needs?
  • Do you want to free your product teams from some administration and automation tasks and focus them on product development?

Take the entrance test at link and join the new recruitment!.

Did the article help? Subscribe to author’s telegram channel

Announcements of all articles, plus a lot of other useful and interesting information that does not get on the site.

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgent=low * Initial release. -- Alexey Lukyanchuk Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ./
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc
cat </dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF
source ~/.bashrc

Second step is config for quilt:

cat </dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^setmouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don’t forget to set up your git variables. Full explanation may be found here, I will do it in the fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

  • build-essential
  • debmake
  • quilt
  • devscripts

I will do all work in podman container because it’s comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Adding third party repositories

You can add repositories to the main config: /etc/apt/sources.list or create separate configs in the /etc/apt/sources.list.d/ directory. I myself think that it is more correct to create separate configs for each third-party repository.

For example, to connect the nginx repository, create the following config. For Debian:

alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian bullseye nginx 

Or for Ubutnu:

alex@ubu:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/ubuntu jammy nginx

Let’s say we have an additional repository for nginx, but how does the system know which repository to take the package for installation from? After all, there are packages for nginx both in the system repository and in the repository from nginx itself. To answer this question, we came up with repositories priorities.

To set the repository priority, you need to create the file /etc/apt/preferences.d/XX, where XX is the file number, the higher it is, the later it will be processed, that is, it will have priority above other settings.

According to our example, for nginx we need to create the following file:

$ sudo nano /etc/apt/preferences.d/99nginx
Package: *
Pin: origin nginx.org
Pin: release o=nginx
Pin Priority: 900
  • Package: The name of the package. You can put an * sign to apply priority to all packages from this repository. You can also specify multiple names separated by spaces;
  • Pin: Pin options. There are many options, I’ll just go over a few:
    • origin «author or vendor name»;
    • release o=nginx — means that the repository’s Release file has a provider (Origin = o) named nginx;
    • release l=Debian means that the repository’s Release file has a Label (l) named Debian;
  • Pin-Priority: Priority.

That is, Package and Pin are the conditions for setting the priority, and Pin-Priority is the action (setting the priority). In our example, the result is the following: if the package name is anything, but the owner of the repository is nginx.org and the Release file says «Origin: nginx«, then for such packages we set priority 900.

Priority can be in the following ranges:

  • P >= 1000 — the package will be installed from this repository even if it would downgrade an already installed package;
  • 990 <= P < 1000 — the package will be installed from this repository if no newer version is installed;
  • 500 <= P < 990 — the package will be installed if there is no package owned to target release or no newer version is installed;
  • 100 <= P < 500 — the package will be installed if there are no candidates from other repositories or installed package of a newer version;
  • 0 < P < 100 — the package will be installed only if it is not already installed (of any version) and if there are no candidates from other repositories;
  • P < 0 — the package will not be installed under any circumstances;
  • P = 0 — not used.

Priorities 500 to 990 and 990 to 1000 are very similar. To tell them apart, you need to understand what a target release is. For Ubuntu or Debian, this is the version name of the distribution. For example, for Ubuntu — jammy, and for Debian — bullseye. But this name still needs to be set in this way:

alex@deb:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "bullseye";
alex@ubu:~$ sudo nano /etc/apt/apt.conf.d/default
APT::Default-Release "jammy";

Repository public key

And so, we added the repository, set the priority. Let’s try to apply the changes:

alex@deb:~$ sudo apt update
Gender:1 http://nginx.org/packages/debian bullseye InRelease [2860 B]
Error:1 http://nginx.org/packages/debian bullseye InRelease The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
Gender:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Ex:3 http://deb.debian.org/debian bullseye InRelease
Gender:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Reading Package Lists... Done
W: GPG error: http://nginx.org/packages/debian bullseye InRelease: The following signatures cannot be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
E: Repository "http://nginx.org/packages/debian bullseye InRelease" is not signed.
N: This repository cannot be updated securely, so it is disabled by default.
N: See the apt-secure(8) man page for information about creating a repository and user settings.

And here we see an error that we do not have enough public key. I gave an example for Debian, but Ubuntu will have a similar situation. The fact is that modern repositories are encrypted using a private key, and in order to use it, we need to install a public key in the system.

To do this, first install the necessary tools:

### For Debian ###
alex@deb:~$ sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
### For Ubuntu ###
alex@ubu:~$ sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring
Attention! The gnupg2 utility for Ubuntu is only available in the universe repository, if you commented out these sources, then uncomment them. These are the lines:
deb http://ru.archive.ubuntu.com/ubuntu jammy universe
deb http://ru.archive.ubuntu.com/ubuntu jammy-updates universe
And don't forget to apply the changes by running:
$ sudo apt update

And then download the public key (using wget):

$ wget https://nginx.org/keys/nginx_signing.key

Next, a command pipeline is executed, this is when the output of one command goes to the input of another command. We will go through such conveyors later. But I’ll try to explain the following command anyway. With the help of cat we read the key file and pass the read to the gpg utility. The gpg utility translates the read into the required format and passes the output to the next tee command. The tee utility (under sudo) saves the resulting text to a file. At the end we add >/dev/null so that there is no output to the terminal. Вот сама команда:

$ cat nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для wget ###
$ wget -O- https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
### Пример для curl ###
$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
alex@deb:~$ sudo nano /etc/apt/sources.list.d/nginx.list
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

И пробуем ещё раз применить изменения:

alex@deb:~$ sudo apt update
Сущ:1 http://security.debian.org/debian-security bullseye-security InRelease
Сущ:2 http://deb.debian.org/debian bullseye InRelease
Сущ:3 http://deb.debian.org/debian bullseye-updates InRelease
Пол:4 http://nginx.org/packages/debian bullseye InRelease [2 860 B]
Пол:5 http://nginx.org/packages/debian bullseye/nginx amd64 Packages [7 633 B]
Получено 7 633 B за 1с (9 420 B/s)
Чтение списков пакетов… Готово
Построение дерева зависимостей… Готово
Чтение информации о состоянии… Готово
Все пакеты имеют последние версии.

На этот раз всё прошло успешно.

Кстати, если вы хотите в источнике пакетов прописать архитектуру и открытый ключ, то это делается через пробел:

### Пример ###
deb [arch=amd64 signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/debian bullseye nginx

Проверка добавленного репозитория

Ну и чтобы понять из какого репозитория будет установлен пакет, выполните команду:

alex@deb:~$ apt-cache policy nginx
nginx: Установлен: (отсутствует) Кандидат: 1.20.2-1~bullseye Таблица версий: 1.20.2-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.20.1-1~bullseye 990 990 http://nginx.org/packages/debian bullseye/nginx amd64 Packages 1.18.0-6.1 990 990 http://deb.debian.org/debian bullseye/main amd64 Packages

Из вывода становится ясно что пакет nginx ещё не установлен в систему, а кандидатом на установку является пакет с версией 1.20.2 из репозитория http://nginx.org/packages/debian. Приоритет у всех пакетов, кстати стал равным = 990. Это произошло после того, как мы установили целевой выпуск = bullseye. Так как все репозитории относятся к этому выпуску, то на назначенный мною приоритет система перестала смотреть, а назначила репозиториям для bullseye такой приоритет.

Зеркало официального репозитория yandex mirror

В рунете популярен репозиторий Яндекса под названием andex. Mirror — https://mirror.yandex.ru. Это зеркало популярных дистрибутивов Linux, Freebsd и других проектов, в том числе и Debian. Работает по протоколам HTTP, FTP и rsync.

deb http://mirror.yandex.ru/debian bullseye main
deb-src http://mirror.yandex.ru/debian bullseye main
deb http://mirror.yandex.ru/debian bullseye-updates main
deb-src http://mirror.yandex.ru/debian bullseye-updates main
deb https://mirror.yandex.ru/debian-security bullseye-security main
deb-src https://mirror.yandex.ru/debian-security bullseye-security main

Repository yandex mirror можно так же использовать для сетевой установки систем.

Локальный репозиторий

Есть несколько способов создать локальный репозиторий Debian. Из того, что я пробовал, самым простым и удобным мне показался apt-mirror, но у него есть один баг, если его использовать как зеркало официальных репозиториев. It does not download .gz and .xz translations, only .bz2. As a result, when you use the local repository as a mirror of the official one, you will get an error:

File not found updates/main/i18n/Translation-en (2: No such file or directory)

Another easy option is to use reprepro. I won’t go into detail on setting up a local repository for Debian, as that’s a separate topic. For good, the repository must be signed with a gpg key, published using http or ftp, maybe somehow. I will only briefly show how this is done so that you understand what it is all about. And if you really need a local repository, you can easily find its detailed settings. There is nothing complicated.

# apt install reprepro

Next, create a directory for the local repository and config.

# mkdir -p /mnt/repo/debian/conf
# touch /mnt/repo/debian/conf/distributions

We make the config approximately as follows.

Codename: bullseye
Suite: stable
Version: 11.x
Origin: Debian
Label: Debian 11.x
Description: Debian Stable Updates Repository
Architectures: amd64 source
Components: main
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release . .gz .bz2
Contents: . .gz .bz2

Initializing the repository.

# cd /mnt/repo/debian
# reprepro export
# reprepro createsymlinks

You can now add packages to your local repository with the following command.

# reprepro -b /mnt/repo/debian --ask-passphrase includedeb bullseye /home/package.deb

In order to connect a new repository locally, you need to add it to sources.list.

deb [trusted=yes] file:/mnt/repo/debian bullseye main

After that, update the package cache and you will see your local one in the list of repositories.

Local Debian repository

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

  • control: file with mandatory information like package name, version, source, checksums, other data
  • rules: make-file with instructions how to build software
  • patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Option to use the official repositories

In order to reduce the chance of your system breaking due to untested updates, you can slightly reduce the number of repositories on Debian and Ubuntu systems.

In general, Debian lists the most secure repositories by default. We can only comment out the source repositories, since most likely you will not need them yet. Let me remind you that such lines begin with the word deb-src. And if needed, you can simply uncomment them. After editing, we are left with 3 package sources:

alex@deb:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://deb.debian.org/debian/ bullseye main 2 deb http://security.debian.org/debian-security bullseye-security main 3 deb http://deb.debian.org/debian/bullseye-updates main

Ubuntu checked in many more of its repositories during installation. But they can also be reduced to three. For example, I find it necessary to disable the universe, multiverse and jammy-backports repositories on the server. After editing, the list of repositories also consists of 3 lines:

alex@ubu:~$ egrep -v '^#|^$' /etc/apt/sources.list | cat-n 1 deb http://ru.archive.ubuntu.com/ubuntu jammy main restricted 2 deb http://ru.archive.ubuntu.com/ubuntu jammy-updates main restricted 3 deb http://ru.archive.ubuntu.com/ubuntu jammy-security main restricted

To apply the changes, run the following command on both systems:

$ sudo apt update

This command will connect to each repository, see which packages can be updated and from which sources. And save the local cache. After executing this command, the system will know which packages can be obtained from which repositories, as well as the versions of these packages. But if a newer version of some package is added to the repository, the system will know about it only after the next execution of this command.

And if you want to update the system, then run the command:

$ sudo apt upgrade

This command already downloads all updates and installs them.

By the way, the apt utility is the package manager. We will explore it and other package managers in the following articles.

Store changes in git

Now we come to the important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It’s universal way that can find needed commit in branch, by commit, elsewhere.

Also don’t forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

  • version before release: v${version}-rc${number}
  • release version: v$(version)-release

$ git tag -a v1.26.0-rc0
$gittag
v1.26.0-rc0

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use these tips:

  • store ready src-deb and binary-deb in repo (everybody does it)
  • make separate git for your debian-dir with files for source-package
  • create and use additional target in rules:
    • src-clean to clean «vanilla» source codes
    • src-get to get «vanilla» source codes from original package git
    • build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

  • original software git, called vanilla
  • your git with debian dir for package creation

Add new repository to debian

Now let’s move from theory to practice. Let’s manually add a new repository to Debian. For example, we need to install a stable version of MariaDB on the server. To do this, add its repository. This can be done either in the sources.list file, but it is better to create a separate one in sources.list.d. Let’s call it MariaDB.list.

deb [arch=amd64,arm64,ppc64el] http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main
deb-src http://mirror.mephi.ru/mariadb/repo/10.6/debian bullseye main

After connecting the repository, you need to add its gpg key.

# curl -LsSO https://mariadb.org/mariadb_release_signing_key.asc
# chmod -c 644 mariadb_release_signing_key.asc
# mv -vi mariadb_release_signing_key.asc /etc/apt/trusted.gpg.d/

Now let’s update the package cache. This must be done every time after connecting a new repository.

# apt update

Adding repository in Debian

You can search for the package to make sure the new repository is connected.

# apt search mariadb-server

New repository

As I said, to set up a new repository, you could simply add the same 2 lines with parameters to sources.list directly. There is no difference.</p

Читайте также:  Размещение серверa в Москве
Оцените статью
Хостинги