How to Easily Patch Fedora Packages

While many Linux users are developers, a rather small part of the developers feels easy to meddle or extend their distribution's packaging. When there is a pending feature from an upstream package, or when there is a bug in a package, users usually wait for the distribution maintainers to fix it, or resort to use independent solutions such as homebrew, for rebuilding upstream projects.

Software packaging in distributions entails dealing with the build system of an unknown subproject, and it is sometimes a learning curve that may deter developers from tackling it. Luckily, in Fedora Linux, we have good tools allowing to deal each package's own mess, and make it surprisingly easy to build modified versions.

Obtaining the source

First, we begin by installing several base system package that will assist us in dealing with package building:

$ sudo dnf install fedpkg make

Let's say we want to consider a patch or a feature for zsh (Z shell).

We need to consider what is the source package project from which the resultant binaries were made. It's easy to query the rpm tool for that. Based on a filename of an installed package, the name of the source package RPM can be revealed. For example:

$ rpm -qif /usr/bin/zsh | grep 'Source RPM'
Source RPM  : zsh-5.7.1-4.fc30.src.rpm

It is not surprising that the source RPM name is zsh too. It may not be the case for other packages though.

Normally, the source package RPM name matches the name of the Git repository in which Fedora maintains the scripts that allow building it. Cloning a source package from Fedora's Git server is very easy, and does not require becoming a Fedora community member or any other credential. For zsh, we can perform the following command using fedpkg:

$ fedpkg co -a zsh
Cloning into 'zsh'...
remote: Counting objects: 1023, done.
remote: Compressing objects: 100% (777/777), done.
remote: Total 1023 (delta 556), reused 423 (delta 216)
Receiving objects: 100% (1023/1023), 235.29 KiB | 271.00 KiB/s, done.
Resolving deltas: 100% (556/556), done.

For each package, the source for each version of Fedora is maintained in a different branch. We can therefore check-out the branch that is matching the version of the distribution for which we are building:

$ cd zsh
$ git checkout -b f31 origin/f31
Branch 'f31' set up to track remote branch 'f31' from 'origin'.
Switched to a new branch 'f31'

From now on we will invoke fedpkg in the working directory of the package.

Setting up a build environment

Different packages will have varying requirements for a functioning development environment. Luckily, Fedora's dnf assists us in bringing in the dependencies needed for any of the packages that it can build. This can be done via the dnf builddep command. For our use case, we can bring in zsh dependencies:

$ sudo dnf builddep zsh

Directly building a distributable RPM

Before modifying the package, it is worth testing to see if we are able to build it correctly even without any modification. The following command will try to build the binary RPMs from the current state of the code:

$ fedpkg local

We can observe that the following RPMs have been generated:

$ ls -l1 x86_64/ noarch/
noarch/:
total 452
-rw-rw-r--. 1 user user 459748 Jan 14 13:58 zsh-html-5.7.1-4.fc31.noarch.rpm

x86_64/:
total 5476
-rw-rw-r--. 1 user user 2999294 Jan 14 13:58 zsh-5.7.1-4.fc31.x86_64.rpm
-rw-rw-r--. 1 user user 1771784 Jan 14 13:58 zsh-debuginfo-5.7.1-4.fc31.x86_64.rpm
-rw-rw-r--. 1 user user  829306 Jan 14 13:58 zsh-debugsource-5.7.1-4.fc31.x86_64.rpm

Alternatively, building the RPM in a mock container

Even before docker containers were popular, Fedora provided us with a tool named mock that creates a separate environment for building packages. Thus, it can be used to build the package independently of the development environment.

First, we need to make sure that mock is installed.

$ sudo dnf install mock

Then, we can tell fedpkg to use mock in order to build the package:

$ fedpkg mockbuild

The build outputs of mock are all moved to a directory containing the log files of the build, and the version and name of the package:

$ ls -lR results_zsh/*/*
results_zsh/5.7.1/4.fc31:
total 9852
-rw-rw-r--. 1 user user  190779 Jan 14 14:10 build.log
-rw-rw-r--. 1 user user    2744 Jan 14 14:03 hw_info.log
-rw-rw-r--. 1 user user   52642 Jan 14 14:08 installed_pkgs.log
-rw-rw-r--. 1 user user  614047 Jan 14 14:10 root.log
-rw-rw-r--. 1 user user     998 Jan 14 14:10 state.log
-rw-r--r--. 1 user mock 3146067 Jan 14 14:06 zsh-5.7.1-4.fc31.src.rpm
-rw-r--r--. 1 user mock 2999346 Jan 14 14:10 zsh-5.7.1-4.fc31.x86_64.rpm
-rw-r--r--. 1 user mock 1772488 Jan 14 14:10 zsh-debuginfo-5.7.1-4.fc31.x86_64.rpm
-rw-r--r--. 1 user mock  829174 Jan 14 14:10 zsh-debugsource-5.7.1-4.fc31.x86_64.rpm
-rw-r--r--. 1 user mock  459682 Jan 14 14:10 zsh-html-5.7.1-4.fc31.noarch.rpm

Using mock to perform the build has the advantage in so that it verifies that the dependencies of the build are properly specified, and it can also be used to build for different versions of the distribution on the same machine. It's also the working horse behind Fedora's own build servers, and namely Copr.

Adding a patch

To produce a patch for a package, we need the source of the package itself, rather than the sources for the Fedora scripts that tell how to build it.

There are several ways to obtain the source, and one of them is by using fedpkg. We can tell it to create a directory containing the patched sources of the package, as they are ready to be built. This process will execute the prep stage of the RPM spec source, and the result will usually be a directory directly under our working tree.

$ fedpkg prep

Because Fedora source packages do not assume about the source control aspects of upstream projects, they contain just archives of a certain version's source. The created directory is not tracked in source control, and if we want to modify it, usually it is best to move it aside to a different directory, and initialize it with Git. For example:

$ mv zsh-5.7.1 ../zsh-5.7.1
$ cd ../zsh-5.7.1
$ git init && git add -f . && git commit -m "Base version"

Here, our zsh-5.7.1 is only a representation of that Fedora-maintained version, that is probably already patched by Fedora to some degree, but it can and should be used as a base for our further patching. Though, we may want to have a clone of the upstream project, zsh in that case, handy for full Git history browsing.

$ cd ..
$ git clone git://git.code.sf.net/p/zsh/code zsh-upstream
Cloning into 'zsh-upstream'...
remote: Enumerating objects: 94026, done.
remote: Counting objects: 100% (94026/94026), done.
remote: Compressing objects: 100% (25128/25128), done.
remote: Total 94026 (delta 73505), reused 87930 (delta 68487)
Receiving objects: 100% (94026/94026), 16.60 MiB | 1.07 MiB/s, done.
Resolving deltas: 100% (73505/73505), done.

Going back to either of our source clones we can proceed to committing changes and generating patches from commits. Here is an example for a trivial patch:

$ git diff HEAD
diff --git a/Src/hist.c b/Src/hist.c
index dbdc1e4..cdb1dd1 100644
--- a/Src/hist.c
+++ b/Src/hist.c
@@ -580,7 +580,7 @@ histsubchar(int c)
      */
     lexraw_mark = zshlex_raw_mark(-1);

-    /* look, no goto's */
+    /* look, no goto's! */
     if (isfirstch && c == hatchar) {
        int gbal = 0;

$ git commit -m "Adding an exclamation mark to Src/hist.c"
[f31 c00d68b] Adding an exclamation mark to Src/hist.c
 2 files changed, 100 insertions(+), 1 deletion(-)
 create mode 100644 0001-zsh-5.7.1-zle-history-avoid-crash.patch

$ git format-patch HEAD~1 -o ../zsh
../zsh/0001-Adding-an-exclamation-mark-to-Src-hist.c.patch

The git format-patch is a handy command that exports Git commits as files, and its output can be used as input to the package building process, as the packaging standard in Fedora mandates separation of the upstream sources from the patches that were made on them.

Adding the patch files to the Fedora package's source specification may be a bit tricky, but after a few times you understand that the packaging format is simpler than it may seem at first. In the case of zsh, we only need to specify the newly created patch in a Patch<number>: line in the beginning of zsh.spec.

$ git diff
diff --git a/zsh.spec b/zsh.spec
index 0d77f70..0022a90 100644
--- a/zsh.spec
+++ b/zsh.spec
@@ -14,6 +14,7 @@ Source6: dotzshrc

 # make failed searches of history in Zle robust (#1722703)
 Patch1:  0001-zsh-5.7.1-zle-history-avoid-crash.patch
+Patch2:  0001-Adding-an-exclamation-mark-to-Src-hist.c.patch

 BuildRequires: autoconf
 BuildRequires: coreutils

Older packages may require some more changes, such as adding extra %patch lines further in the file, relating to the aforementioned prep stage.

Identifying a patched package

If no other .spec fields are changed, our modified package would appear mostly indistinguishable in meta-data from the original package. Usually this is not wanted. Therefore, it is best to modify the .spec to include a string in the package's Release field. For example:

diff --git a/zsh.spec b/zsh.spec
index 0022a90..78c362e 100644
--- a/zsh.spec
+++ b/zsh.spec
@@ -1,7 +1,7 @@
 Summary: Powerful interactive shell
 Name: zsh
 Version: 5.7.1
-Release: 4%{?dist}
+Release: 4%{?dist}.daloni
 License: MIT
 URL: http://zsh.sourceforge.net/
 Source0: https://downloads.sourceforge.net/%{name}/%{name}-%{version}.tar.xz

We can see that following a build, the <name>-<version>-<release> triplet is extended:

$ ls -l1 x86_64/ noarch/

noarch/:
total 452
-rw-rw-r--. 1 user user 459790 Jan 14 15:15 zsh-html-5.7.1-4.fc31.daloni.noarch.rpm

x86_64/:
total 5476
-rw-rw-r--. 1 user user 2999482 Jan 14 15:15 zsh-5.7.1-4.fc31.daloni.x86_64.rpm
-rw-rw-r--. 1 user user 1773094 Jan 14 15:15 zsh-debuginfo-5.7.1-4.fc31.daloni.x86_64.rpm
-rw-rw-r--. 1 user user  829124 Jan 14 15:15 zsh-debugsource-5.7.1-4.fc31.daloni.x86_64.rpm

Installing the package

The new packages can be installed via dnf install. Once the patched packages are installed, it is easy to spot them by grep-ing over the output of dnf list installed by various means.

$ sudo dnf install x86_64/zsh-5.7.1-4.fc31.daloni.x86_64.rpm

$ dnf list installed | grep @@commandline
zsh.x86_64              5.7.1-4.fc31.daloni                    @@commandline

$ dnf list installed | grep daloni
zsh.x86_64              5.7.1-4.fc31.daloni                    @@commandline

Avoiding accidental overrides from a newer package version

When Fedora releases newer version of the package, automatic system upgrades may override the patched one due to providing a higher version. There are several ways to prevent this issue, but the one I prefer is excluding upgrades on such packages from dnf's configuration:

$ grep exclude /etc/dnf/dnf.conf
exclude=zsh

Furthermore, there are various aspects of dealing with packages that are outside the scope of this post. An exercise left for the reader is to create an RPM repository containing the patched package, host it on the network, and for client machines - devise a repository description file to be placed in /etc/yum.repos.d, and configure it to have a higher priority than the original distribution packages.

Keep the debuginfo!

If a patched package has crashed and generated a corefile, the special debuginfo packages that were produced in the build may be handy in examining this corefile. Therefore, you may want to keep these debuginfo packages, especially because a reproduced build is sometimes difficult to match in complete binary compatibility to the original one. This depends on how deterministically the package is being built, a matter that is intrinsic to the package's own build system.