Embedded Build System Voodoo

Qt has this nice feature that you can embed arbitrary files into your binary, and access these so called Resources via special paths starting with a colon. For the longest time, Quassel has exclusively relied on resources and embedded all its data files, including icons, into the binary. This had the advantage that we could always be sure to find all required files at a certain location and need to provide fallback mechanisms. It also meant that Quassel binaries didn't need to be installed; they were completely standalone and could just be run from any location. Quite nice especially on a platform like Windows that does not have standard locations for all sorts of data files. And on Linux, whose various flavors still do not always agree where stuff should go.

Now, where's the problem with that approach? For once, the binaries get bigger. Maybe not on the file system (having dozens of small files tends to waste a lot of space due to fragmentation and partial blocks), but certainly in RAM. It also increases loading times a bit. Then there was the issue with translations. We would embed all translations into the binary - obviously a waste of space even though each individual language file is surprisingly small. The LINGUAS variable, restricting the languages to be built, helped a bit, but not much. Distros tend to provide language packs, so support for external files would be needed.
But the real killer was KDE integration - KIconLoader, for example, wouldn't look into our binary to find an icon, obviously. So for KDE support, we needed to install the icons separately. Consequently, Quassel gained support for that a while ago, complete with some confusing cmake options. Over time, we got more data files, such as .knotifyrc for KDE's notifications, and networks.ini containing predefined network definitions. As it turned out, our build system installed them in locations that would be re-found at run-time only by pure luck, if at all. And it would fail completely on Mac and Windows.

I have fixed that in the past couple days, and Quassel now has proper data file handling. Read on for more information, in particular if you are a distro packager!

First of all, the old cmake options -DOXYGEN_ICONS=External and -DQUASSEL_ICONS=External are gone. They were confusing at best, and road to failure in the worst case. In particular, we didn't install the Oxygen icons at all if the former option was specified; the rationale being that we would then rely on a system-installed icon theme. Nice idea, but one that goes horribly wrong in a world where icon names and paths change all the time. With Quassel releases not synced to KDE, it would be pure luck to find an Oxygen version that matches all the names used in Quassel. Or one that even contains all the icons, since some of them won't appear in kdebase before KDE 4.3... So we will need to install both our own and the used subset of Oxygen icons in any case, either as a resource, or into the filesystem.

I figured it wouldn't make sense to provide bunch of options to specify if various types of files (icons, translations, other data files) should be embedded or not. There is only two cases: Either you want a standalone binary, in which case everything should be embedded, or you install properly, in which case you can as well install all the files. So now there is only one option: -DEMBED_DATA={ON, OFF}, with ON being the default for now. If you set it to OFF, all files get installed externally. Note that this is not yet fully supported on Windows and Mac. You can influence the install location for data files with -DDATA_INSTALL_DIR. The default should be fine though; things go into $PREFIX/share/apps/quassel on Linux for non-KDE, and into whatever-KDE-specifies with KDE support enabled. On Windows, everything goes into %APPDATA%/quassel-irc.org/share/apps/quassel. Mac still needs some love; I don't know enough about that platform myself.

Both the icon loader and the data file finder have been thouroughly reworked to properly work with scattered files. This includes the search order for icons. A system Oxygen theme (often installed in $data/share/icons) will be preferred over our own copy in $prefix/share/apps/quassel/icons. Afterwards, the hicolor theme will be used as a fallback. Translations go into $prefix/share/apps/quassel/translations.

One side remark probably not interesting for anyone: I have found some cmake feature I didn't know before. PARENT_SCOPE for setting variables in the enclosing scope. This allowed me to greatly simplify some parts of the build system. I also fixed a whole lot of other issues, most but not all of them related to external data files.

I hope this will help in particular distro packagers! Next up: Bumping our version of Oxygen to include the gorgeous artwork Nuno Pinheiro has done for us, and making sure that we only ship the icons we need rather than most of Oxygen.

But now, sleep.

Binary size and RAM

Just keep in mind that a large binary size is not a problem in and of itself. Pretty much any modern operating system, including Windows, will only load the parts of your program into RAM that are actually used (called demand-paging). You won't be executing images and other data so that won't increase the memory pressure of your executable code just by adding it to the binary.

Of course there are other good reasons to split the files out but that's probably not one of them.

Translations location

That's great news but allow me a comment regarding the translations location:

On Linux they shouldn't be in $prefix/share/apps/quassel/translations but in /usr/share/locale/$LOCALE/LC_MESSAGES - e.g. the german vlc translation is /usr/share/locale/de/LC_MESSAGES/vlc.mo.

It would be really fantastic if you could stick with the standards here as well.

Besides that I would just use the opportunity to thank you for Quassel. I really like it and am absolutely looking forward to the KDE4 version :)

Is that true for non-gettext

Is that true for non-gettext stuff too? We're using .qm files rather than .mo. Somehow I feel we shouldn't clutter the system locale directory with that, should we?

Your thanks are very appreciated :)

You are probably right

You are probably right, every .qm I have on my system is bellow /usr/share/ so putting it into /usr/share/locale/$LOCALE/LC_MESSAGES probably isn't the best idea.

Sorry for the wrong information.