GNU/Linux bundled application ramblings

It’s impressive how in the last few months (and especially the last few weeks) the discussion around bundled applications for the GNU/Linux Desktop has sparked.

It’s especially interesting because:

  • The problem is not new.
  • The solutions that have attempted to tackle the problem in the past have been ignored (both by us developers and by distributions).

The TLDR

First, let me try to subjectively summarize the problem: Historically, the resources we get in GNU/Linux come from the distributions. Anything: executables, libraries, icons, wallpapers, etc. There’s been alternatives to all of those, but none has flourished as a globally adopted solution.

This guarantees that everyone using a distribution will have access to the resources the distribution can offer. The more powerful the distribution is, the more we get. There’s limitations nevertheless, so some restrictions have to get in place. The ensemble of limitations and technologies adopted will effectively define the user’s experience.

This works. It has worked for years and, given the technology is in place, it could easily keep working. Like in most engineering solutions there’s drawbacks and properly addressing them can bear some goodness. It seems like now it’s the moment to review this situation. Let’s enumerate some of the problems we have nowadays:

  • We have users using really old versions of our software with issues we’ve solved in versions they can’t use.
  • It’s really hard for GNU/Linux users to get users to test unstable versions of our software.
  • We have users who want to use fresh versions of some software but not in the whole system.

There’s been many solutions to fix those, some easily come to mind: ArchLinux’s AUR (with yaourt), Ubuntu’s PPAs, big-tar application packages, OpenSuse’s OBS, and possibly others.

Far from showing the maturity of the Linux desktop, what this depicts is the deep fragmentation we’re into: we have come up with different solutions that break the established distribution paradigm by lowering the restrictions and considering the resources offered as unsupported (often tainting the whole system).

What has appeared recently is sandboxing. It’s especially interesting because by letting the users execute any binaries we’re increasing the exposition of their systems. Hence, jumping from our distributions’ nest into the lions. As always, sandboxing creates new challenges: It requires changes in applications (or frameworks) to adapt, often creating a user interaction fence (e.g. a popup asking if you let Kamoso access the webcam). For what it’s worth, that’s not new: Android does it, OS X does it, Windows does it (from the Store), Chrome OS does it, etc.

Now where are we?

We need to decide about GNU/Linux’s future. Or at least, we need to understand what Plasma users will have available. So far, most of the noise comes from the big players in the business trying to differentiate their products, meaning incompatible versions.

Without an agreed unified solution, we’ll have to assume we’ll end up having installed snappies, flatpaks, AppImages as well as applications from the distribution. Then it’s just a matter of:

  • Presenting it properly so that the user knows the risks taken by executing an application (!)
  • Make sure we don’t lose many features by sandboxing.

Still, one of the good things of this new approach is that it shouldn’t have to be necessary to have several people dedicated to build every single application and component. If the solution is to add 3 more solutions that will need dedicated people, we’re not really moving forward.

Building

As soon as we’ve decided how we want to work, then the interesting stuff needs to appear. If this is properly engineered, it can bring really interesting possibilities that now we hardly ever find:

  • Newer versions of applications on administered systems (e.g. universities).
  • Enabling stable distributions on professional environments.
  • Beta channels.
  • Binary application 3rd party extensions.
  • Provision of debug symbols (some distros don’t offer them).

To finish the fantastic post, a note for the dreamers:
How easier would all that be in a microkernel architecture?

We need you!

Of course this will be a long journey and we need your collaboration. This year in Randa we started working on all these problems in several different angles. It’s important for the KDE Community to have your support, so we can keep providing quality software. Consider donating, doesn’t need to be a lot, everything counts.

KDE on Flatpak in Randa

I talked about KDE on Flatpak before (called xdg-app then). Lots happened since: new name, fancy new website and a couple of releases shows it’s getting quite stable.

Progress on the KDE side of Flatpak

It’s been some months, and this time has been mostly about maturing what we already had and making it useful for others:

  • Improved the runtime, updates to newer versions of Qt and KDE Frameworks. Some functionality issues were fixed.
  • We published the runtime so that developers can test their applications against it.
  • Added several recipes for KDE Applications (help! testers required).
  • We got some initial documentation for developers.

Next steps

Now it’s time to make this work. I find it already close to magic how we get to compile in one distro and works on another. I must admit, I’m excited. But then many things need work, should be simple, but we need to spend the time.

Also we need to compile the applications, start using them and see where’s the limitations, especially regarding the sandboxing. In the end, we also want to bring KDE applications to our GNU/Linux users who cannot reach our stable releases.

Most of it will happen in Randa, let’s see how far we get!

Join us!

Fundraiser

Discover in Plasma 5.7

I haven’t talked about what’s going on in the Discover front for a while, here’s a small heads up on what you’ll get in the next Plasma 5.7 release.

Back Ends

  • PackageKit (and AppStream): This one has received most of the care. It has been rearranged to instant load. Also we made it possible to integrate this backend with KNS/OCS, to show when there’s add-ons ready for an application.
  • KNSBackend: It’s being polished and adapted for new use-cases. First it used to focus on Plasma-centered assets, but we’ve worked towards making it possible to become a viable solution for Application add-ons as well. Furthermore it has been trimmed down to also provide fast startup while keeping its features.
  • QApt: The traditional QApt backend used in Kubuntu is now considered deprecated in favor of PackageKit, which I understand should be available properly on every GNU/Linux distribution, including Kubuntu. We kept it around because there’s been longstanding issues with PackageKit on Ubuntu, but I’m confident these will be solved soon.

Front End

We used to have many of them, they have now been reduced to two: Discover and the Notifier Plasmoid. This allows us to focus one GUI. Of course, you can still use Muon, Apper and Synaptic together with it, for more precise package management.

Regarding Discover, the main focus this year has been: performance, stability and Add-ons.

  • Performance: Mostly back-end work, and it has also benefited from improvements in KF5 and upstream Qt. Updating the dependencies will improve your Discover experience as well as that of other similar applications. Better usage of some model classes was also a nice improvement in some places.
  • Stability: We worked on making sure that things not only are sturdy but that the development process doesn’t get in the way of stability. To that end, I set up a testing framework so that changes are automatically tested, additionally to the manual tests the developers will do at some point.
  • Add-ons: One of the important parts of Discover, is to make sure the user is aware of the nice resources he has available. KNewStuff resources (mostly coming from kde-look and kde-files so far) have been available for a while, albeit mostly for Plasma enhancements. In this next release we allow these to extend applications themselves as well.

Future

Discover‘s future is Kirigami. We have already some work lined up (see the kirigami branch), we are pending some design work so that it shines, but the approach is already quite promising!

Also I’d like to start working on the Flatpak backend, if nobody beats me to it. 😀

Stay tuned!

A special sprint at CERN

Plasma

I came to the sprint hoping I could iterate 2 issues I’ve had been trying to get sorted with Plasma for a while:

  1. Integration testing for plasmoids
  2. Discuss how to keep pushing Discover for the next releases both with Plasma and the VDG.

IMG_20160308_112502

The new testing infrastructure is in place and I hope I’ll get to merge soon this week. Feel free to peek: patch1 and patch2.
This is very important because it will allow us to make the testing of plasmoids systematic, to some extent, making it easier to develop plasmoids and containments while maintaining and increasing the delivered quality.

Regarding Discover, there’s lots of ideas. Definitely far more than we can materialize. For now, we’ll be delivering a much more polished experience for 5.6 both on the strictly technical side (better appstream and PackageKit integration, better use of QtQuick in some areas) as well as on the looks side, mainly thanks to Andrew and Thomas who keep pushing me whenever something is not entirely perfect, which is awesome.

Visual Design Group (VDG)

The first special thing was that there was a VDG sprint in parallel. It wasn’t VDG people who joined us, but an actual sprint. This was especially good because it allowed for a fluid back-and-forth of ideas. It’s especially important both to have some designers full of dreams kicking our asses from time to time, as well as some kind of pragmatism back to them so the good ideas can end up being part of our workflows.

Plasma + VDG

WikiToLearn

Another special thing about the sprint was having the WikiToLearn team over. It was their first actual sprint as well as first time we all met together in a KDE event. I think it worked great, despite being different communities in nature, it’s interesting to see how synergies kept happening. I’m sure there’s good space for collaboration and I’m looking forward to keep working together.

IMG_20160307_104326

LHC

Apparently they have a particle accelerator there. What?!

Akademy-es 2016 in Madrid

Akademy-es 2016 will be taking place during the next 15th to 17th of April. Here you can learn about the conference’s program.

Akademy-es is the annual meeting for KDE contributors and users in Spain, which is organized every year since 2006.

During the conference, there will be talks, workshops and other activities where we can:

  • Get in touch with KDE and Free Software developers and users from around Spain and discuss the projects and trends that are being developed as well as sharing code, experiences and knowledge.
  • Showcase the KDE Community to people with a non-technical profile, showing the different opportunities it offers.
  • Discuss the new technologies KDE and Free Software offers, both for new developers and for users who want to know what’s really available to them.
  • Enjoy learning more about Free Software.

Join us!

Antonio and St Ignucius

KDE Applications on XDG App

Some days ago I started wondering about containerized applications, I looked at a few alternatives and then decided to give xdg-app a go. It took a while, not really because it’s especially hard, but mostly because I’m rather stubborn, then I decided to ask Alexander Larsson and he guided me quite well.

All in all, it’s rather easy, it implements a packaging format, but one that I feel KDE can maintain reasonably easily. I set it up in 2 chunks, copying GNOME’s approach: one repository for the run-time (that includes Qt5 and most of the KDE Frameworks 5) and then another one that can build applications. One thing we could do now is set up servers that built nightly versions of our applications, I’ll have to ask the sysadmins how do they feel about it.

Anyway, you’re waiting for screenshots and I’ll give you screenshots. Some problems are still obvious, but it’s a first step. Patches welcome! \o/


Getting Qt 5 right in your application

I’m sure many of you already read about Clazy. For those who didn’t, it’s a quite convenient tool for checking your code and making sure you’re getting Qt’s API right. There’s some non-trivial quirks here and there that aren’t very important but once polished can push your system considerably, especially on those cases that are ran repeatedly.

Recently, I introduced a -DENABLE_CLAZY=ON setting on extra-cmake-modules, that comes quite handy so if you want to run it on your project you just need to enable it and then Clazy will get rather verbose (although with arguably no false positives). It will work on any KDE project right away, since it comes from KDECMakeSettings.

Note: This will work provided clazy is installed and you’re running Clang as a compiler.

AppStream offers a Cross-Distribution identifier for applications

One of the problems we’ve historically had in the GNU/Linux world (and especially in KDE, since we’re cross-distribution) is that we’ve always had a hard time at explaining how to install our applications. Basically because it’s different on each distribution.

My first approach to this problem was creating Muon Discover but that’s clearly not enough because, in the end, we seldom look for new applications. Often they will be recommended on a blog post or forum and we’ll want to install it. But then, while recommending it, it will either be a vague “you should try KMail” or a “you should ‘apt-get install kmail’ or whatever applies to your distribution.

What I’m proposing today is to embrace AppStream at the level of URL handlers (for web browser integration). AppStream shows GNU/Linux distributions maturity by offering a solution by letting the upstream projects set the identifier in their appdata.xml file (among other things) and then let distributions name the package however they want, rather keeping to push different naming schemes down each other throats.

AppStream has been supported in Discover for over a year now (only for PackageKit so far) and, since introducing this feature is straightforward, I implemented it in master.
If somebody wants to give it a go, test master and click here: KMail

I think this will be beneficial not only for KDE but any project packaged on GNU/Linux distributions that have a presence on the web and need to find their way in users’ installations. This will of course be limited to getting the URL scheme adopted by our peers.

/me winks at Gnome Software and Ubuntu Software Center.

Sprinting KDE Connect

What will happen?

KDE Connect is nowadays on a sweet moment where many things can happen. This is an interesting moment to sprint, because it will allow all of us to work together on interesting features that can then be merged at once with greater impact.

The things that are the most important to me are:

  • The Plasma Phone port. See more below.
  • Improvements on the Plasma Desktop integration. Currently it’s a bit weird. It mostly works, but then it’s currently lacking any interaction with the remote devices. We should review what it does and what can we do to offer what the user actually needs.
  • Using the phone (or any other device) to transparently provide data to the desktop system. We have many interesting listed ideas. Use the phone to provide location, the phone’s microphone and camera, etc.
  • File system synchronization

What will I do?

One of the things I’ve been working on lately is a new KDE Connect client. Initially with the Plasma Phone in mind, this new application opens a new window of opportunity also on other platforms. First of all, it should be right away usable on Jolla (when they adopt a saner set of dependencies) and the Ubuntu Phone.

From our point of view, one of the biggest problems of KDE Connect is that we have a hard time to pivot over to other deskop-based platforms, mostly because our desktop approach so far is a plasmoid, which isn’t very portable. Having a standalone application will help as well to reach such platforms as well.

Another thing I’d like to work on further is KDE software integration on Android. Things should be set by now, but we need work to shape the path further. Things like setting up the KDE Frameworks 5 CI for Android for example will be a huge step forward. Also building some other application for Android, will help us get a better overview of the current state.

Support us!

All of this will happen in the next KDE Connect sprint within the Randa Meetings. If you like KDE Connect and wish it to bloom into the tool we’re all looking forward to, consider donating!

Support KDE Sprints 2015!

Plasma Mobile SDK

Where are the giants?

When approaching this issue I had been thinking about the issue for a while. I had mainly 2 problems: I was rather frustrated with previous Linux-based systems so far and the one I liked didn’t really scale for us. One thing was clear: We had to stand on the shoulder of giants.

  • The first one to think about was the N9 SDK (and by extension N900). It used to have a scratchbox-based system that emulated the one on the phone. It was useful for testing the applications locally on the device (although I actually never used that), I think that this one had both cross-compilation toolchain and base system, as well as the host and it used QEmu if a host executable was run. It felt weird because it shoved you in a weird system and you had to pull your code in weird ways to fetch it. Otherwise it worked great. Afterwards, madde came. It’s what I was looking for really and played quite well with cmake, actually when I published the steps on how to develop for the N9 it’s what I used back then, but it probably was too late for it to become a thing I guess. Also many people weren’t too fond of it, as I learned after some time.
  • The Android NDK is the other I took into account. I don’t think it would be fair to compare anything we could do with the actual Android SDK, so I’ll limit myself to this one. This one ships the complete cross-compilation toolchain and runs native (similar to the BBX SDK, IIRC).

For the N9 approach, I would have had to concentrate on figuring out technologies that were long dead (and probably should remain). For the Android approach I found 2 big problems: We have to actually work on generating the actual binaries (which means, from any platform to a specific ubuntu-vivid-arm target) plus all the dependencies. This meant, creating a new distro, and we already had debian/ubuntu for that, let’s use debootstrap! Oh, wait…

Old school

For a start, I took what sebas already started, in fact. Using debootstrap to create a chroot jail that could cross-compile the projects into our platform. This started to prove feasible really soon, as 2 or 3 days after working on it (and after fixing some issues that kept arising, mostly on KF5 itself and packaging), I already started to output binaries that could be deployed on the device.

Problems:

  • My IDE is outside of the jail, so there isn’t much we can do to integrate at all (we can’t access the build directory, and most of the data in it isn’t too meaningful for the outside anyway). A solution would be to ship and run the IDE from within the SDK though.
  • You need to be root. Not only to generate the system, but also to run it.

New school

An idea I wanted to approach was docker. Everyone on the web world is shit-crazy about it and it’s deeply based on traditional Linux, so there’s quite in common already. Doing something similar to the deboostrap there was a piece of cake, and I managed to re-use the set up code I already had for the previous version.

Problems:

  • Also everything is inaccessible from the IDE
  • I’m less aware of the limitations I’ll find.

Still, this second approach feels lighter and it’s quite fun to investigate something new like that.

Tooling

It seems that the jailed systems are the way to go, at least for now, so the tools I’ve created so far assume that they live in the jail as well. So, what do we have?

  • createpkg: Makes it possible to create a deb package that can be sent to the device for testing. It’s much simpler than the simplest deb package ever, but it works. Much better than sending executables over, much better than learning how to package for Debian.
  • deploypkg: sends the deb file, installs it and starts the application.
  • click-get: Downloads a click file from the Ubuntu Store, it’s harder than you’d think.
  • kdev-debdeploy: Does it all, inside your good ol’ IDE.

To be done

  • Workflow: Figure out a way to deploy KDevelop within either jail, but then we’ll also be able to use kde-debdeploy, that does mostly the same as these tools, integrated on the IDE.
  • Emulation: Testing: QEmu + docker anyone?

FreeSoftware to the fullest!