FreeSoftware to the fullest!

Category: bluesystems (Page 1 of 3)

Developing KWin Wayland

On the last few weeks I’ve been looking at KWin more closely than in the past. It’s definitely a special beast within KDE and I figured it could be useful to give some hints on how to develop and test it.

When developing something, first step is always to compile and get the code installed and usable. It’s especially delicate because when we mess up our system becomes quite unusable so it needs to be done with care. To prevent major damage, we can probably try installing it into a separate prefix (See this blog post, change kate for kwin).
Second step is to make sure that modifying the code will modify the behaviour you perceive. This is what we’ll focus on in this piece.

Bear in mind most of the things I’m saying here are possibly obvious and not news, but it’s still good to have it written in case you feel like building on this (not fun to come up with) experience.

Testing KWin, Nested

That’s the first that we’ll probably try. It’s simple: we run a kwin_wayland instance within our system, it shows a window with whatever we’re running inside. I find it useful to run a konsole inside to trigger other applications. It’s also possible to point processes run elsewhere to that socket too but I haven’t really come across the necessity.

We’ll run something like:
kwin_wayland --exit-with-session konsole

Or even:
kwin_wayland --exit-with-session "konsole -e kate"

In any case, a good old kwin_wayland --help. The --exit-with-session bit isn’t necessary but it makes it useful to easily close it.

One of the biggest advantages of using it like this is that it’s very easy to run through gdb (just prefix with gdb --args kwin_wayland...), so still something to remember.
This works, but it’s usefulness is limited because important parts won’t be run: input, which will come from the actual session rather than from the kernel; and drm, which will go into the virtual session than to the kernel for the same reason.

TL;DR: if you’re modifying KWin to talk to the kernel, this won’t work.

Testing KWin, standalone

Next thing to try is to run it separately. Now you’ll want to move to a tty (e.g. ctrl+alt+F2) and run the same command as earlier. Now we’ll run the same code as above. Instead of getting a window now we get it full screen and using DRM and libinput straight from the source. *yah!*

It’s still quite uncomfortable though: the output will be going to the tty which we can’t see because we have kwin on top and if it crashes we’re at a loss. Also running it with gdb is a pain because we can’t interact with it.

That’s why for this I’d recommend to ssh into the system from another one. I’ve found the sweet spot there to use a combination of ssh and tmux (screen should work as well, there’s tons of doc on how to use either online). Using ssh we find 2 main problems: if the connection breaks the whole set up is left in a weird situation and from ssh we don’t really access the tty.

If you don’t have 2 computers available, you can ssh from a phone as well, should be good enough, especially if you can get an external keyboard, although a nice screen is always useful.

So what we’ll do is:

  • Make sure our system has sshd running, check the IP
  • host$ tmux
  • othercomputer$ ssh $IP
  • othercomputer$ tmux a

And then we’re seeing the same on both, and we can run kwin_wayland on either and everything will work perfectly: we’ll see debugs we may be adding, we’ll get to kill it with Ctrl+C if we please, we can run gdb, add breakpoints. Anything. Full fun.

Problem being it’s just KWin and there’s more to life than a black screen with a dancing window.

Testing a full Plasma Session

To start Plasma Wayland from a tty we run startplasma-wayland. This gives us a complete Plasma instance on Wayland. The same as if we were running it from a display manager like SDDM. Running it from another system tmux still has advantages, like we’ll get to see the debug output. Starting a kwin_wayland session under gdb is way beyond this blog post and barely useful considering the standalone approach I mentioned earlier. What we can do though is to attach to the running gdb.

From the other system (on tmux or not, although I find it useful to just put it in another tab) we can run sudo gdb --pid `pidof kwin_wayland` and it will attach. Then you can add your gdb business: breakpoints, make it crash, look at backtraces, print variables or just stare at it hoping things get fixed automatically.

tmux with logs

Other

There’re other things that have proved useful to me like running it on valgrind or with a custom mesa build + modified LD_LIBRARY_PATHS. This works as with any other project, except it doesn’t by default.

KWin has a safety system set in place that prevents running it on emulators like valgrind and prevents LD_LIBRARY_PATHS from being tampered with. Which is great in general but prevents us to do certain things.

To address this in our build, we can comment out the code in kwin/CMakeLists.txt where ${SETCAP_EXECUTABLE} is called on kwin_wayland. You’ll need to make sure kwin_wayland is recreated, so remember to host$ rm bin/kwin_wayland. This bit here made me waste a fair amount of hours mind you, so it’s worth keeping in mind.

It’s also worth remembering there’s a ton of debugging tools that can be used. Here’s some:

  • WAYLAND_DEBUG=1 environment variable to see how the wayland protocols are flowing.
  • gpuvis, useful to get an idea of how rendering is performing.

Also note that all of this applies for Plasma Mobile as well, where it’s also useful for applications as well, where it’s most convenient since you get a proper keyboard and screen to look at debug output and gdb traces.

Hope this was any useful to you.

Happy hacking, stay safe!

Learning about our users

In a product like Plasma, knowing the kind of things our existing users care about and use sheds light on what needs polishing or improving. At the moment, the input we have is either the one from the loudest most involved people or outright bug reports, which lead to a confirmation bias.

What do our users like about Plasma? On which hardware do people use Plasma? Are we testing Plasma on the same kind of hardware Plasma is being used for?

Some time ago, Volker Krause started up the KUserFeedback framework with two main features. First, allowing to send information about application’s usage depending on certain users’ preferences and include mechanisms to ask users for feedback explicitly. This has been deployed into several products already, like GammaRay and Qt Creator, but we never adopted it in KDE software.

The first step has been to allow our users to tune how much information Plasma products should be telling KDE about the systems they run on.

This mechanism is only integrated into Plasma and Discover right now, but I’d like to extend this to others like System Settings and KWin in the future too.

Privacy

We very well understand how this is related to privacy. As you can see, we have been careful about only requesting information that is important for improving the software, and we are doing so while making sure this information is as unidentifiable and anonymous as possible.

In the end, I’d say we all want to see Free Software which is respectful of its users and that responds to people rather than the few of us working from a dark (or bright!) office.

In case you have any doubts, you can see KDE’s Applications Privacy Policy and specifically the Telemetry Policy.

Plasma 5.18

This will be coming in really soon in the next Plasma release early next February 2020. This is all opt-in, you will have to enable it. And please do so, let it be another way how you get to contribute to Free Software. 🙂

If you can’t find the module, please tell your distribution. The feature is very new and if the KUserFeedback framework isn’t present it won’t be built.

KDE on Android: CI, CD & SDK

I guess we all agree that one of the biggest stoppers to get a contribution out is the ability to get the system ready to start working on the contribution. Today I want to talk a bit about generating Android binaries from our machine.

In the KDE Edu sprint we had the blatant realisation that it’s very frustrating to keep pushing the project while not being swift at delivering fresh packages of our applications in different systems. We looked into windows, flatpak, snap and, personally, I looked into Android once again.

Nowadays, KDE developers develop the applications on their systems and then create the binaries on their systems as well. Usually it’s a team effort where possibly just one person in the team will be familiar with Android and have the development combo in place: Android SDK, Android NDK, Qt binaries and often several KDE Frameworks precompiled. Not fun and a fairly complex premise.

Let’s fix that.

Going back to the initial premise of delivering KDE Edu binaries, the first thing we need is a continuous distribution system for Android, much like we already had for binary-factory.kde.org.

Our applications will be bundling some KDE Frameworks, we need to know which work, to make sure they don’t break to some extent, much like what build.kde.org is already doing.
And last but not least, while at it, we need to create a system that makes it simple for developers to do the same that our automatic systems are doing.

Docker to the rescue

We needed a way to pull everything we will need into a system: SDK, NDK, Qt binaries, few build dependencies. For this we created the following image:
https://hub.docker.com/r/kdeorg/android-sdk/ (Source)

With this image one can create binaries for his application locally.

We have also deployed it in build.kde.org to build some KDE Frameworks. Note that we are lacking some dependencies on Android so our offer is limited in comparison to traditional GNU/Linux. Also we are not running tests, it’s something we could look into but I’m not yet sure how useful that would be.

I want to develop

Here you have a wiki that explains how to use the docker image to build your KDE application.

I want to test

I set up an F-Droid repository that offers the binaries listed in the binary-factory conveniently, so that when there is a new version we will get the update.

Future

All of this ties directly to the runtime integration I mentioned already in this blog post.

Docker as an SDK for ARM distributions

Lately I found myself working on an ARM64 (aka aarch64) based system which in turn I don’t own. So I needed to get a system to build and test things on.

How?

First of all, you need to have qemu static builds installed. For example, for ArchLinux you need to get them from AUR:
$ yaourt -S qemu qemu-user-static binfmt-support

And then enable the aarch64
# update-binfmts --enable qemu-aarch64

$ docker run -ti --rm -v /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static apol/test bash

X11

Once set up, it can be used as any regular docker image. For example passing -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix will give us access to X11, allowing to run apps:
$ docker run -ti --rm -v /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix apol/test bash

One possibly wants to use Xephyr to test some stuff, probably on the safer side.

Similarly, you’ll get to use gdb or any tooling you need and is available for the architecture.

KDevelop

You get to code against your project against the platform you need by passing the extra required arguments, then most features discussed here apply.

Beyond…

Granted it will work rather slow, as it’s emulation, but given the alternative is not being able to work at all, I’m happy.

Note that most of this applies for any architecture, replacing aarch64 for any architecture supported by qemu (and your distro of choice): You can check /usr/bin/qemu-*-static.

Tracking a QObject’s life

Why?

One of the interesting things about QML is that it leverages OOP property semantics of QObject to drive its declarative workflow.

How?

We attach into any QObject the developer requests and monitor it (and optionally all it’s children).

To gather the information we need, we will collect information for every property and connect to the property’s notification signal to see when it changes.

What?

This is the repository:https://phabricator.kde.org/source/kobjecttracking/.

This allows us to see:

  • Spurious property change notifications
  • Property changing bursts (i.e. same property changing values repeatedly in a short amount of time)
  • How a change in a value can have an effect on other properties

At the moment we have two ways of inspecting an application:
On one hand we have warnings that allow us know specific things and even add breakpoints to the sore points. The warnings at the moment look like this:
repeated value ScrollView_QMLTYPE_43(0x5648c6d97b50) width QVariant(double, 320) 8 times
repeated value Kirigami::BasicTheme(0x5648c6986ac0) backgroundColor QVariant(QColor, QColor(ARGB 1, 0.988235, 0.988235, 0.988235)) 11 times

Additionally, we have a timeline view output from tracking. You can see an example of plasmashell run here.

Future

An idea would be to integrate automatically ModelTest, so if a QAbstractItemModel is found, a ModelTest instance is attached that makes sure the model is true to its promises.

Another thing that bothers me is that we are forced to compile it into the application. If we could make it possible to have processes attached or start the application somewhat automatically (like GammaRay does, wink-wink) we would ease the adoption of these tools.

What do you think? Do you think it’s useful?
Feedback very welcome.

KDevelop runtimes: Docker and Flatpak integration

On my last blog post I discussed about how some assumptions such as the platform developed on can affect our development. We need to minimize it by empowering the developers with good tools so that they can develop properly. To that end, I introduced runtimes in our IDE to abstract platforms (much like on Gnome’s Builder or Qt Creator).

There are different platforms that we’ll be developing for and they need to be easily reachable when coding and testing. Both switching and interacting transparently with the different platforms.

To that end I implemented 4 approaches that integrate different runtimes:

  • Docker, allows you to develop directly against virtually any system. This is especially interesting because it enables to reproduce the environment our users are having: behavior on execution and project information (i.e. the imports are the ones from the target rather the ones on our local system). Docker is a wide-spread technology in the cloud, I hope many developers will see the value in integrating the deployed environment into the IDE while they are coding.
  • Flatpak, is a solution that targets specifically desktop Linux applications. We are talking about distributing bundled applications to users, there we have the opportunity to integrate the tooling specifically to that end: from fetching dependencies to testing on other devices (see videos below).
  • Android, as you know it’s something I’ve been pushing for years. Finally we are getting to a space where the IDE can help get some set up troubles out of the way.
  • The local host, i.e. what we have now.

And remember KDevelop is extensible. Do you want snapcraft?, vagrant?, mock? Contributions are very welcome!

If there’s something better than a list of technologies and buzzwords, that’s videos. Let’s see why this could change how you develop your software.

One development, any platform

We get to develop an application and switch back and forth the target platform we are developing for.

Here I put together a short video that tests Blinken on different platforms:

One development, any device

Using the right SDK is not enough proof that the application will work as expected on every device, especially those our users will be using. Being able to easily send our application to another device to test and play around with is something I had needed for longtime. Especially important when we need to test different form factors or input devices.

In this video we can see how we can easily test an application locally and when it works just switch to Android and send to the device for proper test on the smaller touch screen.

Here we can see how we can just test an application by executing it remotely on another device. This is done by creating a bundle of the application, sending it to the device where we want to test it and executing it there.

Hassle-free contributions

You can’t deny it. You’ve wanted to fix things in the past, but you couldn’t be bothered with setting up the development environment. Both Flatpak and Docker offer the possibility to maintainers to distribute recipes to set up development platforms that can and should be integrated so that we can dedicate this 1 hour in the week-end to fixing that bug that’s been annoying us rather than reading a couple of wikis and – oh, well, never mind, gotta make dinner.

We can do this either by providing the flatpak-builder json manifest (disclaimer: the video is quite slow).

Or a Dockerfile.

You can try this today by building kdevelop git master branch, feedback is welcome. Or wait for KDevelop 5.2 later this year. 🙂

Happy hacking!

Getting Free Software into our users’ hands

In KDE we cover a mix of platforms and form factors that make our technology very powerful. But how to reach so many different systems while maintaining high quality on all of them?

What variables are we talking about?

Form factors

We use different form factors nowadays, daily. When moving, we need to be straight-forward; when focusing we want all functionality.

Together with QtQuick Controls, Kirigami offers ways for us to be flexible both in input types and screen sizes.

Platforms

We are not constantly on the same device, diversity is part of our lives. Recommending our peers the tools we make should always be a possibility, without forcing them into major workflow changes (like changing OS, yes).

Qt has been our tool of choice for years and it’s proven to keep up with the latest industry changes, embracing mobile, and adapting to massively different form factors and operating systems. This integration includes some integration in their look and feel, which is very important to many of us.

Devices & Quality Assurance

We are targeting different devices, we need to allow developers to test and make it easy to reproduce and make the most out of the testing we get, learn from our users.

Whatever is native to the platform. APK (and possibly even Google Play) on Android, Installers on Windows and distribution packages for GNU/Linux.
Furthermore, we’ve been embracing new technologies on GNU/Linux systems that can help a lot in this front including Snap/Flatpak/AppImage, which could help streamline this process as well.

What needs to happen?

Some of these technologies are slowly blooming as they get widely adopted, and our community needs as well to lead in offering tooling and solutions to make all of this viable.

  • We need straightforward quality assurance. We should ensure the conditions under which we develop and test are our users’ platforms. When facing an error, being able to reproduce and test is fundamental.
  • We should allow for swift release cycles. Users should always be on fresh stable releases. When a patch release is submitted, we should test it and then have it available to the users. Nowadays, some users are not benefiting from most stable releases and that’s makes lots of our work in vain.
  • Feedback makes us grow. We need to understand how our applications are being used, if we want to solve the actual problems users are having.

All of this won’t happen automatically. We need people who wants to get their hands dirty and help build the infrastructure to make it happen.

There’s different skills that you can put in practice here: ranging from DevOps, helping to offer fresh quality recipes for your platform of choice, improving testing infrastructure, or actual system development on our development tools and of course any of the upstream projects we use.

Hop on! Help KDE put Free Software on every device!

Discover more in 2017

With 2017 starting, we’re getting ready for the next Plasma 5.9 release and with it a new Discover release.

This will be a special release for two main reasons: further add-ons integration and Kirigami.

New Stuff

One thing pending for a long time was to actually better integrate the different parts of the system that can be integrated. To do so, Discover now will automatically import all knsrc files present on the system and offer them as categories.
We’ll be able to go beyond Plasmoids and Comics (!) and make it possible to explore: Plasma Look and Feel themes, cursors, icon themes, window manager add-ons and different application-specific resources.

This won’t have a big impact on Discover’s performance because the backend code was refactored to be able to have several backends loaded even without knowing all of the resources available. Queries are now asynchronous and parallel.

Discover KNS Addons

Kirigami

Kirigami was adopted in the previous release already, and this release will use Kirigami 2, the port to which did not require much work.

The most significant improvement is the inclusion of keyboard navigation, which was requested by several people and is now finally available. Do you hate it when you’re required to use the mouse? Rejoice!

Extra: Snappy support

It’s 2017, not all of the applications are coming from your distribution anymore. To start getting things in place for different software distribution sources I started working on a Snap backend, which allows us to manage applications coming from this system.
Some work will still be required and it won’t be included by default, since it doesn’t yet support AppStream (although we discussed it and it seems it will happen soon), but if you’re curious feel free to take a look and give your feedback!

GNU/Linux bundled application ramblings

It’s impressive how in the last few months (and especially the last few weeks) the discussion around bundled applications for the GNU/Linux Desktop has sparked.

It’s especially interesting because:

  • The problem is not new.
  • The solutions that have attempted to tackle the problem in the past have been ignored (both by us developers and by distributions).

The TLDR

First, let me try to subjectively summarize the problem: Historically, the resources we get in GNU/Linux come from the distributions. Anything: executables, libraries, icons, wallpapers, etc. There’s been alternatives to all of those, but none has flourished as a globally adopted solution.

This guarantees that everyone using a distribution will have access to the resources the distribution can offer. The more powerful the distribution is, the more we get. There’s limitations nevertheless, so some restrictions have to get in place. The ensemble of limitations and technologies adopted will effectively define the user’s experience.

This works. It has worked for years and, given the technology is in place, it could easily keep working. Like in most engineering solutions there’s drawbacks and properly addressing them can bear some goodness. It seems like now it’s the moment to review this situation. Let’s enumerate some of the problems we have nowadays:

  • We have users using really old versions of our software with issues we’ve solved in versions they can’t use.
  • It’s really hard for GNU/Linux users to get users to test unstable versions of our software.
  • We have users who want to use fresh versions of some software but not in the whole system.

There’s been many solutions to fix those, some easily come to mind: ArchLinux’s AUR (with yaourt), Ubuntu’s PPAs, big-tar application packages, OpenSuse’s OBS, and possibly others.

Far from showing the maturity of the Linux desktop, what this depicts is the deep fragmentation we’re into: we have come up with different solutions that break the established distribution paradigm by lowering the restrictions and considering the resources offered as unsupported (often tainting the whole system).

What has appeared recently is sandboxing. It’s especially interesting because by letting the users execute any binaries we’re increasing the exposition of their systems. Hence, jumping from our distributions’ nest into the lions. As always, sandboxing creates new challenges: It requires changes in applications (or frameworks) to adapt, often creating a user interaction fence (e.g. a popup asking if you let Kamoso access the webcam). For what it’s worth, that’s not new: Android does it, OS X does it, Windows does it (from the Store), Chrome OS does it, etc.

Now where are we?

We need to decide about GNU/Linux’s future. Or at least, we need to understand what Plasma users will have available. So far, most of the noise comes from the big players in the business trying to differentiate their products, meaning incompatible versions.

Without an agreed unified solution, we’ll have to assume we’ll end up having installed snappies, flatpaks, AppImages as well as applications from the distribution. Then it’s just a matter of:

  • Presenting it properly so that the user knows the risks taken by executing an application (!)
  • Make sure we don’t lose many features by sandboxing.

Still, one of the good things of this new approach is that it shouldn’t have to be necessary to have several people dedicated to build every single application and component. If the solution is to add 3 more solutions that will need dedicated people, we’re not really moving forward.

Building

As soon as we’ve decided how we want to work, then the interesting stuff needs to appear. If this is properly engineered, it can bring really interesting possibilities that now we hardly ever find:

  • Newer versions of applications on administered systems (e.g. universities).
  • Enabling stable distributions on professional environments.
  • Beta channels.
  • Binary application 3rd party extensions.
  • Provision of debug symbols (some distros don’t offer them).

To finish the fantastic post, a note for the dreamers:
How easier would all that be in a microkernel architecture?

We need you!

Of course this will be a long journey and we need your collaboration. This year in Randa we started working on all these problems in several different angles. It’s important for the KDE Community to have your support, so we can keep providing quality software. Consider donating, doesn’t need to be a lot, everything counts.

Discover in Plasma 5.7

I haven’t talked about what’s going on in the Discover front for a while, here’s a small heads up on what you’ll get in the next Plasma 5.7 release.

Back Ends

  • PackageKit (and AppStream): This one has received most of the care. It has been rearranged to instant load. Also we made it possible to integrate this backend with KNS/OCS, to show when there’s add-ons ready for an application.
  • KNSBackend: It’s being polished and adapted for new use-cases. First it used to focus on Plasma-centered assets, but we’ve worked towards making it possible to become a viable solution for Application add-ons as well. Furthermore it has been trimmed down to also provide fast startup while keeping its features.
  • QApt: The traditional QApt backend used in Kubuntu is now considered deprecated in favor of PackageKit, which I understand should be available properly on every GNU/Linux distribution, including Kubuntu. We kept it around because there’s been longstanding issues with PackageKit on Ubuntu, but I’m confident these will be solved soon.

Front End

We used to have many of them, they have now been reduced to two: Discover and the Notifier Plasmoid. This allows us to focus one GUI. Of course, you can still use Muon, Apper and Synaptic together with it, for more precise package management.

Regarding Discover, the main focus this year has been: performance, stability and Add-ons.

  • Performance: Mostly back-end work, and it has also benefited from improvements in KF5 and upstream Qt. Updating the dependencies will improve your Discover experience as well as that of other similar applications. Better usage of some model classes was also a nice improvement in some places.
  • Stability: We worked on making sure that things not only are sturdy but that the development process doesn’t get in the way of stability. To that end, I set up a testing framework so that changes are automatically tested, additionally to the manual tests the developers will do at some point.
  • Add-ons: One of the important parts of Discover, is to make sure the user is aware of the nice resources he has available. KNewStuff resources (mostly coming from kde-look and kde-files so far) have been available for a while, albeit mostly for Plasma enhancements. In this next release we allow these to extend applications themselves as well.

Future

Discover‘s future is Kirigami. We have already some work lined up (see the kirigami branch), we are pending some design work so that it shines, but the approach is already quite promising!

Also I’d like to start working on the Flatpak backend, if nobody beats me to it. 😀

Stay tuned!

« Older posts

© 2024 TheBlindCow

Theme by Anders NorenUp ↑