If you haven’t heard yet, the first monthly update to KDE4.1 is out.
4.1.1 has lots of performance improvements and bugfixes and is a recommended update for all KDE4.1 users.
For more info, check out the release announcement.
Sep 032008
About a week ago my subversion account was approved and I am now officially a KDE code contributor. Tonight I finally had a chance to check out playground and add kde3to4 to the tree, it is under the utils subdirectory.
Now I will not claim that there isn’t any place I may have made a mistake, I read the appropriate techbase docs but the codebase I am submitting into is heavy C++ which is not my first language so if I did get anything wrong this initial commit may end up being redone, nevertheless I believe it represents a milestone for KDE3TO4 to be in the official KDE repo’s even if it is, for now, in playground.
UPDATE: I added some links to the post and prettied it up a bit.
Introduction: What this post is about
I’ve seen a few comments and blogposts about KDE4 along the lines of:
“You promissed us a semantic desktop – but there isn’t one. KDE4 is just KDE3 made pretty
where is the new semantic interaction with data then ?”
Well in this post, I will try to answer that question. I’ll be talking a lot about other people’s projects, so if I get anything wrong – I hope those people will correct me.
The short version is: it’s coming. KDE4 laid the architectural grounds for it, but it’s not
yet how you visibly interact with it for two main reasons.
The first is simple: applications have to be adapted to start utilising advanced semantic search features for finding data, and that takes time. But this is actually a fairly small factor as about 90% of the adaptation lies in KDELIBS. The big one is the second reason: semantic desktops require lots of metadata, which need to exist first.
The chicken-and-egg problem: Egg-laying
Let’s assume that we had shipped KDE4.0/1 with semantic access as the default in all apps, you would try to find files and constantly not find them -because the metadata doesn’t exist. It has to be added.
This creates a bit of a chicken-and-egg problem, nobody wants to go and spend a month carefully tagging all their old data to be semantically searchable. But if data isn’t tagged, it’s not possible to find it by tags now is it ?
So how do you address this. You start by working behind the scenes, build a desktop that uses traditional file access to get files, but as you create and work with them automagically tag as much as you can. A lot of data can be searched and refferenced by content – in fact almost anything that contain words can be, but music and picture files don’t. Video files don’t. These are the problem right now.
As you use KDE4.1 your data you create is starting to get tagged to some extent automatically. Digikam has powerful image tagging support already and it works easilly enough for manual tagging. But let’s say you save a picture from an e-mail. Kmail should now automatically tag it with the address it was sent from, the date you got it and a few things it already knows. Maybe the e-mail subject ? (I would love to know more about this part – what autotags are automatically created).
Filenames only give you a single refference for data, tags give you the ability to cross refference. Let’s say I open digikam, and select all the pics in an album with my dog in it and tag them “leela”, now I select everything (and it’s easy with thumbnails) that has my wife in them and tag those “silvia”.
This now opens the possibility to later find all pics of my wife with our dog by crosschecking for the Silvia and Leela tags – this is where it gets powerful. By adding more tags, I can start to get ever better results.
The chicken-and-egg-problem: add the chicken
The next phase then, will be to start letting everyhing use these tags. When I am sending a mail and I want to attach a file, the open dialog will let me find files by their tags, rather than the directory/filename – so ultimately I can easilly find the picture which foo@bar mailed me of Silvia playing with Leela on the beach by checking for “silvia”,”leela”,”beach” and “foo@bar”. This is not there yet, because the tag creation part needs to be happening first to allow this to work. Ultimately though, it should become the default way to open files in anything – so that you only ever have to browse for the few things you don’t have searchable.
How it is done:
So how is this being implemented ? There are a few different projects that are working together to make the semantic desktop work. A core one is strigi. Strigi is a desktop search engine. It is powerful with multiple backends. Sadly, quite a few KDE4 distro’s currently disable it ! Or ship it broken (this needs to be considdered an urgent issue as it will be a block to the future of the semantic desktop if it’s not there now). Assuming though that your strigi is working on your desktop, as you start to KDE4, your initial indexing will begin to run (which is slow the first time and uses a lot of resources, this is an unfortunate side-effect). Strigi can index files by image tags, music tags, content and a few other kinds of tags to. So it starts to create a search tool that can ultimately locate any file by every piece of available meta-data.
Strigi will form the heart of how we find things in the future, right now it’s there mostly to start getting the indexing up to speed and it’s not yet integrated well into apps. There is a strigiclient that comes with it though which lets you search through the index already though.
The next level is nepomuk. Nepomuk is not exclusive to KDE though KDE is probably the first desktop to fully utilise it. Nepomuk is a platform independent semantic desktop architecture. It provides an interface between your desktop search engine on one side (in our case strigi) and your desktop on the other. Nepomuk is how apps will find and tag things.
Right now the most visible user of Nepomuk from a user point of view is upcoming amarok2.0 which uses it extensively to find and manage your music files. Amarok has a unique advantage for this purpose as virtually all music files are already well tagged with some essential data – this is usually done when they are created automatically and has been for years. Unlike all other apps, almost all our music data has been solidly tagged for years.
Amarok is using nepomuk to start building things like biassed playlist generation. So you could start to select music from certain era’s by using a bias from a certain year (say not out by more than 4 years in either direction) the further to the limit a song is, the lower it’s bias scoring, the later it gets in the playlist (this is an oversimplification but the amarok blog has a very detailed description of the bias system).
Finally, there is KDELIBS, which is what ties nepomuk to the application level. This is where apps will start to get tag-enabled file dialogs for example, without the apps even having to change. Things like amarok or digikam which work directly with one specific set of data implement semantics in their core interfaces directly. Things like krita or kword which work with one file at a time will use it in other ways, through smart dialogs and automatic tagging and indexing.
Conclusion:
I am well aware that this post does not cover the full detail of the semantic desktop in it’s entirety but that would take a book anyway. More-over it’s not meant to be technical doc, it’s meant to expand on our examples of what a semantic desktop can do with an explanation of how we intend to get to it from the current way KDE4 looks and works and why you don’t really see it yet. I invite comments, especially from developers who can explain parts of the semantic desktop idea in more detail and correct any misunderstandings on my part.
Further reading:
Mandriva demonstrates KDE4 Semantic Desktop
Sebastian Trüg on the semantic desktop ad Akademy 2008
Aaron Seigo on Nepomuk context
Nepomuk for integration (also from Akademy)
A blog post by Sebastian on the Strigi, Nepomuk and KDE4.2
Aug 212008
This is a minor update adding one new feature to correctly migrate the user’s search provider settings for konqueror. Thanks to NhJM who contributed the plugin.
I am working on building slackware packages for it, because it’s a script they will also be compatible with bluewhite64.
Aug 202008
I am canceling the slamdunk project, sorry to anyone who was hoping on it. My reasons are simple: slamd64′s multilib setup is just too far from slackware, for most packages it’s fine but for kde4.1 – it’s a disaster and makes it very hard, 3 days later and I still cannot get QT to build.
Add to that, slamd64 is just not up to date enough, there isn’t a current and this meant it was a backport too. It also means that, since Fred is low on time, I have no idea when the next slackware will become slamd64.
Some slamd64 users have flamed bluewhite64 as a ripoff, and it’s true that the early version used a lot of Fred’s work – but the fact is- bluewhite64 is up-to-date (meaning KDE4.1 packages are already there). The pure 64bit distro can be much closer to it’s 32-bit cousin so it’s also easier to port packages. So tonight, I’ll be replacing slamd64 with bluewhite. I know I’ll pay a price as the ia32 emulation layer for 32bit apps is not complete (you need a multilib to do it right) but everything I use either already has a native 64bit version or I can build one.
Ultimately, it was just more work than I could handle, and too long without a KDE desktop to get there.
I would rather donate time hacking my box and working on KDE and since this is all hobby-stuff I do not want it to be bogged down with hundred hour wastes that doesn’t seem to get anywhere.
Kudos to Fred for his great work, but if he cannot maintain the current tree he should allow somebody else who has time to do it, even if he maintains an oversight role – that’s just my feeling. I was a distro developer for five years and if you develop a distro you take on a responsibility to the users who depend on you. If you are not able to maintain that responsibility anymore you should quit and let somebody else take over. Sometimes, stepping aside from your own pet project is just a part of the way free software works.
Just a heads up to everyone that the new beta Nvidia driver has fixed most of the 2D bugs plaguing KDE users in particular, it’s version: 177.67 and I highly recommend upgrading as almost the entire changelog are fixes for the RENDER bugs that made our lives miserable.
Kudo’s to NVidia for finally dealing with the bug reports- but I stand by my earlier claim: I will never buy another NVidia card unless they make a free driver (or nouveau becomes fully stable). This entire debacle just highlighted why proprietary drivers are so evil.Some say they are less evil than apps – I am thinking they are worse, because when they screw up – it can make everything else broken.
There are now solid 3D acceleration options with free drivers – why would I ever buy NVidia again ?
The announcement of KDE4.1 in slackware-current was the push I needed to go back to my old friend. But, woe is me, slackware does not support 64bit platforms natively and 32bit software on a 64bit CPU is slow.
Even if that software is slackware :p
So what to do ? Well there are two major unofficial slackware ports for 64bit platforms. SlamD64 and BlueWhite. I chose Slam64 primarily because it’s not a pure 64bit OS and has 32bit compatibility libs included. A lot of work by one very cool student named Fred. Now just one major dev may be scary with some distro’s but it didn’t bother me too much about slamd64, after all slackware itself only has one major dev and it’s the oldest surviving distro in the world ! And after all these years, still cool.
But there is one catch, Fred has not had time to keep slackware-current up to date so right now slamd64 doesn’t have KDE4.1 available, since the testing tree doesn’t exist.
I decided to do something about it. Now I don’t have that much time either so I sure wasn’t going to port the entire slackware-current to the slamd64 structure – but I did want KDE4.1
I decided to build it. I am about halfway now, with a really proper QT4 package finally compiled. It’s going to take a few days still as every package needs to be compiled and tested many, many times as I hack at the slackbuilds.
I used the slackware current sources to base my packages on, but I have made some crucial decisions.
1) Since I didn’t want to port the whole slackware-current alone – or live without KDE for that many weeks – my packages are not only being crossported to X64, but also backported to slamd12.1. Only where a library absolutely has to be upgraded am I building anything outside the testing/kde tree. So far the only libraries that are upgraded beyond pure slamd64 as it comes from the disks is fontconfig and freetype – needed for nice antialiassing in the new QT.
2) My packages do not include a QT3 backward compatibility lib as a separate package. I tried a dosen times and the changes Pat made in the compat package just isn’t compatible with the slamd64 way of doing things, it keeps breaking all your libs meaning lots of reinstalls. Instead, I enabled qt3support in QT4 which is mostly SOURCE backward compatible. This means it will take some more work than usual to get KDE3 packages to play nice because if they are compiled against stock QT3 it isn’t binary compatible. Sorry folks, nothing I can do about that – I spent hours trying. If somebody else feels like giving it a go once the packages are out, I’ll be happy to include it. I’m going to compile my kdelibs3 against the backward compatible QT4 though, for what it helps. To make a package work, you will need a dedicated KDE3 machine to build it on (as with Pat’s packages) but you will need to install my QT4 package on it – this means it has to be a 64bit machine, running slamd64 -then you will have to change the QTDIR path to make sure you link against my QT in your SlackBuild script.
3) This is not an official part of slamd64 and I have no expectations of support from Fred, he works hard enough already, it’s something I am doing because I want it badly, and I’ll be sharing the results because that way I may save some other people from having to either forgo KDE4.1 or change distros (equal tragedies methinks).
Once all the packages are built, I’ll create a proper slapt-get friendly repo and put it up on this site for others to use. I am calling my project slamDUNK. Which stands for slamd64-UNoficial-K (the K of course for KDE).
Thanks to Fred personally for his advice right at the start which got all this going in the first place.
The packages are not dependency tracked (sheez, I only have so much hobby time) but I will add a metapackage which will have slapt-get dependencies on all the others in the right order so that you can install KDE4.1 as easily as possible from slamDUNK.
I also intend to add a few other interesting package I build to the repo over time, though they will be in an extras directory as this repo will remain primarily focussed on providing KDE packages. I will also share my modified SlackBuild scripts but beware – they are UGLY right now (I am pushing for time here – I’ll clean them up for round 2) – hopefully that will help others who wish to expand it.
And yes, this is going to be an ongoing project for some time – at least until KDE4.X is part of a mainstream slackware release with an official slamd64 port. Even then I will seriously consider doing weekly packages from trunk or something for bleeding-edge people if there’s enough demand.
That’s the great thing about free software. Sometimes you do it because you should. Sometimes you do it because it’s fun and challenging – and sometimes… it’s both – and you have a great time with not nearly enough sleep.
I have been using end-user aimed desktop GNU/linux distro’s for so long that without me realizing it I had begun to get bored with GNU/Linux… things always just working is convenient and was important when I ran a company – but now my computer at home is mostly a place to play and learn again, and it took going back to slackware to remind me how much I actually like fiddling, making hard things work and figuring out tough challenges.
Using KDE is easy – packaging it is challenging, but some of us enjoy that challenge. The lovely thing about GNU/Linux is- whichever you are, we have a distro for you.
I’ll keep everyone posted on the progress, I am hoping to have a working system in another day or two, then just some testing and if all goes well – the repo should be up by the weekend.
So this week I doubled the ram in my system, and decided to try the NVidia drivers again.
With the new ram and the known tweaks, it got really quite usable and I have most of the desktop effects working lovely.
Here is my main desktop as it looks now: 
I am running the Heron oxygen theme which is one of the few that plays nice on almost any wallpaper (nice for people with slideshows.
More-over, this allowed me to start playing with twinview again and try to get my secondary display to the TV working the way I want. Now in the past I had deliberately avoided both twinview and xinerama on my setup so I could FORCE apps not to run on the TV unless I specifically launched them. KDE4 does not yet support such setups and a lot of attempts to get around it were fruitless, but it turns out the KDE4 does handle twinview/xinerama in a way that completely removes the need for it.
The answer lies in the wonderful way KDE integrates multiple displays with the ZUI (Zooming User Interface) a feature of KDE4 that has gotten far less press than it deserves. The idea of multiple specialized workspaces is powerful and new – and that’s just the start of it.
In a twinview setup, your secondary display automatically becomes a sepparate ZUI workspace – which is great for managing the distant TV-out from my local screen, I just zoom out and do what I need there.
I cannot unfortunately zoom INTO the tv from the local screen (since it’s already there), but the ZUI does let me create setups there and launch apps.
I set up a folderview there to my movies directory – and voila -instant access ! I can even launch them from here without looking over my shoulder into the other room.
Here is a shot of the entire desktop – the right hand block is my TV desktop:
As you can see – it is a really nice way to work with my kind of setup and I am starting to fall in love with the power that ZUI’s can offer to truly orient your current workspace to your actual current activities.
Kudos to the plasma team, the ZUI took some getting used to but I definitely see the point, once you do get used to it, you wonder how you ever lived without it.
