Joachim Breitner's Homepage
The Automatic Rule-Based Time Tracker
A while ago, I thought: „I seem to be less productive than I used to be. Why might that be? Do I spend more and more time with e-Mails? Or is it something else?“ I couldn’t tell, so I needed a time tracking tool. There are a few of those for Linux that allow you, while working, specify what you are doing and then generate statistics about it. This approach has a few disadvantages:
- It is distracting (and I want to increase productivity).
- You can only record one kind of information. So either, you record what you are doing (writing e-Mails, surfing the web, programming), or why you are doing it (work, study, leisure). You can not easily record both.
- If you are lazy and don’t keep updating it, the statistics will be useless.
- You won’t be able to catch a little thing like quickly answering an e-Mail or looking for the weather report.
So I created arbtt, the Automatic Rule-Based Time Tracker. It comes with a background program that is started with your desktop session and will, each minute, record what windows are open, which one is active, what their titles and corresponding programs are. It also checks how long the user has been idle. No interaction required, no distraction possible. This information is stored in a log file. A separate tool allows the user to investigate this data. It is called rule-based because the mapping from the raw data to sensible “tags” that give information about the time sample is specified by a simple, but hopefully sufficiently powerful language.
A simple example for a rule that indicates the currently used program would be:
tag Program:$current.program,
The prefix “Program:” is a category. Arbtt will ensure that for each time sample and category, at most one tag is specified. A more complex rule is
current window $title =~ m!(?:~|home/jojo)/projekte/(?:programming/(?:haskell/)?)?([^/)]*)!
==> tag Project:$1,
which makes use of the fact that both GVim and gnome-terminal display the full path to the currently edited file resp. to the working directory in the window title. This rule will track all my projects separately, and even automatically pick up new projects when they appear! You can see more rules in the example file.
The statistics program then allows you to query the tags with some options:
Usage: arbtt-stats [OPTIONS...]
-h, -? --help show this help
-V --version show the version number
-x TAG --exclude=TAG ignore samples containing this tag
-o TAG --only=TAG only consider samples containing this tag
--also-inactive include samples with the tag "inactive"
-m PERC --min-percentage=PERC do not show tags with a percentage lower than PERC% (default: 1)
-i --information show general statistics about the data
-t --total-time show total time for each tag
-c CATEGORY --category=CATEGORY show statistics about category CATEGORY
For example, if I want to know what folders I have open while using evolution, I can run arbtt-stats -o Program:evolution -c Evo-Folder -m 3
to see this:
Statistics for category Evo-Folder
==================================
|| Time Percentage
=========================++======================
Evo-Folder:Eingang || 22h20m00s 17.0%
Evo-Folder:Bekannte || 16h40m00s 12.7%
Evo-Folder:d-haskell || 9h40m00s 7.4%
Evo-Folder:Itomig || 8h20m00s 6.4%
Evo-Folder:Verschickt || 7h40m00s 5.8%
Evo-Folder:pkg-fso-maint || 7h00m00s 5.3%
Evo-Folder:Bugs || 6h00m00s 4.6%
Evo-Folder:planet debian || 4h00m00s 3.0%
(60 entries omitted) || 4h49m00s 36.7%
(unmatched time) || 8m00s 1.0%
One big advantage of this approach is that you do not need to know in advance what queries you are interested in. Since the rules are applied when you are evaluating your data, and not when recording it, you can add more tags and forgotten special cases later.
At this point, a big warning is due: This program will record a lot of very sensitive information about you. Be aware of this before you start using arbtt, and make sure you protect your data. You can get rid of all logs by deleting ~/.arbtt/capture.log.
I have published arbtt on hackage. If you have cabal-install installed, you can install it with cabal install arbtt. See the README file for more information about setting it up. Depending on the feedback I get I will also consider packaging it for Debian.
If you want to contribute, you are very welcome. The code is available at the darcs repository http://darcs.nomeata.de/arbtt/ (DarcsWeb). See the README for some ideas what to implement and feel free to come up with new ideas.
Comments
http://blogs.igalia.com/mrego/2008/05/30/mytime-free-software-to-improve-time-management/
* Name mapper, you need to group a bunch of poorly named misnamed windows together
* Realtime updates
* Graphs
* Idle detection, I have hacked up xscreensaver just so it can detect when you have gone idle, then my window recorder can record that idle status has changed. Then when you count time you don't include the huge blocks where you went to the bathroom.
My software is in perl and C at http://churchturing.org/ , I would suggest that idle-watcher is the most useful to you.
I had some Graphs locally already, but nothing releaseable yet. Not sure about realtime updates, they make little sense with textual output. With a GUI, maybe.
http://lists.suckless.org/dev/0909/0996.html
I'll post a link to your blog entry there.
I'm very interested in such an application as well. I currently use hamster-applet but would welcome an application where I don't have to change my status manually. I'm not used to haskell so I'd like to see a package as well. Would be nice to see some reporting tools in a graphical interface. I wonder whether MyTime has the same flexibility as yours...
Thanks for arbtt, I was a user of http://www.rescuetime.com/ and now I can replace it.
However, when I run
$ runghc Setup.hs configure
in arbtt-0.1.1 directory, I get
Setup.hs: arbtt.cabal:30: Parse of field 'build-depends' failed:
In that line, there is a dependency like:
base == 4.*
and I think either my cabal is old or this is a syntax error. When I changed these *-ed rules like to
base >= 4.0
it successfully configured, though reporting unmet dependencies.
I'm using Debian Squeeze/Unstable and tested the file also on another machine. GHC version is (apt provided) 6.8.2.
Thanks.
I’m developing with ghc6-6.10.4. I guess that error comes from an older cabal version.
About the unmet dependencies: I suggest you use cabal-install, as it will handle these for you. See http://haskell.org/haskellwiki/Cabal-Install for more details.
thanks! arbtt will be very useful to me. I have started using it already. I had to fix a few things though:
1. getInputFocus returns a window ID that is off-by-one on my system. No idea why. I had to change (w == fwin) to (w == fwin-1) to get anything to ever be matched.
2. Several of the formatSeconds call sites had time/100 instead of time/1000. So some of the calculated times were off.
Thanks again! Great stuff!
Not sure about the other issue. I sent a mail to the X11 maintainer about it.
Thanks!
$ tail -n 1 ~/.arbtt/capture.log
"TimeLogEntry {tlTime = 2009-09-14 22:05:42.129558 UTC, tlRate = 60000, tlData = CaptureData {cWindows = [(False,\"Linkes Kanten-Panel, ausgedehnt\",\"gnome-panel\"),(False,\"Michael Daub\",\"pidgin\"),(False,\"\",\"galeon\"),(True,\"jojo@kirk: /home/jojo\",\"gnome-terminal\"),(False,\"Blog-Comments (89 insgesamt) - Evolution\",\"evolution\"),(False,\"XChat: nomeata @ FreeNode / #pidgin (+tncL)\",\"xchat\"),(False,\"Thomas Breitner\",\"pidgin\"),(False,\"Buddy-Liste\",\"pidgin\")], cLastActivity = 46}}"
You won’t have much fun parsing that without haskell. But I’m planning to switch to a more extensible and faster parseable format.
Anyway: Parsing will be done in haskell (should compile under windows), but writing it down will be in something else. Or someone tells me how to get the data in haskell, I gave up :-/ .
If you want to do some more fun: try to get the URL from the browsers and write that instead of the window title...
Anyways, my idea for a better syntax would not also involve easier parsing, but also extensibility, e.g. for reading the url from a browser. (But how do you do that easily?)
http://forums.devshed.com/delphi-programming-90/getting-the-current-mozilla-mozilla-firefox-url-or-other-browsers-161991.html
FF on linux seems also to have some way to get at the URL: http://support.mozilla.com/tiki-view_forum_thread.php?locale=de&comments_parentId=189769&forumId=1
Please try again with 0.1.3 from hackage.
I am using awesome as WM, Firefox is actually Iceweasel from Debian with the Vimperator plugin, and have tried with arbtt 0.1.3.
A sample entry from capture.log:
"TimeLogEntry {tlTime = 2009-09-25 17:14:25.503754 UTC, tlRate = 60000, tlData = CaptureData {cWindows = [(False,\"vimperator-www.joachim-breitner.de.tmp + (/tmp) - GVIM\",\"gvim\"),(False,\"\",\"Navigator\"),...], cLastActivity = 8330}}"
xwininfo -root -children says:
0x1000046 "The Automatic Rule-Based Time Tracker - nomeata’s mind shares - Vimperator": ("Navigator" "Iceweasel") 502x343+2456+14 +2456+14
So the title information is missing and - what is really bad - Firefox is never logged as active, and therefore can't be tagged.
If not, can you try if 0.1.2 works better for you? If so, I should implement a fallback.
common/atoms.list:_NET_CLIENT_LIST
common/atoms.list:_NET_CLIENT_LIST_STACKING
ewmh.c: _NET_CLIENT_LIST,
ewmh.c: _NET_CLIENT_LIST_STACKING,
ewmh.c: _NET_CLIENT_LIST, WINDOW, 32, n, wins);
ewmh.c: _NET_CLIENT_LIST_STACKING, WINDOW, 32, n, wins);
So my guess would be, "yes, it does".
wmctrl -l seems to list my Browser just fine:
0x01000046 1 macbork The Automatic Rule-Based Time Tracker - nomeata’s mind shares - Vimperator
I just tried with arbtt 0.1.12 - this did not change anything. However, I noticed that sometimes the Window title is found - just not for this very site. Might the Unicode ’ in the title be a problem? Activity is still always false, regardless of the window title being empty or not.
Btw., there is no way to get notified of new comments here, is it?
About the active window: Probably same problem as CJ’s, a possible fix is known and I’ll try to implement it soon.
Subscription to comments used to be possible, but Serendipity did not support Double-Opt-In, which is a legal requirement in Germany. I could see if I can enable per-post-commets-RSS-feeds.
Just a little thing: when I add the feed to Google Reader, it does not show the title, though it is set. Maybe it is also due to the unicode character in the title, but you specified utf-8 as encoding in the xml header, so I think this should be valid. At least feedvalidator.org does not complain:
http://feedvalidator.org/check.cgi?url=https%3A%2F%2Fwww.joachim-breitner.de%2Fblog%2Frss.php%3Fversion%3D2.0%26type%3Dcomments%26cid%3D336
This might just be Google Reader's fault.
Try with those:
http://yogan.meinungsverstaerker.de/tmp/titletest-ascii.html
http://yogan.meinungsverstaerker.de/tmp/titletest-unicode.html
I’ve looked into the fwin-1 issue a little and it seems to me that the problem is that getInputFocus does not always return a top level window. Depending on the toolkit of the application it could return some child window or another. Using queryTree to walk up the tree to the top most parent of the window returned from getInputFocus does the trick here.
I also wanted to have shorter samples, so I just changed the sampleRate in capture-main.hs from 60 to 20. Adding parameter for this would be more convenient, of course.
Too bad I don't know Haskell, so I see no chance implementing this myself.
Making the sample rates configurable would mean that arbtt-capture would read configuration. Currently it does not, which is a nice feature. But then, I see that your point is valid, and I guess it should be possible. (Luckily I planned ahead enough so that the sample rate is stored with the data, so that changing the rate does not break the old data.)
About --last 3h or the like: A more general approach would be to add time related functions to the conditions in the categorizer syntax, and then introduce a new flag "--filter <cond>" where arbitrary complex conditions can be specified. This enables time queries, but also any other stuff that you can do in categorize.cfg itself.
You can now also put fully featured conditions on the statistics command line. To have the desired effect of your proposed "--last 3h", use
arbtt-stats --filter '$sampleage < 1:00'
More stuff could be added, such as day-of-week and more featureful time arithmetic, but that can come later.
cabal install arbtt
and works fine.
To meet dependencies I had to install:
libpcre++-dev
libxrandr-dev
libxss-dev
For tracking Firefox activity I'll try to use customize_titlebar_v2 plugin.
Have no opinion about features but it looks great!! Thanks :-)
Launching arbtt-stats, got always "arbtt-stats: Unknown variable desktop"
Can't find out why
Any help welcome :)
Thx
Have something to say? You can post a comment by sending an e-Mail to me at <mail@joachim-breitner.de>, and I will include it here.
Yes, I'd very much like to see this packaged for Debian. I'm unfamiliar with Haskell, and the current installation process seems complicated and scary, or at least error-prone.