Saturday, December 22, 2007

Removing CHS based access from windows boot loaders

Recently, I had troubles to migrate my Windows installation from VMWare to VirtualBox. When booting the vmware created partition in virtualbox, I got "NTLDR not found". So I sharpened the knives and got down to business with vmware's gdb interface and virtualbox's internal debugger. Tracing the execution showed that the BIOSes of the two products reported different geometries on the INT 13h interface. The generic method contained in the boot loader to read a sector from disk is "clever" as it checks whether the sector is below the maximum sector index that is reachable with the CHS geometry reported by the BIOS. If not, it uses the LBA interface of the BIOS. If yes, the cleverness of the boot loader suddenly vanishes. Instead of using the BIOS reported geometry to break the absolute sector down into its CHS components, the boot loader uses a geometry stored in the so called BIOS parameter block. That's a section of the first sector embedded into the boot loader that hard codes such values as head per cylinder and sectors per heads into the boot loader. If the hard coded values are different from the ones used by the BIOS, the calculation produces wrong values. So, if you move your partition to a BIOS that exposes a different geometry to the boot loader than is hard coded in the boot loader the whole thing blows up. Brilliant Microsoft design, as ever.

My solution is to override the check in the boot loader, so that LBA based access is always used and the CHS code is never touch. This way I'm able to use my partition under vmware (which uses heads=15) and virtualbox (which uses heads=255) simultaneously. Here is my boot loader patcher for FAT32 and NTFS based boot loaders: killchs.c. Use on your own risk. Chances are good that you can restore you boot loader with mbrfix if it breaks your boot loader.

Btw: VirtualBox is available under the GPL, and not only this makes it much more sexy to work with, it is also much faster than VMWare, at least that's my impression. There is also a commercial distribution of VirtualBox.

Thursday, November 15, 2007

X11-xft, also for XMonad

Haskell bindings for Xft are on their way. They don't cover 100% of the API yet, because some of the API relies on the only briefly covered Xrender and FreeType APIs. But the whole thing is sufficiently usable to get nice FreeType rendered fonts in xmonad! Here is the pr0n.

For the moment my Xft patch for xmonad only covers Tabbed.hs, so you get Xft fonts in tab rendering. A few more pieces are missing in Prompt.hs, but I'm sure this will be covered soon.
darcs get http://clemens.endorphin.org/X11-xft/
Enjoy.

Sunday, September 30, 2007

Depressing

After the most expensive car check and maintenance in my car owner history few weeks ago, my car died. Right in the middle of nowhere. And even worse, on my way to ICFP07.

Unfortunately, most of my planning rests on having a car, as my accommodation is not within Freiburg. Taking a plane/train to Freiburg and renting a hotel is simply way over my budget. I hope to have a working car by tomorrow, so I can make at least 50% use of my registration. Otherwise the registration fee will become a write-off in my books.

As I got home (from the middle of nowhere -- story on its own), I was so frustrated that I started to hack C. Here is an updated XFT patch (aka anti-aliasing support) for ion-3rc-20070927. Check the changes the patch does to your system.mk.

Yes, yes, I will switch to XMonad soon.

Tuesday, September 25, 2007

Stabilizing Firefox for beta quality plugins (Flash)

Using the recent Adobe Flash plugin in Firefox is a two edged sword. On one hand, it gives you the never ending joy of useless YouTube videos, but on the other hand it crashes your browser every 24h on average (only if you are also using Flashblocker which reduces flash content activation by 80%, namely all the annoying ads).

Presumably because of Flash, nspluginwrapper was invented to wrap up software of vendors that are incapable of producing 64-bit version of their product. nspluginwrapper works by providing a proxy to Firefox that acts as regular plugin, but actually spawns off a separate process that runs in 32-bit mode that links in the shared object of the plugin. We can use this construction to remove Flash from the intimate dlopen embracement of Firefox and move all negative side effects of beta quality software into a subprocess. The following trick works for all 32-bit browsers, whether the rest of your distro is amd64 based or native i386. If you are
using a 64-bit browser, it's likely that you know nspluginwrapper anyway, so this blog entry is about 32-bit to 32-bit.

Grab a copy nspluginwrapper (at least 0.9.91.5). Untar it. Compile it under an i386 environment. If you don't have this at your disposal, here is it precompiled from my local Gentoo installation: nspluginwrapper-0.9.91.5.tbz2. This is compiled for the amd64 arch layout of Gentoo. Under native i386, just create a link from lib32 to lib in /usr.

Now grab a copy of libflashplayer.so. Drop it into a directory that is not part of your default Firefox plugin path and that does not contain the substring 'netscape' in its absolute path. So,
/opt/netscape/plugins/libflashplayer.so is wrong twice, /usr/lib32/nsbrowser/plugins libflashplayer.so is wrong only with respect to the default plugin path, and /usr/lib32/nsbrowser/wrapped-plugins/libflashplayer.so is correct.

The reason why the path must not contain 'netscape' is that libflashplayer.so presumes its part of a netscape installation when it sees 'netscape' in its path and malfunctions for Flash movies as on YouTube and Google Video under nspluginwrapper and Opera.

Invoke /usr/lib32/nspluginwrapper/i386/linux/npconfig -i /usr/lib32/nsbrowser/wrapped-plugins/libflashplayer.so. As super user, this creates npwrapper.libflashplayer.so in /usr/lib32/nsbrowser/plugins. Invoking this as normal user drops this file into ~/.mozilla/plugins. You are now read to go with your new shielded Firefox. Check that it contains a Section 'Shockwave Flash' under about:plugins referring to the npwrapper.libflashplayer.so file. If it refers to libflashplayer.so directly, you have a copy of libflashplayer.so laying around in a plugin path.

Flash might still cause lock ups in this setting. I presume Flash is getting stuck in an endless loop keeping Firefox busy even through the proxy. However, when that happens just kill the nspluginwrapper subprocess called npviewer. After that Firefox should refresh and you can just hit reload on Flash sites (instead of killing firefox and reloading all websites). Good luck.

Tuesday, September 11, 2007

QT RLE hits ffmpeg SVN

My last drive-by-software-hacking turned out to be useful at the end. I hacked up a basic QT RLE (aka Animation) encoder for ffmpeg, sent the patch to ffmpeg-devel but without further intentions to work on merging. Thanks go to Alexis Ballier for stepping up and refining my patch until it was polished enough.

With recent version of ffmpeg, you can create a 1280x1024 screencast for your running X session at display :0.0 with
ffmpeg -f x11grab -s 1280x1024 -i :0.0 -f oss -i /dev/dsp -vcodec qtrle /tmp/lala.mov
after running configure
./configure --enable-gpl --enable-x11grab --enable-libfaad --enable-libfaac
Unfortunately rgb555 encoding was removed, so you are only able to create videos rgb32 video streams (aka 24-bit color depth).

Friday, May 18, 2007

Lisplab.at dies

lisplab.at dies in 3 days. I can't justify spending 25 EURs on extending the domain subscription, as nobody of my co-developers showed any partical interested in fueling this social attractor.

If anyone is interested in reviving a Common Lisp developer group in Austria, feel free to contact me (preferably before the 21th of May).

Thursday, May 17, 2007

Liskell at the International Lisp Conference 2007, Cambridge, UK

It's been over a month since I returned from Cambridge and still I have not found time to blog an epilogue for this trip.

First, here is the material:
Liskell presentation-ILC 2007.mov.torrent.
This is a screencast of my talk at the International Lisp Conference 2007; one of the two presentations I have given in Cambridge. Notice that this talk was prepared for a Lisp audience, so my job was to highlight the features of Haskell. The other talk in Cambridge was at Microsoft Research and I plan to make this talk available in a similar style (but with the objective to sell Lisp ideas to a Haskell audience).

A few remarks:
  • The implementation of cond as defmacro is a pedagogical code mock-up. It operates on the wrong data structure. The Liskell paper has a correct implementation of cond as defmacro.

  • There are a few errors in the codec stream for qtrle. For some reasons, my recent sync with ffmpeg subversion HEAD caused this trouble. There are only 3-4 errors, and they are usually corrected in a few seconds with the next Pframe in the stream.

  • Levelator rox. I spent about 2 hours finding suitable settings for mastering my voice track in Audacity with no useful results at all. The first try with Levelator was a perfect astering. Lovely.

  • The liskell-mode you see in this video is just a tiny minor mode wrapping haskell-mode with scheme indention. A video on how to set up this development environment is also on my todo list. Grab liskell-mode.el.
Audio and video recordings of the original session are available. But unfortunately, the video is incomplete and the quality is insufficient for reading anything from the projection, also the background noise of the audio recording is quite high. I recommend to download the screencast above. The slides and source code files are available here.

And finally a few photos in my semi-private flickr stream.

Monday, May 14, 2007

liskell.org


Liskell — my favorite Lisp+Haskell pet — has a new website, namely http://liskell.org. There you will find a new branch of Liskell based on GHC-6.6 and a new Liskell development mailing list liskell-devel@liskell.org.

Monday, May 7, 2007

Happy Birthday LUKS!


Two years ago on the 25th of March 2005, I bumped the version of cryptsetup-luks to 1.0. With the release of cryptsetup-luks 1.0.5, the LUKS reference implementation -- and arguably LUKS itself -- is more than 2 years old! With this release, the LUKS branch of cryptsetup is relabelled (in accordance with Christophe Saout) and cryptsetup-luks becomes the original cryptsetup.

Number one hit on Google for "LUKS" was once the American painter George Luks. I'm sorry George, but we have taken over. The term "LUKS" is now a synonym for "Linux Unified Key Setup". LUKS is now part of any major distribution, Debian includes it with their installation suite, Gnome can talk to it via a HAL module. I am quite satisfied with my first free software project, although my recent addiction to Lisp/Haskell/Liskell makes it a bit harder for me to code C.

It's interesting, how live changes in just two years. It's interesting how your personal interests change in just two years. I shifted my attention from capital markets, to Lisp, to Genetic Programming, to Typing, to Haskell, to Liskell, to static reasoning about programs in a non-turing complete language. "Making good cookies" and "world domination" are the next on my list.

Thursday, February 22, 2007

Liskell docs, finally


As the ACM SIGPLAN template gave my Latex document much more space, I decided — instead of writing a separate piece of documentation — to enrich my ILC07 submission with more interesting chapters about meta-programming and the Liskell prelude. This paper is much richer in content as the original (about half the content is new) and it finally starts to explain the mechanisms behind defmacro and how you can use them in the Haskell/Liskell world. So if you are interested in Liskell, the revised paper certainly deserves a second look.

Tuesday, February 13, 2007

Pattern guards in Liskell

Yes, I'm working on documentation this very moment, but after seeing A missing Haskell feature on Planet Haskell, I was unable to resist. I need my daily dosage of coding.

Here is working Liskell syntax sugar for pattern guards as described by Neil Mitchell.
(defmacro (~= pts)
`(let (((comp ,pts) True)
((comp _) False))
comp))
It can be used right-away as in:
(all (~= Star _) ([] (Star "x") (Atom)))       -> False
(all (~= Star _) ([] (Star "x") (Star "foo"))) -> True
To give you a brief idea why this works: The function body of defmacro is called with a list (called pts) that is bound to ([] (PSym "Star") (PSym "_")) which is equivalent to [PSym "Star", PSym "_"] written in Haskell syntax. This parse tree part is substituted into the backquoted code template at ,pts, resulting in the parse tree:
(let (((comp (Star _) True))          | let comp (Star _) = True
((comp _) False)) | comp _ = False
comp) | in comp
The Haskell equivalent of the generated Liskell parse tree is shown on the right hand side for our Planet Haskell readers. This generated parse tree is the result of the compile-time "~=" macro transformation, and instead of the original (~= Star _) expression the generated parse tree is compiled to object code.

You can compile Star.lsk with "ghc Star.lsk -o Star -main-is Star -package LskPrelude".

(For those who already have a local Liskell branch, please pull the latest changes and redo ./darcs-all get. I moved the LskPrelude into a separate new core package. Also I vastly simplified the parse tree type, so the Liskell paper is not correct anymore wrt this.)

Wednesday, January 31, 2007

Macros for Haskell? Done.

Why is Lisp special? Because of macros. This changes today.

For the last three month, I have been working on Liskell, a language extension for GHC. Some people might be tempted to call it a new language because of documents talking about language definition. But in fact, it is just a new dress for something we already embosomed: Haskell.

Liskell brings symbolic expression (aka Lisp syntax) to Haskell. This is no Lisp interpreter written in Haskell, but GHC itself becomes the part that processes Lisp like syntax. After parsing, the Liskell extension transforms this sexp parse tree into a regular Haskell parse tree, as it would result from any regular Haskell file. From then it takes the regular path through type inference, desugaring, compiling core. So, if there was not another important thing, this would just be a new syntax for Haskell.

Within Liskell, we get parse tree transformers, a generalization of macros. Before the conversion to the Haskell parse tree takes place, you are free to do any arbitrary manipulation to the Lisp-styled parse tree of sexps by injecting your own parse tree transformers into the compiler. The possibilities with such a facility ranges from minor syntactic sugar transformations such as field syntax, 'deriving' syntax or another do-notation expansion, to more grave extensions to Liskell, such as embedded sub-languages. An example for the latter is my embedded Prolog version, that can be seen in a little appetizer (Flash Video, 99mb, direct, torrent (preferred). To play mplayer, for Windows especially: FLV Player.).

The official page is http://clemens.endorphin.org/liskell. The material I release for the moment is my darcs repository of GHC containing Liskell and a modified testsuite containing a small set of regression tests and the Liskell Prelude. The Liskell Prelude defines a lot of syntax sugar, so this might be the first place to look for reasonable first-steps examples into meta-programming. But before jumping into meta-programming, you might want to look at the basic language first. The Liskell page contains a draft for a Liskell paper submitted for ILC07 and it also features a 15 minute tour of Liskell
syntax.

How fast the documentation grows, depends on the community reaction and your feedback. For the moment, please be content with the basics, I hope to introduce you to meta-programming in a more systematic way next week.

Monday, January 8, 2007

hoogle.el


Here comes a short emacs helper for Haskell coders. It allows you to hoogle lookup the symbol currently under your cursor. Drop hoogle.el into your ~/(x)emacs dir, and add

(require 'hoogle)
(define-key haskell-mode-map [?\C-c ?h] 'hoogle-lookup)

to your init.el. For educational purposes, here is the function that does all the (pretty-simple) magic.

(defun hoogle-lookup (p)
(interactive "p")
(let ((symbol-name (thing-at-point 'symbol)))
(unless (and (= 1 p) (stringp symbol-name))
(setq symbol-name (read-from-minibuffer "Hoogle lookup name: " ""
nil nil 'hoogle-history)))
(browse-url (concat hoogle-url-base symbol-name))))