Posts in Category: programming

Prettifying forms with jQuery UI

I’m still adding little features and working on the design of my lunch organizer app. I’ve added nice javascript widgets for handling the input of dates and times. jQuery UI is a nice library that is the UI extension of the excellent jQuery library. Among many other things it features a very nice widget for picking dates:

Although the HTML5 input type “date” hopefully brings us the functionality with simple HTML, we have to settle for this version for now (only Opera displays a special widget for the “date”-type). Unfortunately there’s no default widget for time fields in jQuery UI. I’m not sure why that is so. Maybe it’s selection process is a bit slow? I’ve looked at a number of widgets for choosing time and it’s very refreshing to see how many interesting approaches have been invented for this:

In the end I chose the excellent widget by François Gélinas.

When you download jQuery UI, you can pick and choose which functionality you need. There’s a simple form, so you can minimize the size of the scripts each user has to download. The full package is over 1MB, thus this is really necessary. The simple selection with the datepicker and the position library is only 75KB (+25 KB for the images). The timepicker weighs in at an additional 60KB, which should still be tolerable.

The default size of the datepicker is much too big and it’s not very obvious how you can reduce it. Fortunately the always helpful stackoverflow comes to the rescue. Add this simple style to your stylesheet and you will get nicely size date widgets:

The same technique works for the timepicker:

The next problem I had with these widgets were their default positions. Their default is just below the input field at its lower left corner. That way the datepicker covers the rest of the form and lowers the user experience. Of course there’s a stackoverflow page that helps with that, too:

For the timepicker there’s not such an easy way, but it has two position attributes that let me achieve the same effect (at least in Chrome):

When using the datepicker with Django, I had to make use Django’s id format. It’s always “id_“. Thus it was id_time for the field time. Of course I also had to add the stylesheets and the css elements to the head element:

UPDATE: The new code is available at github. I’ve also added the ability to delete entries and I am preparing to add permissions and user management soon.

Improving design

I haven’t used CSS in a while and it’s amazing what effects you can achieve with some simple stylesheets. One thing I really like are the “intelligent” CSS selectors. Instead of dealing awkwardly with classes and polluting your Django views with design decisions, you can simply include some cool CSS.


Image from W3

Take for example the alternating row colors. In the olden times you used to define classes for each second row and use a different background color. Now you can just write:

Pretty cool, he?
In my lunch organizer web app I also used this code to create borders between the different weeks and the header:

It’s nice how easy it is to create fance drop-shadow effects, too:

All in all it looks like this now:

Getting external static files to work with Django on Google App Engine

It’s surprisingly hard to reference simple static files such as css stylesheets or images. When you upload your project to GAE the usual simple file links stop working. Even referencing a file in the same directory doesn’t work. That stumped me a bit. Why shouldn’t it work? I can only imagine that there are security reasons for this. After all you don’t want anybody to be able to access your actual python scripts either. So how does Google solve this? They implemented a technology on top of the web app framework that redirects access on specific paths. This is the same mechanism that’s used for Django’s urls.py.

Now my apps.yaml looks like this:

Any path that starts with /media is translated to the path static/media in your project folder then. It’s advised to distinguish between the actual path and the path used in your templates this way. I imagine the reason is again security, but isn’t this security by obscurity and not even that much obscurity because it’s rather obvious? I’ll have to do some more research on this. One disadvantage of this approach is that a local django test server of course can’t see the files because the app.yaml configuration is specific to GAE. I hope there’s an easy fix for this and you can actually use both with the same set of files.

Now that my site uses some fancy external css file, that defines the default font, it looks like this:

Working with Google App Engine

Google App Engine (GAE) is a service that lets you run your web apps in The Cloud. It’s a magical place where everything runs faster and better.

Cumulus cloud from WikipediaCumulus cloud from Wikipedia

In reality the cloud is not really one technique but rather a slightly overused term for running applications on remote computer in a data center facilitated by some framework that allows easy setup and abstracts away all the small hindrances of dealing with the underlying infrastructure. Google App Engine is a service that lets you put web apps on the web with the touch of a button. The apps run in a sandbox and as long as you don’t overuse the resources you can even put your web app online for free. They estimate that the free limits of the service are enough for 5 millions views per month, which should be true for websites that do not primarily offer large files like videos or music.

The real beauty of all this nice abstraction comes in when you use GAE together with one of the supported frameworks. I’ve tested it with Django and my simple webapp from the earlier post. Django uses a small python script to manage a project and to test a web app locally all you have to do is run “python manage.py runserver”, which starts a web server on your computer that you can access with 127.0.0.1:8000. When you change any of your project’s files Django recognizes the changes and immediately reacts to it. This way you can develop very fast! This is one of the aspects I really love about web development; it’s all so fast and you get to the see the results directly. I wish I had this kind of responsiveness when developing desktop apps. Pushing your changes into The Cloud is as easy as “python manage.py deploy”. How awesome is that?

Ok, after all this good stuff here are the downsides. The documentation for getting Django running is extremely outdated. You only get a link to some other project that supposedly works much better. This project is called django-nonrel, which stands for non-relational meaning a No-SQL database. My first reaction was that it seemed totally unrelated to what I wanted to do. I didn’t care what kind of database was used, but I was much more familiar with some relational DB and I had used SQLite before trying to convert my project for Google App Engine. Then I tried to follow their tutorials, but I soon encountered an annoying problem with GAE. When you upload your app and you make a mistake in the most basic setup; django, google app engine, python version mismatches, nothing works. The only information you get is a useless internal server error 500 error page. There’s no documentation on how to deal with this and it took me an hour to find someone mentioning the google app engine administration to obtain logs. Even then it’s all quite cryptic.

Apparently the default python version is the ancient 2.5, django’s default is 0.96 and then you need “some” version from django-nonrel. After lots of searching I found out that Python 2.7, Django 1.3 and the latest django-nonrel is a working option. The way to go is to install Python 2.7, Django 1.3 and get django-nonrel from the project website. You have to obtain 6 packages (the last characters are the git hash, which reference the exact revision):

  • twanschik-django-autoload-1698ab544030
  • wkornewald-django-dbindexer-ae5f7638827e
  • wkornewald-django-nonrel-be48c152abc6
  • wkornewald-django-testapp-3dd32705c980
  • wkornewald-djangoappengine-60c2b3339a9f
  • wkornewald-djangotoolbox-a8cdf61ba9c0

Then get the test app to work. Fortunately I found this stackoverflow thread that addressed the very problem. Get the packages from this website. Then follow the the instructions for the test app. This youtube tutorial lists the necessary steps in more detail. Then start modifying the test app to include your own project. The youtube video goes through these steps very quickly too. The main idea is to make a subfolder for your project that is used from the modified test app. Fortunately the test app is already configures for django 1.3 and python 2.7 and the only thing you have to do is modify the app.yaml of the test app with your own application name. When you’re done you’ve joined the ranks of the hip cloud people :-) !

Dabbling in web development with Django

I’ve wanted to develop for the web for quite some time. When I was younger I did some web design. But back then it meant that you probably had to learn ActionScript to get any kind of visually impressive design. Today most of it can be done with simple HTML, CSS and JavaScript. Fortunately there’s also been a big trend towards simpler usable and clean web design approaches. Because all that seemed so interesting to me, I’ve begun to learn some bits in web development.

Spider web taken or created by Fir0002Image taken or created by Fir0002


Now that you can make pages look nice, you gotta start your journey to web design guru somewhere and instead of starting from scratch learning HTML5, CSS3 and JavaScript, I’ve taking the less steep road of using a ready-made web app framework. Python being my language of choice for learning new concepts it was only natural to start with its most popular web app framework Django — after spending days researching this and deciding against “cooler” and less feature-rich alternatives like flask, bottle or cherrypy.

One of the mantras of creating good software it to build something that you need yourself. It’s not very deep but still true, as there’s tons of abandonware of software projects programmers worked on but never bothered to finish or improve so they become generally usable. At my current company there’s a group of people that alternate in cooking lunch for the group. Often the coordination who cooks lunch when and where is difficult and requires emails to everyone, that everyone has to respond to interrupting whatever you did before. It’s no big surprise that email is broken. A google search for “email is broken” yields 991.000 results and most of the time instant messaging is touted as its rightful heir. I don’t like that solution very much because it interrupts you in an even more annoying way than email. My small web app tries to fix it by switching from pushing email to pulling the info from the publicly reachable web app.

table view
edit date form

I’ve made a small html table interface that is fed by an ORM that holds Lunch objects and can be edited using forms generated directly from the model. All this is surprisingly easy with django and the only hindrance for me was learning how to represent the control flow in a web app. I especially like Django’s easy setup of the different ulrs that control the different functions of the website. This focus on urls also means that you have really nice clean urls instead of ugly php?id=jfkdfjd. Using regular expression you can match the parameters to the so-called “view function” and just use it as any old python method parameter

from urls.py:

and from views.py:

The code is all at github — which is an awesome site and deserves a later post of its own. I’ve taken the opportunity to create my first open source this way ;-) :
github.com/kossmoboleat/lunch-organizer

New C++

The new standard that was approved last year, will bring a lot of improvements to C++ and reduce some of the gap to Java. Some of Java’s newest additions like lambda and the foreach loop have been taken over to C++ without much change. Watch the above video for some mindblowing (at least for programmers) example of these new features in combination with super-easy use of asynchronous multi-threading. The example begins around minute 15,16.

Installing your own email and calendar server

Instead of giving all your data to other companies like Google or Apple, you can keep it to yourself and create your own email, address and calendar server from which you can control all your data and export it whichever way you want.

If you want an integrated solution for emails, addresses and calendar, you have two choices give up all your data to some big mega corp or setup a home server with Windows and install Exchange on it. Fortunately there are some less well-known open source alternatives to that. You can install them on an old computer that runs Linux and have a much cheaper solution than with Windows. In fact there are even solutions that work on a measly file server or NAS (Network Attached Storage). Such a device has very limited processing power and ram, but it may be enough for an open source group ware solution (that’s what email+addresses+calendar actually is).

Since a few days Zarafa is available for Synology NAS boxes with more than 256 MB RAM. Setting the machine up is relatively easy. There’s a guide on their community site.

I just had a few problems that held me up along the way:

  • If you try to execute postfix reload and it does not work. Check /var/log/messages. If you get this message:
    Mar 18 00:02:24 postfix/postfix-script[11456]: starting the Postfix mail system
    Mar 18 00:02:26 postfix/master[11466]: fatal: open lock file /var/lib/postfix/master.lock: cannot open file: Permission denied

    check the permissions of each directory level up to the file and make sure it’s at least 755.

  • Here are some things (link in German) you can check if something goes wrong.
  • Here’s a guide in German on how to setup different SMTP servers for different sender addresses
  • If you wan to change the default behavior that deletes fetched mail from the server, check this (in German) out. It’s a thread in a QNAP forum that describes how to change fetchmail’s default behavior such that it only copies mail from a mailbox and does not delete it. You have to modify the tip a bit to match Synology’s paths. Use /var/lib/postfix/atc/fetchmailrc and /usr/local/zarafa/bin/fetchmail instead of the QNAP paths mentioned in the thread.

Unfortunately I realized that my machine is a bit too light for this workload. It does not have 256 MB of ram and Zarafa is not very usable, because it’s much too slow.

Zarafa community website

open-source firmware for canon camera

Free open-source software always fascinates me. You have a product that was designed and produced by an industrial manufacturer and then you have a bunch of free-minded people that just decide to take this peace of plastic — or metal — and turn it into something different; supercharging/pimping it. In an earlier post I’ve written about Rockbox, which managed to free some limited devices from their simplistic firmware and added lots of features and the ability to play all kinds of other audio formats that were not playable before because of weird music industry politics.

CHDK is a bit different in that it only adds some features to your camera and runs beside the original firmware. Like Rockbox it adds another format, in this case RAW, and it makes a whole lot of other ways to use the device. CHDK gives you more control over all details of the picture generation too and allows you a closer look at the inner workings of a camera. You can control the details of exposure and bracketing. I’m curios about how you can use another feature; scripts. You can write scripts that tell the camera when and how to make pictures. If you combine this with the motion detection, you get a pretty sophisticated camera. Maybe, I’ll use it for some timelapse experiments. It seems like that should be easy. It’s also possible to make HDR pictures (blog post in German) like this.

OSD view augmented with CHDK elements

Installing the firmware is not soo easy, but doable. Find your camera’s installation page, download the modified firmware, extract it on a SD-card, change to playback mode, press the firmware-update entry in the menu and be happy. Upon restart the firmware is gone; it’s only temporarily loaded to memory.

Have Fun!

Simple Sudoku-Solver

I’ve always wanted to program a sudoku solver. It’s a funny little problem which is essentially a computation with a little logic mixed in. There are many ways to solve a sudoku. Some of the more involved are constraint programming and stochastic search.

There are actually approaches that just fill the numbers brute force. So they start with e.g. 1 in the first cell and then try 1 again in the next cell. Because it was already used they advance the counter to 2 and go on. If none of the 9 numbers works for a cell they go back one cell and increase its counter by one. Surprisingly this approach is actually viable, although there 6.67 x 10^21 different possible grids.

As a middle way, I’ve implemented a backtracking solution that at least takes into account the rules for rows, columns and 3×3-squares. I haven’t optimized the program for speed and it can take a couple of seconds for the harder sudokus.

The main magic happens in the recursiveSolve() method:

First the program tries to derive all the numbers that follow logically from existing numbers. These “derivations” are then stored as are the different choices that are tried in each backtracking step. If a backtracking step turns out to be invalid it returns false and its calling step tries another value for the cell in question.

You can download the whole thing here. It’s a Qt project that you can easily import into Qt Creator that’s part of Qt-SDK.