Sourcepole - Aktuell

Shaded relief maps with QGIS

2012-01-16T07:51:00Z

Creating a shaded relief map from digital elevation data is a nice way to create a backround map for web mapping or other GIS work. Thanks to the know-how and the funding from the Swiss Humanitarian Aid Unit, QGIS now has a sophisticated function for relief map generation. The method is described in detail in an paper by Marc-André Bünzli. An important part of the method is the choice of the elevation colors. The QGIS plugin has the possibility to analyse the frequency distribution of the elevation values in the DEM and to propose color changes where significant changes in the histogram occure. It is of course also possible to modify the color scheme, to insert a completely different one in the dialog or to generat a shaded maps without color.

As illustrated below, the relief map is composed by three components modulated onto the final map to give it a more three-dimensional appearance. These intermediate steps are shown here to get a better understanding of the method. In QGIS, the user directly gets the combination as a result.

The first component consists of a hillshade from north-west (300 degree) and an elevation color

The second component is a hillshade and a gray value depending on the slope angle (darker is steeper). The hillshade angle of this second component differs to the first component by 15% to have better contrast in faces towards the light source. The second component is merged to the relief map with 30% weight.

The third component consists of a hillshade from 270 degree and a yellow color in cells facing towards the light source. It is merged to the relief map with 10% weight

And the final relief map looks like this:

Raster resampling in QGIS

2011-12-30T13:31:00Z

QGIS already offers a lot of possibilities to visualize raster data (contrast enhancement, color map, handling of transparent pixels, …) Last year, Radim Blazek refactored the raster provider interface and added on-the-fly reprojection support for rasters to QGIS. Very cool!

One of the few things currently missing in QGIS raster layer is the possibility to have other resampling types than nearest neighbour. The problem is that rasters appear pixelated when zooming further than the source raster resolution. So for applications like web mapping, it is important to interpolate the pixel colors and to have a broader display scale range for layers. In the last few weeks, I’ve added bilinear and cubic raster resampling to QGIS (thanks to Canton Solothurn for funding these activities!).

A lot of refactoring was necessary in the raster layer code to add resampling in a clean way. Therefore these changes are available in branch ‘raster_resampler’ of the QGIS github clone (git://github.com/mhugent/Quantum-GIS.git). It probably needs a longer period of testing to make sure every feature of the raster layer class still works properly.

The resampling option can be chosen in the raster properties dialog:

And here is the effect for a three band color image.

Nearest neighbour:

Bilinear:

Cubic:

And here for a palletted background map

Nearest neighbour:

Bilinear:

per user X11 options

2011-09-19T21:27:00Z

One would expect that considered the complexity of gdm there would be a way to have per user X11 options. So no, there isn’t one. At least not after the great rewrite after V2.22.

However good old Unix paradigms can help us (this is all under Debian, other Unices will allow a similar trick):

$ vim /usr/local/bin/X

Add something similar like this:

#!/bin/sh
#
# start X with different options depending on user

# check if the parent gdm process that started us contains
# "--username le_gamer" in its commandline
#
if ps -p $PPID -o args= | grep -q 'username le_gamer';
then
  exec /etc/X11/X.orig $* -config /etc/X11/xorg.conf.le_gamer
else
  exec /etc/X11/X.orig $*
fi

We check whether gdm is starting us and using the parameter “–username le_gamer”. If it is, then we’re using a different config file for X.

Of course you’ll need to adapt all this, unless you are using Debian or Ubuntu. You need to adapt the path to the X Server, to the config files etc.

Why did I do that? The problem is that Intel’s X server in Ubuntu Lucid Lynx 10.04 is really unstable with DRI on a Intel GM965/GL960 graphics card.

So unless I’m playing 3D games which are a lot faster with DRI, I don’t want to enable DRI (more over, “NoDRI” is pulling less power out of my battery). So my normal config contains this:

$ cat /etc/X11/xorg.conf
[...]
Section "Device"
        Identifier      "Configured Video Device"
        Option          "NoDRI"
EndSection

So now, the only thing you need to do is replace your “normal” X Server with the new “adaptable” one:

$ sudo mv /etc/X11/X /etc/X11/X.orig
$ ln -s /usr/local/bin/X /etc/X11/X

That’s it,

Tomáš Pospíšek

Celebrating 20 years of Linux!

2011-08-22T08:17:00Z

SVG symbols in QGIS with modifiable colors

2011-06-30T08:02:00Z

SVG markers are a popular way to symbolise points in QGIS. Predefined markers are available in $PREFIX/share/qgis/svg and it is straightforward to add new symbols or to create own symbols with a vector graphics program (e.g. Inkscape). A disadvantage so far was the need to create different versions of an svg file to have the same symbol in several colors. A recent change in QGIS now introduces the possibility to insert parameter tags into the svg file and QGIS is going to replace them with the values for fill color, outline color and outline width.

It works with a syntax similar to the svg params working draft. Let’s say we have the following simple svg file:

<svg width="100%" height="100%">
<rect fill="#ff0000" stroke="#00ff00" stroke-width="10" width="100" height="100">
</rect>
</svg>

To have the possibility to change the colors of the marker, we have to add the placeholders ‘param(fill)’ for fill color, ‘param(outline)’ for outline color and ‘param(outline-width)’ for stroke width. These placeholders can optionally be followed by a default value:

<svg width="100%" height="100%">
<rect fill="param(fill) #ff0000" stroke="param(outline) #00ff00" stroke-width="param(stroke-width) 10" width="100" height="100">
</rect>
</svg>

Now it is possible to change fill color, outline color and stroke width using the new elements in the QGIS symbol layer dialog.

Replacing parameters and rendering svg can be expensive in terms of processing time. Therefore, an svg cache has been added which accelerates painting of svg markers considerably. For drawing on screen and with QGIS server, the svg markers are painted as rasters. For printing, a vectorized output is generated.

This work was funded by the Canton of Solothurn (Switzerland). Thank you very much!

QGIS at FOSS4G 2011 in Denver

2011-06-16T07:37:00Z

Sourcepole is conducting a QGIS workshop and giving two presentations:

There are more QGIS workshops and presentations:

Case studies:

So if you’re interested in QGIS, don’t miss FOSS4G 2011 in Denver. Early-bird registration ends June 30th.

Reducing magic in Ruby's Forwardable class implementation

2011-06-10T13:07:00Z

Abstract

Using string-eval in Ruby for metaprogramming is unnecessarily obscuring. Ruby’s more modern and specific metaprogramming methods should be used instead whenever possible. This problem is illustrated on the example of Ruby’s Forwardable class.

In detail…

Ruby’s Forwardable class is using metaprogramming to forward calls from a frontend interface to an instance in the back executing the call.

Metaprogramming is the discipline of making code that creates code. This task allready is rather abstract and hard to grasp in itself. Having hard to grasp code is a liability. One of the goals of writing code is allways to keep the code as simple and as well understandable as possible.

Additionaly, metaprogramming code itself is difficult to read and understand: that is because the metaprogramming code will not necessarily express what the code it is creating is about, but only how it is creating that code. As such the code it is creating can be invisible to you as a reader of the source code - the created code will only start to exist at runtime.

One would therefore expect that programmers would try especially hard when they metaprogram to make that particular kind of code expressive and easy to understand.

Another consequence of the fact that the code produced by metaprogramming is not necessarily visible, is that debugging becomes more difficult: when analyzing problems you’ll not only be unsure how the programm works, but in addition, you won’t even be sure how the code that is executed looks like - since it is only generated at runtime.

This post is focusing on the last problem: debugging of metaprogrammed code.

There are two approaches to metaprogramming. One is to have as far as possible compile-time parseable code and the other is to let the code only be parsed at runtime.

As of version 1.9.2, Ruby’s Forwardable class is using the latter. The metaprogramming code in Ruby 1.8.7 looks like this:

    module_eval(<

As said, this has the consequence of the metaprogrammed code being completely invisible to the parser and other tools such as editors and debuggers.

This results in the following:

$ cat queue.rb
require 'rubygems'
require 'forwardable'
require 'ruby-debug'

class Queue
  extend Forwardable

  def initialize
    @q = [ ]    # prepare delegate object
  end

  # setup preferred interface, enq() and deq()...
  def_delegator :@q, :push, :enq
  def_delegator :@q, :shift, :deq

  # support some general Array methods that fit Queues well
  def_delegators :@q, :clear, :first, :push, :shift, :size
end

q = Queue.new
debugger # ------ DEBUGGING FROM HERE ON -----
q.enq 1, 2, 3, 4, 5
q.push 6

q.shift    # => 1
while q.size > 0
  puts q.deq
end

q.enq "Ruby", "Perl", "Python"
puts q.first
q.clear
puts q.first

$ ruby queue.rb

queue.rb:24
q.enq 1, 2, 3, 4, 5

(rdb:1) step
(__FORWARDABLE__):2

(rdb:1) list =
*** No sourcefile available for (__FORWARDABLE__)

(rdb:1) step
(__FORWARDABLE__):3

In other words, you are rather lost allready - otherwise you probably wouldn’t be stepping through your code - and in that situation it happens that your debugger gets completely lost as well, since it does not know any more where in the code it is and what it exactly is executing.

That’s nothing the programmer wishes for. In a situation where you are debugging you want to have a maximally clear view of all state, including what code you are currently executing.

Chaning that situation requires making as much of the metaprogrammed code visible to the parser, which is the second approach to metaprogramming mentioned previously:

$ cat forwardable2.rb
...
    self.send(:define_method, ali) do |*args,&block|
      begin
        instance_variable_get(accessor).__send__(method, *args,&block)
      rescue Exception
        $@.delete_if{|s| /^\\(__FORWARDABLE__\\):/ =~ s} unless Forwardable2::debug
        Kernel::raise
      end
    end

Note that it’s the same code as before, except that we do not do eval("string") any more, but instead are using specific, more modern metaprogramming tools provided by standard Ruby.

The result is the following:

$ ruby queue.rb

queue.rb:22
q.enq 1, 2, 3, 4, 5

(rdb:1) step
/usr/lib/ruby/1.8/forwardable2.rb:149
begin

(rdb:1) list =
[144, 153] in /usr/lib/ruby/1.8/forwardable2.rb
   144      accessor = accessor.id2name if accessor.kind_of?(Integer)
   145      method = method.id2name if method.kind_of?(Integer)
   146      ali = ali.id2name if ali.kind_of?(Integer)
   147
   148      self.send(:define_method, ali) do |*args,&block|
=> 149        begin
   150          instance_variable_get(accessor).__send__(method, *args,&block)
   151        rescue Exception
   152          $@.delete_if{|s| /^\\(__FORWARDABLE__\\):/ =~ s} unless Forwardable2::debug
   153          Kernel::raise

(rdb:1) step
/usr/lib/ruby/1.8/forwardable2.rb:150
instance_variable_get(accessor).__send__(method, *args,&block)

Allready much, much better.

Of course, with the string-eval approach to metaprogramming Ruby itself could do better by saving the string that is being evaled to be able to refer to it later at step-through time. However currently we don’t have this option.

Tomáš Pospíšek

QGIS Mobile GSoC has started

2011-05-25T20:00:00Z

Yesterday, we had our first meeting with Marco Bernasocchi, who just started his Google Summer of Code project. The project goals are:

porting QGIS to the Android platform
adapt the QGIS GUI for tablet computers
write a driver for the built-in GPS
create a QGIS “mini” application for mobile phones

Marco Hugentobler is mentoring the project and updated information will be available on a QGIS Wiki page. We wish Marco good luck and are looking forward to a portable QGIS this year!

extending RedCloth markup

2011-04-27T12:09:00Z

There are various approaches when trying to extend the Textile markup that RedCloth understands with own tags or syntax. Some approaches documented on the net have changed or don’t work any more, since RedCloth has been rewritten in Version 4.

Below is a fairly robust aproach, that is based on the assumption, that RedCloth leaves HTML tags inside the markup untouched and passes them on to the application consuming the translated markup.

The below code has been used in the Madek project. Madek, a Ruby On Rails application, uses irwi which provides a Wiki to Madek. And finally irwi uses RedCloth to do the rendering from Textile to HTML. That’s where we hook in.

We want to have a few special tags, which make the life of the Madek Wiki admin simpler, by letting him write the following inside the Textile markup:

[media=210      | Das Huhn]
[screenshot=210 | Das Huhn]
[video=210      | Das Huhn]

That will finally produce this HTML output:

<a href="/media_entries/210">Das Huhn</a>
<img src="/media_entries/210/image" title="Das Huhn"/>
<video src="/media_entries/210/image" title="Das Huhn"/>
  <a href='/media_entries/210'>(see video)</a>
</video>

Here’s the implementation:

class RedClothMadek

  ActionView::Base.sanitized_allowed_tags << 'video'

  def initialize
    require 'redcloth'
  end

  def format( text )
    ::RedCloth.new( replace_madek_tags(text) ).to_html
  end

  # Transforms the follwing Textile markups:
  # 
  #   [media=210      | Das Huhn] -> <a href="/media_entries/210">Das Huhn</a>
  #   [screenshot=210 | Das Huhn] -> <img src="/media_entries/210/image" title="Das Huhn"/>
  #   [video=210      | Das Huhn] -> <video src="/media_entries/210/image" title="Das Huhn"/>
  #                                    <a href='/media_entries/210'>(see video)</a>
  #                                  </video>
  #
  def replace_madek_tags( text )

    # unfortunately having multiple matches in gsub doesn't seem to work, therefore
    # we fall back to $1 $2
    #
    text.gsub(/[\s*media\s*=\s*(\d+)\s*\|\s*([^]]+)\s*]/) { |number,txt|

           "<a href='/media_entries/#{$1}'>#{h($2)}</a>"                 }.

         gsub(/[\s*screenshot\s*=\s*(\d+)\s*\|\s*([^]]+)\s*]/) { |number,title|

           "<img src='/media_entries/#{$1}/image' title='#{h($2)}'/>"    }.

         gsub(/[\s*video\s*=\s*(\d+)\s*\|\s*([^]]+)\s*]/) { |number,title|
           "<video src='/media_entries/#{$1}/image' title='#{h($2)}'>" +
             "<a href='/media_entries/#{$1}'>(see Wideo)</a>" +
           "</video>"  }
  end

end

And finally, irwi needs to be told to use that formatter instead of RedCloth directly. From config/environment.rb:

require "#{Rails.root}/lib/red_cloth_madek.rb"
Irwi.config.formatter = RedClothMadek.new

Tomáš Pospíšek

annotating third party web pages

2011-04-17T09:02:00Z

Problem: I have a web site/page that I visit regularily which I want to annotate with my notes.

More specifically, I was regularly searching through the Homegate real estate hub looking for a new home. It goes without saying that I was again and again forgetting which objects I had already looked at, which objects were really interesting and I should check out more closely.

Therefore the need to annotate search results.

The here presented approach should be applicable for annotation of other web pages as well. It’s based on the observation, that restful web applications need to operate with asset IDs. These asset IDs can be reused to enrich the asset locally with additional data, such as notes. Thus we’re looking for those IDs in specific elements of the page and add a bit of HTML markup to those places.

Of course that aproach only works as long as the web site doesn’t heavily change its markup and doesn’t rendomly change asset IDs.

Here’s a screenshot of regular Homegate search results:

And here’s the same page after scripting it with Greasemonkey:

You’ll notice the input field with the comment in it.

The idea is simple: add an input box to each search result, where you write your comment. When the focus leaves the input box, the comment is stored to localStorage.

Starting with Greasemonkey is quite easy. There are howtos and templates to start from, such as this one.

However, regular JavaScripting and Greasemonkey JavaScripting do not work in exactly the same way:

One of the differences is that Greasemonkey creates a separate JavaScript environment, in which the Greasemonkey scripts are executed. That is calling Javascript contained in the page from Greasemonkey and the inverse calling Greasemonkey scripts from the page is not possible by default. This is on purpose, so that the web page can not detect and not interfere with the Greasemonkey scripts, because you want your Greasemonkey scripts to work allways on some page, whether or not that page likes it or not. Therefore no interference is possible.

Greasemonkey however provides a standard way to access the web page’s scripts, and that’s through the “unsafeWindow” object, which is a reference to the web page’s environment.

I had two mechanisms I had to make accessible using the “unsafeWindow” handle:

the first was accessing JQuery, which is included by default by the Homegate page. Since I needed to use JQuery functions in my Greasemonkey script, I got a reference to it via the standard Greasemonkey precedure:

    var jQuery = unsafeWindow['jQuery'];

the second mechanism that needed to cross the boundaries between the web page and Greasemonkey was callbacks from the web page to my Greasemonkey script. This is necessary, because I’m attaching “input” elements to each search result, which contain a note and which, “onblur”, need to call a function that saves the content of the input box. Here’s part that constructs the input element:

    jQuery("<input onclick='event.cancelBubble = true;'" +
                 " onblur='saveComment(this, immoID);'>").insertAfter(immoElement);

And this is the function that gets called back by “onblur”:

    unsafeWindow.saveComment = function(element, immoID) {
      unsafeWindow.localStorage.setItem(immoID, element.value);
    };

The next interesting thing you’ll note is usage of ‘locaStorage’. Support for the latter in browsers does not seem mature yet. One problem I’ve encountered when developing under Firefox 3.6 was that saving to ‘localStorage’ was not possible when cookies were disabled (see this report). Thus you’ll need to permanently enable cookies for Homegate in order for the script to be able to save its data.

Finally, while developing, a major problem was, that Firefox did not show me errors in the Greasemonkey scripts. Thus either the script would work or not work and fail completely silently. That made debugging a bit painful.

So now, here’s the script.

Tomáš Pospíšek

PS: This script also lives at userscripts.org

QGIS Anwendertag, 6.5., HSR Rapperswil

2011-04-11T14:26:00Z

Am Freitag dem 6. Mai, findet an der Hochschule für Technik Rapperswil das 2. deutschsprachige QGIS Anwendertreffen statt. Quantum GIS (oder kurz QGIS) ist ein benutzerfreundliches Open Source Desktop- und Server-GIS welches sich einer stark wachsenden Anwendergruppe erfreut. Sie finden Infos zu QGIS unter www.qgis.org

Nach dem erfolgreichen ersten deutschsprachigen QGIS Anwendertreffen am 21.4.2010 in Bern findet das zweite deutschsprachige QGIS-Anwendertreffen an der HSR in Rapperswil statt. Alle aktuellen Infos zur Veranstaltung, wie auch zur Anmeldung, finden Sie auf der QGIS Seite.

Der Vormittag besteht aus zwei Blöcken mit Vorträgen und Demos, am Nachmittag werden drei parallele Workshops/Kurse für Anfänger, Fortgeschrittene und Programmierer durchgeführt. Der Vormittag ist für alle Teilnehmer gratis, die Kurse am Nachmittag kosten CHF 120.-

Wir würden uns freuen, möglichst viele QGIS-Anwender und Interessierte begrüssen zu dürfen.

Visually sorting images under Linux

2011-04-10T13:51:00Z

to visually sort images under Linux doesn’t seem to be a trivial task. I duckduckgo‘ed for a long time and had a look at various image and file managing applications before finding gthumb. And even there, you first need to create a “catalog” and within the catalog a “library” which will finally allow you to manually sort your images. All of which is not documented.

Once you’ve sorted your images, you’d possibly want to export the sorting? Again, no trace of any help or documentation: gthumb catalogs are saved under $HOME/.local/share/gthumb/catalogs/foobar.catalog.

Tomáš Pospíšek

FOSSGIS 2011: Vergleich von Open Source Virtual Globes

2011-04-08T09:45:00Z

Mittlerweile gibt es mehrere Open Source Alternativen zu Google Earth, wie z.B. Nasa World Wind Java SDK, ossimPlanet oder osgEarth. Dieser Vortrag soll eine Übersicht aktueller Open Source Virtual Globes bieten und einige Vorschläge zu erwünschten Funktionen liefern.

Präsentation

Demos:

NASA World Wind Java SDK

FOSSGIS 2011: Webgis mit QGIS und GeoExt

2011-04-07T08:24:00Z

Neues vom QGIS Server

Der Vortrag zeigt die Fortschritte von QGIS Server auf, illustriert auch anhand von Beispielen aus der Praxis. QGIS Server kann neu .qgs Projektfiles aus dem Desktop-GIS-Bereich als zentrale Konfigurationsdatei von Webdiensten (derzeit nur WMS) verwenden. Das bietet verschiedene Vorteile:

Desktop-GIS, Web-GIS und WMS aus der gleichen Konfigurationsdatei
kein manuelles Editieren von .map-Files - auch Mapserver-Laien können nun sehr einfach Webdienste erstellen
Übernahme von Druck-Layouts für das Drucken im Web (derzeit in Arbeit)
Verwendung der neuen Label-Engine und der erweiterten Symbolisierungsmöglichkeiten aus der neuen Symbology-Engine

Die Client-Komponente wurde mit GeoExt (OpenLayers, ExtJS,GeoExt) realisiert. Es werden soviele Konfigurationen wie möglich aus dem Desktop-GIS-Bereich in den Web-GIS-Bereich übernommen, wie z.b. der Projektname, die Attributdatenbehandlung (Spaltenunterdrückung, Spaltenaliases, etc.), das Selektierbarmachen von Layern, sowie das Verwenden bereits vordefinierter Drucklayouts.

Wo möglich wurde der bestehende WMS Standard verwendet, wo nötig, wurden Erweiterungen im WMS-Protokoll vorgenommen, vorwiegend beim Drucken, aber auch bei den getFeatureInfo und getCapabilities Requests.

Der Vortrag zeigt produktive Beispiele aus dem GIS der Stadt Uster und zeigt welche Arbeitsschritte auf dem Server und dem Client notwendig sind, um ein QGIS-Projekt ins Web zu bringen.

Präsentation

FOSSGIS 2011: Neue Entwicklungen und Features vom Quantum GIS Projekt

2011-04-07T08:07:00Z

Das QGIS Projekt hat zum Ziel, ein plattformunabhängiges, einfach zu bedienendes und offenes Desktop GIS zu entwickeln. Im vergangenen Jahr hat die Aktivität in der QGIS-Community erneut zugenommen. Seit der letzten FOSSGIS Konferenz in Osnabrück hat sich QGIS wieder enorm weiterentwickelt. Sucht man sich (aus der Liste der svn commits) alle Neuerungen des letzten Jahres heraus, so kommt man auf eine Liste, die den Rahmen dieses Artikels bei weitem sprengen würde. Daher stellen wir im Rahmen des Vortrags lediglich eine kleine Auswahl der Entwicklungen der aktuellen und der kommenden Versionen vor.

Präsentation