Sunday, April 24, 2011


How about we install and configure a version control system while we are at it. Let's install favorite.
$ sudo apt-get install subversion libapache2-svn
Make the directory that we are going to create the repository in, then create the repository, and finally set the ownership to the user that Apache runs as...
$ sudo mkdir /home/<user>/svn
$ svnadmin create /home/<user>/svn
$ sudo chown -R www-data /home/<user>/svn
Now let's create the password file for accessing the repository and one account. It will ask you to enter a password for the new account twice...
$ sudo htpasswd -c /etc/apache2/dav_svn.passwd <user>
Then let's create an authorization file for giving fine grain read/write permissions to particular directories and files in the repository itself. I am just going to give read/write permissions to the entire repository structure...
$ sudo nano /etc/apache2/dav_svn.authz
We need to configure it all together now so you can do this in any place you like but the easiest is in the file below or virtual host file...
$ sudo nano /etc/apache2/mods-enabled/dav_svn.conf
Add or uncomment this text...
<Location /svn>
  DAV svn
  SVNPath /home/<user>/svn
  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile  /etc/apache2/dav_svn.passwd
  AuthzSVNAccessFile /etc/apache2/dav_svn.authz
  Require valid-user
Let's grab our old repository from another instance of subversion and bring it over...
$ svnadmin dump /path/to/repository > repo_name.svn_dump
Then lets suck it up into our new repository on this new server...
$ svnadmin load /home/<user>/svn < repo_name.svn_dump
How about adding SSL to make it all secure. Load this module so we can use SSL...
$ sudo a2enmod ssl
$ sudo a2ensite default-ssl
You may also need to install a couple of additional mods to make it all work depending on your linux distribution or version
sudo a2enmod authz_svn
sudo a2enmod dav_svn
Now you should be able to open you subversion repository using https://localhost/svn and it should work. Try using curl or something like it to test it out locally on your headless server like...
$ sudo apt-get install curl
$ curl -k https://username:password@localhost/svn

Thursday, April 21, 2011

Now What?

Well I think we have all the software installed that we need to build some pretty nice GIS web applications!!! So, I thought this would be a good time to stop and think about what we have done and all the possibilities from here. In following blog posts, I will talk about ways to develop  custom software solutions using this server setup.

This set of technologies is really quite similar to working with the Microsoft / ESRI platform that I use professionally. The difference is you are not paying for IIS / ASP.Net for the web server, Sql Server for the database, and the ESRI stack of GIS technologies like SDE and ArcGIS Server. With this completely free solution we have a spatially enabled enterprise database with PostgreSQL and PostGIS and an incredibly fast mapping server in Geoserver. With a bit of development you could do anything with this setup.

One of the things that I am really amazed by is how many great technologies we have installed on this server, yet the footprint is still very very small. Here are the specs for this server...

Size of VirtualBox image on disk: 1.9 Gigabytes
RAM used at rest: 382 Megabytes

A server build with the Microsoft / ESRI software would probably be 20 GigaBytes in size and use at least a GigaByte of RAM at rest. This server can run very well on older and probably free hardware. If you are hosting this server it will be cheaper too because Linux servers are cheaper than Microsoft servers. It would also take many days of CPU type to install all of the software needed to build this type of solution using Microsoft and ESRI. But, for this linux server I could easily script everything we have done and run it in less than an hour to rebuild this server!!!! I think that is pretty remarkable.

Below this paragraph is a link to the Geoserver main page running on this linux server that we have just built. This is the administrative interface for Geoserver and without logging in you can still play around with some of the functionality. The "Demos" and "Layer Preview" links allow you to view and query the data on the server. The "Demo Requests" link lets you formulate and send Open Geospatial Consortium requests to the server like WMS, WFS, and WCS requests, so give it a try. Don't bother trying to hack this server because it gets reverted to a previous snapshot of the OS every so often, which will remove any hacks. :)

Here are a few more links of projects using this server. These are just little pet projects that I have built using this system. In fact these applications are running on the same headless linux server that I have been building while writing this blog.

The Appalachian Trail Hiker Map site is an Mono/C# web application that uses the Google Maps API in combination with Geoserver. Geoserver is used to deliver the map tiles for all of the trail, towns, parking lots, shelters, etc. All of the actual data that make up the tiles reside in PostgreSQL/PostGIS, of course. This is an example of a completely asynchronous application because it never does a full post back to the server. All requests are AJAX oriented. It also does asynchronous calls to geoserver to get information about features on the map using the WMS GetFeature request. This is simple stuff, but illustrates some basic usage of this platform for viewing cached tiles and retrieving feature attributes.

The Search OGC website doesn't really do much mapping, but it again is an Mono/C# application running on Ubuntu Linux. It uses an open source GIS component called SharpMap to gather and store information from OGC mapping servers around the world and indexes them for searching. I use OpenLayers to make a simple map of any data that can easily be displayed on a map (this needs a lot of work and doesn't work for all sites yet, but take a look anyway).

Anyway, these are pretty simple mapping applications that don't do any really complex GIS work, but it is a place to start. In the next blog entries I hope to delve deeper into these technologies and explain how to use them to do things that we may take for granted using Microsoft / ESRI technologies; like doing spatial queries, geoprocessing, map overlay, geocoding, etc.

Anyway, enough for now...

Wednesday, April 13, 2011

Geoserver Daemon

Now I am going to setup my geoserver installation to run on system startup as a daemon. This is quite nice so you don't have to manually run the geoserver process every time you log into your server. This is quite easy to do also, but you have to get a few things right or it will not work and it can be difficult to understand why.

First go ahead and download a little script file from the Geoserver website. You can use the handy little wget utility to download the file straight from the web to your choice of directory...
$ cd /full/path/to/geoserver/
$ sudo wget -O
$ sudo ln -s /full/path/to/geoserver/ /etc/init.d/geoserver
$ sudo chmod +x
$ sudo flip -ub
Notice that we named it and then created a link file to it at /etc/init.d/geoserver. You could just copy the script directly to this path, renaming the file as geoserver, but typically the script is kept with the application binaries and a link is used to reference it from this location. We also run flip on this file to make sure Windows hasn't messed up the carriage returns and line feeds.

Now we need to modify some of the settings in this file so open up the /etc/init.d/geoserver file using your favorite text editor. Below is the entire script file with the important lines being 16, 17, 22, 23. These are the settings that I had to change to fit my server configuration. Notice that by default this file is setup to run if the user account you are working under is named "geoserver". Also, notice that I added the " -server" string to the end of the JAVA_OPTS variable on line 23. Honestly, I have no idea what it does, but many resources I read said to do that :) Thank you very much to Lennart for writing this script!

#! /bin/sh
# Provides:          geoserver
# Required-Start:    $local_fs $remote_fs
# Required-Stop:     $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      S 0 1 6
# Short-Description: GeoServer OGC server

# Author: Lennart Jütte <>

# Geoserver configuration - use /etc/default/geoserver to override these vars
# user that shall run GeoServer

DESC="GeoServer daemon"
JAVA_OPTS="-Xms128m -Xmx512m -server"

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME


# Load the VERBOSE setting and other rcS variables
[ -f /etc/default/rcS ] && . /etc/default/rcS

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions


 # Return
 #   0 if daemon has been started
 #   1 if daemon was already running
 #   2 if daemon could not be started

 start-stop-daemon --start --pidfile $PIDFILE --make-pidfile \
  --chuid $USER --chdir $GEOSERVER_HOME \
  -b --test --exec $DAEMON -- $DAEMON_ARGS > /dev/null \
  || return 1

 start-stop-daemon --start --pidfile $PIDFILE --make-pidfile \
  --chuid $USER --chdir $GEOSERVER_HOME \
  -b --exec $DAEMON -- $DAEMON_ARGS \
  || return 2


 # Return
 #   0 if daemon has been stopped
 #   1 if daemon was already stopped
 #   2 if daemon could not be stopped
 #   other if a failure occurred

 start-stop-daemon --stop --pidfile  $PIDFILE \
  --user $USER \

 [ "$RETVAL" = 2 ] && return 2

 # Many daemons don't delete their pidfiles when they exit.
 rm -f $PIDFILE
 return "$RETVAL"


case "$1" in
 [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 log_daemon_msg "Restarting $DESC" "$NAME"
 case "$?" in
  case "$?" in
   0) log_end_msg 0 ;;
   1) log_end_msg 1 ;; # Old process is still running
   *) log_end_msg 1 ;; # Failed to start
    # Failed to stop
  log_end_msg 1
 #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
 echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
 exit 3



Again make sure to flip this file to make it readable on linux and double or triple check your paths and other options.

Next you will need to set the geoserver to launch on startup. I don't understand the details of this technology since I have relatively new to linux, but it is something akin to windows services in the Microsoft world. Regardless, just run the following command and it will setup Geoserver to run as a service when the system starts up...
$ sudo update-rc.d geoserver defaults
You can put a number at the end to set the start order if you have a problem. Adding it to the end is probably a good idea like so...
$ sudo update-rc.d geoserver defaults 99
After doing this you should be able to start Geoserver by starting the daemon...
$ sudo /etc/init.d/geoserver start
Also, you can remove this service using the following command...
$ sudo update-rc.d -f geoserver remove
Make sure that this command them returns something that looks like a geoserver process running as a java application...
$ ps aux | grep geoserver
Now try logging into the administrative website again using a browser pointed to...
If not it could be a million things that went wrong. Check your paths, permission, user, etc. and try again. It took me about three attempts before I stopped making stupid mistakes.

Of course this is not a secure installation so again we need to read the following resources to tighten it all down.

Sunday, April 10, 2011


So we now have a pretty decent little web application platform. We can run ASP.Net / C# web applications from this server using the PostgreSQL as our database backend. There are a couple of really good content management platforms that will run well on this server now...Chayahogo and MojoPortal. I have run Chayahogo and it works well. I hae never setup MojoPortal, but it seems to be the most robust solution for these technologies.

Anyway, we want to get to some GIS so lets move on to installing and running Geoserver. Perhaps I will come back to MojoPortal down the road. So, first we need to download and unpack Geoserver. You can download it from geoserver. I am going to install version 2.0.3, which is the latest stable build available. Also, I am going to install the binary (OS independent) install. I don't have a java servlet engine like Tomcat installed and want to keep this easy anyway.

One last thing...this is not a secure way to install Geoserver. I am just wanting to install it in as few steps possible to illustrate how to do it. We will discuss security later. Actually, you can take a look at this article if you want to get the details on security right away...production install.

So, let's download, unzip, and FTP the unzipped folder to some place on your server's file system. 

Install the Sun Java JRE (sun-java6-jre) (Used to need JDK and native JAI and JAI Image I/O extensions) Use the Sun Java JRE not the OpenJRE because it is much faster.

You have to add a source first to the end of the file /etc/apt/sources.list...

deb lucid partner

This makes it such that we can install from apt-get without downloading it manually from Oracle, etc. Then run the following lines from the terminal in order to install the Java JRE...
$ sudo apt-get update
$ sudo apt-get install sun-java6-jre #(may not need sun-java6-fonts)
$ sudo update-alternatives --config java
Test it out to make sure you have successfully installed the JRE...
$ java -version
Now navigate to the folder that you installed Geoserver to and go into the bin directory. You will see a number of script files for both windows and linux. The file is the one you want to use to start geoserver, but there are a few problems. The file was modified on a windows machine so it has an issue with carriage returns and line feeds. We need to fix this, so install a little program called flip to resolve this issue and run it like so...
$ sudo apt-get install flip
$ sudo flip -u
$ sudo flip -u
If you try to run it now, it will say that the JAVA_HOME variable is not set. That is true we never set it, so we need to do that. I tend to set these variable in the and files themselves so the application is completely self contained so let's do that. Open both files and add the following lines to the top of the file just after all the bash script comments...
If you added these lines to the file on a windows machine, make sure to flip them again like we did above. In fact I suspect that this version was touched by a windows text editor so you need to flip it. But, now it should run using the following command. Notice that I put an ampersand after the command in order to tell the shell to run the process in the background. This way you can continue to use the terminal for other work...
$ sudo sh &
and shut it down with...Ctrl-C

Again, this is not a secure installation at all. Everyone in the world knows your admin password until you change it. So, login to the geoserver administration page at http://localhost:8080/geoserver and change the admin password. The default for Geoserver is admin/geoserver for username/password, so change it first thing. Also, we did not set permissions at all on the file system. You may need to set some permissions if you get an error trying to run it, and then change ownership and permissions to secure it. At the very least set the permissions on the "gwc" and "logs" folders in the data_dir directory such that Geoserver has read and write to them. The performance will be terrible if you do not because the tile server will not cache any files and the logging will not work.

There are a number of ways to run geoserver and this was just the easiest way for me to get it setup. You can run geoserver as a Daemon so it is available on system startup or you can run the war file version from within a Java Servlet container. Either way it should behave exactly the same.

Tuesday, April 5, 2011


Now we need to install PostGIS to add geographic objects to PostgreSQL. PostGIS is pretty much equivalent to SDE in the ESRI world but follows the OpenGIS "Simple Features Specification for SQL". It is really nice that it subscribes to an open standard like this so it can interoperate with so many other systems. We will eventually install Geoserver and configure it to connect to PostgreSQL/PostGIS to get data and utilize funstionality. You can do quite a bit of serious web mapping and GIS with this combination of technologies.

Anyway, lets get PostGIS installed and configured here now...
$ sudo apt-get install postgis postgresql-8.4-postgis
Now we need to create a new database, add support for the PL/pgSQL database language to that database, and then run 3 scripts that add all the geographic database objects to the database.
$ sudo -u postgres createdb my_db
$ sudo -u postgres createlang plpgsql my_db
$ sudo cd /usr/share/postgresql/8.4/contrib/
$ sudo -u postgres psql -d my_db -f postgis.sql
$ sudo -u postgres psql -d my_db -f spatial_ref_sys.sql
$ sudo -u postgres psql -d my_db -f postgis_comments.sql
Notice that I am giving the postgres user superuser rights to run these commands. By default you have to run the psql command as postgres since that is the only user with a database account, but we need to give the OS account permission to do these things as well. This is a bit annoying but it is better than spending a bunch of time changing permissions and settings on the postgres account. That would be less secure as well.

Here are a few other useful programs that may come in handy. These are all programs not commands executed inside psql.

Delete database...
sudo -u postgres createdb dropdb <database>

Sunday, April 3, 2011


Now lets get the database installed. PostgreSQL is a fantastic database and another company has built an "extension" for it called PostGIS that gives it an amazing amount of GIS functionality. This has become the standard database platform for the open source GIS community. Let's install, configure, and discuss PostgreSQL in this post. I'll add another blog entry regarding PostGIS shortly.

Install the database...(or postgresql-8.4 for version specific)
$ sudo apt-get install postgresql 
Configuration files are located at /etc/postgresql/8.4/main...
environment (variables used by postmaster)
pg_ctl.conf (database cluster specific options)
pg_hba.conf (controls access to the database)
pg_ident.conf (username mapping?)
postgresql.conf (general configuration file)
start.conf (assume it has something to do with starting up :) )
First let's enable TCP/IP connections since they are disabled by default by adding the following lines to /etc/postgresql/8.4/main/postgresql.conf...
#For local connection only
listen_addresses = 'localhost'
#For remote connections
#listen_addresses = '*'
Now we need to set a password for the PostgreSQL database user that is automatically created called postgres. There is also an operating system user account named postgres that is added specifically for running the PostgreSQL daemon. By default this OS user is actually set to "login disabled". So, the user is really used only for running PostgreSQL. Anyway let's go ahead and set the password for this database user account now...
$ sudo -u postgres psql template1
# ALTER USER postgres with encrypted password '<password>';
# \q
After configuring the password, edit the file /etc/postgresql/8.4/main/pg_hba.conf to use MD5 authentication with the postgres user. This will allow localhost access only and requires an encrypted password...
#For local connections only
local         all       postgres        md5
#For remote connections from a certain ip address
host          all       postgres       xxx.x.x.x/32        md5
Restart PostgreSQL using this command...
$ sudo /etc/init.d/postgresql-8.4 restart
Now let's connect to a database. Since the postgres OS user is not setup to allow login, you have to connect to the database as that user with sudo as you see below.
$ sudo -u postgres psql postgres
Frankly I think it is pretty confusing and takes a little while to get the hang of navigating PostgreSQL using the psql command, but it works great after a bit of a learning curve.

Now you should see the postgresql command prompt where you can perform SQL and interact with the database. Type \? or \help to get some information on how to work with this program. Most importantly, \q gets you back to the shell prompt...

I'll add some common PostgreSQL tasks cheatsheet below a bit later since the database should be running fine now.


Saturday, April 2, 2011


Now we have the Apache web server running on our server, but we have not installed any of the packages we need to run ASP.Net applications. We could install run-times that would allow us to build web applications in PHP, Perl, or Python, but I am going to install Mono so we can use C# / .Net. This is the language I am most familiar with so it is nice choice for me. Unfortunately, installing and configuring Mono is not quite as easy and straight forward as the PHP runtime. I may write a blog entry for PHP as well since so many of the existing open source GIS technologies favor it.

Make sure to stop Apache using the command below before installing Mono because it can cause problems otherwise.
$ sudo /etc/init.d/apache2 stop
Now run the following commands to install Mono for a production environment.
$ sudo apt-get install mono-apache-server
$ sudo apt-get install mono-apache-server2
$ sudo apt-get install libapache2-mod-mono
I think you could run the following as well, but it installs a bunch of additional packages that you may not need...
$ sudo apt-get install mono-complete
Now we need to configure Mono. This was very confusing to me because I couldn't find really good documentation anywhere, so this section is really a work in progress. This works but I can't say I understand all the options fully.

You can use the a2enmod and a2dismod commands to enable and disable virtual hosts in Apache. These commands create link files in the sites-enabled folder and Apache parses the contents when it is reloaded. 
$ sudo a2dismod mod_mono
$ sudo a2enmod mod_mono_auto
Enabling mod_mono_auto seems to setup the parameters that you would want for all ASP.Net sites that you might host on this server. I think this is called autohosting and basically the remaining configuration options are set in the virtual host file for the specific website so you don't have to make configuration changes to Apache in order to control how Mono works in your web applications. This is nice but requires a few more options to be understood and set correctly. You can see that even the version of the .Net framework is set within the virtual host file.

Now to get a test Mono web application running let's create a new virtual host file, enable it , and have it "point" to a simple web application. You can go to the mono website and use their little wizard to create your virtual host file. This is probably the best way to do it...
I don't understand all of the Mono options for the virtual host file yet but many of them are sort of intuitive. Below is a sample virtual host file that I generated from the website...
<VirtualHost *:80>
  ServerAlias *
  DocumentRoot /home/atv/web/mydomain/www

  # MonoServerPath can be changed to specify which version of ASP.NET is hosted
  # mod-mono-server1 = ASP.NET 1.1 / mod-mono-server2 = ASP.NET 2.0
  # For SUSE Linux Enterprise Mono Extension, uncomment the line below:
  # MonoServerPath "/opt/novell/mono/bin/mod-mono-server2"
  # For Mono on openSUSE, uncomment the line below instead:
  MonoPath "/usr/lib/mono/3.5"
  MonoServerPath "/usr/bin/mod-mono-server2"

  # To obtain line numbers in stack traces you need to do two things: 
  # 1) Enable Debug code generation in your page by using the Debug="true" 
  #    page directive, or by setting <compilation debug="true" /> in the 
  #    application's Web.config
  # 2) Uncomment the MonoDebug true directive below to enable mod_mono debugging
  # MonoDebug true

  # The MONO_IOMAP environment variable can be configured to provide platform abstraction
  # for file access in Linux.  Valid values for MONO_IOMAP are:
  #    case
  #    drive
  #    all
  # Uncomment the line below to alter file access behavior for the configured application
  MonoSetEnv MONO_IOMAP=all

  # Additional environtment variables can be set for this server instance using 
  # the MonoSetEnv directive.  MonoSetEnv takes a string of 'name=value' pairs 
  # separated by semicolons.  For instance, to enable platform abstraction *and* 
  # use Mono's old regular expression interpreter (which is slower, but has a
  # shorter setup time), uncomment the line below instead:
  # MonoSetEnv MONO_IOMAP=all;MONO_OLD_RX=1
  MonoApplications "/:/home/atv/web/mydomain/www"

  <Location "/">
    Allow from all
    Order allow,deny
    SetHandler mono
    SetOutputFilter DEFLATE
    SetEnvIfNoCase Request_URI "\.(?:gif|jpe?g|png)$" no-gzip dont-vary
  <IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript
I added lines 13 and 14 in order to make it run using the 3.5 framework, but now you can run the command below to enable the virtual hostsite...
$ sudo a2ensite <myvirtualhost>
Of course the actual ASP.Net web application needs to be in place and located at the path /home/<user>/www as you can see above in the virtual host file. Create a simple test file in this directory called default.aspx and fill it with some really simple code just to make sure all is working. Here is a little snippet to copy and past...
<%@ Page Language="VB" %>

<script runat="server"> 

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) 
 lblVersion.Text = "Your server is running ASP.NET and the version is " & System.Environment.Version.ToString() 
End Sub


<title>ASP.NET Version</title>
<form id="form1" runat="server"> 
<asp:Label ID="lblVersion" runat="server">
This just writes out some text to a Label webcontrol, but if you see the text, then Mono / ASP.Net is working. If not, you done screwed it up.

Oh, don't forget to restart the Apache webserver firstand see if it will even load using...
$ sudo /etc/init.d/apache2 restart
If you see an error that looks like this you can ignore it. It is an unfortunate error but doesn't actually cause any problems. I think it is fixed in a future release of Apache2 or Mono...not sure and don't really care much since it still works fine.

Now you can pull up a browser and navigate to the URL of the website you setup and it should work.

Friday, April 1, 2011


Now we are starting to get somewhere. Let's install the webserver now so we can host some GIS web-sites  very soon.

Install Apache web server from a terminal...
$ sudo apt-get install apache2
Here is a list of the important paths and files that you may need to edit at some point...
/etc/apache2/ (server root)
/etc/apache2/mods-available (apache modules you can load)
/etc/apache2/mods-enabled (apache modules loaded)
/etc/apache2/sites-available (virtual host files you can load)
/etc/apache2/sites-enabled (virtual host files loaded)
/etc/apache2/apache2.conf (main configuration file)
/etc/apache2/error.log (error log file)
/etc/apache2/port.config (ports file)

You may want to add the server name to the main configuration file at /etc/apache2/apache2.conf
ServerName <Name>

In the /etc/apache2/sites-available folder you will find a default virtual host file. You can enable this website by running the command below. This command simply creates a link file in the /etc/apache2/sites-enabled folder. Restarting Apache will then load this website, making it reachable.
$ sudo a2ensite default
You can disable this virtual host website using this command...
$ sudo a2dissite default
These virtual host files simply contain all the website configuration information needed to make the website live. You can see in the screenshot below that the default virtual host file is pointing to a document root of /var/www. By default there is an index.html file in that path, so once you apply permissions to this folder and restart apache, it should work.

So, let's assign permissions to the document root folder now. This is not the most secure way to do things, but it is good enough for now. This gives all read and execute permissions recursively on all files and folders withing /var/www.
$ sudo chmod -R a=rx /var/www/
Now restart the apache web server so all these changes are loaded...
$ sudo /etc/init.d/apache2 restart
Now go the the URL that points to your web server and you should see...