Wednesday, December 7, 2011

Cheat Sheet

Below is my linux terminal cheat sheet that I use often. I decided to post it mostly so I could easily find it myself. I seem to reference my own blog quite often to remember how I did something in the past. These are all just examples not command descriptions. I hope it is helpful...

Users, Groups, & Permissions
  • Create User Account
    • sudo adduser <username> # program that does everything for you
  • Modify User Account
    • sudo usermod -a -G <group> <username> #append user to a group
  • Delete User Account
    • ?
  • Change Permissions
    • chmod 755 <myfile>
    • chmod -R 755 <mydir> #applied permissions recursively on directory
  • Change Ownership
    • chown root:root /my/dir/path/
Working w/ Files
  • Create Empty Text File
    • >> <filename>
    • touch <filename>
  • Move Files
    • mv <oldfile> <newfile>
  • Rename Files
    • mv <oldfile> <newfile>
  • Delete File
    • rm <filename>
  • Delete Recursively w/ Wildcard
    • find . -type f -name '*.htm' -delete
  • Copy Files
    • cp <existing> <new>
  • Change line terminators
    • flip -ub <textfile>
  • Search results of standard output
    • grep <searchstring>
Working w/ Directories
  • Create Empty Directory
    • mkdir
  • Move or Rename
    • mv
  • Delete Directory Recursively
    • rm -R <folder> <newfolder>
  • Copy Recursively
    • cp -r <dir1> <dir2>
    • scp ...
    • rsync
Misc. Tools
  • Search Results of Standard Output:
    • grep <searchstring>
System Resources
  • View Memory Usage
    • free
  • Process Monitor
    • top
    • htop
  • Processes
    • ps -A
  • System Activity
    • vmstat
  • User Monitor
    • who
    • whoami
  • View Device Ids (UUID)
    • blkid
Package Management

  • Install
    • apt-get install <packagename>
  • Remove
    • apt-get remove <packagename>
  • Remove completely
    • apt-get remove --purge <packagename>
  • List all installed package
    • dpkg -l
  • See if a package is installed
    • dpkg -l | grep <packagename>
  • Find sources here:
    • sudo nano /etc/apt/sources.list
  • Find computers on network:
    • arp -av
  • Schedule tasks:
    • crontab -e
Tar and Zip Files & Directories
  • Tar Files
    • tar -cvpf <file.tar> <files or folder>
  • Tar Files and Zip
    • tar -zcvpf <file.tar.gz> <files or folder>
  • View  Tar
    • tar -ztvf <file.tar>
  • Untar Files
    • tar -xvpf <file.tar> <files or folder>
  • Untar Files w/ Zip
    • tar -xvpzf <file.tar.gz>


Thursday, August 11, 2011


So I setup a Samba server running on an Ubuntu headless machine. From the terminal go ahead and install the server portion. You could also install smbclient if you need to.
sudo apt-get install samba
Edit the /etc/samba/smb.conf file in your favorite text editor. Make sure the following two lines are not commented out and have the right values for your network.
 workgroup = MyNetwork
 security = user
Now your linux machine will be visible on the windows workgroup you specified and logging on will be secured by a unix account.

You will need to create linux user accounts on the machine for any user you want to use a samba share under this configuration. Also, you will need to add the users to the samba password database. I assume you can figure out how to create user account and add them to the appropriate groups, but this is how you add and enable those users to the samba password file...
sudo smbpasswd -a username
sudo smbpasswd -e username
This allows you to have a lot of user accounts on the computer but control access using samba instead of the OS only.

Now Create a new share at the bottom of the file, or uncomment any of the examples. Here is an example folder share from the Ubuntu server manual...
    comment = Ubuntu File Server Share
    path = /srv/samba/share
    browsable = yes
    guest ok = yes
    read only = no
    create mask = 0755
comment: a short description of the share. Adjust to fit your needs.

path: the path to the directory to share.

This example uses /srv/samba/sharename because, according to the Filesystem Hierarchy Standard (FHS), /srv is where site-specific data should be served. Technically Samba shares can be placed anywhere on the filesystem as long as the permissions are correct, but adhering to standards is recommended.

browsable: enables Windows clients to browse the shared directory using Windows Explorer.

guest ok: allows clients to connect to the share without supplying a password.

read only: determines if the share is read only or if write privileges are granted. Write privileges are allowed only when the value is no, as is seen in this example. If the value is yes, then access to the share is read only.

create mask: determines the permissions new files will have when created.

Make sure to create the directory that corresponds to this share and set the proper ownership and permissions...
sudo mkdir -p /srv/samba/share
sudo chown /srv/samba/share/
I wanted a couple of different types of shares, so I create the following shares  for my little file server...
  comment = "Shared by all users on this system that are part of the sambashare group"
  path = /path/to/share
  browsable = yes
  guest ok = no
  read only = no
  create mask = 0770
  directory mask = 0770
  force group = @sambashare
  valid users = @sambashare

  comment = "Only accessable by the user named Rick"
  path = /path/to/Rick
  browsable = yes
  guest ok = no
  read only = no
  create mask = 0770
  directory mask = 0770
  force user = rick
  force group = rick
  valid users = rick
This is a nice way to have a shared directory that all valid users on the system can access and a private share for each user on the system. I have no idea if this is a standard way of accomplishing this, but it works really well for us.

The "create mask" and "directory mask" options are used to define the default permissions applied to any new files or directories that the user creates. Likewise, the "force user" and "force group" options determine the ownership of the files and directories. Notice for the first share I left out the "force user" option and set the "force group" option to @sambashare. This is nice because the ownership of any new files or directories are owned by the user that created them.

You can restart the two samba services like so to get started...
sudo restart smbd
sudo restart nmbd
Also, the following command is useful to parse the configuration file and see if there were any problems are suggestions...
testparm /etc/samba/smb.conf

Monday, August 1, 2011

Raid 1

Wow!! I was doing some research into setting up a linux file and print server and came across the concept of software Raid in linux. Are you kidding me!! Raid 0, 1, 5, etc. can all be done at the linux kernal level without the need of any special Raid hardware. The linux community is amazing! I love it.

So, I have an older desktop I wanted to use as a file and print server for the household and decided to set it up with a simple 2 disk Raid 1. This means the 2 hard drives are mirrored. You can have one hard drive completely fail and lose no data at all. This is perfect for a little light duty file server.  This is a bit slower than a one drive system, but that is not important because I am mostly concerned that I don't lose any data from the past 20 years.

I started by connecting 2 identical 300 gigabyte IDE hard drives that I already had in a ~8 year old computer. An old computer should work fine for this sort of need to spend any money at all.

You can follow the Ubuntu website documentation for setting up the Raid at installation time like I did at...

I setup identical boot and swap partitions on these two identical hard drives, and then created two Raid devices to mirror the respective partitions (md0 & md1). It is quite amazing that you can do all of this at installation time and the drives just start to sink up seamlessly in the background.

Drive 1:
sda1 - /boot
sda5 - /swap

Drive 2:
sdb1 - /boot
sdb5 - /swap

Once you have the Software raid setup, you can use the mdadm software to manage it.

cat /proc/mdstat

sudo mdadm --query --detail /dev/md0


Friday, July 1, 2011

GIS Desktop

This blog started out being mostly about building a Linux GIS server, but I am expanding it now to building a good GIS desktop as well. Below I have an outline of what I hope to cover. As the blog posts are written , I will come back here and add links to those posts. So, these links will either be to my blog posts are external websites.

Great Desktop GIS systems (that run on linux)
Desktop GIS on Ubuntu linux...
Virtual Desktops /w GIS desktop software
Desktop GIS / C# / Mono Software development topics...
Related Development technologies...

Wednesday, June 1, 2011

VirtualBox Daemon

This is a quick post about running VirtualBox guest images as daemons from a headless Ubuntu server host. Most of this can be found in Brendan Kidwell's post located at... I am mostly re-posting it here for myself and others in case his post is ever removed.

First, take a look at the VirtualBox service manager script that is installed with VBox. It is located at /etc/default/virtualbox. This script is run when the computer is shutdown, so it is a good place to tell the computer what to do about an any running virtualbox guest VMs. I modified or added the following lines so it would look for guest images running at my user and save their state, as apposed to just shutting them down.

SHUTDOWN_USERS="user1 user2" # space-delimited list of users who might have runnings vms
SHUTDOWN=savestate # if any are found, suspend them to disk. Could be shutoff too

Next, you need to create a new service manager script to run when the computer starts up. This is a standard startup script in most distributions of linux and usually placed in the /etc/init.d/ directory. You can use the script from the bottom of this post to startup one VirtualBox guest VM, so name it with the guest name and copy it like so...
sudo cp virtualbox-machinename /etc/init.d/virtualbox-machinename

#! /bin/sh
# Provides:          virtualbox-!!SHORTNAME
# Required-Start:    $local_fs $remote_fs vboxdrv vboxnet
# Required-Stop:     $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: !!LONGNAME virtual machine
# Description:       !!LONGNAME virtual machine hosted by VirtualBox
# Author: Brendan Kidwell 
# Based on /etc/init.d/skeleton from Ubuntu 8.04. Updated for Ubuntu 9.10.
# If you are using Ubuntu <9.10, you might need to change "Default-Stop"
# above to "S 0 1 6".
# Do NOT "set -e"
# PATH should only include /usr/* if it runs after the script
DESC="!!LONGNAME virtual machine"
VM_NAME="!!LONGNAME" #This has to be the name exactly as it appears in your VirtualBox GUI control panel.
# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
# Load the VERBOSE setting and other rcS variables
[ -f /etc/default/rcS ] && . /etc/default/rcS
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
# Function that starts the daemon/service
 # Return
 #   0 if daemon has been started
 #   1 if daemon was already running
 #   2 if daemon could not be started
 sudo -H -u $VM_OWNER $MANAGE_CMD showvminfo "$VM_NAME"|grep "^State:\s*running" >/dev/null && {
     echo "$VM_NAME" is already running.
     return 1
 sudo -H -u $VM_OWNER $MANAGE_CMD startvm "$VM_NAME" -type vrdp >/dev/null || {
     echo Failed to start "$VM_NAME".
     return 2
 echo "$VM_NAME" started or resumed.
 return 0
# Function that stops the daemon/service
 # Return
 #   0 if daemon has been stopped
 #   1 if daemon was already stopped
 #   2 if daemon could not be stopped
 #   other if a failure occurred
 sudo -H -u $VM_OWNER $MANAGE_CMD showvminfo "$VM_NAME"|grep "^State:\s*running" >/dev/null || {
     echo "$VM_NAME" is already stopped.
     return 1
 sudo -H -u $VM_OWNER $MANAGE_CMD controlvm "$VM_NAME" savestate || {
     echo Failed to stop "$VM_NAME".
     return 2
 echo "$VM_NAME" suspended.
 return 0
# Display "State" field from showinfo action
 sudo -H -u $VM_OWNER $MANAGE_CMD showvminfo "$VM_NAME"|grep "^State:\s*.*$"
case "$1" in
 [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 # If the "reload" option is implemented then remove the
 # 'force-reload' alias
 log_daemon_msg "Restarting $DESC" "$NAME"
 case "$?" in
  case "$?" in
   0) log_end_msg 0 ;;
   1) log_end_msg 1 ;; # Old process is still running
   *) log_end_msg 1 ;; # Failed to start
    # Failed to stop
  log_end_msg 1
 #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
 echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload|status}" >&2
 exit 3

Then set the permissions so it can be executable and leave the owner set to root

sudo chmod 755 virtualbox-machinename

Now you can test it with...

./etc/init.d/virtualbox-machinename status # Is the VM running?
./etc/init.d/virtualbox-machinename start  # Start the VM
./etc/inid.d/virtualbox-machinename stop   # Stop the VM

Run the command top to see if a VBoxHeadless process is running. Now we need to install the script with the host computer. Use the update-rc.d command to install the script. I like to add another argument (99) to this to make sure that the script is made to run late in the bootup process. That way there is not a problem if this process relies on some other service that has not been run yet.

sudo update-rc.d -f virtualbox-machinename defaults 99

You are done! You can repeat this process for any other virtual machines you would like to run on this host computer.


Sunday, April 24, 2011


How about we install and configure a version control system while we are at it. Let's install favorite.
$ sudo apt-get install subversion libapache2-svn
Make the directory that we are going to create the repository in, then create the repository, and finally set the ownership to the user that Apache runs as...
$ sudo mkdir /home/<user>/svn
$ svnadmin create /home/<user>/svn
$ sudo chown -R www-data /home/<user>/svn
Now let's create the password file for accessing the repository and one account. It will ask you to enter a password for the new account twice...
$ sudo htpasswd -c /etc/apache2/dav_svn.passwd <user>
Then let's create an authorization file for giving fine grain read/write permissions to particular directories and files in the repository itself. I am just going to give read/write permissions to the entire repository structure...
$ sudo nano /etc/apache2/dav_svn.authz
We need to configure it all together now so you can do this in any place you like but the easiest is in the file below or virtual host file...
$ sudo nano /etc/apache2/mods-enabled/dav_svn.conf
Add or uncomment this text...
<Location /svn>
  DAV svn
  SVNPath /home/<user>/svn
  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile  /etc/apache2/dav_svn.passwd
  AuthzSVNAccessFile /etc/apache2/dav_svn.authz
  Require valid-user
Let's grab our old repository from another instance of subversion and bring it over...
$ svnadmin dump /path/to/repository > repo_name.svn_dump
Then lets suck it up into our new repository on this new server...
$ svnadmin load /home/<user>/svn < repo_name.svn_dump
How about adding SSL to make it all secure. Load this module so we can use SSL...
$ sudo a2enmod ssl
$ sudo a2ensite default-ssl
You may also need to install a couple of additional mods to make it all work depending on your linux distribution or version
sudo a2enmod authz_svn
sudo a2enmod dav_svn
Now you should be able to open you subversion repository using https://localhost/svn and it should work. Try using curl or something like it to test it out locally on your headless server like...
$ sudo apt-get install curl
$ curl -k https://username:password@localhost/svn

Thursday, April 21, 2011

Now What?

Well I think we have all the software installed that we need to build some pretty nice GIS web applications!!! So, I thought this would be a good time to stop and think about what we have done and all the possibilities from here. In following blog posts, I will talk about ways to develop  custom software solutions using this server setup.

This set of technologies is really quite similar to working with the Microsoft / ESRI platform that I use professionally. The difference is you are not paying for IIS / ASP.Net for the web server, Sql Server for the database, and the ESRI stack of GIS technologies like SDE and ArcGIS Server. With this completely free solution we have a spatially enabled enterprise database with PostgreSQL and PostGIS and an incredibly fast mapping server in Geoserver. With a bit of development you could do anything with this setup.

One of the things that I am really amazed by is how many great technologies we have installed on this server, yet the footprint is still very very small. Here are the specs for this server...

Size of VirtualBox image on disk: 1.9 Gigabytes
RAM used at rest: 382 Megabytes

A server build with the Microsoft / ESRI software would probably be 20 GigaBytes in size and use at least a GigaByte of RAM at rest. This server can run very well on older and probably free hardware. If you are hosting this server it will be cheaper too because Linux servers are cheaper than Microsoft servers. It would also take many days of CPU type to install all of the software needed to build this type of solution using Microsoft and ESRI. But, for this linux server I could easily script everything we have done and run it in less than an hour to rebuild this server!!!! I think that is pretty remarkable.

Below this paragraph is a link to the Geoserver main page running on this linux server that we have just built. This is the administrative interface for Geoserver and without logging in you can still play around with some of the functionality. The "Demos" and "Layer Preview" links allow you to view and query the data on the server. The "Demo Requests" link lets you formulate and send Open Geospatial Consortium requests to the server like WMS, WFS, and WCS requests, so give it a try. Don't bother trying to hack this server because it gets reverted to a previous snapshot of the OS every so often, which will remove any hacks. :)

Here are a few more links of projects using this server. These are just little pet projects that I have built using this system. In fact these applications are running on the same headless linux server that I have been building while writing this blog.

The Appalachian Trail Hiker Map site is an Mono/C# web application that uses the Google Maps API in combination with Geoserver. Geoserver is used to deliver the map tiles for all of the trail, towns, parking lots, shelters, etc. All of the actual data that make up the tiles reside in PostgreSQL/PostGIS, of course. This is an example of a completely asynchronous application because it never does a full post back to the server. All requests are AJAX oriented. It also does asynchronous calls to geoserver to get information about features on the map using the WMS GetFeature request. This is simple stuff, but illustrates some basic usage of this platform for viewing cached tiles and retrieving feature attributes.

The Search OGC website doesn't really do much mapping, but it again is an Mono/C# application running on Ubuntu Linux. It uses an open source GIS component called SharpMap to gather and store information from OGC mapping servers around the world and indexes them for searching. I use OpenLayers to make a simple map of any data that can easily be displayed on a map (this needs a lot of work and doesn't work for all sites yet, but take a look anyway).

Anyway, these are pretty simple mapping applications that don't do any really complex GIS work, but it is a place to start. In the next blog entries I hope to delve deeper into these technologies and explain how to use them to do things that we may take for granted using Microsoft / ESRI technologies; like doing spatial queries, geoprocessing, map overlay, geocoding, etc.

Anyway, enough for now...

Wednesday, April 13, 2011

Geoserver Daemon

Now I am going to setup my geoserver installation to run on system startup as a daemon. This is quite nice so you don't have to manually run the geoserver process every time you log into your server. This is quite easy to do also, but you have to get a few things right or it will not work and it can be difficult to understand why.

First go ahead and download a little script file from the Geoserver website. You can use the handy little wget utility to download the file straight from the web to your choice of directory...
$ cd /full/path/to/geoserver/
$ sudo wget -O
$ sudo ln -s /full/path/to/geoserver/ /etc/init.d/geoserver
$ sudo chmod +x
$ sudo flip -ub
Notice that we named it and then created a link file to it at /etc/init.d/geoserver. You could just copy the script directly to this path, renaming the file as geoserver, but typically the script is kept with the application binaries and a link is used to reference it from this location. We also run flip on this file to make sure Windows hasn't messed up the carriage returns and line feeds.

Now we need to modify some of the settings in this file so open up the /etc/init.d/geoserver file using your favorite text editor. Below is the entire script file with the important lines being 16, 17, 22, 23. These are the settings that I had to change to fit my server configuration. Notice that by default this file is setup to run if the user account you are working under is named "geoserver". Also, notice that I added the " -server" string to the end of the JAVA_OPTS variable on line 23. Honestly, I have no idea what it does, but many resources I read said to do that :) Thank you very much to Lennart for writing this script!

#! /bin/sh
# Provides:          geoserver
# Required-Start:    $local_fs $remote_fs
# Required-Stop:     $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      S 0 1 6
# Short-Description: GeoServer OGC server

# Author: Lennart Jütte <>

# Geoserver configuration - use /etc/default/geoserver to override these vars
# user that shall run GeoServer

DESC="GeoServer daemon"
JAVA_OPTS="-Xms128m -Xmx512m -server"

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME


# Load the VERBOSE setting and other rcS variables
[ -f /etc/default/rcS ] && . /etc/default/rcS

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions


 # Return
 #   0 if daemon has been started
 #   1 if daemon was already running
 #   2 if daemon could not be started

 start-stop-daemon --start --pidfile $PIDFILE --make-pidfile \
  --chuid $USER --chdir $GEOSERVER_HOME \
  -b --test --exec $DAEMON -- $DAEMON_ARGS > /dev/null \
  || return 1

 start-stop-daemon --start --pidfile $PIDFILE --make-pidfile \
  --chuid $USER --chdir $GEOSERVER_HOME \
  -b --exec $DAEMON -- $DAEMON_ARGS \
  || return 2


 # Return
 #   0 if daemon has been stopped
 #   1 if daemon was already stopped
 #   2 if daemon could not be stopped
 #   other if a failure occurred

 start-stop-daemon --stop --pidfile  $PIDFILE \
  --user $USER \

 [ "$RETVAL" = 2 ] && return 2

 # Many daemons don't delete their pidfiles when they exit.
 rm -f $PIDFILE
 return "$RETVAL"


case "$1" in
 [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
 case "$?" in
  0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
  2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
 log_daemon_msg "Restarting $DESC" "$NAME"
 case "$?" in
  case "$?" in
   0) log_end_msg 0 ;;
   1) log_end_msg 1 ;; # Old process is still running
   *) log_end_msg 1 ;; # Failed to start
    # Failed to stop
  log_end_msg 1
 #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
 echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
 exit 3



Again make sure to flip this file to make it readable on linux and double or triple check your paths and other options.

Next you will need to set the geoserver to launch on startup. I don't understand the details of this technology since I have relatively new to linux, but it is something akin to windows services in the Microsoft world. Regardless, just run the following command and it will setup Geoserver to run as a service when the system starts up...
$ sudo update-rc.d geoserver defaults
You can put a number at the end to set the start order if you have a problem. Adding it to the end is probably a good idea like so...
$ sudo update-rc.d geoserver defaults 99
After doing this you should be able to start Geoserver by starting the daemon...
$ sudo /etc/init.d/geoserver start
Also, you can remove this service using the following command...
$ sudo update-rc.d -f geoserver remove
Make sure that this command them returns something that looks like a geoserver process running as a java application...
$ ps aux | grep geoserver
Now try logging into the administrative website again using a browser pointed to...
If not it could be a million things that went wrong. Check your paths, permission, user, etc. and try again. It took me about three attempts before I stopped making stupid mistakes.

Of course this is not a secure installation so again we need to read the following resources to tighten it all down.

Sunday, April 10, 2011


So we now have a pretty decent little web application platform. We can run ASP.Net / C# web applications from this server using the PostgreSQL as our database backend. There are a couple of really good content management platforms that will run well on this server now...Chayahogo and MojoPortal. I have run Chayahogo and it works well. I hae never setup MojoPortal, but it seems to be the most robust solution for these technologies.

Anyway, we want to get to some GIS so lets move on to installing and running Geoserver. Perhaps I will come back to MojoPortal down the road. So, first we need to download and unpack Geoserver. You can download it from geoserver. I am going to install version 2.0.3, which is the latest stable build available. Also, I am going to install the binary (OS independent) install. I don't have a java servlet engine like Tomcat installed and want to keep this easy anyway.

One last thing...this is not a secure way to install Geoserver. I am just wanting to install it in as few steps possible to illustrate how to do it. We will discuss security later. Actually, you can take a look at this article if you want to get the details on security right away...production install.

So, let's download, unzip, and FTP the unzipped folder to some place on your server's file system. 

Install the Sun Java JRE (sun-java6-jre) (Used to need JDK and native JAI and JAI Image I/O extensions) Use the Sun Java JRE not the OpenJRE because it is much faster.

You have to add a source first to the end of the file /etc/apt/sources.list...

deb lucid partner

This makes it such that we can install from apt-get without downloading it manually from Oracle, etc. Then run the following lines from the terminal in order to install the Java JRE...
$ sudo apt-get update
$ sudo apt-get install sun-java6-jre #(may not need sun-java6-fonts)
$ sudo update-alternatives --config java
Test it out to make sure you have successfully installed the JRE...
$ java -version
Now navigate to the folder that you installed Geoserver to and go into the bin directory. You will see a number of script files for both windows and linux. The file is the one you want to use to start geoserver, but there are a few problems. The file was modified on a windows machine so it has an issue with carriage returns and line feeds. We need to fix this, so install a little program called flip to resolve this issue and run it like so...
$ sudo apt-get install flip
$ sudo flip -u
$ sudo flip -u
If you try to run it now, it will say that the JAVA_HOME variable is not set. That is true we never set it, so we need to do that. I tend to set these variable in the and files themselves so the application is completely self contained so let's do that. Open both files and add the following lines to the top of the file just after all the bash script comments...
If you added these lines to the file on a windows machine, make sure to flip them again like we did above. In fact I suspect that this version was touched by a windows text editor so you need to flip it. But, now it should run using the following command. Notice that I put an ampersand after the command in order to tell the shell to run the process in the background. This way you can continue to use the terminal for other work...
$ sudo sh &
and shut it down with...Ctrl-C

Again, this is not a secure installation at all. Everyone in the world knows your admin password until you change it. So, login to the geoserver administration page at http://localhost:8080/geoserver and change the admin password. The default for Geoserver is admin/geoserver for username/password, so change it first thing. Also, we did not set permissions at all on the file system. You may need to set some permissions if you get an error trying to run it, and then change ownership and permissions to secure it. At the very least set the permissions on the "gwc" and "logs" folders in the data_dir directory such that Geoserver has read and write to them. The performance will be terrible if you do not because the tile server will not cache any files and the logging will not work.

There are a number of ways to run geoserver and this was just the easiest way for me to get it setup. You can run geoserver as a Daemon so it is available on system startup or you can run the war file version from within a Java Servlet container. Either way it should behave exactly the same.

Tuesday, April 5, 2011


Now we need to install PostGIS to add geographic objects to PostgreSQL. PostGIS is pretty much equivalent to SDE in the ESRI world but follows the OpenGIS "Simple Features Specification for SQL". It is really nice that it subscribes to an open standard like this so it can interoperate with so many other systems. We will eventually install Geoserver and configure it to connect to PostgreSQL/PostGIS to get data and utilize funstionality. You can do quite a bit of serious web mapping and GIS with this combination of technologies.

Anyway, lets get PostGIS installed and configured here now...
$ sudo apt-get install postgis postgresql-8.4-postgis
Now we need to create a new database, add support for the PL/pgSQL database language to that database, and then run 3 scripts that add all the geographic database objects to the database.
$ sudo -u postgres createdb my_db
$ sudo -u postgres createlang plpgsql my_db
$ sudo cd /usr/share/postgresql/8.4/contrib/
$ sudo -u postgres psql -d my_db -f postgis.sql
$ sudo -u postgres psql -d my_db -f spatial_ref_sys.sql
$ sudo -u postgres psql -d my_db -f postgis_comments.sql
Notice that I am giving the postgres user superuser rights to run these commands. By default you have to run the psql command as postgres since that is the only user with a database account, but we need to give the OS account permission to do these things as well. This is a bit annoying but it is better than spending a bunch of time changing permissions and settings on the postgres account. That would be less secure as well.

Here are a few other useful programs that may come in handy. These are all programs not commands executed inside psql.

Delete database...
sudo -u postgres createdb dropdb <database>

Sunday, April 3, 2011


Now lets get the database installed. PostgreSQL is a fantastic database and another company has built an "extension" for it called PostGIS that gives it an amazing amount of GIS functionality. This has become the standard database platform for the open source GIS community. Let's install, configure, and discuss PostgreSQL in this post. I'll add another blog entry regarding PostGIS shortly.

Install the database...(or postgresql-8.4 for version specific)
$ sudo apt-get install postgresql 
Configuration files are located at /etc/postgresql/8.4/main...
environment (variables used by postmaster)
pg_ctl.conf (database cluster specific options)
pg_hba.conf (controls access to the database)
pg_ident.conf (username mapping?)
postgresql.conf (general configuration file)
start.conf (assume it has something to do with starting up :) )
First let's enable TCP/IP connections since they are disabled by default by adding the following lines to /etc/postgresql/8.4/main/postgresql.conf...
#For local connection only
listen_addresses = 'localhost'
#For remote connections
#listen_addresses = '*'
Now we need to set a password for the PostgreSQL database user that is automatically created called postgres. There is also an operating system user account named postgres that is added specifically for running the PostgreSQL daemon. By default this OS user is actually set to "login disabled". So, the user is really used only for running PostgreSQL. Anyway let's go ahead and set the password for this database user account now...
$ sudo -u postgres psql template1
# ALTER USER postgres with encrypted password '<password>';
# \q
After configuring the password, edit the file /etc/postgresql/8.4/main/pg_hba.conf to use MD5 authentication with the postgres user. This will allow localhost access only and requires an encrypted password...
#For local connections only
local         all       postgres        md5
#For remote connections from a certain ip address
host          all       postgres       xxx.x.x.x/32        md5
Restart PostgreSQL using this command...
$ sudo /etc/init.d/postgresql-8.4 restart
Now let's connect to a database. Since the postgres OS user is not setup to allow login, you have to connect to the database as that user with sudo as you see below.
$ sudo -u postgres psql postgres
Frankly I think it is pretty confusing and takes a little while to get the hang of navigating PostgreSQL using the psql command, but it works great after a bit of a learning curve.

Now you should see the postgresql command prompt where you can perform SQL and interact with the database. Type \? or \help to get some information on how to work with this program. Most importantly, \q gets you back to the shell prompt...

I'll add some common PostgreSQL tasks cheatsheet below a bit later since the database should be running fine now.


Saturday, April 2, 2011


Now we have the Apache web server running on our server, but we have not installed any of the packages we need to run ASP.Net applications. We could install run-times that would allow us to build web applications in PHP, Perl, or Python, but I am going to install Mono so we can use C# / .Net. This is the language I am most familiar with so it is nice choice for me. Unfortunately, installing and configuring Mono is not quite as easy and straight forward as the PHP runtime. I may write a blog entry for PHP as well since so many of the existing open source GIS technologies favor it.

Make sure to stop Apache using the command below before installing Mono because it can cause problems otherwise.
$ sudo /etc/init.d/apache2 stop
Now run the following commands to install Mono for a production environment.
$ sudo apt-get install mono-apache-server
$ sudo apt-get install mono-apache-server2
$ sudo apt-get install libapache2-mod-mono
I think you could run the following as well, but it installs a bunch of additional packages that you may not need...
$ sudo apt-get install mono-complete
Now we need to configure Mono. This was very confusing to me because I couldn't find really good documentation anywhere, so this section is really a work in progress. This works but I can't say I understand all the options fully.

You can use the a2enmod and a2dismod commands to enable and disable virtual hosts in Apache. These commands create link files in the sites-enabled folder and Apache parses the contents when it is reloaded. 
$ sudo a2dismod mod_mono
$ sudo a2enmod mod_mono_auto
Enabling mod_mono_auto seems to setup the parameters that you would want for all ASP.Net sites that you might host on this server. I think this is called autohosting and basically the remaining configuration options are set in the virtual host file for the specific website so you don't have to make configuration changes to Apache in order to control how Mono works in your web applications. This is nice but requires a few more options to be understood and set correctly. You can see that even the version of the .Net framework is set within the virtual host file.

Now to get a test Mono web application running let's create a new virtual host file, enable it , and have it "point" to a simple web application. You can go to the mono website and use their little wizard to create your virtual host file. This is probably the best way to do it...
I don't understand all of the Mono options for the virtual host file yet but many of them are sort of intuitive. Below is a sample virtual host file that I generated from the website...
<VirtualHost *:80>
  ServerAlias *
  DocumentRoot /home/atv/web/mydomain/www

  # MonoServerPath can be changed to specify which version of ASP.NET is hosted
  # mod-mono-server1 = ASP.NET 1.1 / mod-mono-server2 = ASP.NET 2.0
  # For SUSE Linux Enterprise Mono Extension, uncomment the line below:
  # MonoServerPath "/opt/novell/mono/bin/mod-mono-server2"
  # For Mono on openSUSE, uncomment the line below instead:
  MonoPath "/usr/lib/mono/3.5"
  MonoServerPath "/usr/bin/mod-mono-server2"

  # To obtain line numbers in stack traces you need to do two things: 
  # 1) Enable Debug code generation in your page by using the Debug="true" 
  #    page directive, or by setting <compilation debug="true" /> in the 
  #    application's Web.config
  # 2) Uncomment the MonoDebug true directive below to enable mod_mono debugging
  # MonoDebug true

  # The MONO_IOMAP environment variable can be configured to provide platform abstraction
  # for file access in Linux.  Valid values for MONO_IOMAP are:
  #    case
  #    drive
  #    all
  # Uncomment the line below to alter file access behavior for the configured application
  MonoSetEnv MONO_IOMAP=all

  # Additional environtment variables can be set for this server instance using 
  # the MonoSetEnv directive.  MonoSetEnv takes a string of 'name=value' pairs 
  # separated by semicolons.  For instance, to enable platform abstraction *and* 
  # use Mono's old regular expression interpreter (which is slower, but has a
  # shorter setup time), uncomment the line below instead:
  # MonoSetEnv MONO_IOMAP=all;MONO_OLD_RX=1
  MonoApplications "/:/home/atv/web/mydomain/www"

  <Location "/">
    Allow from all
    Order allow,deny
    SetHandler mono
    SetOutputFilter DEFLATE
    SetEnvIfNoCase Request_URI "\.(?:gif|jpe?g|png)$" no-gzip dont-vary
  <IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript
I added lines 13 and 14 in order to make it run using the 3.5 framework, but now you can run the command below to enable the virtual hostsite...
$ sudo a2ensite <myvirtualhost>
Of course the actual ASP.Net web application needs to be in place and located at the path /home/<user>/www as you can see above in the virtual host file. Create a simple test file in this directory called default.aspx and fill it with some really simple code just to make sure all is working. Here is a little snippet to copy and past...
<%@ Page Language="VB" %>

<script runat="server"> 

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) 
 lblVersion.Text = "Your server is running ASP.NET and the version is " & System.Environment.Version.ToString() 
End Sub


<title>ASP.NET Version</title>
<form id="form1" runat="server"> 
<asp:Label ID="lblVersion" runat="server">
This just writes out some text to a Label webcontrol, but if you see the text, then Mono / ASP.Net is working. If not, you done screwed it up.

Oh, don't forget to restart the Apache webserver firstand see if it will even load using...
$ sudo /etc/init.d/apache2 restart
If you see an error that looks like this you can ignore it. It is an unfortunate error but doesn't actually cause any problems. I think it is fixed in a future release of Apache2 or Mono...not sure and don't really care much since it still works fine.

Now you can pull up a browser and navigate to the URL of the website you setup and it should work.

Friday, April 1, 2011


Now we are starting to get somewhere. Let's install the webserver now so we can host some GIS web-sites  very soon.

Install Apache web server from a terminal...
$ sudo apt-get install apache2
Here is a list of the important paths and files that you may need to edit at some point...
/etc/apache2/ (server root)
/etc/apache2/mods-available (apache modules you can load)
/etc/apache2/mods-enabled (apache modules loaded)
/etc/apache2/sites-available (virtual host files you can load)
/etc/apache2/sites-enabled (virtual host files loaded)
/etc/apache2/apache2.conf (main configuration file)
/etc/apache2/error.log (error log file)
/etc/apache2/port.config (ports file)

You may want to add the server name to the main configuration file at /etc/apache2/apache2.conf
ServerName <Name>

In the /etc/apache2/sites-available folder you will find a default virtual host file. You can enable this website by running the command below. This command simply creates a link file in the /etc/apache2/sites-enabled folder. Restarting Apache will then load this website, making it reachable.
$ sudo a2ensite default
You can disable this virtual host website using this command...
$ sudo a2dissite default
These virtual host files simply contain all the website configuration information needed to make the website live. You can see in the screenshot below that the default virtual host file is pointing to a document root of /var/www. By default there is an index.html file in that path, so once you apply permissions to this folder and restart apache, it should work.

So, let's assign permissions to the document root folder now. This is not the most secure way to do things, but it is good enough for now. This gives all read and execute permissions recursively on all files and folders withing /var/www.
$ sudo chmod -R a=rx /var/www/
Now restart the apache web server so all these changes are loaded...
$ sudo /etc/init.d/apache2 restart
Now go the the URL that points to your web server and you should see...

Thursday, March 31, 2011


OK, if you need to install FTP here is a quick and totally insecure way to do it!! I am just going to get the most simple FTP server running here, but you will probably want to do something to secure it better at some point. Just use OpenSSH if you simply need to move files over from development to production.

$ sudo apt-get install vsftpd
Configure it's configuration file located at /etc/vsftpd.conf
$ sudo restart vsftpd


Well, we need to be able to connect to this new server and we want to do everything from the command-line, so we need to install an SSH server. I am going to use OpenSSH because it just works and is very simple. You can do a lot with OpenSSH including connect to the terminal and transfer files over SFTP, and all very securely. With this there is no good reason to have an FTP server installed.

Install the server...
sudo apt-get install openssh-server
Backup the configuration file...
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original
Configure the server to make it a bit more secure by modifying the file located at /etc/ssh/sshd_config. Change it to something non-standard like...port 2222 if you like, but this is not really necessary since it is already secure...but may prevent denial of service types of attacks...

Restart the server...
sudo /etc/init.d/ssh restart
Now try connecting from another computer using the following command from a terminal...
ssh <username>@<ipaddress>

Now, if you want to log in to your remote servers using ssh and not have to bother entering a password, you can use a private / public key to set that up. Below I have created a set of steps starting with what you do from the local computer...

Local Computer:

1. Set the appropriate ownership for the entire ~/.ssh directory and do it recursively
sudo chown -R user:group ~/.ssh

2. Set the appropriate permissions for ~/.ssh/known_hosts so ssh can write to it.
sudo chmod ~/.ssh/known_hosts 700

3. Now run these commands to generate the private/public keys and to put the public key on the remote computer. Just accept all the defaults (leave blank and press enter) for the first command.
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/ user@computer
Remote Computer:
Run this command on the client as well to make sure the key is now managed by the ssh-agent
On the remote computer you just need to make sure that the configuration file used by the ssh deamon (/etc/sshd/sshd_config) has a couple of lines added or uncommented.
RSAAuthentication yes
PubkeyAuthentication yes

After doing this restart the ssh service and you should be able to connect from the local computer without needing to enter your password. You can repeat these step for another remote computer, but of course you do not need to use the ssh-keygen command because you already have private/public keys generated from running it the first time.


Wednesday, March 30, 2011


Ok, now that we have installed Ubuntu Server 10.04, we need to do a bit of configuration. Let's go ahead and do a few things to make the server accessible over the internet.  There are many ways to do this so I will just share with you what I do to host my servers.

You could set your server up on a computer that is co-located, or dedicated at a professional hosting company, or perhaps you just want to run these servers from a computer at your house. I run my servers from my house, but with business class internet service and a few static IP addresses. If you do not have a static IP address available, you can still us an internal LAN IP address (internal network specific IP address) and use port forwarding on you router to let the outside world access services on your Ubuntu Server. You could even use one of the many services like to simulate a global static IP address. Perhaps this is worth an entire blog entry for later, so lets just continue.

Modify the network settings in the file located at /etc/network/interfaces to suite your situation.
auto eth0
iface eth0 inet static
Set the DNS settings if they are not already set in the file located at /etc/resolv.conf with...
Refresh and check the ethernet settings using the commands below.
$ sudo /etc/init.d/networking restart
$ ifconfig
If you are using VirtualBox to set this up, make sure to setup your guest VM to use the Bridged Network connection so the Ubuntu Server can be reached at the static IP address you assigned it.

So now the computer should be accessible over the internet, but there is almost nothing running on this bare bones server, so we need to install some software next.

Tuesday, March 29, 2011


Canonical Ltd. make it so easy to install a bare bones Ubuntu Server system. Go ahead and download the .iso file from here... if you have not already. There are detailed instructions on this page for downloading, burning to CD, and installing, so go ahead and follow those instructions or follow the screen shots below.

I install the very least amount of software at first in order to take a snapshot of the system in it's simplest form. Then we can build it up from there. Here are the steps you will see while installing Ubuntu Server...

Step 1 (Choose Language)

Step 2 (Start Install)

Step 3 (Installation Language)

Step 4 (Country)

Step 5 (Automatically Detect Keyboard)

Step 6 (Origin of the Keyboard)

Step 7 (Keyboard Layout)

Step 8 (Host Name)

Step 9 (Timezone)

Step 10 (Partition Guide)

Step 11 (Disk to Partition)

Step 12 (Confirm Write to Partition)

Step 13 (Full User Name)

Step 14 (User Account Name)

Step 15 (Password)

Step 16 (Confirm Password)

Step 17 (Encrypt Home Directory)

Step 18 (HTTP Proxy Server)

Step 19 (Automatic Updates)

Step 20 (Software Installs)

Step 21 (Configure Grub)

Step 22 (Installation Complete)

Step 23 (First Reboot)

Update all the software packages and then reboot with...
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo shutdown -h now

Monday, March 28, 2011


I run all of my development, test, and production computers as virtual machines using the free / open source virtualization package called VirtualBox OSE. You can download it from here... if you would like. VirtualBox is fantastic open source software that you can run on any host operating system. I do this primarily so I can take periodic snapshots of my operating system and role back to a previous state if it crashes or I just dirty up the virtual machine. There are many reasons to use virtualization.

If you are running Ubuntu Desktop 10.04 on your host computer and want to use VirtualBox to virtualize the Ubuntu Server operating system that we are going to build, then you can follow these instructions. Or, you can just skip this blog entry and go to the next one about installing Ubuntu Server like normal on a physical machine. Below is a screen shot of the Applications > Ubuntu Software Center window. Open this window and search for "VirtualBox" and install it like you see below.

Then you can start VirtualBox from Applications > Accessories >VirtualBox OSE. You will see the window below, but will not have any VMs of course. Go ahead and create a new virtual machine using the "New" button and step through the wizard.

Once you have created the new VM, you can go into the storage item and add set the CDROM to point to your Ubuntu Server .ISO file so it will begin the installation process the first time you start the VM. I recommend doing this because it works well and allows you to avoid burning a CD with the operating system on it.

Now you are ready to install the Ubuntu Server 10.04 operating system.

Sunday, March 27, 2011

The Plan

I guess the best place to start is with a plan, so here it is...

I am going to write a bunch of blogs over the next few days / weeks explaining how I built a nice little Linux GIS web server. I am going to build it with the technologies I have become familiar with over the last couple of years and cover each technology as a separate blog entry. Please give me some feedback or corrections so we can document these steps together. I will update the blog entries with any good ideas or corrections.

Since professionally I'm primarily a Microsoft developer myself, the selection of technologies may not satisfy the most hardcore purist Linux geek, but that is not my goal. I want to lay out a solution that anyone (that is relatively comfortable with linux) can put together to create a good platform for GIS web no cost.

I have been using the following technologies for a little while now (maybe a few years). Until recently I have been developing my applications using Ubuntu Desktop for both development and production. It has been nice using the Gnome GUI, but now I am going to try my hand at building this as a bare bones headless Ubuntu Server strictly from the terminal. Oh, and I am creating this system as a VirtualBox guest operating system on an Ubuntu Desktop 10.04 host operating system. VirtualBox and virtualization in general is a great tool for doing this kind of experimentation so you can role back if you make mistakes. Here is the set of snapshots for the system we are building...

So, this is the system we are going to build and document in the coming blog entries...

VirutalizationVirtualBox OSEv3.1.6_r59338
Operating SystemUbuntu Serverv10.04 LTS
Web ServerApache2v2.2.14-5ubuntu8.4
Spatial Database Ext.PostGISv1.4.0-2
Mapping/GIS ServerGeoserverv2.0.3
Client GIS APIOpenLayersv?

The version numbers were obtained using...
$ sudo dpkg -s <packagename> | more 
I think this is a great little system and I am currently running these technologies to host a couple of websites with GIS functionality. I am amazed at the things you can do with a completely free solution like this. The next blog entries will cover each of these technologies.

Again, you may want to look at OpenGeo Suite if you want to do a lot of what I am going to cover here, but in a much quicker and easier way. You may not learn as much, but can get started in just a few minutes.


    Friday, March 25, 2011

    Hello World

    I have been building GIS software solutions for over 10 years now and have never blogged about it. I don't think I have ever blogged about anything technology related actually. Not sure why...probably just too tired at the end of the day and want to think about other things (I am pretty damn tired right now and not sure I want to even do this!!) Well I'm not exactly going to start blogging about what I do for a living now. As interesting as it can be, 8 hours a day is enough for me. I am going to blog about the free / open source GIS software that I don't use professionally for building GIS solutions.

    Most professional GIS software developers (including myself) use software components, built by a fine company called ESRI, as a starting point for building their solutions. ESRI builds great software, but it is very expensive and is primarily built to run on the Windows operating systems... which is also very expensive software. You usually need a database, and most people naturally choose a Microsoft database like SQL Server...which is also very expensive. The price tag for running a GIS solution can become huge before you even start to develop your custom software solution on top of it.

    For the last couple of years I have been playing around with free / open source technologies that one can use to build GIS software solutions as well. I guess the community calls it FOSS4G (Free Open Source 4 GIS). I am starting this blog to document my experiments using these opens source equivalents to ESRI software. I am not exactly sure why I have been so drawn to these technologies lately. It is probably because of the global cooperative effort and the more idealistic approach to building open source technologies. It seems somehow very pure and honest. Some of these technologies are really very good as well...if not better than their commercial counterparts. I think taking the capitalistic motivation out of the equation can result is some really great software.

    So here is the goal of this blog. I want to document the process of building a completely free / open source platform for creating web-based GIS software solutions. Most of the proprietary GIS software products out there are amazingly expensive and it is nice to know that there are a number of free community driven solutions that work well. The problem is that the information on how to use them is sometimes hard to find or laden with unnecessarily technical jargon. I am hoping to document a few things that may help others get started using these technologies.

    Luckily there are a number of technologies in the open source world that many seem to agree on as the best in class. I am going to blog about the following technologies and try to explain how to set them all up in a simple and reproducible way, and how to use them to build great GIS software solutions. I am not a master of any of these technologies but hopefully I will learn a lot more going through this process.
    • Linux
    • Apache
    • C# / Mono
    • PostgreSQL / PostGIS
    • Geoserver
    • OpenLayers
    • etc
    On last thing, for most of my career I have been a Microsoft developer and have become very comfortable using C# / .Net to build applications.  Because of this I have been drawn to Mono so I can continue to use my experience with the C# language and .Net framework but on the Linux operating system. I will probably spend a lot of time covering topics related to Mono with C# as the language of choice...because it makes a lot of sense in my case, with so many years of experience invested. Many of the Open Source GIS software available is written in C, C++, Java, and PHP, so this will make things a bit interesting and challenging. I'll probably cover the use of some other languages as well though.

    Ok, I am really tired of talking about technology now....good night.

    EDIT: Take a look at OpenGeo Suite if you want to do a lot of what I am going to cover here, but in a much quicker and easier way. You may not learn as much, but can get started in just a few minutes.