Google My Maps to Galileo Offline Maps

As I mentioned in a previous post, I’m using Google My Maps as my primary tool for creating my map for Te Araroa, mapping estimated day stopping points, resupply points, etc. This tool has been invaluable for this purpose. It’s very easy to use, allows me to add custom icons to the points on the map to differentiate huts and resupply points and everything else, and, most importantly, it has the ability to export KML.

On the trail itself I’ll be using Galileo Offline Maps as my primary navigation tool. This app has the ability to download vector maps of all of New Zealand, import KML to display on the map, and allows for custom raster map tile sources. Sadly, but understandably, it won’t download the entire set of topo maps for the whole country in one shot, but what it does do is have a cache that can be used offline, so I just need to make sure I have the maps I need to get me to the next place I’ll have internet in my cache, and I’m set. This is easily done by simply looking at the map and sliding around, picking up the tiles I need and caching them in the process. I can then flip to airplane mode, verify I still have the tiles I need, and I’m set.

A problem I ran into is that while Galileo also supports customizing the icons or “Category” thats get displayed for a “Bookmark” (their name for what KML calls a Placemark), it doesn’t support the ones exported by Google Maps. Fortunately, it does support its own categories. As in, I can create some bookmarks, export the KML, import that same KML, and the categories are preserved. This means I should be able to convert the KML from Google Maps into something Galileo can use.

So I wrote a script which does just that. One of the things I noticed in my experimenting with the KML I’m importing into Galileo is that it actually doesn’t seem to care about any of the Style information in the file, it just looks at the styleUrl on the Placemark and uses that to determine which category to apply to the bookmark. This allowed me to quick-and-dirty modify each Placemark and import it into Galileo.

This allows me to turn this:

Into this:

And this:

Right now this is super super hacky. It uses a static mapping of icons from Google Maps to Galileo (including colors), because I’d like to be able to convert back to Google Maps from Galileo, so I came up with a mapping from Galileo’s icons to a subset of Google Maps icons. It’s not perfect, but it works.

It also only touches the styleUrl contents on each Placemark. This means the file is actually completely broken for any other purpose than using it with Galileo. I’d like to change that so it converts it to something closer to what Galileo itself exports.

This is a huge step, though. It means I can now, with minimal fuss, export my Google Map to my phone and use it offline. Something that is going to be incredibly valuable to me on the trail!

DreamHost, rvm, capistrano, fun!

I recently started setting up ktc.hn as a url shortener for my blog. I decided to make it a little Sinatra app, which meant I got to play with my standard box of tools for deploying Ruby applications: rvm, capistrano, and passenger. Since things weren’t super straightforward, I figured I’d write up this post to share what I ended up with, things I ran into, and things I’m hoping to improve upon.

The Setup

RVM

I use rvm, if for no other reason than it’s what I’m familiar with. It makes it really easy to install and manage multiple rubies, manage gems with bundler, it’s well maintained, and I can do it locally on my laptop as well as remotely on my deployment targets.

Capistrano

Capistrano is an ssh-based deployment tool, useful for deploying applications to remote servers. I like it because, while it has great integrations for ruby projects, it can also be used to deploy pretty much any software. I’ve used it for PHP as well as golang projects, rails and non-rails ruby projects, whatever. I’ve also used its sshkit library for automating tasks via ssh.

Passenger

DreamHost supports running ruby applications using Phusion Passenger. The details of how this all works are outside of the scope of this post, but it’s similar to running an application using FCGI. You point passenger at the ‘public’ directory of your application, and it goes looking for your config.ru file, which is the interface defined by rack, which Passenger is an implementation of, fires up the app, and starts sending HTTP requests to it.

Configuration

After installing rvm locally, I drop a .ruby-version file in my project root, and a .ruby-gemset file as well. This makes my zsh shell integration do the thing it needs to do to select the correct ruby version and gemset. Note that since I’m using Bundler to manage my gem dependencies, I don’t really need to specify a different gemset, but it’s nice to have so I don’t have to constantly re-run bundle install every time I switch between projects.

Since I’m deploying using capistrano, and want to use rvm on the remote side to manage my ruby install, I need to add a couple of gems to my Gemfile: capistrano  rvm1-capistrano3

Then I run cap installto “capify” (that actually used to be the command to run to do this) my project. This installs the following files:

  • Capfile
  • config/deploy.rb
  • config/deploy/production.rb
  • config/deploy/staging.rb

The main purpose of the Capfile is to load in gem-based plugins for capistrano. The config/deploy.rb file is for global configuration of capistrano, such as declaring new tasks, configuring the source control repository, and other things. Overrides or stage-specific settings go into the config/deploy/*.rb files, with each file declaring a different stage capistrano can deploy.

Capfile

Since I’m using the rvm1-capistrano3 gem, I need to load it by adding a line to the Capfile:

This loads up all of the tasks in the plugin, sets some of them up to be executed, and defines others to be used by the user of the plugin. I’ll be using some of those later.

Note: the gem is called rvm1-capistrano3, but you need to require rvm1/capistrano3, this is because gem names can’t contain slashes, but require is working with filesystem paths and the file being loaded is lib/rvm1/capistrano3.rb. Fun!

config/deploy.rb

In config/deploy.rb I set the name of the application, which is really pretty arbitrary, but should be unique, as this is used in temporary directories and such.

I also need to set the path to the git repo where the code to be deployed is stored.

Capistrano supports several different source control systems, but git is the default, and it’s where my application’s code is stored.

For various historical reasons, I like my application’s tmp directory to be shared amongst all deployments. This also seems to play a little easier with the tmp/restart.txt functionality passenger provides to allow the user to restart the application on demand (such as when new code is deployed).

Two things to note here:

  1. The linked directory will be created by capistrano in the deploy path and symlinked into the release path.
  2. The reason to use fetch() here is to not stomp on :linked_dirs settings declared elsewhere in the capistrano configuration (e.g., a deploy stage configuration file, or by a plugin)

Since ideally I’m installing onto a clean slate, and want to have capistrano managing the entirety of the installation, I set up some of the tasks from rvm1-capistrano3 to be run during deployments automatically:

Here’s where I ran into some trouble. Previously, I was using the capistrano-bundler gem, which will run bundle install to install your gems, among other things. The problem here is that bundler is no longer installed by default alongside ruby, so when the plugin goes to run bundle install, it does so using the system’s /usr/bin/bundle, which on DreamHost (at least my server, a VPS, at time of writing) is running against ruby version 1.8.7. My Gemfile apparently isn’t loved by that version of bundler, so what happened is the bundle install command took a long time, ended up consuming all of the memory on my instance, and caused it to get rebooted by DreamHost. Oops.

The fix here appears to be to not use bundler for deployment and instead use the deploy task provided by rvm1-capistrano3 to install the required gems:

The reason I can’t just gem install bundler is because the capistrano-bundler gem wraps any gem command with bundle exec so there’s a bit of a chicken and egg problem. This approach seems to work, however, so I’m going to stick with it for now.

In the DreamHost control panel, I ticked the boxes next to Passenger and RVM and supplied the path to rvm:

Screen Shot 2016-05-14 at 11.20.56 PM

The ‘ktchn’ in that path is an rvm alias set up for me in capistrano. I do this so I don’t have to hardcode the ruby version and gemset name into the path in DreamHost’s configuration.

Sadly, there appears to be a bug in the rvm1-capistrano3 gem, or perhaps rvm itself, that makes it not alias the version of ruby in the specified path when setting up an alias, so I also had to explicitly set the ruby version here, which I don’t like, as it’s redundant, and buried in a config file.

Since I’m running on a managed DreamHost VPS, I don’t have root, and rvm install fails because it tries to use sudo to check for and install library dependencies. Fortunately, they seem to have installed them already, so I just disable the autolibs functionality:

And finally, to tell Passenger to restart the app after deployment, I touch tmp/restart.txt:

config/deploy/production.rb

Finally, the deploy stage. Capistrano does a great job of making this pretty straightforward.

The fun part here was the :tmp_dir thing. DreamHost’s VPS product mounts /tmp (which is the default tmp dir in capistrano) as noexec, which capistrano doesn’t like, as it uses that directory for a couple of things and needs to execute at least one of them!

Issues

There are 2 major unresolved issues I have with this setup.

First is not being able to use bundler for managing my gems on the remote server. I am not certain if the correct versions are being installed by the rvm1:install:gems task. I feel like this is not an insurmountable task, I just need to poke around a bit in capistrano to see how I can do it!

The other is having to hardcode the ruby version into my config/deploy.rb file for the rvm1:alias:create command to work. The default for :rvm1_ruby_version is ., which is causing it to run rvm alias create ktchn . which rvm isn’t very happy about. I’m sure this is also something fairly easy to fix!