Vector tile serving: Some introductory thoughts & hints for a fast way to serve vector tiles

I. Introduction

In the last few weeks I played around and tested some approaches to  self hosted serve (and style) vector tiles. In this blog entry I share some of my observations to you. Don’t expect a fully designed tutorial, it’s more a collection of starting points for your own tests to serve vector tiles. There are several approaches how to serve vector tiles. Mapbox [1] can be used, also GeoServer  has a vector tile extension [2] In my experiments I used tileserver-gl, an open source tile server developed by Klokantech [3] . Tileserver-gl is a great way to quickly serve vector tiles on an own web server, escaping typically pay per view scenarios. Mapbox for example, which is today’s big player in serving web maps, uses a pay per map-view payment model, charging 0.50 US$ per 1000 map views [4]. With tileserver-gl it is possible to self host you vector map tiles and later embed them in your web map using e.g. Leaflet.js or OpenLayers and tileserver.gl has the great advantage, that it allows using GL styles, a JSON style specification, basically telling your map how it should look [5]

For those of you that never heard about vector tiles, I recommend reading this entry of the OpenStreetMap Wiki: [6]. In its nutshell, vector tiles are a vector based representation of geo-objects. The idea is rather old and appeared shortly after the appearance of GIS systems and first was used by the US Wildfire Service in 1975, in their software  Wetlands Analytical Mapping System (WAMS) [7]. Vector tiles started to “revolutionize” the way how map data is served in the WWW, when Google introduced vector tiles to serve Google maps in 2010  [7].

II. Prerequisites

If you plan to serve some vector tiles and would like to follow the suggestions of this blog post, you need a running LINUX server with docker installed on the server. For my tries I use a LINUX V-Server from Strato [8], a Berlin based company belonging to 1&1 (as far as I know). They offer some interesting and cheap virtual server packages. I recently  switched from UnitedHoster to STRATO . Of course, you can use your own server, or Ubuntu from the Amazon AWS market place [9].  As you can see, there are plenty of other hosting companies out there. For some testing, a starter package normally is sufficient, if bandwidth and speed matters, you have to spend more bucks for the fun. 

III. Use OpenMapTiles to get OpenStreetMap(OSM) vector tiles

OpenMaptTiles [10 ] allows downloading vector tiled OSM data in the MBtiles format free of charge with smaller regions; for customized and bigger regions, the service charges a fee, e.g. to download the whole planet.osm in MBtiles format costs 2048 US$ for business users (1024 for individual users, as free lancers).

IV. How can you create vector tiles of your own map data?

The makers of OpenMaptiles published a nice introduction how to create vector tiles using the command line tool Tippecanoe developed by Mapbox [11] What isn’t answered is, how to install Tippecanoe on a LINUX system. First you have to clone the Git repository, Next change to the folder where the repository was cloned to, compile the software and install it.

git clone mapbox/tippecanoe.git 
cd tippecanoe
make -j
make install

After the installation you need files in the GeoJSON format as an input of Tippecanoe.

Ogr2ogr easily allows to convert file formats as the still widely popular ESRI Shapefiles to GeoJSON:

ogr2ogr -f GeoJSON your_data_in_4326.json -t_srs EPSG:4326 your_data.shp

Next Tippecanoe converts the GeoJSON in the MBtiles format in 14 different zoom levels, the MBtiles later can be served using a tileserver.

tippecanoe -o your_data.mbtiles your_data_in_4326.json

If you just need raster tiles for you web map?

In some cases vector tiles aren’t necessary and raster tiles are all you need to quickly visualize something on a slippy map. A quick way to get raster tiles, without zoom-level (scale) dependent rendering of objects would be:  1. Style your data in QGIS . 2. Export your styled map as high resolution png and 3. Tile your date with gdal2tiles:  

gdal2tiles.py -s EPSG:4326  -z 10-16 yourdata.png yourdata_xyz=4

Gdal2tiles creates a folder structure, with subfolders for each zoom level, depending on how much zoom levels you have defined. For quite a while I used the great software Tilemill [12]  to style geodata using carto.css and to create raster tiles. Unfortunately active development of Tilemill stopped a few years ago.

V. How to install and use tileserver.gl?

First the good news, tileserver.gl is available as docker container. In case you want to fight through your way  to install the full stack and not to use docker, you can install the server this way:

  1. Install node.js on Ubuntu Server: DigitalOcean [13]  offers a nice introduction on how to install node.js. To use tileserver.gl, at least the version 6 of node.js is necessary.
  2. Install Tileserver.gl: For the installation this tutorial found on “Ralphs blog” (interesting blog, by the way) was used [14]

Use Klokantechs Docker container

As mentioned above, it is not necessary to install the full stack. You can also use Klokantechs Docker container. If you haven’t installed Docker, Digitalocean also offers great tutorials on how to install Docker on your Ubuntu server [15]Start the container in the same folder your MBTiles are located.  After installing docker, start the docker container from the directory, where you host your MBtiles with the command:

 sudo docker run --rm -it -v $(pwd):/data -p 8080:80 klokantech/tileserver-gl -c config.json

The map servers frontpage, with access to the styles and the data,  can be opened via your servers IP and the port number 8080 by default. Here you can see the frontpage of my test server: http://h2800220.stratoserver.net:8080/ The behavior of the map, e.g. the map style can be controlled in the style.json file. In the configuration file, the style.json file has to be referenced.  

VI. Style your map with Maputnik

Styles are controlled by changing the style.json file and have to be referenced in the config.json. How can you change this file, to get an own style for a custom web map? Styles can be changed using Maputnik [16], an open source style editor for Mapbox styles. Maputnik can be used in the browser as online editor, but does not allow adding vector tiles served with http, just https. On my test server I did not activate https, but the offline version of Maputnik also accepts http as data source. Maputnik just hast to be downloaded from: https://github.com/maputnik/editor/releases/tag/v1.5.0 and extracted, than it can be used locally in a browser. Edit the style as you want and then export the style.json and upload it to your server. I recommend to start with an existing style and to adapt it to your needs. After modifying the style, upload the style.json file to your server, don’t forget to reference the config.json and restart the docker container, so that the changes take effect.  Just an observation, Maputnik looks and feels a lot like the Mapbox Studio [17] , the usage is quite intuitive.

And finally I got  a self hosted vector map (of Managua, Nicaragua) with some edits of the nice dark matter style (which is just a product of playing around a little with the style):  http://h2800220.stratoserver.net:8080/styles/test/#15/12.14186/-86.28059

In my next blog post I plan to write a little more about the GL style definition.

 

References

[1] https://www.mapbox.com/vector-tiles/

[2] https://docs.geoserver.org/stable/en/user/extensions/vectortiles/

[3] https://github.com/klokantech/tileserver-gl

[4] https://www.mapbox.com/pricing/?utm_source=chko&utm_medium=search&utm_content=MapboxPricePlan&utm_campaign=CHKO-PR01-BR-Mapbox-INT-Exact&gclid=CjwKCAiAlvnfBRA1EiwAVOEgfMMmTtJem8_gOUFz8gE5S3RshxEqdPOON_Z8OiCGzV6Ht1O2BmzI6xoCb5EQAvD_BwE

[5]  https://www.mapbox.com/mapbox-gl-js/style-spec/

[6] https://wiki.openstreetmap.org/wiki/Vector_tiles

[7] https://en.wikipedia.org/wiki/Vector_tiles#History

[8] https://www.strato.de/server/linux-vserver/

[10  ]https://openmaptiles.com/

[11] https://openmaptiles.org/docs/generate/custom-vector-from-shapefile-geojson/

[12] https://github.com/tilemill-project/tilemill

[13] https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04

[14] https://golb.hplar.ch/2018/07/self-hosted-tile-server.html

[15]https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04

[16] https://maputnik.github.io/

[17] https://www.mapbox.com/studio/