AvnNav Charts

AvNav Charts and Overlays

Introduction

Karten

To use charts in AvNav they have to be available in a tile format. This is a format used by services like OpenStreetMaps or GoogleMaps. A tile has (normally) a size of 256x256 pixel. The world is projected to a flat surface (imagine a paper cylinder wrapped around the earth at the aequator). Each point with it's coordinates (longitude/latitude) will pe projected at this cylinder. How this is performed in detail, which units will be used and whether the earth is considered as an ellipsoid or a sphere is described by the projection. AvNav is using the so called google mercator projection (earth considered a sphere) with the EPSG code 900913. The projected units are always meters (but can be converted to longitude and latitude). If you have charts using a different projection they have to be reprojected beforehand.

The whole projection area is split into tiles. The zoom level determines the number of tiles for the complete area. Zoom level 0 means:  complete earth (from -85°...+85°, outside these limits the projection is undefined) is shown in one tile of 256x256 pixel. With each other level there will be more tiles - level 1: 2x2 tiles, level 2: 4x4 tiles. The most relevant zoom levels for our usage typically range from 7 to 18..19. That means at level 19:2^19x2^19 tiles.
The openlayers library is used to display the charts. This library loads the tiles based on zoom level from the server (raspberry) and displays them on the screen. It is often used in OpenStreatMap applications.
You can easily imagine the huge numbers of tiles required for higher zoom levels (if you try e.g. zoom 19 for all). Therefore it makes sense to follow the same approach we take for paper charts: for an overview a small zoom level is used, detail charts a bit more and e.g. port approaches with the highest levels 18 or 19 (60cm/pixel and 30cm/pixel). To still maintain a nice display experience the different detail levels can overlay the other tiles of less detail. If there is a better (higher) zoomlevel for an area this one will be displayed - otherwise the one with a lower zoom level (potentially up scaled). To not overload the display devices, the number of layers should be limited to 3...5 (depending on the device).

For chart conversion the goal is to assign available charts to such layers, to potentially reproject them and finally create the chart tiles (maybe together with a description). Typically this requires a lot of computing power (and time) and should be better run on a desktop system. But typically this should be no problem as you will do this only once. After you will be rewarded with a continous chart view with no gaps inbetween.
The same has to be considered when it comes to downloading charts. You need to carefully select areas and zoom levels.

The download procedure depends on the source. I currently support 2 options to create charts for AvNav:

  • Converting charts using GDAL (like BSB) – you can use everything that GDAL can read

  • Downloading charts using the Mobile Atlas Creator.

Chart Formats

AvNav uses GEMF as its primary format. It is a compact binary format merging all tiles into one file with the necessary meta data added. This format permits special functions like automatic zoom level adaptation.  
Since version 20200325 AvNav can directly use the mbtiles format. To use this format you potentially have to select the correct variant of internal tile sorting. The default format is "xyz" but there is also a "tms" format available. You can switch the format at the Files/Download page. You can download MbTiles directly e.g. from OpenSeamap.

Hint: Up to version 202011xx AvNav was reading an entry "scheme" in the mbtiles metadata. Unfortunately the usage of this entry is not well defined and different sources are using differently (see Issue #63). In earlier editions AvNav internally inverted the meaning of xyz and tms. Since version 202011xx AvNav ignores this value (and assumes tms). If the value is set in the metadata, the chart will be displayed in red and you can visit the Download-Page to choose the type to be applied for this specific chart.

Additionally (since 20200325) you can also use online chart sources, provided they support the default url format. You have to configure this with an xml file. An example can be found in the provided online source for OpenSeaMap.

Since version 20200515 you can use a plugin to process charts from o-charts as well.

Since version 202011xx you can use charts from the signalK chart provider   (if the signalK integration is active).

With the MapProxy Plugin you can utilize and download charts from various online chart sources.

Installation of Charts

After installing AvNav there is one/a couple of online demo charts available. For real usage you have to install charts in AvNav.
You can do this either by directly copying charts into the chart directory (/home/pi/avnav/data/charts) or  to the external chart directory on android. You can also upload the charts directly within the WebApp (some only starting from 20200325). Upload can be executed via the Files/Download page.
For android you have to consider that mbtiles can only be handled after explicit upload  - instead of simply storing them in the external chart directory.
For mbtiles consider the option to switch the internal scheme at the Files/Download page .

In standard versions (not android) starting from 20200325 you can also upload charts that still require conversion. The converter as described in  Converting is always included in AvNav. It will read files from it's import directory (/home/pi/avnav/data/import) or from sub directories. When uploading a file with a known filename extension (like .kap) you will be asked if you want this file to be loaded to the converter import directory. You can monitor the working state of the importer at the status page. You should keep in mind that converting requires a lot of CPU power and can last for hours on the raspberry pi. Probably it is more efficient to run this on a desktop system.

If the MapProxy plugin is installed its charts will directly become visible in AvNav without any further installation.

Overlays

Since version 20201219 AvNav can display further information as overlay (or underlay). You can as well combine different charts.

For details see Overlays.

Converting Charts

As described you can run the conversion directly on the pi - but it will take time.

It is potentially more appropriate to run the conversion on a desktop computer. You can do this on Linux or Windows machines. For the necessary installations see:

The central part of the conversion will be handled by tile tools (https://code.google.com/p/tilers-tools/) - a copy is included in AvNav.
On a desktop the charts will be created at <UserHome>/AvNavCharts/out  (e.g. c:\Users\Andreas\AvNavCharts\out). In the UI you can select the output path. Normally one gemf file is created per run. The name is derived from the first selected input file or directory.
There will be a work directory within AvNavCharts - you should not delete this as it would speed up subsequent conversion runs (if you e.g. receive some corrective input charts).
The conversion consists of the steps:

  • sorting the charts into layers (and potentially converting them)

  • creation of tiles

  • creation of the gemf file

After creation of the gemf file just install it on the pi.
You can also test the created chart directly on your desktop. The converter will invoke the AvNav server to which you connect by browser.

Windows

There is a Net-Installer (AvNavNet...) for windows to install all the necessary software:

  • python 2.7.10 (32 Bit)
  • GDAL 1.11 for python 2.7 (32 Bit)
  • Pillow 2.7.0
  • pyserial 2.7
The installer will be installed within ProgramFiles\AvNav. If you uninstall AvNav just the installer will be removed.
All added components are installed in the user home diectory. They have to be uninstalled via the UI.
After installation you can start with AvChartConvert.

There is just one screen:

windows gui

Click (1) to select the files/directories to convert. Click (2) to start the conversion process. The "view" button will display the logfile as the conversion proceeds. That's something you certainly like to see if something goes wrong.
Per default the AvNav server will automatically start up and invoke a browser after the conversion has finished.
If the necessary software has not yet been installed,  a dialog is brought up to prompt if it should be installed right now. Clicking (5) you can repeat this step for an update. At (4) you can remove the downloaded software.
Some demo NMEA data are included in AvNav installation (southern baltic, Greifswalder Bodden), You may also use any other NMEA0183 file as input.
As the server on desktop computer is a fully featured AvNav you can also read NMEA data from connected devices, from bluetooth or IP. To meet special requirements you can even create your own avnav_server.xml and set up things there.

This way it is also possible to use AvNav e.g. on a laptop on board.
If you only want to start up the server - click (3) "startServer". Has to be repeated after every configuration change.

Linux

As described at Installation start up AvNav using avnav -g and select the charts you would like to convert.

Details

Here are some details about the conversion steps. Normally you do care about them. You only need them if some of the default settings do not fit your needs.

The first step is fast. All chart data will be read, resolution and range will be detected. If necessary charts would be converted/reprojected. As a result there will be a chartlist.xml in workdir/<name> . The command line would be:

read_charts.py -g -o name -m chartlist inputdir [inputdir...]

Afterwards you can check the chartlist.xml and you could e.g. move some charts into a different layer.

The second step will take more time -  now the tiles are created:

read_charts.py -g -o name -m generate inputdir [inputdir...]

The file chartlist.xml must already exist at  workdir/<name>. The creation will run multi threaded.

At the end you need to create the gemf file:

read_charts.py -g -o name -m gemf inputdir [inputdir...]

You could also combine all steps:

read_charts.py -g -m all [-o name] inputdir [inputdir...]

MapProxy plugin

A plugin is available to embed MapProxy,  providing access to various online chart sources. The plugin will support you in configuring MapProxy and allows to comfortably download charts. For details refer to the plugin documentation.

The plugin offers a lot of functionality very similar to Mobile Atlas Creator and it can replace it in most cases.

Download using Mobile Atlas Creator

To use the Mobile Atlas Creators you only need Java and the MOBAC itself. It is important to follow a certain procedure when selecting the ranges for download  in order to fit the charts to the described layer concept and to limit the amount of data.

Typically, I recommend to use 3 layers: overview( zoom level 7-10), navigation (level 10-15), details (Level 16-18). Afterwards just proceed in MOBAC layer by layer. Select the zoom levels for the layer, select the areas you want and add them with an arbitrary name. Repeat this for all layers. Now save your selection using a meaningful name (the xml file could be reused later). Select OsmDroid GEMF (File->convertAtlasFormat) as output format and start atlas creation. In the output directory a xxx.gemf file will be created. Install this on the pi.
At the mapsources page I present a collection of useful map sources.