This past week we released four new services in the Vexcel API. I’ll provide a brief description of each here, but be sure to click through into our full documentation for all of the details
Digital Terrain Model (DTM) This service provides access to our DTM via a convenient tiled interface, very much like our Digital Surface Model (DSM) which was released earlier this year. Requests are made by passing in the X/Y/Z tile location, and the response is a single 256×256 geotiff.
Coverage API’s When working with our imagery it is important to be able to query the catalog to find out what imagery and data products are available for a given location or even for an entire country. Our ImageLayerCoverage service has provided this capability for our discrete imagery (individual oblique and nadir images). This week we added three new services that work in the same manner, allowing you to query Orthomosaic, DSM, and DTM coverages.
The response for these API’s is JSON with metadata indicating attributes like the name, resolution and date range of each Area of Interest (AOI). the Geometry of each AOI in the coverage is also returned. In each case you can pass in a point or polygon in the optional WKT parameter to limit the response to just coverage over the specified location. If you omit this parameter, coverage will be returned for the entire layer allowing you to cache coverage for offline use.
Here are links to the documentation for each of these endpoints:
With our tile services its easy to add layers of imagery to applications that support WMTS and X/Y/Z tiles. In these applications, a ‘template’ URL is used with placeholders for the tile coordinates that are filled in by the application at runtime. A typical template url would look something like this one for Vexcel’s blue sky ultra layer:
In both cases, the host application will fill in the X, Y, and Z values dynamically to request the tiles needed to fill the map area. This is the most common use of tile services, but they can also be accessed programmatically to create maps by stitching a series of tiles together. Many map providers such as Google and Bing provide static map api’s that do this for you. In this article we’ll build a static map service that does exactly that.
Lets start by looking at the completed service, and trying it out. In its most basic form it will return a standard map image in PNG format but you can optionally pass in a polygon with the WKT parameter and have that polygon drawn on the map as an overlay or have the map cropped to that polygon. Lets look at examples of all three, then we’ll dive into the code to implement it.
Here is a simple request to the service along with the resulting stitched and cropped image. The required parameters are the latitude, longitude and zoom level of the desired map. layer is also a required parameter and should be one of the Vexcel imagery layers. width and height are optional and represent the size in pixels of the resulting bitmap. If not passed, these parameters default to 400 pixels. the authtoken parameter is also required and carries the standard authentication token generated by the login method of the Vexcel API.
Next we’ll pass in a polygon to be drawn over the stitched map image. The rest of the parameters remain the same. The polygon itself is passed in the wkt parameter, and surprise, should be in wkt format. The wktaction parameter specifies whether the polygon should be drawn or used for cropping. in this first case we set its value to ‘draw’.
And finally, in this third variant of the call we set the wktaction parameter to a value of ‘crop’ causing the resulting image to be cropped to include only the area within the polygon.
Alright, on to the code to make all of this happen. the main workflow is in the get() method. In it we:
Gather all of the parameters that were passed in and set defaults for any that were omitted.
Calculate the pixel dimension of the bounding box in global pixel coordinates that will fit the desired image
Calculate the tiles needed to cover that bounding box. the Northwest and southeast tile coordinates.
Request all of the tiles and stitch them into a single bitmap.
If a polygon was passed in the WKT parameter, draw it over the image or mask off the image as specified in the wktaction parameter
Crop the image to the precise pixel size requested in the width and height parameters
Return the bitmap as a PNG
All of the code below is written in C# and built to run as a web service. The portions related to threading and drawing rely on the .NET framework, but all of this should be easily adapted to any other language and execution environment.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.Threading;
using System.Web;
using System.Net;
using System.IO;
using System.Drawing;
namespace GICPropertyInfo.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class staticMapController : ControllerBase
{
TileImage[] tileImages;
public int tileSizeInPixels = 256;
public MemoryStream Get()
{
//Get URL parameters
double cntrLatitude = double.Parse(HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("latitude"));
double cntrLongitude = double.Parse(HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("longitude"));
string vexcelToken = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("authtoken");
if (vexcelToken==null || vexcelToken=="")
return new System.IO.MemoryStream(System.Text.Encoding.UTF8.GetBytes("authtoken is a required parameter."));
int zoom;
string tmpZoom = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("zoom");
if (tmpZoom == null || tmpZoom == "")
zoom = 21;
else
zoom = int.Parse(tmpZoom);
int widthInPixels;
string tmpWidth = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("width");
if (tmpWidth == null || tmpWidth == "")
widthInPixels = 400;
else
widthInPixels = int.Parse(tmpWidth);
int heightInPixels;
string tmpHeight = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("height");
if (tmpHeight == null || tmpHeight == "")
heightInPixels = 400;
else
heightInPixels = int.Parse(tmpHeight);
string layername = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("layer");
if (layername ==null || layername=="")
layername = "bluesky-ultra";
string aoiName = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("aoi");
if (aoiName == null || aoiName == "")
aoiName = "";
string wktpolygon = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("wkt");
if (wktpolygon == null || wktpolygon == "")
wktpolygon = "";
string wktAction = HttpUtility.ParseQueryString(Request.QueryString.ToString()).Get("wktaction");
if (wktAction == null || wktAction == "")
wktAction = "draw";
//Calculate the pixel dimension bounding box
TileSystem.LatLongToPixelXY(cntrLatitude, cntrLongitude, zoom, out int pixelX, out int pixelY);
int minPixelX = pixelX - (widthInPixels / 2);
int maxPixelX = pixelX + (widthInPixels / 2);
int minPixelY = pixelY - (heightInPixels / 2);
int maxPixelY = pixelY + (heightInPixels / 2);
//calculate the tiles needed to cover that bounding box
TileSystem.PixelXYToTileXY(minPixelX, minPixelY, out int minTileX, out int minTileY);
TileSystem.PixelXYToTileXY(maxPixelX, maxPixelY, out int maxTileX, out int maxTileY);
//Get the Pixel location of the Upper Left Tile
TileSystem.TileXYToPixelXY(minTileX, minTileY, out int PixelXorigin, out int PixelYorigin);
//build an image from all the tiles
Image imgUncropped = getPropertyTiles(minTileX, maxTileX- minTileX+1, minTileY, maxTileY - minTileY + 1,
zoom, vexcelToken, layername,aoiName);
//If a polygon was passed in, this is the place to apply it...
//Either Draw it...
if (wktpolygon!="" && wktAction=="draw")
{
List<Coordinate> polygonCoordsLatLong = UtilityFunctions.convertWKTPolygonToCoordinates(wktpolygon);
imgUncropped = renderPolygonToBitmap(polygonCoordsLatLong, PixelXorigin, PixelYorigin, zoom,
imgUncropped, Color.Red, Color.Yellow);
}
//or Crop with it
if (wktpolygon != "" && wktAction == "crop")
{
List<Coordinate> polygonCoordsLatLong = UtilityFunctions.convertWKTPolygonToCoordinates(wktpolygon);
//Convert the polygon's lat/long coordinates to pixel coordinates
PointF[] polygonPixels = new PointF[polygonCoordsLatLong.Count];
PointF thisPointPixel = new PointF();
for (int thiscoord = 0; thiscoord < (polygonCoordsLatLong.Count); thiscoord++)
{
TileSystem.LatLongToPixelXY(polygonCoordsLatLong[thiscoord].latitude, polygonCoordsLatLong[thiscoord].longitude, zoom, out int drawPix1X, out int drawPix1Y);
thisPointPixel.X = drawPix1X - PixelXorigin;
thisPointPixel.Y = drawPix1Y - PixelYorigin;
polygonPixels[thiscoord] = thisPointPixel;
}
//test each pixel in the bitmap and black out any pixel outside of the polygon
Bitmap bmpCropped = new Bitmap(imgUncropped.Width, imgUncropped.Height);
Bitmap gUncropped = (Bitmap)(imgUncropped);
for (int locrow = 0; locrow < imgUncropped.Height; locrow++)
{
for (int loccol = 0; loccol < imgUncropped.Width; loccol++)
{
thisPointPixel = new PointF(loccol, locrow);
bool inside = UtilityFunctions.IsPointInPolygon(polygonPixels, thisPointPixel);
if (inside)
{
bmpCropped.SetPixel(loccol, locrow, gUncropped.GetPixel(loccol, locrow));
} else
{
bmpCropped.SetPixel(loccol, locrow, Color.FromArgb(0, 0, 0));
}
}
}
imgUncropped = bmpCropped;
}
//crop the image down to the desired pixel width and height.
//IMPORTANT: offset it to crop the 'centered' area as expected
Rectangle cropRect = new Rectangle(minPixelX - (minTileX * tileSizeInPixels), minPixelY -(minTileY * tileSizeInPixels),
widthInPixels, heightInPixels);
Bitmap target = new Bitmap(cropRect.Width, cropRect.Height);
Bitmap src = (Bitmap)imgUncropped;
using (Graphics g = Graphics.FromImage(target))
{
g.DrawImage(src, new Rectangle(0, 0, target.Width, target.Height),
cropRect, GraphicsUnit.Pixel);
}
//Now return the reulting Cropped final bitmap as a PNG
MemoryStream memstream = new MemoryStream();
target.Save(memstream, System.Drawing.Imaging.ImageFormat.Png);
memstream.Seek(0, SeekOrigin.Begin);
Response.ContentType = "image/png";
return memstream;
}
//This function takes a List of lat/longs, converts them to pixel coords, and draws them over the map
private Image renderPolygonToBitmap(List<Coordinate> polygonCoordsLatLong,int PixelXorigin, int PixelYorigin ,int zoom,
Image bmp, Color edgeColor, Color vertexColor)
{
var g = Graphics.FromImage(bmp);
for (int thiscoord = 0; thiscoord < (polygonCoordsLatLong.Count - 1); thiscoord++)
{
TileSystem.LatLongToPixelXY(polygonCoordsLatLong[thiscoord].latitude, polygonCoordsLatLong[thiscoord].longitude, zoom, out int drawPix1X, out int drawPix1Y);
TileSystem.LatLongToPixelXY(polygonCoordsLatLong[thiscoord + 1].latitude, polygonCoordsLatLong[thiscoord + 1].longitude, zoom, out int drawPix2X, out int drawPix2Y);
g.DrawLine(new Pen(edgeColor, 2), new Point(drawPix1X - PixelXorigin, drawPix1Y - PixelYorigin), new Point(drawPix2X - PixelXorigin, drawPix2Y - PixelYorigin));
int radius = 3;
g.DrawEllipse(new Pen(vertexColor, 2), (drawPix1X - PixelXorigin) - radius, (drawPix1Y - PixelYorigin) - radius,
radius + radius, radius + radius);
}
return bmp;
}
//this function requests the needed tiles from vexcels tile services. for efficiency, each tile
//is requested on a separate thread to gather all the tiles in parrallel
private Image getPropertyTiles(int originXtile, int numXtiles, int originYtile,
int numYtiles, int originZtile, string VexcelToken, string layerName, string AOIname)
{
//Create a Bitmap for the RGB tiles
var bmp = new System.Drawing.Bitmap(tileSizeInPixels * numXtiles, tileSizeInPixels * numYtiles);
var g = Graphics.FromImage(bmp);
//we'll need a task per tile
int numerOftiles = numYtiles * numXtiles;
Task[] tasks = new Task[numerOftiles];
tileImages = new TileImage[numerOftiles];
int tileCnt = -1;
int colIndx = 0;
int rowIndx = 0;
//First, loop through each desired tile and create a TileImage object to be filled asynchronously
for (int row = originYtile; row < originYtile + numYtiles; row++)
{
colIndx = 0;
for (int col = originXtile; col < originXtile + numXtiles; col++)
{
tileCnt++;
TileImage thisTileImage = new TileImage();
thisTileImage.xIndx = colIndx;
thisTileImage.yindx = rowIndx;
thisTileImage.URL = "https://api.gic.org/images/GetOrthoImageTile/" + layerName + "/" + originZtile + "/" +
col + "/" + row + "/true?format=image%2Fjpeg&token=" + VexcelToken;
if (AOIname != "")
thisTileImage.URL += "&AOI=" + AOIname;
thisTileImage.requestID = "t" +tileCnt;
tileImages[tileCnt] = thisTileImage;
colIndx++;
}
rowIndx++;
}
//Now go fetch eeach tile asynchronously
tileCnt = 0;
DateTime startTime = DateTime.Now;
foreach (TileImage thisTI in tileImages)
{
tasks[tileCnt] = Task.Run(() =>
{
fetchATile(thisTI);
int tileindx = tileCnt;
});
tileCnt++;
}
//wait here for all threads to complete.
Task.WaitAll(tasks);
//loop again to stitch the tiles together into a single bitmap
colIndx = 0;
rowIndx = 0;
tileCnt = -1;
for (int row = originYtile; row < originYtile + numYtiles; row++)
{
colIndx = 0;
for (int col = originXtile; col < originXtile + numXtiles; col++)
{
tileCnt++;
g.DrawImage(tileImages[tileCnt].imgTile, tileSizeInPixels * colIndx, tileSizeInPixels * rowIndx);
colIndx++;
}
rowIndx++;
}
Image imageRGB = bmp;
return imageRGB;
}
private TileImage fetchATile(TileImage thisTileImage)
{
thisTileImage.imgTile = fetchImageFromURL(thisTileImage.URL);
return thisTileImage;
}
private Image fetchImageFromURL(string imageURL)
{
Image imageRGB =null;
if (imageURL == null || imageURL == "")
return imageRGB;
//fetch it
using (WebClient webClient = new WebClient())
{
DateTime startTime = DateTime.Now;
try
{
byte[] data = webClient.DownloadData(imageURL);
MemoryStream memstream = new MemoryStream(data);
imageRGB = Bitmap.FromStream(memstream);
}
catch (Exception ex)
{
imageRGB = null;
}
}
return imageRGB;
}
}
}
and here are two utility functions used by the code above
There are a number of conversion methods that were taken from this excellent article on Bing Maps tile system. FOr convenience, I’ve included these helper function here as well:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Text;
namespace GICPropertyInfo
{
static class TileSystem
{
private const double EarthRadius = 6378137;
private const double MinLatitude = -85.05112878;
private const double MaxLatitude = 85.05112878;
private const double MinLongitude = -180;
private const double MaxLongitude = 180;
/// <summary>
/// Clips a number to the specified minimum and maximum values.
/// </summary>
/// <param name="n">The number to clip.</param>
/// <param name="minValue">Minimum allowable value.</param>
/// <param name="maxValue">Maximum allowable value.</param>
/// <returns>The clipped value.</returns>
private static double Clip(double n, double minValue, double maxValue)
{
return Math.Min(Math.Max(n, minValue), maxValue);
}
/// <summary>
/// Determines the map width and height (in pixels) at a specified level
/// of detail.
/// </summary>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <returns>The map width and height in pixels.</returns>
public static uint MapSize(int levelOfDetail)
{
return (uint)256 << levelOfDetail;
}
/// <summary>
/// Determines the ground resolution (in meters per pixel) at a specified
/// latitude and level of detail.
/// </summary>
/// <param name="latitude">Latitude (in degrees) at which to measure the
/// ground resolution.</param>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <returns>The ground resolution, in meters per pixel.</returns>
public static double GroundResolution(double latitude, int levelOfDetail)
{
latitude = Clip(latitude, MinLatitude, MaxLatitude);
return Math.Cos(latitude * Math.PI / 180) * 2 * Math.PI * EarthRadius / MapSize(levelOfDetail);
}
/// <summary>
/// Determines the map scale at a specified latitude, level of detail,
/// and screen resolution.
/// </summary>
/// <param name="latitude">Latitude (in degrees) at which to measure the
/// map scale.</param>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <param name="screenDpi">Resolution of the screen, in dots per inch.</param>
/// <returns>The map scale, expressed as the denominator N of the ratio 1 : N.</returns>
public static double MapScale(double latitude, int levelOfDetail, int screenDpi)
{
return GroundResolution(latitude, levelOfDetail) * screenDpi / 0.0254;
}
/// <summary>
/// Converts a point from latitude/longitude WGS-84 coordinates (in degrees)
/// into pixel XY coordinates at a specified level of detail.
/// </summary>
/// <param name="latitude">Latitude of the point, in degrees.</param>
/// <param name="longitude">Longitude of the point, in degrees.</param>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <param name="pixelX">Output parameter receiving the X coordinate in pixels.</param>
/// <param name="pixelY">Output parameter receiving the Y coordinate in pixels.</param>
public static void LatLongToPixelXY(double latitude, double longitude, int levelOfDetail, out int pixelX, out int pixelY)
{
latitude = Clip(latitude, MinLatitude, MaxLatitude);
longitude = Clip(longitude, MinLongitude, MaxLongitude);
double x = (longitude + 180) / 360;
double sinLatitude = Math.Sin(latitude * Math.PI / 180);
double y = 0.5 - Math.Log((1 + sinLatitude) / (1 - sinLatitude)) / (4 * Math.PI);
uint mapSize = MapSize(levelOfDetail);
pixelX = (int)Clip(x * mapSize + 0.5, 0, mapSize - 1);
pixelY = (int)Clip(y * mapSize + 0.5, 0, mapSize - 1);
}
/// <summary>
/// Converts a pixel from pixel XY coordinates at a specified level of detail
/// into latitude/longitude WGS-84 coordinates (in degrees).
/// </summary>
/// <param name="pixelX">X coordinate of the point, in pixels.</param>
/// <param name="pixelY">Y coordinates of the point, in pixels.</param>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <param name="latitude">Output parameter receiving the latitude in degrees.</param>
/// <param name="longitude">Output parameter receiving the longitude in degrees.</param>
public static void PixelXYToLatLong(int pixelX, int pixelY, int levelOfDetail, out double latitude, out double longitude)
{
double mapSize = MapSize(levelOfDetail);
double x = (Clip(pixelX, 0, mapSize - 1) / mapSize) - 0.5;
double y = 0.5 - (Clip(pixelY, 0, mapSize - 1) / mapSize);
latitude = 90 - 360 * Math.Atan(Math.Exp(-y * 2 * Math.PI)) / Math.PI;
longitude = 360 * x;
}
/// <summary>
/// Converts pixel XY coordinates into tile XY coordinates of the tile containing
/// the specified pixel.
/// </summary>
/// <param name="pixelX">Pixel X coordinate.</param>
/// <param name="pixelY">Pixel Y coordinate.</param>
/// <param name="tileX">Output parameter receiving the tile X coordinate.</param>
/// <param name="tileY">Output parameter receiving the tile Y coordinate.</param>
public static void PixelXYToTileXY(int pixelX, int pixelY, out int tileX, out int tileY)
{
tileX = pixelX / 256;
tileY = pixelY / 256;
}
/// <summary>
/// Converts tile XY coordinates into pixel XY coordinates of the upper-left pixel
/// of the specified tile.
/// </summary>
/// <param name="tileX">Tile X coordinate.</param>
/// <param name="tileY">Tile Y coordinate.</param>
/// <param name="pixelX">Output parameter receiving the pixel X coordinate.</param>
/// <param name="pixelY">Output parameter receiving the pixel Y coordinate.</param>
public static void TileXYToPixelXY(int tileX, int tileY, out int pixelX, out int pixelY)
{
pixelX = tileX * 256;
pixelY = tileY * 256;
}
/// <summary>
/// Converts tile XY coordinates into a QuadKey at a specified level of detail.
/// </summary>
/// <param name="tileX">Tile X coordinate.</param>
/// <param name="tileY">Tile Y coordinate.</param>
/// <param name="levelOfDetail">Level of detail, from 1 (lowest detail)
/// to 23 (highest detail).</param>
/// <returns>A string containing the QuadKey.</returns>
public static string TileXYToQuadKey(int tileX, int tileY, int levelOfDetail)
{
StringBuilder quadKey = new StringBuilder();
for (int i = levelOfDetail; i > 0; i--)
{
char digit = '0';
int mask = 1 << (i - 1);
if ((tileX & mask) != 0)
{
digit++;
}
if ((tileY & mask) != 0)
{
digit++;
digit++;
}
quadKey.Append(digit);
}
return quadKey.ToString();
}
public static void MetersXYToLatLong(double x, double y, out double latitude, out double longitude)
{
latitude = 90 - 360 * Math.Atan(Math.Exp(-y / EarthRadius)) / Math.PI;
longitude = 360 * x / (EarthRadius * 2 * Math.PI);
}
/// <summary>
/// Converts a QuadKey into tile XY coordinates.
/// </summary>
/// <param name="quadKey">QuadKey of the tile.</param>
/// <param name="tileX">Output parameter receiving the tile X coordinate.</param>
/// <param name="tileY">Output parameter receiving the tile Y coordinate.</param>
/// <param name="levelOfDetail">Output parameter receiving the level of detail.</param>
public static void QuadKeyToTileXY(string quadKey, out int tileX, out int tileY, out int levelOfDetail)
{
tileX = tileY = 0;
levelOfDetail = quadKey.Length;
for (int i = levelOfDetail; i > 0; i--)
{
int mask = 1 << (i - 1);
switch (quadKey[levelOfDetail - i])
{
case '0':
break;
case '1':
tileX |= mask;
break;
case '2':
tileY |= mask;
break;
case '3':
tileX |= mask;
tileY |= mask;
break;
default:
throw new ArgumentException("Invalid QuadKey digit sequence.");
}
}
}
}
}
This week we launched a new API providing access to our high resolution Digital Surface Model (DSM) at 7.5cm! This data has been available to our customers over the past years via download of large GeoTiffs from our cloud storage, ideal for offline use. But now with the DSM API a whole world of applications opens up with on-demand access to the Surface Model on an individual property basis. With the surface model, you know the elevation in meters above the Earth’s surface of every pixel. Whether you simply need to know the height of an exhaust pipe on a building or you need to accurately measure the north facing facet of a roof, our DSM provides you with the necessary data. Detecting the footprint of a building, calculating the volume of the building, or identifying and measuring individual roof slopes and sizes are all straightforward operations with the our DSM.
And it couldn’t be any easier to use. The API returns standard 256X256 tiles in Geotiff format. As these tiles conform to the WMTS spec, they align perfectly with other Vexcel data like our True Orthomosaic tiles and Near-Infrared tiles, as well as tiled data from other providers in the industry. Here is an excellent overview on the standard tiling system from Microsoft, which also includes some helpful sample code for converting between lat/long and tile coordinates. If you know how to request an X/Y/Z tile from another provider, you already know how to request data from our DSM API. From street map tiles from Open Street Map to Land Parcels from Landgrid, our DSM aligns perfectly without any painful up/down sampling or transform of the data. and of course it aligns with our own high quality aerial imagery.
Here are two tiles at location Tile X = 359441, Tile Y = 836948, Zoom=21. The first tile is from the Vexcel Blue Sky Ultra Orthomosaic. The second image is a simple rendering of the SAME geography’s DSM. Note the clean alignment between them. the black areas are flat to the Earth’s Surface, with the brightest red areas representing the highest points. the Delta here is 11.87 meters from the ground to the flat roof. the elevation of EVERY pixel is easily read from the Geotiff.
Here is a series of tiles stitched together showing the orthomosaic and DSM together. use the slider to compare.
Getting started with the DSM API is easy. Begin with the documentation which you can access here. If all you need to do is display the DSM data in an application like QGIS that knows how to render DSM, simply download some tiles and drop them into a supported application. But if you want to access the elevation data programmatically in your own applications, you’ll likely want to use one of the many code libraries out there that making working with Tiff images (or the GeoTiff extension) quite easy. Here is a link to LibTiff, a very popular library for opening Tiff Files: http://libtiff.org/ And for .NET developers, here is a port of Libtiff for .NET: https://bitmiracle.com/libtiff/ LibTiff is so popular that many of the Geotiff libraries available rely on it under the hood. So you can’t go wrong by looking into LibTiff to speed your development process with our DSM.
Below is some sample code written in C# that shows how to open a DSM Geotiff tile and read the matrix of elevation values out of it. It relies on the aforementioned LibTiff libarary, but you should be able to adapt this code to any language and library of your choice. There are two important extended tags in the Geotiff that you will want to be aware of. Extended Tag 33550 contains the Pixelsize information. Tag 33922 contains the X/Y location on earth of the tile, in Meters. These tags are highlighted in the comments in the code below.
This should get you off to a good start working with the DSM API, but as always reach out to our support team with any questions. support@vexcelgroup.com In future articles here, we’ll build on the code below to do more interesting things like building footprint detection and 3D extrusion.
using System;
using BitMiracle.LibTiff.Classic;
using System.Net;
namespace DSMtileConsole
{
class Program
{
private const double EarthRadius = 6378137;
static void Main(string[] args)
{
string outputFolder = System.IO.Path.GetTempPath();
string VexcelToken = "YOURTOKENHERE";
//Request a DSM Geotiff tile from the service and save it as a local file
string filename = outputFolder + "TestDSMTile.tiff";
string imageURL_DSM = "https://api.gic.org/images/GetDSMTile/21/359441/836948/?layer=bluesky-ultra" +
"&token=" + VexcelToken;
fetchImageFromURL(imageURL_DSM, filename);
//Open and Parse the Geotiff file
using (Tiff tiff = Tiff.Open(filename, "r"))
{
if (tiff == null)
{
// Error - could not open
Console.WriteLine("ERROR: Couldn't open the file. ");
return;
}
Console.WriteLine("Input File: " + filename);
//Read the image Height and width from standard tags
int imageWidth = tiff.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int imageHeight = tiff.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
Console.WriteLine("Image width: " + imageWidth + " Image height: " + imageHeight);
//Parse the Pixel Width and Height. this information is contained in Extended Tag 33550
//These values are each 8 byte doubles. the first double is PixelsSizeX and the second is PixelsSizeY
byte[] destin = new byte[8];
FieldValue[] values = tiff.GetField((TiffTag)33550);
byte[] bytesGeo = values[1].ToByteArray();
Array.Copy(bytesGeo, 0, destin, 0, 8);
double PixelSizeX = BitConverter.ToDouble(destin, 0);
Array.Copy(bytesGeo, 8, destin, 0, 8);
double PixelSizeY = BitConverter.ToDouble(destin, 0);
Console.WriteLine("Pixel Size X: " + PixelSizeX + " Pixel Size Y: " + PixelSizeY);
//Parse the X and Y origin. this represents the coordinate of the upper left pixel of the geotiff
//these values are in meters and found in Extended Tag 33922
//These values are each 8 byte doubles. the 4th double is OffsetX and the fifth is OffsetY
values = tiff.GetField((TiffTag)33922);
bytesGeo = values[1].ToByteArray();
Array.Copy(bytesGeo, 24, destin, 0, 8);
double OriginMetersX = BitConverter.ToDouble(destin, 0);
Array.Copy(bytesGeo, 32, destin, 0, 8);
double OriginMetersY = BitConverter.ToDouble(destin, 0);
Console.WriteLine("X Origin: " + OriginMetersX + " Y Origin: " + OriginMetersY);
//Convert the X,Y from meters to decimal degrees
MetersXYToLatLong(OriginMetersX, OriginMetersY, out double originLatitude, out double originLongitude);
Console.WriteLine("Origin Longitude: " + originLongitude + " Origin Latitude: " + originLatitude);
//Now we can start reading the body of the geotiff containing the elevation data
//the body is organized as a series of 4 byte Singles
Console.WriteLine("Reading the body of the file...");
int stripSize = tiff.StripSize();
int stripMax = tiff.NumberOfStrips();
int bufferSize = stripMax * stripSize;
byte[] buf = new byte[bufferSize];
int result = tiff.ReadEncodedTile(0, buf, 0, stripSize);
//Scan the file. Report each pixel's Elevation and find the min/max elevation along the way
Single minElevation = 999999;
Single maxElevation = 0;
Single curEvelation = 0;
FieldValue[] value = tiff.GetField(TiffTag.IMAGELENGTH);
int imageLength = value[0].ToInt();
string outline = "";
for (int row = 0; row < imageLength; row++)
{
outline = "Row " + row.ToString().PadRight(4,' ');
for (int col = 0; col < (buf.Length / (imageWidth * 4)); col++)
{
int offset = (row * (imageWidth * 4)) + (col * 4);
byte[] bytes = { buf[offset], buf[offset + 1], buf[offset + 2], buf[offset + 3] };
curEvelation = BitConverter.ToSingle(bytes, 0);
minElevation = Math.Min(minElevation, curEvelation);
maxElevation = Math.Max(maxElevation, curEvelation);
outline += Truncate( curEvelation.ToString(),6).PadLeft(8, ' ');
}
//Console.WriteLine(outline);
}
Single elevationDelta = maxElevation - minElevation;
Console.WriteLine("Elevation range: " + minElevation + " to " + maxElevation);
Console.WriteLine("Elevation delta: " + elevationDelta );
}
}
public static void fetchImageFromURL(string imageURL, string outputFilePath)
{
using (WebClient webClient = new WebClient()) {
try {
webClient.DownloadFile(imageURL, outputFilePath);
}
catch (Exception ex)
{
}
}
return;
}
public static void MetersXYToLatLong(double x, double y, out double latitude, out double longitude)
{
//Adapted from https://docs.microsoft.com/en-us/bingmaps/articles/bing-maps-tile-system Thanks Joe!!
latitude = 90 - 360 * Math.Atan(Math.Exp(-y / EarthRadius)) / Math.PI;
longitude = 360 * x / (EarthRadius * 2 * Math.PI);
}
public static string Truncate(string source, int length)
{
if (source.Length > length)
{
source = source.Substring(0, length);
}
return source;
}
}
}
A question came in to support this week on how to select the best image for each day of collection. When we fly and collect aerial imagery, we will typically end up with many shots of a property, each just a second or 2 apart. Add in the fact that we have multiple images in each oblique orientation of north, south, east and west and you can see how we have on the order of 50 views of each property every time we fly over. All of these images can be very helpful in doing detailed analysis of a property, but sometimes you want to present just the best image on each day of flight in your application.
Here is an easy solution that takes advantage of the date filters on the ExtractImages endpoint. there are three steps in the process.
Query the catalog to get all of the metadata for all collections at your coordinate of interest
Parse the results to create a list of dates that have imagery
Make a follow up call to ExtractImages for EACH of the days found in step two and ask for the single best image on that day
Here is a sample app showing the three steps together. You need to plug in your Vexcel account credentials, then specify any lat/long coordinate and hit go! The app is all client-side JavaScript so just view source to see the API calls being made.
Using this coordinate (36.175216, -86.73649), lets look at the actual API calls for steps 1 and 3
Step 1. Find all Image metadata a the point
We’re going to focus on just the Nadir (vertical) imagery. Here is the API call to query the Vexcel catalog at this coordinate. in the response you’ll see we have 57 Nadir images for this property!
Step 2. Parse the Json from step one to find the dates with at least 1 image.
We want to build a list of days that have at least one image. The best way to do this is to sort the 57 images by capture date, then iterate through and build a list of unique dates. In this case we will end up with a list of 5 unique dates from September 9, 2012 to March 4, 2020
Step 3. Request the best image on each date found in step 2
For each of these calls, note the initDate and EndDate parameters. Also, note the mode=one which will return the best image in the date range. We can make a call to ExtractImages like this for each of the 5 dates we found in step 2. Here are a couple of them with their resulting image.
A nice new feature was added to the Vexcel API this week; the ability to crop an image to an arbitrary shape, masking off the rest of the image in black. This is very useful in computer vision applications where you don’t want neighboring pixels to interfere in the analysis. A couple of example use cases:
when looking for rooftop solar panels, you can crop the image to the building footprint or parcel boundary of the target property so that an adjacent home with solar panels will not interfere.
when assessing damage after a hurricane, cropping to the building footprint will prevent the debris field from the storm from coloring the analysis of the target structure.
Lets look at an example. This is a typical image with three homes in it.
If you want to limit the analysis to the building at the center, you can pass a polygon in well known text format as shown in this URL. Also, you’ll want to set the ‘crop’ parameter to ‘exact’.
Here is another example utilizing recent imagery from the Oregon wildfires. In this case, cropping to the building footprint prevents neighboring properties from adversely affecting the results of analysis to determine if the building has been destroyed or not. Here is the API call:
Here are a couple of additional helpful resources. Microsoft has made large sets of building footprints available for the United States and Canada. You can download them in convenient zip files here.
When passing in a polygon representing a land parcel or building footprint, you might want to buffer or ‘inflate’ the polygon before passing it in. This can be helpful as your geometry might not match up with our imagery perfectly; allowing a bit of fudge factor can insure the entire intended area is included in the cropped image. I’ve used the Clipper library for this and have found it convenient and easy to use. It can also be installed via the NuGet package manager. In the second example above, Clipper was used to inflate the footprint by 20%.
This video tutorial shows how to use the Vexcel API and Azure’s Custom Vision service to find damage after a fire or wind catastrophe across an arbitrary area. You can download the app from the previous post where we looked at how to analyze specific properties coming from a CSV file. The big difference here is that you don’t need a list of properties to analyze. instead you set the corners of a bounding box, and all of the properties within the box are analyzed. Microsoft’s building footprint files are used to find the candidate properties to test. In the image above, the white pins represent ALL of the building structures in the tested region. Each was run through the damage model with the structures determined to be destroyed showing as red pins.
Try it out and send your feedback along. you will need to have a Vexcel API account in order to run the app.
Over the last weeks, I shared a three part tutorial on training a model to identify homes damaged by wildfire, hurricane or tornado using Azure Cognitive services. In the final part we looked at how to use the Cognitive Services API to automate analysis of properties. Building on that, I’ve used the API to create an app to read a CSV file of coordinates, request the imagery from Vexcel API, and run each through the Cognitive services model. the results are written out to a CSV file that you can then do further analysis with, as well as a KML file to load into a GIS. You will need your Vexcel Platform account credentials to run the app.
You can download a zip file containing the app (runs on Any flavor of Windows) and a couple of sample CSV files to run through it. Unzip everything to a folder and launch the .exe to get started.
Choose Your Input File Use the … button to browse and select your input CSV. The first column should contain latitude and the second column, longitude. Additional fields can follow. A couple of sample files are included in the zip that you can use as a template. (I’ll cover the building footprints option in the next article.)
Set a name for your output file a CSV and KML file will be created in the same folder as your input file. Specify a name for the root of the file. The appropriate extensions will be appended on.
Choose the appropriate model and processing threads there are two choices for model; Wildfire and Wind Damage. choose the one appropriate for the type of damage you are analyzing. You can choose from 1 to 8 processing threads.
Finally enter your Vexcel Platform credentials and hit the Go! button to get things started.
While the app is running, have a look in the output folder. You’ll see some temporary files being created there of the form OutputFilename_thread#. To prevent IO conflict, each thread writes to it’s own files and when the run is finished, these files will be appended together to form your final output KML and CSV files. You can then delete these temp files if you wish.
After processing is complete, open up the .CSV file in Excel or a text editor. You’ll see three fields have been appended to your input file representing the prediction from the model, the date of the image used in the analysis and a link to the image itself. In the screenshot below you can see this in columns D, E, and F respectively. If you were running the wind damage model, the predictions will go from Damage0 (no visible damage) to Damage4 (complete loss). The score shown is the probability the prediction is correct. For Wildfire, there are just 2 possible tags; Fire0 (undamaged) and fire2 (burned).
Let’s look at row 4, which scored Damage4 with very high probability. the ImageURL in Column F returns this image:
You can load the CSV or KML file into just about any mapping application which will certainly have support for either CSV or KML files. Lets use the Vexcel viewer to visualize a file of Fire Damaged properties in Otis Oregon. After the run was complete, I loaded the CSV file into Excel, sorted by the AI_Probability column, then cut all of the damage2 records into 1 file, and damage0 into another. this allows me to load them as two sperate layers in the Vexcel viewer.
In this first image you can see all of the properties in the region that were run through the app.
And in these next two, you can see the properties tagged Damage2 in red.
and here zoomed in to the damaged cluster in the northeast
KNOWN APP ISSUES
After appending the individual KML file there is a stray character on the last line of the file that prevents it from loading in some client apps. if you encounter this, load the file in a text editor and delete the last line with the odd control character. I’ll fix this as soon as I figure out what’s causing it 🙂
Finally! In the first 2 parts of this tutorial series we focused on training models on Azure’s Custom Vision platform to perform recognition on Vexcel imagery. But now the best part; We’ll use the REST API exposed by Custom Vision to handle the repetitive task of running multiple properties through the model. This opens up use cases like analyzing a bunch of property records after a tornado or wildfire using Vexcel graysky imagery. Or checking to see which homes have a swimming pool using Vexcel Blue sky imagery.
In this tutorial we’ll use c# to call the Vexcel API and Custom Vision API, but you should be able to adapt this to any language or environment of your choosing. The application will make a call to the Vexcel platform to get an auth token, subsequent calls to generate an ortho image of a given property, then pass that image to our model on Custom Vision for recognition. Once you have this working, its easy to take it to the next step to open a CSV file containing a list of locations, and perform these steps for each record..
Step 1: Publish your model
In the previous tutorials we trained a model to recognize objects or damage in aerial imagery. We can now make programmatic calls to the model using the Custom Vision API, but first we need to publish the trained iteration, making it accessible by the API.
This is easy to do. In the Custom vision Dashboard, go to the Performance tab, select your iteration, and hit the ‘publish’ button as highlighted here.
Once the publish is complete, the ‘Prediction URL’ link will become active. Click it to view the parameters for your model that you will need when making calls with the API. The ‘Iteration ID’ is shown on the main dashboard page. The prediction key is visible in the dialog that pops up, as well as the REST URL which will contain the project ID. Take note of all of these values. We’ll use them in a moment.
Step 2: Generate an Authentication token with the Vexcel API
Each API call to the Vexcel platform requires and auth token to be passed. When your app starts up, you can call the login service to generate one and use it for all subsequent calls. An auth token is good for up to 12 hours.
The fetchURL() method is used to make an HTTP request and return the response as a string. Here is a simple implementation for C#.
string html = "";
try
{
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
using (StreamReader sr = new StreamReader(data))
{
html = sr.ReadToEnd();
}
}
catch (Exception ex)
{
//handle the error here
}
return html;
Step 3: Generate a URL to request an image of a property
There’s generally two steps to request an image from the Vexcel library; first query the catalog to see what is available, then request the appropriate image. Lets do exactly that for this coordinate damaged in the Oregon wild fires recently: 45.014910, -123.93089
We’ll start with a call to FindImages(). This service will return a JSON response telling us about the best image that matches our query parameters. Those parameters include the coordinate, a list of layers to query against, and the orientation of the image we want returned. For the layer list we are passing in Vexcel’s two gray sky layers; we want the best (most recent) image across any catastrophe response layer. We’ll set orientation to Nadir as we want a traditional vertical image, but you can also query for Vexcel’s oblique imagery with this parameter.
In the Json response, we’ll have all of the information we need to request a snippet of imagery with the ExtractImages() method. This workhorse provides you access to all of the pixels that make up the Vexcel library, one snippet at a time carved up to your exact specification. As you can see below in the code, the first bit of metadata that we’ll grab is the date the image was taken. This is one of the most important pieces of metadata regardless of what kind of application you are building; you’ll always want to know the date of the image being used. And then most importantly, we’ll form a URL to the ExtractImages endpoint with all of the parameters needed to get the image we need, as provided by the FindImage() call above.
Step 4: Pass the image to Custom vision for analysis
Its finally time to pass the Image snippet to Custom Vision for recognition. You’ll need the details from step 1 above where you published your model. You can return to the Custom Vision dashboard to get them. Here is the c# to make the API call and get back a JSON response indicating what tags were found in the image.
The last bit of code is to parse the returned JSON to find the tags discovered in the image. Keep in mind that there can be multiple tags each with their own probability score returned. We’ll keep it simple and loop through each tag looking for the highest probability, but in your implementation you could choose to be more precise than this, perhaps by considering the position of each discovered tag relative to the center of the image.
That’s it! Now that you can programmatically analyze a single image, its a small step to put a loop together to step through a large table of properties. In a future tutorial here on the Groundtruth, we’ll do something similar building on the code above to create a highly useful application.
In Part One of this three part tutorial, we trained a model using Azure’s Custom Vision platform to identify Solar Panels on rooftops using Vexcel’s Blue sky imagery. Here in part two we are going to work with disaster response imagery (aka graysky imagery) to identify buildings damaged in Wildfires.
The main difference here is that we will train the model on two tags, one representing buildings that have not been burned, and a second tag representing buildings that have been destroyed in the fire. Other than that, the steps are identical to what we did in Part One.
In this image you can see a good example of both tags that we will be training.
Step 1: Create a Custom Vision Account
If you completed part one of the tutorial, you’ve already setup your Custom Vision account and can proceed to step two below. But if you have not set up you Custom Vision account yet, Go back to part 1 of the previous tutorial to complete step 1 (account setup) then return here.
Step 2: Collect a bunch of images for tagging
You’ll need to have 15 or more images of buildings NOT damaged by fire and 15 showing damaged buildings. Its important that both sets of images are pulled from the graysky data.
Create a folder on your local PC to save these images too. There are several ways you can create your images. One easy way is to use the GIC web application, browse the library in areas where there is wildfire imagery, then take a screen grab and save it to your new folder. Here are some coordinates to search for that will take you to areas with good wildfire coverage:
42.272824, -122.813898 Medford/Phoenix Oregon fires 47.227105, -117.471557 Malden, Washington fires
Here are two good representative images similar to what you are looking for. First an example of a destroyed building
and an example of a structure still standing after the fire:
When you have 15 or more good examples of each in your folder, move on to the next step. Its time to tag our images!
Step 3: Create a new project in your Custom vision account
Click the ‘New project’ button and fill in the form like this:
If this is your first project you’ll need to hit the ‘create new’ link for the Resource section. Otherwise you can select an existing Resource. Hit the ‘Create project’ button to complete the creation process. You now have a new empty project. In the next step we’ll import the images we created previously and tag them.
Step 4: Upload and Tag these images.
Your new empty project should look something like this:
Hit the ‘Add Images’ button and import all of the images you saved earlier. You should see all of your untagged images in the interface like this:
Click on the first image of an undamaged property to begin the tagging process. Drag a box around the building structure. Its OK to leave a little buffer, but try to be as tight as possible to the building footprint. Enter a new tag name like ‘firenotdamaged’ And hit enter. If your image contains more than one structure, you can tag more than one per image.
Next choose an image with a building destroyed by fire and tag it in the same manner giving it a descriptive tag name like ‘firedamaged’.
Continue to click through all of your images and tag them. Some images might have a mix of burned and not burned structures. That’s OK, just tag them all appropriately.
Step 5: Train the model
If you click the ‘tagged’ button as highlighted below, you will see all the images you have tagged. You can click on any of them to edit the tags if needed. But if you are happy with your tags, its time to train your model!
Hit the ‘Train’ button and select ‘quick training’ as your training type. Hit the ‘Train’ button to then kick off the training process. This will take around 5 minutes to complete, depending on how many images you have tagged.
Step 6: Test the model
When training completes, your screen will look something like this:
Its time to test your model! The easiest way to do so is with the ‘quick test’ button as highlighted above. Using one of the techniques used in step 2 to gather your images, go and grab a couple more and save them to the same folder. Grab a mix of buildings, some destroyed and some not.
Hit the ‘Quick test’ link, and browse to select one of your new images. Here I selected an image that contained two adjacent structures, one destroyed and one not. You can see that both were correctly identified, although the probability on the burned building is a little low. this can be improved with tagging more images and retraining the model.
In Part three of this tutorial, we’ll use the API exposed by the Custom Vision Platform to build an app that can iterate through a list of properties and score each one
At Vexcel, we collect and process our aerial imagery with an eye towards much more than just traditional visual inspection scenarios. Our Ultracam line of camera systems are engineered from the ground up (punny!) with precise photogrammetry and computer vision applications in mind. Until recently it took a room full of data scientists and lot’s of custom application development to tap into the power of AI analysis over imagery, but today off the shelf tools on Amazon’s AWS platform and Microsoft’s Azure have democratized this technology making it available and easy to use by anyone.
In this multipart tutorial we’ll look at how easy it is to use aerial imagery in your own computer vision systems built on Azure’s Custom Vision platform. Custom Vision provides a web application for image tagging and training your model, as well as a simple REST api to integrate your model into any kind of application. And it couldn’t be easier! You’ll have your first computer vision system working end to end in just a few hours with part one of this tutorial. Stick around for all three parts and this is what we’ll cover
Part 1. Train a model that works with Vexcel’s Blue Sky Ultra-high resolution imagery to detect solar panels on Rooftops.
Part 2.Train a model utilizing Vexcel Gray Sky (disaster response) imagery to detect Fire damage after wildfires. Or you could choose to focus on Wind Damage after a tornado or hurricane.
Part 3. Classify gray sky images using the Custom Vision REST api. We’ll build an app to iterate through a list of properties from a CSV file, classify each one based on the wind damage level, and save the results to a KML file for display in any mapping application
Part 1: Solar Panel detection in Blue Sky Imagery
This tutorial will show you how to Utilize GIC Aerial Imagery to detect objects like solar panels or swimming pools using AI. We’ll build a system to detect the presence of solar panels on a roof, but you can easily customize to add other objects you would like to detect.
We’ll be using Microsoft’s Custom Vision service, which runs on the Azure platform. If you already have an Azure account, you can use it in this tutorial. If not, we’ll look at how to create one along the way. Keep in mind that although there is no charge to get started with Azure and Custom Vision, during Azure signup a credit card is required.
At the end of this section of the tutorial, you’ll have a trained model in the cloud that you can programmatically pass an image to and get back a JSON response indicating if a Solar panel was found in the image along with the probability score.
Hit the ‘Sign in’ button. You can then either sign in with an existing Microsoft account, or create a new one. If you sign in with a Microsoft account already connected to Azure, you won’t need to create an Azure account.
If there isn’t an Azure subscription attached to the Microsoft account you are using, you’ll see a dialog like the one shown here. Click ‘sign up for azure’ and follow the steps to create your azure account.
You should see something like the image shown here. Great! You got all of the account creation housekeeping out of the way, now on to the fun stuff!
Step 2: Collect a bunch of images showing solar panels
In this step, we’ll collect images pulled from the Vexcel image library that feature homes with solar panels on the roof. These images will be used in step 3 to train the AI model.
TIP: You need a minimum of 15 images to get started. More images will yield better results, but you can start with 15 and add more later if you like. As you collect them, try to pull a sample from different geographic regions. Rooftops in Phoenix are very different than those in Boston; try to provide diversity in your source images to ensure that the resulting model will work well in different regions.
Create a folder on your local PC to save these images too. There are several ways you can create your images of rooftops with Solar panels. One easy way is to use the GIC web application, browse the library looking for solar panels, then take a screen grab and save it to your new folder.
Here is an address to try this on: 11380 Florindo Rd, San Diego, CA 92127
Use a screen clipping tool to grab an image and save it to your folder. It should look something like this:
When you have 15 or more good examples of rooftops with solar panels in your folder, move on to the next step. Its time to tag our images!
Step 3: Create a new project in your Custom vision account
Click the ‘New project’ button and fill in the form like this:
For Resource, hit the ‘Create new’ link.
Your ‘New project’ form will ultimately look something like this:
Hit the ‘Create project’ button to complete the creation process. You now have a new empty project. In the next step we’ll import the images we created previously and tag them.
Step 4: Upload and Tag these images.
Your new empty project should look something like this:
Hit the ‘Add Images’ button and import all of the images you saved earlier. You should see all of your untagged images in the interface like this:
Click on the first one to begin the tagging process. Drag a box around the area of the image with solar panels, enter a new tag name of ‘solarpanel’ and hit enter.
You’ve tagged your first solar panel! Continue tagging each of the remaining images, one at a time until you have no untagged images remaining.
Step 5: Train the model
If you click the ‘tagged’ button as highlighted below, you will see all the images you have tagged. You can click on any of them to edit the tags if needed. But if you are happy with your tags, its time to train your model!
Hit the ‘Train’ button and select ‘quick training’ as your training type. Hit the ‘Train’ button to then kick off the training process. This will take around 5 minutes to complete, depending on how many images you have tagged.
Step 6: Test the model
When training completes, your screen will look something like this:
Its time to test your model! The easiest way to do so is with the ‘quick test’ button as highlighted above. Using one of the techniques used in step 2 to gather your images, go and grab a couple more and save them to the same folder. Grab some images of rooftops with solar panels of course, but also save a few that don’t have panels on the roof.
Hit the ‘Quick test’ link, and browse to select one of your new images.
As you can see here, the new model identified the correct location of the solar panels with a 71% confidence. Adding more images and running the training again will improve this. and you can go back to step 4 and do this anytime.
very cool! you just taught a machine how to identify solar panels on a roof. You can not only tag more solar panels, but you can add new tags for other entities you want to recognize in aerial imagery… Pools, trampolines, tennis courts…
In Part 2 of this tutorial series, we’ll use the same technique to operate on our disaster response imagery to identify differing levels of damage after wind events. I’ll add a link here as soon as Part 2 is online.
In Part 3, we’ll start to access the model’s we trained using the REST Api. But if you’d like to get a headstart on that and try the API out, here is a good tutorial on the custom vision website. You’ll find all of the access details you need to integrate in your app on the ‘Prediction URL’ dialog on the performance tab: