With tools designed to turn anyone into a data scientist, Tableau Software is all about data. The company’s signature product pulls information from databases, spreadsheets, cloud-based data warehouses and servers, and visualizes the data in graphs, maps, tables and more. Even the company’s ticker symbol on the New York Stock Exchange — Tableau went public in 2013 — is DATA.
In any other town, Tableau — with its elegant headquarters stretched along the northern shores of Lake Union, four acquisitions under its belt and thousands of employees all over the world — would be a central focus of the local tech scene. But in Seattle, the wider community’s attention remains focused on Amazon and Microsoft, so Tableau tends to fly under the radar.
To learn more about this tech giant living amid tech titans, we spoke with team members who worked on a pair of recently-completed projects rolled out in March as part of the latest version of Tableau’s signature software. The result is a window into the problems the company is working on, the challenges they encountered along the way and the technology they use to solve them.
Ask Data
Put simply, Tableau’s Ask Data function acts as an intermediary between a user and a dataset. The company allows users to pose questions they need answered to Ask Data, which then organizes and presents the data as a graph or map. The idea is to get data-driven answers fast, without needing to understand the underlying structure or pesky integration issues.
To let anyone — not just data scientists — find data-driven answers to vexing business problems, Development Manager Vidya Setlur said her team had to go deep on natural language processing technology.
What new functionalities did you add to Ask Data, and what kinds of technology did you use in the process?
Ask Data supports natural language interaction between people and their data sources. The system has an underlying parser and grammar where natural language input utterances are translated into Tableau’s visual query language — known as VizQL — in order to generate interactive visualizations. The translation of natural language input into VizQL is performed using an intermediate query language called Arklang that was developed in-house. Ask Data supports five analytical expression types – grouping data into partitions, aggregations, filtering, limiting to a subset of data rows and sorting.
We needed to develop techniques to parse ambiguous input and fill in any missing information.”
What challenges did you encounter during this project, and how did you overcome them?
One of the challenges for any natural language system is underspecified and ambiguous input utterances. People have certain mental models of their intent when using such systems and we needed to develop techniques to parse ambiguous input and fill in any missing information. For example, a user could say “most expensive products.” The system needs to identify that the concept of “expensive” is related to currency and find an attribute in the data source, such as “sales” to resolve the concept to.
Another challenge is that our Tableau customers come from various industries and backgrounds and the data sources vary in scale, characteristics and semantics. We needed to develop infrastructure so that Ask Data was robust both in performance and quality to handle the high degree of versatility.
Vector Maps
Tableau’s Vector Maps product allows users to visualize their data through maps. To boost the product’s functionality, the company added a range of background map styles — including streets and detailed satellite imagery — and map layers with handy features like building footprints and topographical lines. The team also used this update to introduce Vector Tiles, a feature that allows for a smoother mapping user interface.
To learn more, we had Product Manager Ashwin Kumar fill us in on how they pulled it off.
What kinds of technology did you use in the process?
On the back end, we’ve completely changed the technology that powers our map service. Our legacy map service used raster tiles, which work by asking for individual square images that each make up a small part of the background map, and then stitching all of these pictures together to make a single map, like a puzzle.
Our new service leverages vector tile technology, which means that the drawing instructions for what should appear on each map tile is actually contained within each map tile. These tiles then get drawn on the client machine, resulting in significantly improved map performance and interactivity that gets better as the user’s hardware improves. The experience that our end users will see is akin to the experience you get when you zoom and pan on a mapping application like Google Maps.
We worked closely with the development team at Mapbox to bring vector tile technology to Tableau. Specifically, we made great use of the Mapbox vector tile specification, Mapbox GL JS, Mapbox GL Native and several of Mapbox’s Maps SDKs and APIs to make this feature work. Our team also dug deep into web service technology, standing up services in AWS to enable core feature functionality and monitoring success after rollout.
We were suddenly faced with a far more complex background map dataset, having to contend with detailed cities, building footprints, postcodes and points-of-interest datasets.”
What challenges did you encounter during this project, and how did you overcome them?
A project of this size and complexity is definitely not without its share of challenges. I’ll name a few here:
Technology. Our development team had to ramp up quickly on cutting-edge vector tile technology and libraries, and understand the impact of this technology across different platforms: desktop, server, Linux, Mac — this feature impacted them all. When you’re dealing with a codebase as complex and large as Tableau’s, it takes a sharp attention to detail and a team of talented engineers to integrate external technology properly.
Quality. Our team members also built load-testing environments and telemetry infrastructure from scratch, to give ourselves the confidence that the new maps could handle anything our users could throw at it.
Cartography. One of the key differences between vector tile and raster tile technologies is that with vector tiles, map features can be loaded continuously at any zoom level, whereas with raster tiles we were restricted to quarter-zoom increments. This had major ramifications on the cartography of our ‘Light,’ ‘Dark’ and ‘Normal’ map styles, which we had to redesign from the ground up to ensure that every background map feature rendered appropriately for the given zoom level. In addition, we were suddenly faced with a far more complex background map dataset, having to contend with detailed cities, building footprints, postcodes and points-of-interest datasets, which all required specific instructions on styling and appearance.
Finally, performance was a key consideration. With so much data packed into each tile, we were suddenly downloading larger tiles than previously. We had to make intentional and subtle adjustments to what datasets we included in each tile, experiment with progressive map loading techniques and make use of the expressions feature in Mapbox Studio to ensure consistent, aesthetic and performant map cartography.
