Scala studies, Part 1: Seattle engineers describe their adventures in functional programming

by Quinten Dol
March 25, 2019

Originally designed in Switzerland, Scala was released to the public in 2004 to address Java’s famous verbosity, its focus on object-oriented programming and other issues. Since then, the language has grown out of its status as a niche skill favored by hobbyists and is now used by startups large and small to build a wide range of tools and services.

With a number of local tech companies looking for engineers with Scala skills, we decided to investigate what prompts them to choose functional programming over more traditional object-oriented languages. In part one of a two-part series, we spoke to techies at four Seattle tech startups to learn why they have adopted functional programming, and the challenges and benefits associated with it.

 

esentire seattle tech scala coding
photo via esentire

By combining artificial intelligence technology with professional experience, eSentire builds cybersecurity solutions designed to hunt for, investigate and respond to threats in real time. The company opened its Seattle office after acquiring local startup Versive, which used artificial intelligence to search for intruders within a system.

Senior Software Developer Tim Steinbach said the company has used Scala to build big data pipelines at the core of their product.  

 

What have you built using Scala, and what criteria do you use when determining whether you’ll use it in any given circumstance?

When it comes to deploying high-quality, dependable software, functional programming in Scala has proven itself to be invaluable in our environment. We have access to the wealth of the JVM ecosystem while using a high-velocity, high-quality programming language. There are very few ecosystems in the functional programming community that provide us with the tools and libraries required for our day-to-day business.

During the entire time that eSentire has had Scala applications in production, we have not had any major outage of said applications.”

 

What benefits and challenges does your team see in using a functional language like Scala?

The entry barrier into functional programming as a whole is still surprisingly high. Most developers have had very limited exposure to proper functional programming and are hesitant to explore its benefits. We have been lucky to have a group of developers and stakeholders that were excited about the journey, and convinced of the eventual benefits.

During the entire time that eSentire has had Scala applications in production, we have not had any major outage of said applications — and we largely attribute this to the transparency and “code cleanliness” of principled functional programming. The Scala community has created a rich set of tooling to ensure adherence to proper conventions and quality guidelines.

 

How can larger organizations improve their adoption of functional languages?

All of our projects make use of an internal SBT plugin, which propagates code conventions, test coverage goals, linting rules and much more. This has allowed developers to get automated feedback as soon as they write code on whether the code is formatted properly, or if there’s proper test coverage.

The biggest benefit, however, is the mental model that functional programming brings. Once understood, the benefits of a simpler, constrained, easy-to-follow programming paradigm will make developers more productive. Here at eSentire, we have accepted that it is quicker to produce a quick script in Python or Bash. But with principled Scala, the maintenance cost and confidence that the application can withstand error cases, invalid data and high throughput has certainly paid off.

 

nortal seattle tech startup kirkland dev9
photo via dev9

Estonian software company Nortal builds a wide range of tools and services for enterprise and mid-market clients. The company arrived in the Pacific Northwest with its acquisition of Kirkland-based Dev9 last year, and maintains that office to focus on building custom cloud and software products.

Because Nortal works with a variety of customers, Solutions Architect Nathan Skone said his company must consider the practicality of adopting Scala-based tools within a client organization.

 

What have you built using Scala, and what criteria do you use when determining whether you’ll use it in any given circumstance?

We have built some really cool Apache Spark data pipelines that use machine learning algorithms to remove human bottlenecks from our clients’ business processes. As consultants, we consider ease-of-adoption and our clients’ skill sets when we are choosing a language to use. Scala is often a perfect fit for our big data projects.

Functional concepts such as immutability, pure functions and separating program data from behavior help us reason about data transformations and ensure data correctness.”

 

What benefits and challenges does your team see in using a functional language like Scala?

Functional languages like Scala fit big data workflows very well. Functional concepts such as immutability, pure functions and separating program data from behavior help us reason about data transformations and ensure data correctness. One of the main challenges we have faced is training developers that have been taught and raised in an object-oriented programming world.

 

How can larger organizations improve their adoption of functional languages?

It is critical to focus on more than just language syntax when teaching a functional language like Scala to developers. Developers can end up trying to apply their OOP techniques instead of taking advantage of the functional benefits of the language. For this reason, we are careful to teach general functional concepts and skills along with the specific language syntax.

 

curalate seattle tech engineers scala
photo via curalate

Curalate helps businesses with their marketing efforts through building and engaging social media followers, stimulating the production of user-generated content, driving traffic to websites and helping to grow influencer networks. The company is headquartered in Philadelphia, and has offices in New York, London and downtown Seattle.

Director of Engineering Mike Cook says his team enjoys the conciseness of coding in Scala.

 

What have you built using Scala, and what criteria do you use when determining whether you’ll use it in any given circumstance?

We use Scala for everything server-side at Curalate, including building and deploying hundreds of microservices. These range from very high-scale stream listeners responsible for processing social media posts or web metrics events (at over one billion per week) in real time, down to infrequently occurring event triggers where we build small serverless Scala apps triggered by AWS lambdas.

 

What benefits and challenges does your team see in using a functional language like Scala?

Our team loves how concise and compact code written in Scala can be. More concise code means that it’s easier for developers to see the ideas in the code in their entirety more often. Concepts that would otherwise need to be pulled into a separate method can often be articulated by a single line of functional code. Scala’s more powerful language features and elegant syntax make it easier to spot defects in our code since there’s less context to read within.

On the challenges front, it’s a lot less often that we come across software engineering candidates with Scala or FP experience than the more common imperative programming experience.

 

A good set of core libraries and service scaffolds has made for a fantastic developer experience here at Curalate.”

 

How can larger organizations improve their adoption of functional languages?

Invest in infrastructure to support your team. A good set of core libraries and service scaffolds has made for a fantastic developer experience here at Curalate. This drums up enthusiasm within the team and encourages further learning and adoption of the language and its concepts.

 

porch seattle tech startup
photo via porch

Porch’s simple interface belies complex technology that assesses the needs of homeowners based on a simple questionnaire, then connects them with service providers that can repair, maintain or add to their home. Those services include everything from garden mulching and lawn mowing to ceiling fan installation and deck repairs.

Staff Engineer Jim McCabe said Porch uses Scala for many of its back end services.

 

What have you built using Scala, and what criteria do you use when determining whether you’ll use it in any given circumstance?

Some of our backend Scala services include our communication platform, which manages email, SMS and web-based chat; our real-time phone call routing platform; our event scheduling system; our file storage platform; our partner-facing API layer; and our project service, which manages the lifecycle of a project between a homeowner and pros.

We love Scala as a general purpose back end language. The main criteria is, “is this a backend service? Great, let’s use Scala instead of Java.”

 

What benefits and challenges does your team see in using a functional language like Scala?

The benefits of Scala accumulate over time, as it slowly changes the patterns we use to solve problems. Our first early steps were mostly just trying to avoid clumsy Java syntax, but gradually we started to appreciate other things. At this point, I would say that we’re a good way down the path but not all the way. Our Scala code is asynchronous, uses immutable data structures with a declarative style instead of imperative, and with functional error handling instead of throwing exceptions.

The big challenge is that most new hires don’t have any background in this stuff, and there’s a steep learning curve. We organically learned these things over a few years, but a new hire has to dive into the deep end all at once. It can take a few weeks or even months until they feel totally comfortable.

 

A microservice architecture really goes a long way, because a team can experiment with Scala in a tiny corner of the system with little risk... After a while, people come to trust it and the excitement spreads.”

 

How can larger organizations improve their adoption of functional languages?

This happened organically at Porch, where a few early adopters were able to get a skeletal system running. The enthusiasm spreads before long, and as our skills grew we also fine tuned that platform.

Code reviews also play a big role here at Porch, and that continues to be a big factor because it is a natural place to teach and learn and grow. We also have a Scala chat room where people ask questions throughout the day. If a question can’t be answered in that forum, people frequently pair up to solve the problem together.

 

Jobs from companies in this blog

Seattle startup guides

LOCAL GUIDE
Best Companies to Work for in Seattle
LOCAL GUIDE
Coolest Tech Offices in Seattle
LOCAL GUIDE
Best Benefits at Seattle Tech Companies
LOCAL GUIDE
Women in Seattle Tech