Today, we’re very happy to announce that we’re open sourcing the results from Thumbtack’s monthly Economic Sentiment Survey (ESS) series. The ESS captures the attitudes and perspectives of thousands of business owners across the country to gauge how they feel about the economy and their businesses. Now in its fifth year, this survey provides a unique vantage point on the economy, as respondents are largely mobile service professionals with five or fewer employees who operate in households across the United States. Because they are hard to reach, these professionals are frequently overlooked in other surveys of small businesses.
When we built our first machine learned model in Spark to predict pros’ interests in customer requests, we loaded pros’ historical engagement features directly into an online Golang application, which used the features to determine which pros are potentially interested in a given request. This approach had a few limitations:
- Other applications could not reuse the model.
- The feature loading process was not scalable due to the ever-increasing number of features.
- The interoperability between Scala (for offline Spark jobs) and Golang (for online applications) led to a substantial amount of duplicated code for common data models,
The data infrastructure team at Thumbtack has just completed the process of migrating all of our production Spark jobs from Spark 1.6.1 to Spark 2.0.0. We deployed Spark 2 and all of our updated jobs to production after a few weeks of testing in our development environment.
As we went through the Spark 2 migration process, we realized that many of the issues we encountered were not well documented, but had relatively straightforward solutions. In this post, we wanted to share some of the common problems we found during the migration, and the solutions that we identified.
By: Marco Zappacosta
Working with so many curious, ambitious, and kind humans is one of the best things (if not the best thing) about being at Thumbtack. This month, I’ve had the privilege of welcoming another such person to the team: Noam Lovinsky as our new VP of Product. We’ve got a long road ahead still, and I couldn’t be more excited to have him leading the charge.
In a lot of ways, we’re entering a new era of Thumbtack. We’ve always been focused on our vision — we want to be the destination for getting anything done.
When we first developed our mobile apps in 2014, we decided to use a third-party service to deliver push notifications. It was fast, easy and at the time we did not have enough engineers to implement sending push notifications ourselves. However, as we have grown, both in number of engineers and in mobile app users, that company no longer gives us the level of control, data analysis, and stability that we desire. (Plus, it charges us per push notification.) Therefore, we decided to implement our own way of sending push notifications.