For as long as large datasets have existed, people have been trying to figure out how to accurately detect their anomalies. ‘Normal’ data points are often less interesting, and typically don’t have the same financial impact as their abnormal brethren…
Before we get into things, let me start by saying that this isn’t going to be about authorization or any of the many, many mechanisms for authenticating a user and authorizing that user to access your system. While those mechanisms…
This month’s dev blog comes from one of our senior software engineers, Landon Robinson, our very own Hadoopster. Check out his site for more great Hadoop tutorials and tips. I’ve been working with Hadoop, Map-Reduce, and other “scalable” frameworks for…
Multi-armed bandits (MABs) are powerful algorithms to solve optimization problems that have a wide variety of applications in website optimization, clinical trials and digital advertising. In this blog post, we’ll: Explain the concept behind MABs Present a use case of…
For many, random forests are a go-to model for a variety of machine learning tasks such as binary classification, multi-class classification and regression. It’s easy to understand why: out of the box, random forests are strong predictors that generalize well…
At SpotX, we are always looking for new and innovative ways to support employee growth and development. Our participation in Bertelsmann’s MEDIAN initiative is the latest way we live one of our seven core values: “We believe in our people.”…
You’ve just made a hit new platforming game. Millions of people are playing it. The future looks bright. But wait! There’s just one problem – your app is free and you’re not making any money. Time to change that and…
Today we’ll briefly showcase how to join a static dataset in Spark with a streaming “live” dataset, otherwise known as a DStream. This is helpful in a number of scenarios: like when you have a live stream of data from Kafka…
This blog post is an excerpt taken from the SpotX Big Data talk at BELFOSS 2018. The full talk can be viewed here: https://www.youtube.com/watch?v=OlmbuyCH4lQ SpotX is a video ad serving platform based out of Colorado, USA and chances are, you…
Creating Hive tables is a common experience to all of us that use Hadoop. It enables us to mix and merge datasets into unique, customized tables. And, there are many ways to do it. We have some recommended tips for…