Category: Big data architecture

As you probably know, the data-science world is at war: half of it uses Python, the other half R. The Internet is full of comparisons and long discussions. However, all of this becomes irrelevant when your data becomes Big. The first road bump occurs when your data don’t fit in your laptop’s memory anymore. What …

Read more “Gradient Boosted Trees in Spark on AWS”

Last month we went as fast as we could with a client to set up an infrastructure for huge amounts of data. The goal was to show that we could (a) handle all these events (b) make real-time reports and (c) combine this data with offline data for reporting. One amazing aspect of this project …

Read more “Fast iteration with Kafka, Druid and Airflow”