Migration How Miro migrated its analytics event tracking system In this post, we’ll take you behind the scenes of how we migrated our analytics event system — responsible for handling ~3.5 billion events per day — without losing a single event. We’ll go into the technical details and the strategy we used that allowed us to leverage Kubernetes,
My Talks Big Data LDN 2019: Taking control of user analytics with Snowplow User analytics is well established at Auto Trader but breakthroughs in the cloud, big data and streaming technologies offer clear benefits above and beyond what we could do today. This talk describes how we migrated to Snowplow on Google Cloud for our user-analytics platform. Snowplow is an event streaming pipeline
My Talks Continuous deployment of machine learning models This talk describes how Auto Trader launched a suite of new machine learning models with the ability serve low-latency predictions in real time. These models are automatically retrained and redeployed using continuous deployment pipelines in our existing deployment infrastructure, making use of technology including Apache Spark, Airflow, Docker and Kubernetes.
Ways Of Working How we use Architectural Decision Records (ADRs) on Data Engineering When I joined our data engineers over a year ago they had already adopted Architectural Decision Records (ADRs) to document architectural decisions made whilst building Auto Trader’s Data Platform. ADRs are listed in ThoughtWorks’ Technology Radar [https://www.thoughtworks.com/radar/techniques/lightweight-architecture-decision-records] as a technique to adopt; our
