Spydra – Ephemeral Hadoop Clusters Using Google Compute Platform

We are happy to announce a new open sourced project from Spotify called Spydra. Spydra makes it easy to run data processing jobs in Google Cloud Platform (GCP) Dataproc, automating the cluster life-cycle management and making troubleshooting of failed jobs easier. Spydra supports usage of ephemeral clusters created only for the duration of a single […]