Some of the premises mentioned above have since changed, but the power of open source and the core principles on which it’s built have only grown. That's without mentioning the fact that for a cloud-native company, Tableau's Windows-centric approach at the time didn't work well for the team. Secondarily, we had a limited number of licenses for Tableau, and generally had an order of magnitude more employees that wanted/needed access to our internal than our contract allowed. Extracts crumbled under the data volumes we had at Airbnb, creating a whole lot of challenges around non-additive metrics (think distinct user counts) and forcing us to intricately pre-compute multiple “grouping sets” which broke down some of the Tableau paradigms and confused users. With Tableau’s “Live Mode” misbehaving in intricate ways at the time (I won’t get into this!), we were steered towards using Tableau Extracts. More specifically for Superset, the main driver to start the project at the time was the fact that Tableau (our main data visualization solution at the time) couldn’t connect natively to Apache Druid and Trino / Presto, our data engines of choice that provided the properties and guaranties that we needed to satisfy our data use cases.
Looking back, the main reason why I originally created both Apache Airflow and Apache Superset while I was at Airbnb in 2014-17 is because the vendors in the data space were failing to:
For those reasons, it is not surprising that it has taken over most of the modern data stack: infrastructure, databases, orchestration, data processing, AI/ML and beyond. Simply put, open source is a superior approach at building and distributing software because it provides important guaranties around how software can be discovered, tried, operated, collaborated on and packaged. While “software is eating the world”, it’s also clear that open source is taking over software.
0 kommentar(er)
