Level: IntermediateEnvironment: Oracle Cloud ARM VM, Ubuntu 24.04, Docker, Caddy (non-Docker), Cloudflare Introduction: What is RedPanda and Why Should Analytics Engineers/Data Scientists Care? In the world of data, moving information quickly and reliably is crucial. Enter RedPanda, a modern streaming data platform designed for simplicity and performance. Think of it as a Kafka-compatible alternative, but...
Continue reading...Hector Sanchez
Installing Apache Airflow with Docker and Caddy on Oracle ARM (Ubuntu 24.04)
Running data pipelines often requires a robust orchestrator like Apache Airflow. Setting it up on modern ARM-based cloud infrastructure, such as Oracle Cloud’s Ampere A1 instances, can be efficient and cost-effective. This guide walks you through installing Apache Airflow using Docker Compose on an Ubuntu 24.04 ARM VM, using Caddy...
Continue reading...Installing Self-Hosted Airbyte on Oracle ARM Ubuntu 24.04 with Caddy & Cloudflare
Airbyte is a powerful open-source data integration platform, allowing you to sync data between various sources and destinations. Self-hosting Airbyte gives you full control over your data pipelines. This guide walks through installing Airbyte on an Oracle Cloud ARM Ampere VM running Ubuntu 24.04, making it accessible via a secure...
Continue reading...The Self-Hosted Power Trio: Integrating Airflow, Airbyte, and n8n for Ultimate Data & Automation
In today’s data-driven world, moving information, processing it, and acting upon insights are critical. But stitching together these processes can be complex. Thankfully, a powerful trio of open-source tools, when used together, can create incredibly robust and flexible data and automation pipelines, especially when you prefer the control of a...
Continue reading...Self-Hosting ChromaDB on Oracle ARM Ubuntu with Docker, Caddy & Cloudflare
ChromaDB is a powerful open-source embedding database, essential for building AI applications involving semantic search, retrieval-augmented generation (RAG), and more. While Chroma offers a managed service, self-hosting provides greater control, privacy, and potentially lower costs, especially when leveraging free-tier resources like Oracle Cloud’s ARM VMs. This guide details how to...
Continue reading...