The COPY command in DuckDB and MotherDuck is a versatile tool for importing and exporting data. This guide provides a concise overview of how to use COPY both from the DuckDB CLI (SQL only) and from Python, including workflows with Ibis and pandas. Use this as a quick reference for your data engineering tasks!...
Continue reading...SQL
Building a Complete DuckLake Solution: From Local Development to Cloud Production
Introduction DuckLake is revolutionizing the lakehouse architecture by combining the simplicity of DuckDB with the power of modern data lake formats. In this comprehensive guide, I’ll walk you through building a complete DuckLake solution in two parts: first creating a local development environment, then scaling it to a cloud-based production...
Continue reading...KQL and Azure Data Explorer – Reference Guide
Recently I’ve been learning a new database query language, KQL which stands for “Kusto Query Language”. It is the language used by Azure Data Explorer, a tool in Microsoft’s Azure Cloud that helps query Kusto Databases. These databases run on server clusters so they are mighty as they can query...
Continue reading...Advanced SQL Server Guide
This is a short quick guide on advanced SQL Server topics, that I recently learned in this course from Udemy. The examples run in this post, use the AdventureWorks2019 sample database provided by Microsoft. Window Functions OVER() Function OVER() is a function used after an aggregate function like SUM() to...
Continue reading...SQL Server Integration Services (SSIS) Guide
Here is a Guide on SSIS Integration Services Tasks Service Type Service Name Explanation Data Flow Task Data Flow Task * Extract data from a variety of sources, apply column-level transformations, and load data into a database Data Preparation Tasks * FTP Task* Hadoop File System Task* File System Task...
Continue reading...