Systems in Practice · Library

Data platforms

Pipelines, lakehouses, streaming, and reliable data operations.

15 items · newest first

Oct 19, 2026Guide

15 high-value data engineering use cases for proactive durable AI agents

Fifteen proposed agent workflows for data operations, with a reusable architecture, an incident-investigation example, and a staged approach to autonomy.

Data platforms AI & ML systems

Jul 05, 2026Guide

Data System Design Interview Glossary

The core concepts and tradeoffs used in data-platform architecture interviews.

Data platforms Engineering practice

Jul 05, 2026Guide

Study Guide: Data Operations Architecture at Scale

A study guide to ingestion, enrichment, quality, governance, and operations at scale.

Data platforms AI & ML systems

Jul 04, 2026Guide

Study Guide: The Nemotron 3 Super Data Engineering Pipeline

A structured reference to the stages and tradeoffs in the Nemotron pretraining data pipeline.

AI & ML systems Data platforms

Jul 01, 2026Article

Inside Nemotron 3 Super: The Data Engineering Behind a 25-Trillion-Token Model

Examining the data preparation behind the Nemotron 3 Super pretraining corpus.

AI & ML systems Data platforms

Jun 30, 2026Article

The Data Operations Cycle

The connected systems that acquire, validate, enrich, serve, and monitor production ML data.

Data platforms AI & ML systems

Jun 27, 2026Article

Batch, Streaming, and Merged Views in ML Data Operations

How batch and streaming paths combine into consistent views for machine learning.

Data platforms AI & ML systems

Jun 23, 2026Article

Machine Learning System Design: A Comprehensive Guide to Data-Centric Feature Stores

Designing the ingestion, serving, and monitoring systems behind production feature stores.

AI & ML systems Data platforms

Jun 18, 2026Article

Training-Ready Multimodal Data: Shards and Loaders

Turning immutable dataset manifests into WebDataset shards and efficient training loaders.

AI & ML systems Data platforms

Jun 18, 2026Article

From Search Demo to Data Infrastructure

Precomputation, provenance, and observability turn a search demo into dependable data infrastructure.

Data platforms Engineering practice

Jun 15, 2026Article

What Comes Next for the Multimodal Lakehouse

The next design questions for a multimodal lakehouse: quality, deduplication, and evaluation.

AI & ML systems Data platforms

Jun 11, 2026Article

Scaling ETL Pipelines: From One Machine to Distributed Systems

Moving from single-machine ETL to distributed pipelines without losing correctness or control.

Distributed systems Data platforms

Jun 11, 2026Article

Multimodal Lakehouse Implementation Notes

Content-addressed storage, dataset versioning, and deduplication in a multimodal lakehouse.

Databases & querying Data platforms

Jun 01, 2026Article

Serverless Multimodal Data Lakehouse

A 12-stage pipeline for turning text, images, video, and audio into traceable training data.

Data platforms AI & ML systems

Apr 30, 2021Article

DATA PIPELINE ORCHESTRATION

Orchestrating a batch analytics pipeline with Airflow, AWS EMR, and Redshift.

Data platforms