AI & Data
Databricks Data Engineer Associate: Delta Transactions & Medallion Design
Delta Lake transactions and the medallion pattern solve two different problems that become more valuable when they are used together.…
Explore / Technology subjects
AI & Data
Data engineering, analytics, databases, machine learning, artificial intelligence and AI systems.
The reading room
All insights
Latest in AI & Data
139 published articles in this category.
AI & Data
Databricks Data Engineer Associate: Incremental Delta Lake Processing
Within Databricks, incremental data processing is the practice of computing what changed instead of repeatedly rebuilding everything. At small scale,…
AI & Data
Databricks Data Engineer Associate: Lakeflow Jobs and Orchestration
Lakeflow Jobs is Databricks' workflow orchestration layer for coordinating notebooks, Python code, SQL, pipelines, and other tasks as dependable production…
AI & Data
Databricks Data Engineer Associate: PySpark DataFrames
PySpark DataFrames are one of the main programming interfaces for transforming structured and semi-structured data on Databricks. They let engineers…
AI & Data
Databricks Data Engineer Associate: Schema Evolution & Data Quality
Schema change is normal in data engineering. Sources add fields, rename concepts, change optionality, widen numeric ranges, and occasionally send…
AI & Data
Databricks Data Engineer Associate: Troubleshooting Slow Spark Jobs
A slow Spark job is rarely fixed by immediately choosing a larger cluster. Performance problems usually come from a specific…
AI & Data
Databricks Data Engineer Associate: Unity Catalog Governance
Unity Catalog is the governance layer that connects identity, permissions, lineage, auditing, discovery, and policy across data and AI assets…
AI & Data
Databricks GenAI Engineer Associate: Building RAG Systems
Retrieval-augmented generation, or RAG, improves a generative AI application by retrieving relevant enterprise information and placing that information into the…
AI & Data
Databricks GenAI Engineer Associate: Unity Catalog for GenAI
Generative AI governance must cover more than model access. Production applications depend on prompts, evaluation datasets, vector indexes, tools, functions,…
AI & Data
Databricks GenAI Engineer Associate: MLflow GenAI Evaluation
Generative AI evaluation is difficult because many important qualities are not captured by one accuracy number. A response can be…
AI & Data
Databricks GenAI Engineer Associate: Serving & Monitoring GenAI Models
Putting a generative AI model behind an endpoint is only the start of production operations. A reliable serving design must…
AI & Data
Databricks GenAI Engineer Associate: Vector Search for Enterprise AI
Enterprise AI applications often need to find information by meaning rather than exact keywords. Vector retrieval represents text, images, or…
AI & Data
Databricks ML Associate: Batch vs Real-Time Model Inference
Batch and real-time inference solve different operational problems. Batch scoring processes many records on a schedule or as a dataset…
AI & Data
Databricks ML Associate: MLflow Experiment Tracking & Model Registry
Machine learning development is iterative. Engineers change training data, features, parameters, algorithms, code, and environments, and each choice can affect…
AI & Data
Databricks Data Engineer Professional: Lakehouse Change Data Capture
Change data capture is the point where a lakehouse stops behaving like a sequence of full reloads and starts behaving…