Snowflake Advanced Data Engineer: Snowpark for Data Engineering
Snowpark brings DataFrame-style programming into Snowflake so engineers can build transformations in languages such as Python while keeping execution close…
Browse practical guides, technical explanations and certification coverage.
Snowpark brings DataFrame-style programming into Snowflake so engineers can build transformations in languages such as Python while keeping execution close…
CI/CD for data engineering is the practice of treating notebooks, Python packages, SQL, job definitions, pipeline resources, tests, and environment…
A lakehouse architecture combines data-lake scale and openness with warehouse-style reliability, governance, and analytics. On Databricks, the architecture is not…
Databricks SQL warehouses provide managed compute for SQL analytics, dashboards, BI tools, and ad hoc queries over lakehouse data. The…
Delta Lake transactions and the medallion pattern solve two different problems that become more valuable when they are used together.…
Within Databricks, incremental data processing is the practice of computing what changed instead of repeatedly rebuilding everything. At small scale,…
Lakeflow Jobs is Databricks' workflow orchestration layer for coordinating notebooks, Python code, SQL, pipelines, and other tasks as dependable production…
PySpark DataFrames are one of the main programming interfaces for transforming structured and semi-structured data on Databricks. They let engineers…
Schema change is normal in data engineering. Sources add fields, rename concepts, change optionality, widen numeric ranges, and occasionally send…
A slow Spark job is rarely fixed by immediately choosing a larger cluster. Performance problems usually come from a specific…
Unity Catalog is the governance layer that connects identity, permissions, lineage, auditing, discovery, and policy across data and AI assets…
Retrieval-augmented generation, or RAG, improves a generative AI application by retrieving relevant enterprise information and placing that information into the…
Generative AI governance must cover more than model access. Production applications depend on prompts, evaluation datasets, vector indexes, tools, functions,…
Generative AI evaluation is difficult because many important qualities are not captured by one accuracy number. A response can be…
Putting a generative AI model behind an endpoint is only the start of production operations. A reliable serving design must…