Architecture

The accelerator separates the reusable platform foundation from each client implementation. That keeps the core logic portable while allowing Databricks and Fabric implementations to diverge where they must.

Design principles

1. Separation of concerns: reusable framework logic stays separate from client overlays.
2. Infrastructure as code first: Bicep defines resources; Python handles post-deployment configuration.
3. Security by default: managed identities, RBAC, and Key Vault take priority over shared secrets.
4. Incremental evolution: environment changes should be redeployable and reviewable rather than hand-tuned in the portal.

Design boundaries

• Rockdata maintains the shared platform foundation.
• Client or project-specific behavior belongs in the example implementations as a basic and ready-to-go implementation.
• Data contracts define how source data should behave across landing, Bronze, and Silver.

Deployment flow

Phase 1: Infrastructure provisioning

• Provision storage for landing and catalog or metadata concerns.
• Provision secret storage and identity boundaries.
• Provision the platform workspace and any required managed identities.
• Apply RBAC so storage and secret access are ready before data-plane setup begins.

Phase 2: Post-deployment configuration

• Authenticate with Azure AD rather than personal tokens.
• Register external locations, credentials, or workspace access.
• Keep post-deployment scripts idempotent so pipelines can rerun safely.

Phase 3: Data-plane rollout

• Deploy notebooks, jobs, and contract-driven pipelines.
• Connect source contracts to landing, Bronze, and Silver.
• Schedule recurring execution only after validation and operational visibility are in place.

Medallion responsibilities

• Landing stores raw ingress with minimal opinion.
• Bronze preserves operational history and normalizes physical structure.
• Silver applies casting, constraints, and contract-driven silver write behavior.
• Gold stays intentionally client-specific but also contract driven.

Component architecture

• The example workspaces provide the environment overlays, notebooks, jobs, and contract files your team will configure.
• The reusable infrastructure assets provide the deployment baseline for storage, identities, security boundaries, and platform services.
• The shared framework packages provide the reusable ingestion and medallion behavior and normally remain unchanged in client implementations.