chore: Automate fabric workspace & fabric capacity creation in Workshop mode#343
chore: Automate fabric workspace & fabric capacity creation in Workshop mode#343Prajwal-Microsoft wants to merge 18 commits into
Conversation
When isWorkshop=true and azureEnvOnly=false, Fabric Capacity is automatically provisioned via AVM Bicep module and Fabric Workspace is created via Fabric REST API in post-provision hook. Users can opt to use existing resources via env vars: - EXISTING_FABRIC_CAPACITY_NAME: use existing capacity - EXISTING_FABRIC_WORKSPACE_ID: use existing workspace - FABRIC_CAPACITY_SKU: configure capacity SKU (default: F2) Changes: - infra/main.bicep: Add Fabric Capacity AVM module (conditional) - infra/main.parameters.json: Add Fabric parameter mappings - infra/scripts/fabric/: New Fabric workspace automation scripts - azure.yaml: Add post-provision hook for workspace setup - docs: Update workshop guide and parameters documentation Resolves: ADO US #40274 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| @@ -0,0 +1,80 @@ | |||
| """ | |||
There was a problem hiding this comment.
Lets add it the changes with in the 02 script, lets not have sub folder everything goes under scripts
There was a problem hiding this comment.
Done — consolidated everything into scripts/02_create_fabric_items.py, deleted scripts/fabric/ subfolder.
- Inline workspace setup/cleanup into scripts/02_create_fabric_items.py - Remove scripts/fabric/ subfolder (workspace_setup.py, cleanup_workspace.py) - Remove infra/scripts/fabric/ (fabric_api.py, helpers/) - Add --cleanup flag for azd down predown hook - Update azure.yaml predown hook path - Match existing coding style (flat functions, AzureCliCredential) - Clear FABRIC_WORKSPACE_ID from .env on cleanup Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s and fix indentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to automate Microsoft Fabric “workshop mode” provisioning by creating a Fabric capacity via IaC and auto-creating (or reusing) a Fabric workspace during the data/load step, plus adding a teardown hook to optionally delete the workspace on azd down.
Changes:
- Add Bicep parameters/outputs and an AVM module to create or reuse a Fabric capacity in workshop mode.
- Update the Fabric item creation pipeline to auto-create/reuse a Fabric workspace and add a
--cleanupmode for teardown. - Update docs and
azure.yamlhooks to reflect (and trigger) the new automated workflow.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| workshop/docs/01-deploy/02-setup-fabric.md | Documents the new “automated Fabric setup” workflow and env vars. |
| scripts/02_create_fabric_items.py | Adds workspace auto-create/reuse logic, Fabric API helpers, and --cleanup deletion flow. |
| scripts/00_build_solution.py | Adjusts prompting/persistence for workspace id and improves quiet-mode subprocess output handling. |
| infra/main.parameters.json | Adds parameters for existing Fabric capacity name and capacity SKU. |
| infra/main.bicep | Adds Fabric capacity parameters, optional AVM capacity module, and outputs consumed by scripts. |
| documents/CustomizingAzdParameters.md | Documents new azd parameters related to Fabric capacity/workspace. |
| docs/Fabric_workspace_changes.md | Adds an explanatory doc describing the workspace creation/cleanup changes. |
| azure.yaml | Adds a predown cleanup hook and tweaks postprovision messaging logic. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| !!! tip "Automated Fabric Setup (Recommended)" | ||
| When running in **workshop mode** (default), Fabric Capacity and Workspace are **automatically created** during `azd up`. No manual steps are needed unless you want to use existing resources. | ||
|
|
There was a problem hiding this comment.
The doc says Fabric Capacity and Workspace are automatically created during azd up, but in this PR only the capacity is created by Bicep; the workspace is created later by scripts/02_create_fabric_items.py when step 02 runs (or when FABRIC_WORKSPACE_ID is provided). Please update this wording so users don’t expect a workspace to exist immediately after azd up.
There was a problem hiding this comment.
Fixed — updated wording to clarify that capacity is created during azd up and workspace is created later by the build script when CREATE_FABRIC_WORKSPACE is enabled.
| ``` | ||
| - To use an **existing Fabric workspace**, set: | ||
| ```bash | ||
| azd env set EXISTING_FABRIC_WORKSPACE_ID "your-workspace-id" |
There was a problem hiding this comment.
This page instructs users to set EXISTING_FABRIC_WORKSPACE_ID, but no code/infra in this PR reads that variable (workspace reuse is driven by FABRIC_WORKSPACE_ID). Either wire EXISTING_FABRIC_WORKSPACE_ID through to FABRIC_WORKSPACE_ID (or a new infra param) or change the docs to reference the supported variable.
| azd env set EXISTING_FABRIC_WORKSPACE_ID "your-workspace-id" | |
| azd env set FABRIC_WORKSPACE_ID "your-workspace-id" |
There was a problem hiding this comment.
Fixed — docs now reference FABRIC_WORKSPACE_ID consistently.
| | `DEPLOYING_USER_PRINCIPAL_TYPE` | string | `User` | Principal type of deployer (allowed: `User`, `ServicePrincipal`). | | ||
| | `EXISTING_FABRIC_CAPACITY_NAME` | string | ` ` | Use an existing Fabric capacity instead of creating a new one. If empty, a new capacity is auto-created in workshop mode. | | ||
| | `FABRIC_CAPACITY_SKU` | string | `F2` | SKU tier for the Fabric capacity (allowed: `F2` through `F2048`). | | ||
| | `EXISTING_FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. | |
There was a problem hiding this comment.
The parameter table adds EXISTING_FABRIC_WORKSPACE_ID, but it is not referenced anywhere else in the repo (scripts use FABRIC_WORKSPACE_ID, infra only supports existing capacity). Please remove/replace this entry or implement the end-to-end support so the documented parameter actually works.
| | `EXISTING_FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. | | |
| | `FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. | |
There was a problem hiding this comment.
Fixed — replaced FABRIC_WORKSPACE_ID and FABRIC_WORKSPACE_NAME with CREATE_FABRIC_WORKSPACE flag and FABRIC_ADMIN_MEMBERS parameter.
| def make_request(method, url, **kwargs): | ||
| """Make request with retry logic for 429 rate limiting""" | ||
| max_retries = 5 | ||
| for attempt in range(max_retries): | ||
| response = requests.request(method, url, headers=get_headers(), **kwargs) | ||
| if response.status_code == 429: | ||
| retry_after = int(response.headers.get("Retry-After", 30)) | ||
| print(f" Rate limited. Waiting {retry_after}s...") | ||
| time.sleep(retry_after) | ||
| continue | ||
| return response | ||
| return response |
There was a problem hiding this comment.
make_request() does not set a network timeout, so a hung TCP connection can block the entire provisioning flow indefinitely. Consider adding a default timeout (and allowing override via kwargs) so CI/automation doesn’t stall on transient network issues.
There was a problem hiding this comment.
Fixed — added kwargs.setdefault("timeout", 60) as default, overridable via kwargs.
| run: python scripts/02_create_fabric_items.py --cleanup | ||
| shell: pwsh | ||
| continueOnError: true | ||
| interactive: true | ||
| posix: | ||
| run: python scripts/02_create_fabric_items.py --cleanup |
There was a problem hiding this comment.
The new predown hook runs workspace cleanup unconditionally and is marked interactive: true, but the cleanup script prompts for confirmation. In non-interactive runs (CI or azd down --no-prompt), this can block teardown. Consider gating the hook on AZURE_ENV_ONLY/IS_WORKSHOP, or passing a non-interactive flag to the script once supported.
| run: python scripts/02_create_fabric_items.py --cleanup | |
| shell: pwsh | |
| continueOnError: true | |
| interactive: true | |
| posix: | |
| run: python scripts/02_create_fabric_items.py --cleanup | |
| run: | | |
| $isWorkshop = $env:IS_WORKSHOP | |
| $azureEnvOnly = $env:AZURE_ENV_ONLY | |
| if ($isWorkshop -eq "true" -and $azureEnvOnly -ne "true") { | |
| python scripts/02_create_fabric_items.py --cleanup | |
| } else { | |
| Write-Host "Skipping Fabric cleanup for non-workshop or Azure-env-only teardown." | |
| } | |
| shell: pwsh | |
| continueOnError: true | |
| interactive: true | |
| posix: | |
| run: | | |
| IS_WORKSHOP_VAL="${IS_WORKSHOP:-}" | |
| AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}" | |
| if [ "$IS_WORKSHOP_VAL" = "true" ] && [ "$AZURE_ENV_ONLY_VAL" != "true" ]; then | |
| python scripts/02_create_fabric_items.py --cleanup | |
| else | |
| echo "Skipping Fabric cleanup for non-workshop or Azure-env-only teardown." | |
| fi |
There was a problem hiding this comment.
Fixed — predown hook is now gated on CREATE_FABRIC_WORKSPACE=true. Also added --yes flag for non-interactive cleanup.
| confirm = input(f"\n Delete workspace '{ws_name}' ({workspace_id})? [y/N]: ").strip().lower() | ||
| if confirm != "y": | ||
| print(" Skipped workspace deletion.") | ||
| return |
There was a problem hiding this comment.
--cleanup always prompts for confirmation via input(). When invoked from azd down (or any non-interactive environment), this can hang teardown indefinitely. Suggest adding a non-interactive/auto-confirm option (e.g., --yes or env var) and/or skipping deletion when stdin isn’t a TTY.
There was a problem hiding this comment.
Fixed — added --yes/-y flag that skips the confirmation prompt for non-interactive environments.
| var fabricCapacityDefaultAdmins = deployer().?userPrincipalName == null | ||
| ? [deployer().objectId] | ||
| : [deployer().userPrincipalName] |
There was a problem hiding this comment.
fabricAdminMembers is documented as an array of object IDs, but fabricCapacityDefaultAdmins uses deployer().userPrincipalName when present. This mismatch can break deployments (and also makes the output AZURE_FABRIC_CAPACITY_ADMINISTRATORS ambiguous). Please either pass deployer().objectId consistently or update the parameter description/type expectations to match what the AVM module requires.
| var fabricCapacityDefaultAdmins = deployer().?userPrincipalName == null | |
| ? [deployer().objectId] | |
| : [deployer().userPrincipalName] | |
| var fabricCapacityDefaultAdmins = [deployer().objectId] |
There was a problem hiding this comment.
Fixed — changed to [deployer().objectId] consistently as suggested.
| $isWorkshop = $env:IS_WORKSHOP | ||
| $deployApp = $env:AZURE_ENV_DEPLOY_APP | ||
| $azureEnvOnly = $env:AZURE_ENV_ONLY | ||
|
|
There was a problem hiding this comment.
$azureEnvOnly is assigned but never used in this hook script. Please remove it or use it to conditionally print Fabric/Azure-only guidance so the hook stays easy to maintain.
| run: | | ||
| if [ "$IS_WORKSHOP" != "true" ] || [ "$AZURE_ENV_DEPLOY_APP" = "true" ]; then | ||
| IS_WORKSHOP_VAL="${IS_WORKSHOP:-}" | ||
| AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}" |
There was a problem hiding this comment.
AZURE_ENV_ONLY_VAL is set but never used in the POSIX postprovision hook. Please remove it or use it for conditional messaging to avoid dead variables.
| AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}" |
… Fabric workspace docs
| | `DEPLOYING_USER_PRINCIPAL_TYPE` | string | `User` | Principal type of deployer (allowed: `User`, `ServicePrincipal`). | | ||
| | `EXISTING_FABRIC_CAPACITY_NAME` | string | ` ` | Use an existing Fabric capacity instead of creating a new one. If empty, a new capacity is auto-created in workshop mode. | | ||
| | `FABRIC_CAPACITY_SKU` | string | `F2` | SKU tier for the Fabric capacity (allowed: `F2` through `F2048`). | | ||
| | `FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. | |
There was a problem hiding this comment.
I see that these variables are not used while deployment, we can remove both Workspace id and workspace name
There was a problem hiding this comment.
Done — removed both FABRIC_WORKSPACE_ID and FABRIC_WORKSPACE_NAME from the parameters table. Replaced with CREATE_FABRIC_WORKSPACE flag and FABRIC_ADMIN_MEMBERS.
| param fabricCapacitySku string = 'F2' | ||
|
|
||
| @description('Optional. An array of user object IDs or service principal object IDs that will be assigned the Fabric Capacity Admin role.') | ||
| param fabricAdminMembers array = [] |
There was a problem hiding this comment.
This should be mapped tom main.paramaters.json, else users might need to manaully edit bicep file, also include that in CustominzingAzdParameters.md
There was a problem hiding this comment.
Done — added fabricAdminMembers mapping to main.parameters.json as FABRIC_ADMIN_MEMBERS and documented it in CustomizingAzdParameters.md.
… flag, timeout, --yes flag, fix docs - Add CREATE_FABRIC_WORKSPACE env flag (default false) to control workspace creation - Add default timeout (60s) to make_request() to prevent hanging - Add --yes/-y flag for non-interactive cleanup in CI - Gate predown hook on CREATE_FABRIC_WORKSPACE - Fix fabricCapacityDefaultAdmins to use objectId consistently - Map fabricAdminMembers to main.parameters.json - Remove unused azureEnvOnly vars from postprovision hooks - Update docs: clarify workspace creation timing, add new params Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update Fabric_workspace_changes.md to reflect CREATE_FABRIC_WORKSPACE flag - Restore AZURE_ENV_ONLY vars in postprovision hooks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…workspace creation options
… clarity and consistency
…e clarity in main.bicep and main.parameters.json
| ### 5. Build the solution | ||
|
|
||
| #### Retrieve your Fabric workspace ID | ||
| This step runs `scripts/00_build_solution.py`, which creates (or uses) the Fabric workspace, loads the sample data, creates the Ontology and Data Agent, and publishes the MCP endpoint. |
There was a problem hiding this comment.
No need to mention details of the script it runs
|
|
||
| Let the build script create the workspace for you. Set this flag before running Section 5: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
This has to be ran before azd up
…an docs - Remove fabricWorkspaceId param from main.bicep - Add createFabricWorkspace bool param to main.bicep - Update shouldCreateFabricCapacity condition and outputs - Update main.parameters.json mapping - Recompile main.json from Bicep - Remove script implementation details from docs - Rewrite Section 1b/1c options and automation tip - Add optional env flags before azd up in Section 3 - Fix garbled box-drawing characters in customdata tree Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…wn hook Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Purpose
Fabric Capacity and Workspace Management Enhancements:
AZURE_FABRIC_CAPACITY_NAME) and its SKU (FABRIC_CAPACITY_SKU), as well as additional admin members (FABRIC_ADMIN_MEMBERS), allowing for more flexible and reusable deployments. [1] [2] [3] [4]CREATE_FABRIC_WORKSPACEflag to enable automatic creation of a Fabric workspace during the build script, with corresponding changes in environment variable handling and documentation. [1] [2] [3]Deployment Script and Output Improvements:
00_build_solution.pyto prompt for a workspace ID only if auto-creation is not enabled, persist the workspace ID to both.envand the azd environment, and dynamically adjust step labels and script parameters based on workspace creation mode. [1] [2] [3]Automation and Cleanup Hooks:
predownhooks for both Windows and POSIX systems to optionally clean up Fabric workspaces ifCREATE_FABRIC_WORKSPACEis set, improving resource hygiene during teardown.Documentation Updates:
These changes collectively provide a more robust, flexible, and user-friendly experience for deploying and managing Microsoft Fabric resources in both automated and interactive scenarios.
Does this introduce a breaking change?
Golden Path Validation
Deployment Validation
What to Check
Verify that the following are valid
Other Information