Skip to content

chore: Automate fabric workspace & fabric capacity creation in Workshop mode#343

Open
Prajwal-Microsoft wants to merge 18 commits into
microsoft:mainfrom
Yamini-Microsoft:psl-automate-fabric-workshop
Open

chore: Automate fabric workspace & fabric capacity creation in Workshop mode#343
Prajwal-Microsoft wants to merge 18 commits into
microsoft:mainfrom
Yamini-Microsoft:psl-automate-fabric-workshop

Conversation

@Prajwal-Microsoft
Copy link
Copy Markdown
Collaborator

@Prajwal-Microsoft Prajwal-Microsoft commented Apr 20, 2026

Purpose

  • This pull request introduces several enhancements to the deployment process for Microsoft Fabric resources, focusing on the flexibility and automation of Fabric capacity and workspace management. It adds new parameters and logic for auto-creating or reusing Fabric capacities and workspaces, improves script robustness, and updates documentation and environment configuration to support these new features.

Fabric Capacity and Workspace Management Enhancements:

  • Added support for specifying an existing Fabric capacity (AZURE_FABRIC_CAPACITY_NAME) and its SKU (FABRIC_CAPACITY_SKU), as well as additional admin members (FABRIC_ADMIN_MEMBERS), allowing for more flexible and reusable deployments. [1] [2] [3] [4]
  • Introduced the CREATE_FABRIC_WORKSPACE flag to enable automatic creation of a Fabric workspace during the build script, with corresponding changes in environment variable handling and documentation. [1] [2] [3]

Deployment Script and Output Improvements:

  • Updated 00_build_solution.py to prompt for a workspace ID only if auto-creation is not enabled, persist the workspace ID to both .env and the azd environment, and dynamically adjust step labels and script parameters based on workspace creation mode. [1] [2] [3]
  • Ensured UTF-8 output for script logs, particularly on Windows, to avoid encoding issues with emoji or special characters. [1] [2]

Automation and Cleanup Hooks:

  • Added predown hooks for both Windows and POSIX systems to optionally clean up Fabric workspaces if CREATE_FABRIC_WORKSPACE is set, improving resource hygiene during teardown.
  • Refined post-provisioning instructions and logic in deployment hooks to better reflect the new workspace and capacity management options.

Documentation Updates:

  • Expanded documentation to describe the new parameters and environment variables for Fabric capacity and workspace management, making it easier for users to understand and configure these options.

These changes collectively provide a more robust, flexible, and user-friendly experience for deploying and managing Microsoft Fabric resources in both automated and interactive scenarios.

Does this introduce a breaking change?

  • Yes
  • No

Golden Path Validation

  • I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

  • I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

  • ...

Other Information

Yamini-Microsoft and others added 2 commits April 17, 2026 23:06
When isWorkshop=true and azureEnvOnly=false, Fabric Capacity is
automatically provisioned via AVM Bicep module and Fabric Workspace
is created via Fabric REST API in post-provision hook.

Users can opt to use existing resources via env vars:
- EXISTING_FABRIC_CAPACITY_NAME: use existing capacity
- EXISTING_FABRIC_WORKSPACE_ID: use existing workspace
- FABRIC_CAPACITY_SKU: configure capacity SKU (default: F2)

Changes:
- infra/main.bicep: Add Fabric Capacity AVM module (conditional)
- infra/main.parameters.json: Add Fabric parameter mappings
- infra/scripts/fabric/: New Fabric workspace automation scripts
- azure.yaml: Add post-provision hook for workspace setup
- docs: Update workshop guide and parameters documentation

Resolves: ADO US #40274

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread scripts/fabric/workspace_setup.py Outdated
@@ -0,0 +1,80 @@
"""
Copy link
Copy Markdown
Collaborator Author

@Prajwal-Microsoft Prajwal-Microsoft Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add it the changes with in the 02 script, lets not have sub folder everything goes under scripts

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — consolidated everything into scripts/02_create_fabric_items.py, deleted scripts/fabric/ subfolder.

- Inline workspace setup/cleanup into scripts/02_create_fabric_items.py
- Remove scripts/fabric/ subfolder (workspace_setup.py, cleanup_workspace.py)
- Remove infra/scripts/fabric/ (fabric_api.py, helpers/)
- Add --cleanup flag for azd down predown hook
- Update azure.yaml predown hook path
- Match existing coding style (flat functions, AzureCliCredential)
- Clear FABRIC_WORKSPACE_ID from .env on cleanup

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s and fix indentation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to automate Microsoft Fabric “workshop mode” provisioning by creating a Fabric capacity via IaC and auto-creating (or reusing) a Fabric workspace during the data/load step, plus adding a teardown hook to optionally delete the workspace on azd down.

Changes:

  • Add Bicep parameters/outputs and an AVM module to create or reuse a Fabric capacity in workshop mode.
  • Update the Fabric item creation pipeline to auto-create/reuse a Fabric workspace and add a --cleanup mode for teardown.
  • Update docs and azure.yaml hooks to reflect (and trigger) the new automated workflow.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
workshop/docs/01-deploy/02-setup-fabric.md Documents the new “automated Fabric setup” workflow and env vars.
scripts/02_create_fabric_items.py Adds workspace auto-create/reuse logic, Fabric API helpers, and --cleanup deletion flow.
scripts/00_build_solution.py Adjusts prompting/persistence for workspace id and improves quiet-mode subprocess output handling.
infra/main.parameters.json Adds parameters for existing Fabric capacity name and capacity SKU.
infra/main.bicep Adds Fabric capacity parameters, optional AVM capacity module, and outputs consumed by scripts.
documents/CustomizingAzdParameters.md Documents new azd parameters related to Fabric capacity/workspace.
docs/Fabric_workspace_changes.md Adds an explanatory doc describing the workspace creation/cleanup changes.
azure.yaml Adds a predown cleanup hook and tweaks postprovision messaging logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +8 to +10
!!! tip "Automated Fabric Setup (Recommended)"
When running in **workshop mode** (default), Fabric Capacity and Workspace are **automatically created** during `azd up`. No manual steps are needed unless you want to use existing resources.

Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc says Fabric Capacity and Workspace are automatically created during azd up, but in this PR only the capacity is created by Bicep; the workspace is created later by scripts/02_create_fabric_items.py when step 02 runs (or when FABRIC_WORKSPACE_ID is provided). Please update this wording so users don’t expect a workspace to exist immediately after azd up.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — updated wording to clarify that capacity is created during azd up and workspace is created later by the build script when CREATE_FABRIC_WORKSPACE is enabled.

```
- To use an **existing Fabric workspace**, set:
```bash
azd env set EXISTING_FABRIC_WORKSPACE_ID "your-workspace-id"
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This page instructs users to set EXISTING_FABRIC_WORKSPACE_ID, but no code/infra in this PR reads that variable (workspace reuse is driven by FABRIC_WORKSPACE_ID). Either wire EXISTING_FABRIC_WORKSPACE_ID through to FABRIC_WORKSPACE_ID (or a new infra param) or change the docs to reference the supported variable.

Suggested change
azd env set EXISTING_FABRIC_WORKSPACE_ID "your-workspace-id"
azd env set FABRIC_WORKSPACE_ID "your-workspace-id"

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — docs now reference FABRIC_WORKSPACE_ID consistently.

Comment thread documents/CustomizingAzdParameters.md Outdated
| `DEPLOYING_USER_PRINCIPAL_TYPE` | string | `User` | Principal type of deployer (allowed: `User`, `ServicePrincipal`). |
| `EXISTING_FABRIC_CAPACITY_NAME` | string | ` ` | Use an existing Fabric capacity instead of creating a new one. If empty, a new capacity is auto-created in workshop mode. |
| `FABRIC_CAPACITY_SKU` | string | `F2` | SKU tier for the Fabric capacity (allowed: `F2` through `F2048`). |
| `EXISTING_FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. |
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter table adds EXISTING_FABRIC_WORKSPACE_ID, but it is not referenced anywhere else in the repo (scripts use FABRIC_WORKSPACE_ID, infra only supports existing capacity). Please remove/replace this entry or implement the end-to-end support so the documented parameter actually works.

Suggested change
| `EXISTING_FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. |
| `FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. |

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — replaced FABRIC_WORKSPACE_ID and FABRIC_WORKSPACE_NAME with CREATE_FABRIC_WORKSPACE flag and FABRIC_ADMIN_MEMBERS parameter.

Comment on lines +93 to +104
def make_request(method, url, **kwargs):
"""Make request with retry logic for 429 rate limiting"""
max_retries = 5
for attempt in range(max_retries):
response = requests.request(method, url, headers=get_headers(), **kwargs)
if response.status_code == 429:
retry_after = int(response.headers.get("Retry-After", 30))
print(f" Rate limited. Waiting {retry_after}s...")
time.sleep(retry_after)
continue
return response
return response
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make_request() does not set a network timeout, so a hung TCP connection can block the entire provisioning flow indefinitely. Consider adding a default timeout (and allowing override via kwargs) so CI/automation doesn’t stall on transient network issues.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — added kwargs.setdefault("timeout", 60) as default, overridable via kwargs.

Comment thread azure.yaml Outdated
Comment on lines +18 to +23
run: python scripts/02_create_fabric_items.py --cleanup
shell: pwsh
continueOnError: true
interactive: true
posix:
run: python scripts/02_create_fabric_items.py --cleanup
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new predown hook runs workspace cleanup unconditionally and is marked interactive: true, but the cleanup script prompts for confirmation. In non-interactive runs (CI or azd down --no-prompt), this can block teardown. Consider gating the hook on AZURE_ENV_ONLY/IS_WORKSHOP, or passing a non-interactive flag to the script once supported.

Suggested change
run: python scripts/02_create_fabric_items.py --cleanup
shell: pwsh
continueOnError: true
interactive: true
posix:
run: python scripts/02_create_fabric_items.py --cleanup
run: |
$isWorkshop = $env:IS_WORKSHOP
$azureEnvOnly = $env:AZURE_ENV_ONLY
if ($isWorkshop -eq "true" -and $azureEnvOnly -ne "true") {
python scripts/02_create_fabric_items.py --cleanup
} else {
Write-Host "Skipping Fabric cleanup for non-workshop or Azure-env-only teardown."
}
shell: pwsh
continueOnError: true
interactive: true
posix:
run: |
IS_WORKSHOP_VAL="${IS_WORKSHOP:-}"
AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}"
if [ "$IS_WORKSHOP_VAL" = "true" ] && [ "$AZURE_ENV_ONLY_VAL" != "true" ]; then
python scripts/02_create_fabric_items.py --cleanup
else
echo "Skipping Fabric cleanup for non-workshop or Azure-env-only teardown."
fi

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — predown hook is now gated on CREATE_FABRIC_WORKSPACE=true. Also added --yes flag for non-interactive cleanup.

Comment thread scripts/02_create_fabric_items.py Outdated
Comment on lines +232 to +235
confirm = input(f"\n Delete workspace '{ws_name}' ({workspace_id})? [y/N]: ").strip().lower()
if confirm != "y":
print(" Skipped workspace deletion.")
return
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--cleanup always prompts for confirmation via input(). When invoked from azd down (or any non-interactive environment), this can hang teardown indefinitely. Suggest adding a non-interactive/auto-confirm option (e.g., --yes or env var) and/or skipping deletion when stdin isn’t a TTY.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — added --yes/-y flag that skips the confirmation prompt for non-interactive environments.

Comment thread infra/main.bicep
Comment on lines +192 to +194
var fabricCapacityDefaultAdmins = deployer().?userPrincipalName == null
? [deployer().objectId]
: [deployer().userPrincipalName]
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fabricAdminMembers is documented as an array of object IDs, but fabricCapacityDefaultAdmins uses deployer().userPrincipalName when present. This mismatch can break deployments (and also makes the output AZURE_FABRIC_CAPACITY_ADMINISTRATORS ambiguous). Please either pass deployer().objectId consistently or update the parameter description/type expectations to match what the AVM module requires.

Suggested change
var fabricCapacityDefaultAdmins = deployer().?userPrincipalName == null
? [deployer().objectId]
: [deployer().userPrincipalName]
var fabricCapacityDefaultAdmins = [deployer().objectId]

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — changed to [deployer().objectId] consistently as suggested.

Comment thread azure.yaml
Comment on lines 30 to 33
$isWorkshop = $env:IS_WORKSHOP
$deployApp = $env:AZURE_ENV_DEPLOY_APP
$azureEnvOnly = $env:AZURE_ENV_ONLY

Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$azureEnvOnly is assigned but never used in this hook script. Please remove it or use it to conditionally print Fabric/Azure-only guidance so the hook stays easy to maintain.

Copilot uses AI. Check for mistakes.
Comment thread azure.yaml Outdated
run: |
if [ "$IS_WORKSHOP" != "true" ] || [ "$AZURE_ENV_DEPLOY_APP" = "true" ]; then
IS_WORKSHOP_VAL="${IS_WORKSHOP:-}"
AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}"
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AZURE_ENV_ONLY_VAL is set but never used in the POSIX postprovision hook. Please remove it or use it for conditional messaging to avoid dead variables.

Suggested change
AZURE_ENV_ONLY_VAL="${AZURE_ENV_ONLY:-}"

Copilot uses AI. Check for mistakes.
Comment thread documents/CustomizingAzdParameters.md Outdated
| `DEPLOYING_USER_PRINCIPAL_TYPE` | string | `User` | Principal type of deployer (allowed: `User`, `ServicePrincipal`). |
| `EXISTING_FABRIC_CAPACITY_NAME` | string | ` ` | Use an existing Fabric capacity instead of creating a new one. If empty, a new capacity is auto-created in workshop mode. |
| `FABRIC_CAPACITY_SKU` | string | `F2` | SKU tier for the Fabric capacity (allowed: `F2` through `F2048`). |
| `FABRIC_WORKSPACE_ID` | string | ` ` | Use an existing Fabric workspace instead of creating a new one. If empty, a new workspace is auto-created in workshop mode. |
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that these variables are not used while deployment, we can remove both Workspace id and workspace name

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — removed both FABRIC_WORKSPACE_ID and FABRIC_WORKSPACE_NAME from the parameters table. Replaced with CREATE_FABRIC_WORKSPACE flag and FABRIC_ADMIN_MEMBERS.

Comment thread infra/main.bicep
param fabricCapacitySku string = 'F2'

@description('Optional. An array of user object IDs or service principal object IDs that will be assigned the Fabric Capacity Admin role.')
param fabricAdminMembers array = []
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be mapped tom main.paramaters.json, else users might need to manaully edit bicep file, also include that in CustominzingAzdParameters.md

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — added fabricAdminMembers mapping to main.parameters.json as FABRIC_ADMIN_MEMBERS and documented it in CustomizingAzdParameters.md.

… flag, timeout, --yes flag, fix docs

- Add CREATE_FABRIC_WORKSPACE env flag (default false) to control workspace creation
- Add default timeout (60s) to make_request() to prevent hanging
- Add --yes/-y flag for non-interactive cleanup in CI
- Gate predown hook on CREATE_FABRIC_WORKSPACE
- Fix fabricCapacityDefaultAdmins to use objectId consistently
- Map fabricAdminMembers to main.parameters.json
- Remove unused azureEnvOnly vars from postprovision hooks
- Update docs: clarify workspace creation timing, add new params

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update Fabric_workspace_changes.md to reflect CREATE_FABRIC_WORKSPACE flag
- Restore AZURE_ENV_ONLY vars in postprovision hooks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Prajwal-Microsoft Prajwal-Microsoft changed the title Psl automate fabric workshop chore: Automate fabric workspace & fabric capacity creation in Workshop mode Apr 27, 2026
### 5. Build the solution

#### Retrieve your Fabric workspace ID
This step runs `scripts/00_build_solution.py`, which creates (or uses) the Fabric workspace, loads the sample data, creates the Ontology and Data Agent, and publishes the MCP endpoint.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to mention details of the script it runs


Let the build script create the workspace for you. Set this flag before running Section 5:

```bash
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has to be ran before azd up

Yamini-Microsoft and others added 5 commits April 27, 2026 11:46
…an docs

- Remove fabricWorkspaceId param from main.bicep
- Add createFabricWorkspace bool param to main.bicep
- Update shouldCreateFabricCapacity condition and outputs
- Update main.parameters.json mapping
- Recompile main.json from Bicep
- Remove script implementation details from docs
- Rewrite Section 1b/1c options and automation tip
- Add optional env flags before azd up in Section 3
- Fix garbled box-drawing characters in customdata tree

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…wn hook

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants