Skip to content

Conversation

@jiasli
Copy link
Member

@jiasli jiasli commented Mar 18, 2025

Description
msrestazure requires adal. This PR drops msrestazure so that Azure CLI no longer depends on adal.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Mar 18, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️latest
️✔️3.12
️✔️3.9

@azure-client-tools-bot-prd
Copy link

Hi @jiasli,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Mar 18, 2025

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@github-actions
Copy link

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

@yonzhan
Copy link
Collaborator

yonzhan commented Mar 18, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@jiasli
Copy link
Member Author

jiasli commented Mar 18, 2025

Besides managed identity authentication (#25959), msrestazure is still used by

def empty_on_404(ex):
from msrestazure.azure_exceptions import CloudError
if isinstance(ex, CloudError) and ex.status_code == 404:
return None
raise ex

azure.cli.core.util.empty_on_404 is widely used by several modules:

https://github.com/search?q=repo%3AAzure%2Fazure-cli+empty_on_404&type=code

and extensions:

https://github.com/search?q=repo%3AAzure%2Fazure-cli-extensions%20empty_on_404&type=code

@evelyn-ys
Copy link
Member

Besides managed identity authentication (#25959), msrestazure is still used by

def empty_on_404(ex):
from msrestazure.azure_exceptions import CloudError
if isinstance(ex, CloudError) and ex.status_code == 404:
return None
raise ex

Commented in #29856 (comment)

@jiasli
Copy link
Member Author

jiasli commented Jun 5, 2025

Besides managed identity authentication (#25959), msrestazure is still used by

def empty_on_404(ex):
from msrestazure.azure_exceptions import CloudError
if isinstance(ex, CloudError) and ex.status_code == 404:
return None
raise ex

azure.cli.core.util.empty_on_404 is widely used by several modules:

https://github.com/search?q=repo%3AAzure%2Fazure-cli+empty_on_404&type=code

and extensions:

https://github.com/search?q=repo%3AAzure%2Fazure-cli-extensions%20empty_on_404&type=code

Fixed by #31077

@jiasli jiasli marked this pull request as ready for review June 5, 2025 09:19
Copilot AI review requested due to automatic review settings June 5, 2025 09:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes the msrestazure dependency from the project, streamlining the package requirements as it is no longer needed.

  • Removed the msrestazure dependency line from the Windows requirements file
  • Removed the msrestazure dependency line from the Linux requirements file
  • Removed the msrestazure dependency line from the Darwin requirements file
  • Removed the msrestazure dependency from the azure-cli-core setup script

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/azure-cli/requirements.py3.windows.txt Removed msrestazure dependency
src/azure-cli/requirements.py3.Linux.txt Removed msrestazure dependency
src/azure-cli/requirements.py3.Darwin.txt Removed msrestazure dependency
src/azure-cli-core/setup.py Removed msrestazure entry from dependency list

@jiasli
Copy link
Member Author

jiasli commented Jun 5, 2025

Codegen Coverage task failed:

https://dev.azure.com/azclitools/public/_build/results?buildId=248987&view=logs&jobId=9345cfb0-d7b9-509c-bf97-03248f55543a&j=9345cfb0-d7b9-509c-bf97-03248f55543a&t=e1a1fdd4-b888-5743-66dc-7bbee1ce278e

Logs
2025-06-05T10:49:55.1813191Z # CLI repo only
2025-06-05T10:49:55.1818097Z azdev statistics list-command-table CLI --statistics-only
2025-06-05T10:49:55.5022914Z 
2025-06-05T10:49:55.5030324Z ======================
2025-06-05T10:49:55.5030597Z | List Command Table |
2025-06-05T10:49:55.5030818Z ======================
2025-06-05T10:49:55.5030906Z 
2025-06-05T10:49:58.1828764Z Modules: find, iot, rdbms, databoxedge, cosmosdb, cloud, computefleet, extension, role, sqlvm, cdn, compute_recommender, eventgrid, configure, ams, policyinsights, aro, vm, feedback, dms, batchai, privatedns, serviceconnector, util, signalr, servicebus, keyvault, identity, containerapp, config, redis, resource, security, storage, batch, sql, backup, appconfig, consumption, marketplaceordering, eventhubs, botservice, mysql, relay, dls, cognitiveservices, appservice, interactive, billing, search, managedservices, servicefabric, network, apim, acr, netappfiles, container, maps, monitor, synapse, advisor, profile, hdinsight, acs, lab, azure-cli, azure-cli-telemetry, azure-cli-core, azure-cli-testsdk
2025-06-05T10:49:58.1831119Z 
2025-06-05T10:49:58.1845311Z Initializing with command table and help files...
2025-06-05T10:50:13.5944968Z WARNING: Unable to load extension 'apic-extension: No module named 'msrestazure''. Use --debug for more information.
2025-06-05T10:50:17.4622014Z WARNING: Unable to load extension 'keyvault-preview: cannot import name 'KeyVaultArgumentContext' from 'azure.cli.command_modules.keyvault._command_type' (/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/keyvault/_command_type.py)'. Use --debug for more information.
2025-06-05T10:50:25.1100364Z WARNING: Unable to load extension 'spring-cloud: No module named 'msrestazure''. Use --debug for more information.
2025-06-05T10:51:01.6900452Z {
2025-06-05T10:51:01.6903756Z   "codegenV1": 89,
2025-06-05T10:51:01.6910494Z   "codegenV2": 1969,
2025-06-05T10:51:01.6910865Z   "total": 5935
2025-06-05T10:51:01.6911168Z }
2025-06-05T10:51:05.3092248Z # CLI + EXT repos
2025-06-05T10:51:05.3097448Z pip install jinja2 -U
2025-06-05T10:51:06.1357165Z Requirement already satisfied: jinja2 in ./s/env/lib/python3.11/site-packages (3.1.6)
2025-06-05T10:51:06.3074935Z Requirement already satisfied: MarkupSafe>=2.0 in ./s/env/lib/python3.11/site-packages (from jinja2) (3.0.2)
2025-06-05T10:51:06.8817300Z azdev statistics list-command-table --statistics-only > /tmp/codegen_report.json
2025-06-05T10:51:07.1842470Z 
2025-06-05T10:51:07.1847142Z ======================
2025-06-05T10:51:07.1847599Z | List Command Table |
2025-06-05T10:51:07.1847966Z ======================
2025-06-05T10:51:07.1848121Z 
2025-06-05T10:51:07.4176691Z Modules: find, iot, rdbms, databoxedge, cosmosdb, cloud, computefleet, extension, role, sqlvm, cdn, compute_recommender, eventgrid, configure, ams, policyinsights, aro, vm, feedback, dms, batchai, privatedns, serviceconnector, util, signalr, servicebus, keyvault, identity, containerapp, config, redis, resource, security, storage, batch, sql, backup, appconfig, consumption, marketplaceordering, eventhubs, botservice, mysql, relay, dls, cognitiveservices, appservice, interactive, billing, search, managedservices, servicefabric, network, apim, acr, netappfiles, container, maps, monitor, synapse, advisor, profile, hdinsight, acs, lab, azure-cli, azure-cli-telemetry, azure-cli-core, azure-cli-testsdk, azext_offazure, azext_mcc, azext_automation, azext_workloads, azext_carbon, azext_edgezones, azext_webpubsub, azext_managementpartner, azext_alias, azext_nsp, azext_custom_providers, azext_scvmm, azext_load, azext_applicationinsights, azext_confluent, azext_dms, azext_peering, azext_amg, azext_qumulo, azext_healthbot, azext_acrquery, azext_webapp, azext_notification_hub, azext_spring_cloud, azext_support, azext_baremetal_infrastructure, azext_eventgrid, azext_acrcssc, azext_resource_mover, azext_log_analytics_solution, azext_healthcareapis, azext_powerbidedicated, azext_costmanagement, azext_serialconsole, azext_scenario_guide, azext_firmwareanalysis, azext_keyvault_preview, azext_amlfs, azext_ai_examples, azext_quantum, azext_providerhub, azext_amcs, azext_netappfiles_preview, azext_hack, azext_databricks, azext_oracle_database, azext_hdinsightonaks, azext_computeschedule, azext_datashare, azext_nginx, azext_palo_alto_networks, azext_datafactory, azext_monitor_pipeline_group, azext_elastic_san, azext_hardware_security_modules, azext_vm_repair, azext_microsoft_fabric, azext_databox, azext_account, azext_containerapp, azext_billing_benefits, azext_authV2, azext_aosm, azext_dev_spaces, azext_edgeorder, azext_timeseriesinsights, azext_staticwebapp, azext_dataprotection, azext_alb, azext_storage_blob_preview, azext_storagesync, azext_apic_extension, azext_managednetworkfabric, azext_elastic, azext_acat, azext_fleet, azext_acrtransfer, azext_next, azext_fluid_relay, azext_vnettap, azext_pinecone, azext_datadog, azext_devcenter, azext_diskpool, azext_swiftlet, azext_networkcloud, azext_subscription, azext_aks_preview, azext_loganalytics, azext_network_analytics, azext_image_gallery, azext_durabletask, azext_ad, azext_automanage, azext_k8s_extension, azext_traffic_collector, azext_cli_translator, azext_mongo_db, azext_front_door, azext_standbypool, azext_reservation, azext_portal, azext_logz, azext_graphservices, azext_sentinel, azext_change_analysis, azext_hpc_cache, azext_vmware, azext_maintenance, azext_guestconfig, azext_vwan, azext_import_export, azext_expressroutecrossconnection, azext_connectedk8s, azext_imagecopy, azext_mobile_network, azext_azurelargeinstance, azext_mixed_reality, azext_terraform, azext_stack_hci, azext_functionapp, azext_purview, azext_storage_mover, azext_serviceconnector_passwordless, azext_connectedmachine, azext_cosmosdb_preview, azext_scheduled_query, azext_bastion, azext_zones, azext_quota, azext_ssh, azext_mesh, azext_site_recovery, azext_desktopvirtualization, azext_nexusidentity, azext_storage_actions, azext_aem, azext_dynatrace, azext_init, azext_datamigration, azext_redisenterprise, azext_k8s_runtime, azext_alertsmanagement, azext_internet_analyzer, azext_interactive, azext_attestation, azext_command_change, azext_voice_service, azext_gallery_service_artifact, azext_trustedsigning, azext_network_manager, azext_logic, azext_storage_preview, azext_rdbms_connect, azext_weights_and_biases, azext_appservice_kube, azext_stream_analytics, azext_cloudservice, azext_astronomer, azext_footprint, azext_confcom, azext_confidentialledger, azext_dnsresolver, azext_k8s_configuration, azext_playwright_cli_extension, azext_lambda_test, azext_kusto, azext_resourcegraph, azext_communication, azext_arcgateway, azext_dnc, azext_blueprint, azext_mdp, azext_managedccfs, azext_self_help, azext_neon, azext_multicloud_connector, azext_spring, azext_vme, azext_ip_group, azext_informatica, azext_orbital, azext_connectedvmware, azext_firewall, azext_new_relic
2025-06-05T10:51:07.4184321Z 
2025-06-05T10:51:07.4188398Z Initializing with command table and help files...
2025-06-05T10:51:11.0839294Z WARNING: Unable to load extension 'apic-extension: No module named 'msrestazure''. Use --debug for more information.
2025-06-05T10:51:12.2678351Z WARNING: Unable to load extension 'keyvault-preview: cannot import name 'KeyVaultArgumentContext' from 'azure.cli.command_modules.keyvault._command_type' (/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/keyvault/_command_type.py)'. Use --debug for more information.
2025-06-05T10:51:15.0401113Z WARNING: Unable to load extension 'spring-cloud: No module named 'msrestazure''. Use --debug for more information.
2025-06-05T10:51:49.8119245Z ERROR: No module named 'msrestazure'
2025-06-05T10:51:49.8122699Z Traceback (most recent call last):
2025-06-05T10:51:49.8131151Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
2025-06-05T10:51:49.8131740Z     cmd_result = self.invocation.execute(args)
2025-06-05T10:51:49.8132117Z                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8132595Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/knack/invocation.py", line 224, in execute
2025-06-05T10:51:49.8133125Z     cmd_result = parsed_args.func(params)
2025-06-05T10:51:49.8133473Z                  ^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8133937Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/knack/commands.py", line 149, in __call__
2025-06-05T10:51:49.8134433Z     return self.handler(*args, **kwargs)
2025-06-05T10:51:49.8134791Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8135434Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/knack/commands.py", line 256, in _command_handler
2025-06-05T10:51:49.8136029Z     result = op(client, **command_args) if client else op(**command_args)
2025-06-05T10:51:49.8136998Z                                                        ^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8137522Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/azdev/operations/statistics/__init__.py", line 90, in list_command_table
2025-06-05T10:51:49.8138173Z     codegen_info = _command_codegen_info(command_name, command, module_loader)
2025-06-05T10:51:49.8138589Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8139131Z   File "/mnt/vss/_work/1/s/env/lib/python3.11/site-packages/azdev/operations/statistics/__init__.py", line 259, in _command_codegen_info
2025-06-05T10:51:49.8139743Z     op = command_operation.get_op_handler(operation_path)
2025-06-05T10:51:49.8140056Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8140548Z   File "/mnt/vss/_work/1/s/src/azure-cli-core/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
2025-06-05T10:51:49.8141092Z     handler = import_module(mod_to_import)
2025-06-05T10:51:49.8141452Z               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8141930Z   File "/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-06-05T10:51:49.8142522Z     return _bootstrap._gcd_import(name[level:], package, level)
2025-06-05T10:51:49.8142917Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-06-05T10:51:49.8143313Z   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-06-05T10:51:49.8143817Z   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-06-05T10:51:49.8144360Z   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-06-05T10:51:49.8144858Z   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-06-05T10:51:49.8145372Z   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-06-05T10:51:49.8149318Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-06-05T10:51:49.8149935Z   File "/mnt/vss/_work/1/azure-cli-extensions/src/acrcssc/azext_acrcssc/custom.py", line 7, in <module>
2025-06-05T10:51:49.8150578Z     from .cssc import create_acrcssc, update_acrcssc, delete_acrcssc, show_acrcssc, cancel_runs, list_scan_status
2025-06-05T10:51:49.8151224Z   File "/mnt/vss/_work/1/azure-cli-extensions/src/acrcssc/azext_acrcssc/cssc.py", line 10, in <module>
2025-06-05T10:51:49.8151749Z     from .helper._taskoperations import (
2025-06-05T10:51:49.8152285Z   File "/mnt/vss/_work/1/azure-cli-extensions/src/acrcssc/azext_acrcssc/helper/_taskoperations.py", line 47, in <module>
2025-06-05T10:51:49.8152855Z     from ._workflow_status import WorkflowTaskStatus
2025-06-05T10:51:49.8153439Z   File "/mnt/vss/_work/1/azure-cli-extensions/src/acrcssc/azext_acrcssc/helper/_workflow_status.py", line 26, in <module>
2025-06-05T10:51:49.8154362Z     from msrestazure.azure_exceptions import CloudError
2025-06-05T10:51:49.8154833Z ModuleNotFoundError: No module named 'msrestazure'
2025-06-05T10:51:53.3521895Z 
2025-06-05T10:51:53.3576074Z ##[error]Bash exited with code '1'.
2025-06-05T10:51:53.3602487Z ##[section]Finishing: Bash

This indicates these extensions still have dependency on msrestazure:

  • apic-extension
  • spring-cloud
  • acrcssc
  • possibly more

FumingZhang
FumingZhang previously approved these changes Jun 6, 2025
evelyn-ys
evelyn-ys previously approved these changes Jun 10, 2025
@jiasli jiasli dismissed stale reviews from evelyn-ys and FumingZhang via e6ebfe1 June 10, 2025 09:03
@jiasli jiasli merged commit 4152901 into Azure:dev Jun 11, 2025
48 of 49 checks passed
@jiasli jiasli deleted the msrestazure branch June 11, 2025 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot Packaging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants