Skip to content

[VMWare] error when detaching volume #6283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nvazquez merged 3 commits into from
Apr 27, 2022
Merged

[VMWare] error when detaching volume #6283

nvazquez merged 3 commits into from
Apr 27, 2022

Conversation

@SadiJr
Copy link
Contributor

@SadiJr SadiJr commented Apr 18, 2022 •
edited
Loading

Description

Using the VMWare (version 6.5), when trying to detach a volume, vCenter detaches it successfully; however, ACS throws an exception. It already was reported in VMWare and ACS' communities, but it hasn't been fixed yet. For more information about this error, please check this links ([1], [2], [3]).

To know why ACS tries to check folder content, please check these links ([4], [5]). This error apparently only occurs with datastores that use the NFS protocol and n specific versions of VMWare. Users using VMFS or VMWare versions greater than 6.7 have not reported this error. If the operation performed is a volume detach, it was successful. If you want to know why this error occurs in VMWare, please contact VMWare's technical support.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

How Has This Been Tested?

It was tested in a local lab:

  1. I created a new VM with two disks;
  2. I tried to detach the datadisk;
  3. Before, an exception was thrown, and, in the ACS database, the VM was still with two disks, however, in vCenter the volume was already detached;
  4. Now, detach volume process finish with success.

@acs-robot
Copy link

acs-robot commented Apr 18, 2022

Found Java/XML changes, kicking packaging job
@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Apr 18, 2022

@acs-robot a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented Apr 18, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3204

@acs-robot
Copy link

acs-robot commented Apr 18, 2022

PR Coverage Report

CLASS INSTRUCTION MISSED INSTRUCTION COVERED BRANCH MISSED BRANCH COVERED LINE MISSED LINE COVERED
VmwareStorageLayoutHelper 1287 0 114 0 200 0

@nvazquez nvazquez added this to the 4.17.0.0 milestone Apr 20, 2022
@nvazquez
Copy link
Contributor

nvazquez commented Apr 20, 2022

Hi @weizhouapache @Pearl1594 can you please review?

@weizhouapache
Copy link
Member

weizhouapache commented Apr 20, 2022

@SadiJr @nvazquez
there are 5 links in this PR which is difficult to maintain.
would it be better to point to a link which contains all the information ?

@nvazquez
Copy link
Contributor

nvazquez commented Apr 21, 2022

@blueorangutan test centos7 vmware-65u2

@blueorangutan
Copy link

blueorangutan commented Apr 21, 2022

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-65u2) has been kicked to run smoke tests

@nvazquez
Copy link
Contributor

nvazquez commented Apr 22, 2022

@weizhouapache I think the error message is very descriptive as it is but will be +1 to point to a single location with all the information - @SadiJr please advise

@nvazquez
Copy link
Contributor

nvazquez commented Apr 22, 2022

@blueorangutan test centos7 vmware-65u2

@blueorangutan
Copy link

blueorangutan commented Apr 22, 2022

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-65u2) has been kicked to run smoke tests

@DaanHoogland DaanHoogland reopened this Apr 22, 2022
@blueorangutan
Copy link

blueorangutan commented Apr 22, 2022

Trillian Build Failed (tid-3997)

@DaanHoogland DaanHoogland reopened this Apr 22, 2022
@acs-robot
Copy link

acs-robot commented Apr 22, 2022

Found UI changes, kicking a new UI QA build
@blueorangutan ui

@SadiJr
Copy link
Contributor Author

SadiJr commented Apr 22, 2022

@weizhouapache @nvazquez thansk for the suggestion. I changed the description of the PR, adding all the links, and adding the link of this PR in the final message. What do you think?

@weizhouapache
Copy link
Member

weizhouapache commented Apr 22, 2022

@SadiJr thanks.
my concern is, is it good to ignore all the errors in detch volume on vmware ?
can we ignore the error only if some errors happen, for example by checking the error message e.getMessage().contains("xxxxxx") ?

cc @nvazquez

@acs-robot
Copy link

acs-robot commented Apr 22, 2022

PR Coverage Report

CLASS INSTRUCTION MISSED INSTRUCTION COVERED BRANCH MISSED BRANCH COVERED LINE MISSED LINE COVERED
Network 554 0 42 0 107 0
Volume 109 0 2 0 44 0
VirtualMachineGuru 78 0 4 0 15 0
NetworkOrchestrationService 0 101 0 0 0 10
ConnectedAgentAttache 149 0 20 0 40 0
VirtualMachineManagerImpl 15659 0 1500 0 3079 0
NetworkOrchestrator 9806 0 1206 0 1928 0
DataCenterVnetVO 58 0 0 0 24 0
VlanVO 186 0 2 0 72 0
HostDaoImpl 4983 0 180 0 803 0
AccountGuestVlanMapVO 46 0 0 0 19 0
NetworkOfferingVO 414 31 0 0 126 12
VolumeVO 529 133 4 0 179 39
SystemVmTemplateRegistration 1773 0 86 0 376 0
Upgrade41520to41600 262 17 16 0 62 5
Upgrade41610to41700 159 7 6 0 43 2
DomainRouterVO 116 50 0 0 41 14
ConsoleProxyDaoImpl 635 0 20 0 144 0
DomainRouterDaoImpl 1600 0 22 0 228 0
VolumeObject 773 665 77 39 178 117
CloudStackContextLoaderListener 77 0 2 0 21 0
LibvirtComputingResource 8888 1988 1087 147 1983 451
LibvirtStartCommandWrapper 75 295 16 22 15 72
VmwareResource 20395 0 2274 0 4397 0
VmwareStorageLayoutHelper 1263 0 114 0 196 0
VmwareStorageProcessor 9892 9 940 0 2122 2
CitrixResourceBase 14658 557 1456 34 3169 122
CitrixCheckSshCommandWrapper 15 51 3 3 3 14
CitrixNetworkElementCommandWrapper 0 14 0 0 0 4
CitrixRebootRouterCommandWrapper 28 25 3 1 5 7
CitrixStartCommandWrapper 690 62 81 1 116 16
KubernetesClusterManagerImpl 4724 0 480 0 760 0
KubernetesClusterActionWorker 1540 0 114 0 281 0
KubernetesClusterResourceModifierActionWorker 1843 0 140 0 325 0
MetricsServiceImpl 2130 0 134 0 446 0
ClusterMetricsResponse 523 0 132 0 60 0
VmMetricsResponse 142 0 22 0 25 0
VolumeMetricsResponse 69 0 8 0 10 0
ZoneMetricsResponse 501 0 126 0 56 0
DomainChecker 1206 0 300 0 238 0
ApiDBUtils 2374 0 210 0 592 0
ApiResponseHelper 12144 0 1274 0 2779 0
ResponseObjectTypeAdapter 161 8 14 0 37 2
ParamProcessWorker 1050 0 155 0 241 0
QueryManagerImpl 14257 0 1248 0 2409 0
DomainRouterJoinDaoImpl 801 0 80 0 195 0
VolumeJoinDaoImpl 770 0 94 0 171 0
DomainRouterJoinVO 237 0 0 0 80 0
VolumeJoinVO 267 0 0 0 93 0
ConfigurationManagerImpl 18104 0 3032 0 3570 0
ConsoleProxyManagerImpl 3717 0 423 0 726 0
LibvirtServerDiscoverer 994 0 116 0 218 0
IpAddressManagerImpl 4045 0 461 0 806 0
NetworkModelImpl 6182 0 838 0 1300 0
NetworkServiceImpl 13383 0 1864 0 2551 0
GuestNetworkGuru 622 298 98 34 124 64
PrivateNetworkGuru 394 0 46 0 88 0
LoadBalancingRulesManagerImpl 6024 0 666 0 1254 0
NetworkHelperImpl 2073 0 274 0 441 0
VirtualNetworkApplianceManagerImpl 7585 0 780 0 1521 0
RulesManagerImpl 4074 0 492 0 790 0
VpcManagerImpl 6883 0 762 0 1314 0
ConfigurationServerImpl 2061 0 176 0 495 0
ManagementServerImpl 12159 0 1076 0 2428 0
StatsCollector 2178 0 124 0 359 0
StorageManagerImpl 8580 0 976 0 1710 0
VolumeApiServiceImpl 10851 0 1500 0 2035 0
AccountManagerImpl 6446 0 906 0 1351 0
UserVmManagerImpl 20798 0 2566 0 3869 0
CAManagerImpl 633 0 80 0 129 0
MockNetworkManagerImpl 485 0 22 0 83 0
SecondaryStorageManagerImpl 3544 149 343 11 629 32
VirtualMachineMO 8953 135 1063 17 1982 31

@nvazquez
Copy link
Contributor

nvazquez commented Apr 23, 2022

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Apr 23, 2022

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@nvazquez
Copy link
Contributor

nvazquez commented Apr 25, 2022

@blueorangutan test centos7 vmware-65u2

1 similar comment
@nvazquez
Copy link
Contributor

nvazquez commented Apr 25, 2022

@blueorangutan test centos7 vmware-65u2

@blueorangutan
Copy link

blueorangutan commented Apr 25, 2022

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-65u2) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Apr 25, 2022

Trillian test result (tid-4018)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 17646 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6283-t4018-vmware-65u2.zip
Smoke tests completed. 49 look OK, 0 have errors
Only failed tests results shown below:

Test Result Time (s) Test File

@nvazquez
Copy link
Contributor

nvazquez commented Apr 25, 2022 •
edited
Loading

@SadiJr thanks for the changes in the error message - I was not able to reproduce the issue on main branch using vmware 6.5 + NFS following the steps in the PR description, however changes LGTM

@nvazquez nvazquez modified the milestones: 4.17.0.0, 4.17.1.0 Apr 26, 2022
@SadiJr
Copy link
Contributor Author

SadiJr commented Apr 26, 2022

@SadiJr thanks. my concern is, is it good to ignore all the errors in detch volume on vmware ? can we ignore the error only if some errors happen, for example by checking the error message e.getMessage().contains("xxxxxx") ?

cc @nvazquez

Done, thanks for the suggestion.

@acs-robot
Copy link

acs-robot commented Apr 26, 2022

Found UI changes, kicking a new UI QA build
@blueorangutan ui

@blueorangutan
Copy link

blueorangutan commented Apr 26, 2022

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented Apr 26, 2022

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6283 (SL-JID-1474)

weizhouapache
weizhouapache approved these changes Apr 26, 2022
View reviewed changes
Copy link
Member

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@weizhouapache
Copy link
Member

weizhouapache commented Apr 26, 2022

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Apr 26, 2022

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@acs-robot
Copy link

acs-robot commented Apr 26, 2022

PR Coverage Report

CLASS INSTRUCTION MISSED INSTRUCTION COVERED BRANCH MISSED BRANCH COVERED LINE MISSED LINE COVERED
Network 554 0 42 0 107 0
Volume 109 0 2 0 44 0
VirtualMachineGuru 78 0 4 0 15 0
NetworkOrchestrationService 0 101 0 0 0 10
AlertManager 79 0 0 0 7 0
ConnectedAgentAttache 149 0 20 0 40 0
VirtualMachineManagerImpl 15659 0 1500 0 3079 0
NetworkOrchestrator 9910 0 1224 0 1945 0
CapacityVO 261 0 0 0 80 0
DataCenterVnetVO 58 0 0 0 24 0
VlanVO 186 0 2 0 72 0
VlanDaoImpl 1307 0 54 0 218 0
EventVO 140 0 0 0 58 0
HostDaoImpl 4983 0 180 0 803 0
AccountGuestVlanMapVO 46 0 0 0 19 0
NetworkDetailsDaoImpl 30 0 4 0 5 0
NetworkVO 396 159 19 1 140 44
VpcOfferingDaoImpl 137 0 0 0 25 0
VpcOfferingDetailsDaoImpl 173 0 10 0 31 0
NetworkOfferingVO 414 31 0 0 126 12
NetworkOfferingDaoImpl 691 0 22 0 114 0
VolumeVO 529 133 4 0 179 39
SystemVmTemplateRegistration 1773 0 86 0 376 0
Upgrade41520to41600 262 17 16 0 62 5
Upgrade41610to41700 159 7 6 0 43 2
DomainRouterVO 116 50 0 0 41 14
ConsoleProxyDaoImpl 635 0 20 0 144 0
DomainRouterDaoImpl 1600 0 22 0 228 0
NicDaoImpl 1392 0 12 0 193 0
SnapshotObject 748 30 64 0 151 11
VolumeObject 773 665 77 39 178 117
CloudStackContextLoaderListener 77 0 2 0 21 0
LibvirtComputingResource 8888 1988 1087 147 1983 451
LibvirtStartCommandWrapper 75 295 16 22 15 72
MockNetworkManagerImpl 510 0 24 0 88 0
SimulatorManagerImpl 1394 0 214 0 309 0
VmwareResource 20395 0 2274 0 4397 0
VmwareStorageLayoutHelper 1297 0 118 0 201 0
VmwareStorageProcessor 9892 9 940 0 2122 2
CitrixResourceBase 14658 557 1456 34 3169 122
CitrixCheckSshCommandWrapper 15 51 3 3 3 14
CitrixNetworkElementCommandWrapper 0 14 0 0 0 4
CitrixRebootRouterCommandWrapper 28 25 3 1 5 7
CitrixStartCommandWrapper 690 62 81 1 116 16
KubernetesClusterManagerImpl 4724 0 480 0 760 0
KubernetesClusterActionWorker 1540 0 114 0 281 0
KubernetesClusterResourceModifierActionWorker 1843 0 140 0 325 0
CreateKubernetesClusterCmd 215 0 18 0 61 0
MetricsServiceImpl 2130 0 134 0 446 0
ClusterMetricsResponse 523 0 132 0 60 0
VmMetricsResponse 142 0 22 0 25 0
VolumeMetricsResponse 69 0 8 0 10 0
ZoneMetricsResponse 501 0 126 0 56 0
BigSwitchBcfGuestNetworkGuru 662 0 62 0 145 0
CreateServiceInstanceCmd 201 0 16 0 45 0
ContrailManagerImpl 2218 0 234 0 497 0
StopNetScalerVMCmd 114 0 12 0 26 0
OvsGuestNetworkGuru 409 0 48 0 90 0
VxlanGuestNetworkGuru 89 180 11 21 16 39
LinkAccountToLdapCmd 77 105 6 4 18 23
ListAndSwitchSAMLAccountCmd 182 205 30 22 38 34
DomainChecker 1206 0 300 0 238 0
AlertManagerImpl 2133 0 183 0 418 0
ApiAsyncJobDispatcher 212 0 12 0 53 0
ApiDBUtils 2374 0 210 0 592 0
ApiDispatcher 231 0 42 0 55 0
ApiResponseHelper 12400 0 1292 0 2837 0
ApiServer 2868 112 346 0 672 14
ResponseObjectTypeAdapter 161 8 14 0 37 2
ParamProcessWorker 1050 0 155 0 241 0
QueryManagerImpl 14429 0 1270 0 2439 0
DomainRouterJoinDaoImpl 801 0 80 0 195 0
NetworkOfferingJoinDaoImpl 278 0 12 0 57 0
VolumeJoinDaoImpl 770 0 94 0 171 0
VpcOfferingJoinDaoImpl 172 0 6 0 36 0
AsyncJobJoinVO 86 0 0 0 31 0
DomainRouterJoinVO 237 0 0 0 80 0
EventJoinVO 100 0 0 0 35 0
NetworkOfferingJoinVO 219 0 0 0 79 0
VolumeJoinVO 267 0 0 0 93 0
VpcOfferingJoinVO 120 0 0 0 37 0
ConfigurationManagerImpl 18652 0 3092 0 3639 0
ConsoleProxyManagerImpl 3717 0 423 0 726 0
ActionEventInterceptor 398 19 44 2 90 7
ActionEventUtils 929 0 90 0 198 0
EventJoinDaoImpl 329 0 18 0 58 0
HypervisorGuruBase 669 38 52 0 133 5
LibvirtServerDiscoverer 994 0 116 0 218 0
ExternalFirewallDeviceManagerImpl 1885 0 198 0 381 0
ExternalLoadBalancerDeviceManagerImpl 2591 0 288 0 593 0
IpAddressManagerImpl 4084 0 463 0 814 0
Ipv6AddressManagerImpl 371 0 32 0 70 0
NetworkMigrationManagerImpl 1706 0 90 0 355 0
NetworkModelImpl 6182 0 838 0 1300 0
NetworkServiceImpl 13648 0 1902 0 2613 0
FirewallManagerImpl 2600 0 399 0 466 0
ExternalGuestNetworkGuru 849 0 92 0 158 0
GuestNetworkGuru 702 298 114 34 142 64
PrivateNetworkGuru 394 0 46 0 88 0
PublicNetworkGuru 341 0 42 0 77 0
LoadBalancingRulesManagerImpl 6024 0 666 0 1254 0
CommandSetupHelper 4088 0 300 0 771 0
NetworkHelperImpl 2092 0 278 0 445 0
VirtualNetworkApplianceManagerImpl 7767 0 792 0 1544 0
RulesManagerImpl 4074 0 492 0 790 0
NetworkACLManagerImpl 861 0 118 0 182 0
NetworkACLServiceImpl 2701 0 304 0 520 0
VpcManagerImpl 6936 0 770 0 1321 0
Site2SiteVpnManagerImpl 2298 0 214 0 468 0
ResourceManagerImpl 8524 0 982 0 1659 0
RollingMaintenanceManagerImpl 2041 0 192 0 363 0
ResourceIconManagerImpl 328 0 36 0 70 0
ConfigurationServerImpl 2061 0 176 0 495 0
ManagementServerImpl 12197 0 1076 0 2428 0
StatsCollector 2178 0 124 0 359 0
StorageManagerImpl 8580 0 976 0 1710 0
VolumeApiServiceImpl 10873 0 1502 0 2041 0
SnapshotManagerImpl 4195 0 410 0 753 0
SnapshotSchedulerImpl 871 0 81 0 197 0
ResourceManagerUtilImpl 356 0 24 0 71 0
TemplateManagerImpl 5011 0 696 0 1045 0
AccountManagerImpl 6460 0 906 0 1352 0
UserVmManagerImpl 20798 0 2566 0 3869 0
VMSnapshotManagerImpl 3151 0 296 0 628 0
RoleManagerImpl 737 0 114 0 147 0
AffinityGroupServiceImpl 791 0 112 0 172 0
AnnotationManagerImpl 1401 0 167 0 284 0
BackupManagerImpl 2779 0 224 0 489 0
CAManagerImpl 633 0 80 0 129 0
DirectDownloadManagerImpl 1684 0 186 0 361 0
HAManagerImpl 1391 0 194 0 231 0
BasicNetworkVisitor 535 0 28 0 112 0
OutOfBandManagementServiceImpl 1182 0 130 0 237 0
PowerOperationTask 109 0 0 0 17 0
MockNetworkManagerImpl 510 0 24 0 88 0
SecondaryStorageManagerImpl 3544 149 343 11 629 32
VirtualMachineMO 8953 135 1063 17 1982 31

@blueorangutan
Copy link

blueorangutan commented Apr 26, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3312

@nvazquez nvazquez modified the milestones: 4.17.1.0, 4.17.0.0 Apr 27, 2022
@nvazquez nvazquez merged commit 4cc830d into apache:main Apr 27, 2022
@SadiJr SadiJr deleted the vmware-fix-detach-volume-exception branch May 4, 2022 15:19
DaanHoogland pushed a commit to shapeblue/cloudstack that referenced this pull request Jun 14, 2022
@SadiJr @DaanHoogland
[VMWare] error when detaching volume (apache#6283)
43cc933 
* Fix exception, in VMWare 6.5, when trying to detach volume

* Address reviews

* Address reviews

Co-authored-by: SadiJr <sadi@scclouds.com.br>
shwstppr pushed a commit to shapeblue/cloudstack that referenced this pull request Sep 22, 2022
@DaanHoogland
Merge pull request #56 from shapeblue/sb10-ammendment
f9262b6 
[VMWare] error when detaching volume (apache#6283)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nvazquez nvazquez nvazquez approved these changes

@weizhouapache weizhouapache weizhouapache approved these changes

@Pearl1594 Pearl1594 Awaiting requested review from Pearl1594

Assignees

No one assigned

Labels

component:vmware type:bug

Projects

No open projects
Apache CloudStack 4.17.0
Status: Done

Milestone

4.17.0.0

Development

Successfully merging this pull request may close these issues.

6 participants

@SadiJr @acs-robot @blueorangutan @nvazquez @weizhouapache @DaanHoogland