I came across an situation today where I was doing a clean up of Logical Switches in my NSX Lab where I was unable to remove a logical switch due to it apparently still being in use.
Heading to the NSX Manager via SSH I followed the Manager Log and tried to remove the logical switch again:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 |
2015-08-14 14:09:42.533 WST INFO http-nio-127.0.0.1-7441-exec-126 FirewallDao:624 - source virtualwire-7 used in 0 firewall rules {} 2015-08-14 14:09:42.534 WST INFO http-nio-127.0.0.1-7441-exec-126 FirewallDao:633 - Destination virtualwire-7 used in 0 firewall rules 2015-08-14 14:09:42.534 WST INFO http-nio-127.0.0.1-7441-exec-126 FirewallDao:642 - AppliedTo virtualwire-7 used in 0 firewall rules 2015-08-14 14:09:42.535 WST INFO http-nio-127.0.0.1-7441-exec-126 IsVirtualWireObjectInUse:36 - This VirtualWire Object virtualwire-7 is consumed in AppFireall rules: false 2015-08-14 14:09:42.543 WST ERROR http-nio-127.0.0.1-7441-exec-126 VirtualWireServiceImpl:1362 - VirtualWire virtualwire-7 has backing dvportgroup-274862 in use. Cannot delete virtualwire. 2015-08-14 14:09:42.543 WST WARN http-nio-127.0.0.1-7441-exec-126 RemoteInvocationTraceInterceptor:87 - Processing of VsmHttpInvokerServiceExporter remote call resulted in fatal exception: com.vmware.vshield.vsm.vdn.facade.VirtualWireFacade.delete com.vmware.vshield.vsm.vdn.exceptions.XvsException: core-services:841:virtualwire-7 resources are still in use. at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl.deleteInternal(VirtualWireServiceImpl.java:1363) at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl.delete_aroundBody26(VirtualWireServiceImpl.java:1309) at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl$AjcClosure27.run(VirtualWireServiceImpl.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.vmware.vshield.vsm.aspects.security.VsmSecuredAspect.secureCheck(VsmSecuredAspect.java:105) at com.vmware.vshield.vsm.aspects.security.VsmSecuredAspect.ajc$inlineAccessMethod$com_vmware_vshield_vsm_aspects_security_VsmSecuredAspect$com_vmware_vshield_vsm_aspects_security_VsmSecuredAspect$secureCheck(VsmSecuredAspect.java:1) at com.vmware.vshield.vsm.aspects.security.VsmSecuredAspect.secure(VsmSecuredAspect.java:58) at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl.delete(VirtualWireServiceImpl.java:1308) at com.vmware.vshield.vsm.vdn.facade.VirtualWireFacadeImpl.delete(VirtualWireFacadeImpl.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at com.sun.proxy.$Proxy395.delete(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:205) at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:38) at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78) at org.springframework.remoting.support.RemoteInvocationBasedExporter.invokeAndCreateResult(RemoteInvocationBasedExporter.java:114) at org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:73) at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:585) at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:369) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:97) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:100) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:78) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:35) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at com.vmware.vshield.vsm.security.utils.VsmSSOBasicAuthenticationFilter.doFilter(VsmSSOBasicAuthenticationFilter.java:200) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:187) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:109) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:168) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at com.bluelane.em.filter.GenericOperationsOnRequestFilter.doFilter(GenericOperationsOnRequestFilter.java:97) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1736) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1695) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Unknown Source) |
Taking a look through the log output the important information is contained in this line:
|
1 2 |
# 2015-08-14 14:09:42.543 WST ERROR http-nio-127.0.0.1-7441-exec-126 VirtualWireServiceImpl:1362 - VirtualWire virtualwire-7 has backing dvportgroup-274862 in use. Cannot delete virtualwire. |
So, lets search for this PortGroup via PowerCLI:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
PowerCLI C:\> Get-VDPortgroup | where { $_.Key -eq "dvportgroup-274862" } | fl VlanConfiguration : VLAN 769 Name : vxw-dvs-191-virtualwire-7-sid-5003-DC_1120783-NIC2 ExtensionData : VMware.Vim.DistributedVirtualPortgroup Key : dvportgroup-274862 Notes : Datacenter : Production PortBinding : Static NumPorts : 8 VDSwitch : Host01 dvSwitch IsUplink : False Id : DistributedVirtualPortgroup-dvportgroup-274862 Uid : /VIServer=zettagrid\adm.anthony.spiteri@172.17.0.103:443/DistributedPortgroup=DistributedVirtualPortgroup-dvportgroup-274862/ Client : VMware.VimAutomation.Vds.Impl.V1.VDClientImpl VirtualSwitch : Host01 dvSwitch |
In my case the PortGroup in question was part of a vCD OrgvDC Network that still had a VM connected to it. Once I removed the VM, took sharing off and deleted the Direct Network from vCD I was able to go back into the Web Client and successfully remove the Logical Switch.
Of course…there is an easier way than that to see which resources where attached to the Logical Switch…By taking note of the Logical Switch Segment ID you should be able to search through vCenter for the PortGroup that has that Segment ID number in the name and trace back by that…
