Replies: 8 comments 13 replies
-
|
continuing to get a hang on Update-DbaInstance on Windows Failover Cluster Servers. Have to issue a Stop-Operation Ctrl+Break Starting process [E:\TempUpdate\dbatools_KB5005226_Extract_85542f071d8740ee8f1e185722b3a414\setup.exe] with arguments [/quiet /allinstances /IAcceptSQLServerLicenseTerms] on SERVERB through CredSSP protocol |
Beta Was this translation helpful? Give feedback.
-
|
@nvarscar or @andreasjordan can you please take a look? 🙇🏼 |
Beta Was this translation helpful? Give feedback.
-
|
Perhaps try to run the function with |
Beta Was this translation helpful? Give feedback.
-
|
Ok we should have some patching coming up, we will run with DEBUG. Hope to overcome this HANG issue with reusing existing session. |
Beta Was this translation helpful? Give feedback.
-
|
updated dbatools to 1.1.60. I am experimenting with -DEBUG SINGLE SERVER, no hangs when using -DEBUG yet and walking through the Debug steps. |
Beta Was this translation helpful? Give feedback.
-
|
We havent had a hang with -DEBUG, but there have been 2 hangs without -DEBUG. They have occurred on PASSIVE side of WINDOWS CLUSTER (SQL SERVER not running). PS>TerminatingError(): "Connecting to remote server SERVERB.redacted failed with the following error message : What additional debugging can we do? |
Beta Was this translation helpful? Give feedback.
-
|
Do I understand correctly that the messages that you posted are from the terminal where the Update command is hanging and appear there once you have forced the remote server into a restart? And the command is "unstuck" after you restarted the server manually? |
Beta Was this translation helpful? Give feedback.
-
|
yes - "unstuck" after manually restarting target server. Checked with Update-DbaInstance after the "unstuck" |
Beta Was this translation helpful? Give feedback.
-
|
I got one with -DEBUG. It hung at the [16:51:25][Invoke-Command2] Found an existing session. I rebooted the server at 17:09. after checking the target C:\Program Files\Microsoft SQL Server\150\Setup Bootstrap\Log\Summary.txt timestamp 16:54 ( now 17:09) Instance MSSQLSERVER overall summary: |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately, all I can see is that the Invoke-Command2 (which launches Invoke-Command behind the scenes) didn't get any result code, effectively handing for a whole 1h+ until 17:09 when you restarted the server. This seems like it is an issue with Powershell Remoting which fails to return the results. Found exactly one link with similar symptoms to try: https://stackoverflow.com/questions/29638446/powershell-remote-invocation-mysteriously-hangs The only thing that can be done here from the dbatools end, as I don't see other options, is to add a timeout, implemented as an optional |
Beta Was this translation helpful? Give feedback.
-
|
Do you have any helps in looking at Windows Events or anything on the target server where the Invoke-Command2 is running? |
Beta Was this translation helpful? Give feedback.
-
|
Not sure if Windows Events would help you. It's seconds https://github.com/dataplat/dbatools/blob/development/internal/functions/flowcontrol/Invoke-Parallel.ps1#L45 |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, to clarify: this would be a new feature, it doesn't exist just yet. |
Beta Was this translation helpful? Give feedback.
-
|
Can you provide additional explanation of this description in Update-DbaInstance, does CredSSP have something to do with my issue?: |
Beta Was this translation helpful? Give feedback.
-
|
I'm not sure if a CredSSP would be causing all the trouble here, as it's merely a communication protocol. However, you can try to work with Kerberos or Default authentication. As long as the path to the exe file is available to the remote server without any extra auth (i.e. local disk access or network path with anonymous access enabled) |
Beta Was this translation helpful? Give feedback.
-
|
Update-DbaInstance does not like -Timeout or -RunSpaceTimeout parameters VERBOSE: [15:27:54][Invoke-Program] Starting process [E:\TempUpdate\dbatools_KB5008996_Extract_1b2dbaece67d4d4bb6960f577446426d\setup.exe] with arguments [/quie I am Stuck. This command should not use Credssp protocol - that was only needed for the extract>>> |
Beta Was this translation helpful? Give feedback.
-
|
CallStack : At Invoke-Command2, C:\Program Files\WindowsPowerShell\Modules\dbatools\1.1.74\allcommands.ps1: Line 89318 reboot target manually* CallStack : At Invoke-Program, C:\Program Files\WindowsPowerShell\Modules\dbatools\1.1.74\allcommands.ps1: Line 90243 |
Beta Was this translation helpful? Give feedback.
-
|
dbatools 1.1.80 VERBOSE: [13:39:21][Invoke-Program] Starting process [E:\TempUpdate\dbatools_KB5007182_Extract_6c1a087e89ad413ab250df5ef19f35f0\setup.exe] with arguments [/quiet /allinstances /IAcceptSQLServerLicenseTerms] on MYComputer through Credssp protocol ComputerName : MyComputer WARNING: [13:56:50][Update-DbaInstance] Update failed: Update failed with exit code |
Beta Was this translation helpful? Give feedback.
-
|
what is the exit code? VERBOSE: [11:18:06][Invoke-Program] Starting process [E:\TempUpdate\dbatools_KB5010786_Extract_8a51cd036438416e8ed8ef2151b887c5\setup.exe] with arguments [/quiet /allinstances /IAcceptSQLServerLicenseTerms] on UHAIMDBSQA1.umhs.med.umich.edu through Credssp protocol When went to check the script DBAdvanced up associated with this, we see that it’s calling this code: But we’re not seeing this 3010 code when we’re outputting this to the script which makes it confusing on whether or not we’re actually grabbing this next line. I also want to point out that in the Invoke script we see: $updateResult = Invoke-Program @execParams -Path "$spExtractPath\setup.exe" -ArgumentList $programArgumentList -WorkingDirectory $spExtractPath -Fallback |
Beta Was this translation helpful? Give feedback.
-
|
So today it is working. I get this> Somedays it works somedays it hangs. |
Beta Was this translation helpful? Give feedback.
-
|
Removed parameter -ExtractPath. No more hangs. However it appears to be executing from the network -PATH location even though as an intial step the Update-DbaInstance indicates that it is copying the patch executable to a directory created on the target computer e.g. C:\dbatools_KB5number_Extract_string. |
Beta Was this translation helpful? Give feedback.
-
DBATOOLS RELEASE 1.1.39.
I am trying to execute Update-DbaInstance on a Active/Passive Windows Failover Cluster server environment:
patch the passive node and restart (SQL SERVER INSTANCE is up on the ACTIVE NODE)
force failover Active to Passive (newly patched server becomes active)
patch the passive node and restart.
Update-DbaInstance gets hung so after 30 minutes I stop the process. The patch is completed on the server the hang occurred. I restart the server manually and I get 2 restarts in the event log. I dont have a lot to go on, Get-DbatoolsLog
I see it starting process to execute the setup.exe
Then it does
CallStack : {at Get-DbatoolsLog, C:\Program Files\WindowsPowerShell\Modules\dbatools\1.1.39\allcommands.ps1:
line 37261, at , : line 1}
ComputerName : mycomputer
File : C:\Program Files\WindowsPowerShell\Modules\dbatools\1.1.39\allcommands.ps1
FunctionName : Invoke-Command2
Level : Debug
Line : 87953
Message : Found an existing session
dbatools_CredSSP_stuff, reusing it
ModuleName : dbatools
Runspace : ca00a1e8-03a8-4b4f-92fd-f35d63cfd2c3
Tags : {}
TargetObject :
Timestamp : 11/29/2021 6:10:44 PM
Type : Debug
Username : thatsme
and this gets repeated once after I break and execute the get-dbatoolslog 20 minutes later.
Any ideas would be helpful.
Beta Was this translation helpful? Give feedback.
All reactions