I recently observed some strange behavior on a Citrix XenApp 6.5 application server. After some piece of software crashed on this server (still working on this, perhaps csrss.exe contention), I was not able to establish an RDP session to this box. While looking at this server from the Citrix AppCenter, I saw that all 35 sessions were listed as DISCONNECTED. I could not connect from the console or RDP, just the command line. Instead of trying to kill these existing disconnected sessions from the command line, I hard-rebooted this server.
This was a mistake. When the server came back up I could now connect at the console and with SCCM, however I could not RDP to the server. Every time I started an RDP session to the server, the mstsc client would authenticate then immediately close out. It would flash a correctly-sized RDP window momentarily than just disappear and close out.
After troubleshooting for a few hours, I noticed the output of the qwinsta command:
C:\Users\msp>qwinsta /counter SESSIONNAME USERNAME ID STATE TYPE DEVICE services 0 Disc console 1 Conn ica-tcp 65536 Listen rdp-tcp 65537 Listen Total sessions created: 1 Total sessions disconnected: 6 Total sessions reconnected: 0
The “Total sessions disconnected” counter would increment every time I attempted to connect. At least now I knew the RDS services were not crashing, and were actually servicing my requests. I was also seeing various eventlog errors about Citrix HealthMon not allowing this server to accept new connections:
After doing some more digging, I found this excellent article here. It looks like similar issues, which lead to me finding the Citrix ICA Session registry key (HKLM\Software\Citrix\ICA\Session):
From here, I could see all the disconnected sessions that existed on the server before it was hard-rebooted. Citrix did not have a chance to clean this up. I deleted and recreated the Session key. Make sure to re-add permissions for the NT SERVICE\TermService account to read and write to this key. After doing this, I was immediately able to RDP back into this server. It seems as though something in Citrix was still looking at this key to determine either some kind of load balancing or licensing and causing the sessions to disconnect. I believe that if I was able to manually force a high session number I would’ve been able to create a new RDP session.