Friday, April 10, 2009

Fixing XDMCP logins on Solaris

I use XDMCP logins via Xnest quite frequently with X Window to get graphical console on another machine, mostly to do an install that requires a graphical environment (Oracle quickly comes to mind) or launch an application from the point of view of a different machine. With the Solaris Nevada (Solaris Excepress or SXDE/SXCE) builds on x86 I noticed that the XDMCP logins no longer work and when using Xnest querying the other host I end up with an empty window -- in other words the machine is not accepting the XDMCP requests. Scratching my head I decided to poke around the to see if dtlogin process responsible for accepting the XDMCP requests runs with any special arguments. Sure enough for some reason the Nevada builds have "-udpPort 0" argument passed to dtlogin process:

# ps -ef grep dtlogin
root 4919 4838 0 12:37:58 ? 0:00 /usr/dt/bin/dtlogin -daemon -udpPort 0
root 4838 1 0 12:36:50 ? 0:00 /usr/dt/bin/dtlogin -daemon -udpPort 0

The XDMCP requests are usually accepted on udp port 177, so udp port set to 0 would surely remove the dtlogin's ability to accept the requests. I'm not sure why the Solaris developers decided to do that, but I'm guess it was done to improve the security of the installation out of the box. Knowing this fact it is easy to fix this pesky problem. All I had to do is to change the udpPort property value for cde-login service in SMF repository:

# svccfg
svc:> select cde-login
svc:/application/graphical-login/cde-login> listprop *arg*dtlogin/args astring " -udpPort 0"

So here we go, all we need to do is to set the dtlogin/args property to " -udpPort 177" and we should be in business as usual:

svc:/application/graphical-login/cde-login> setprop
dtlogin/args=astring:" -udpPort 177"
svc:/application/graphical-login/cde-login> quit
#

Now we can just restart the cde-login service and XDMCP login should work (Note: if you're doing this in an X Window session, the session will be restarted):
# svcadm restart cde-login

As a configmation, lets see if the dtlogin process is running with correct arguments:
# ps -ef grep dtlogin
root 4919 4838 0 12:37:58 ? 0:00 /usr/dt/bin/dtlogin -daemon -udpPort 177
root 4838 1 0 12:36:50 ? 0:00 /usr/dt/bin/dtlogin -daemon -udpPort 177

Sure enough, we've got the dtlogin process listening on the correct port. We should now be in business as usual:
# /usr/openwin/bin/Xnest :1 -query nevada

gives me a nice login as it should have in the very beginning

No comments: