[X2Go-User] Log-in problem after NVIDIA card update

[Alec Bayo]

Hi all,

Asking for help please troubleshooting an issue with x2go.

Inbound connections stopped working after I updated the kernel and nvidia
drivers on my centos workstation (the x2go server).

On the client, the error message says "Connection failed. Failed to start
X2Go Agent session with ID account-_____. X2Go Agent terminated
unexpectedly. Aborting session startup."

[If I attempt a connecion again the client shows a previous apparently
pre-existing connection, but I cannot resume it.]

On the server, the session log says "Error: Aborting session with 'Unable
to open display 'nx/nx,options=...". I have attempted uninstalling and
reinstalling x2goserver from epel.

This symptoms are similar to an issue Mihai Moldovan resolved for Qianqian
Fang in April:
https://www.mail-archive.com/x2go-***@lists.x2go.org/msg02809.html

However, I do not have the offending line set in LD_LIBRARY_PATH which
solved the issue for Qianqian.

#Contents of LD_LIBRARY_PATH (this is from env after ssh-in).

LD_LIBRARY_PATH=/usr/local/cuda/lib64:/home/[...]/caffe/distribute/lib/::/usr/lib:/usr/lib/python2.7


I also attempted commenting out this setting from .bashrc, and that did not
fix it.

I've copied below answers to some questions Mihai asked Qianqian. The only
strange thing I noticed is that Mihai asked for the contents of
/usr/lib64/nx/X11 , and this is empty. The library Qianqian listed in there
is located at /usr/lib64 on my server.

Some more details below, answering questions that were made on this
earlier, similar troubleshooting request.

Best regards, thanks for any help,
Ale


# Please make sure that nx-libs 3.5.99.16 is installed on the server side:

Package nx-libs-3.5.99.16-1.el7.x86_64 already installed and latest version


# Client up to date:

x2goclient is already the newest version (4.1.1.1-2).


# Perhaps this is useful, these lines are also from env:

QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/1
QT_GRAPHICSSYSTEM_CHECKED=1
QTLIB=/usr/lib64/qt-3.3/lib
QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins



# Contents of /usr/lib64/nx/X11

$ ls /usr/lib64/nx
bin X11

$ ls -lah /usr/lib64/nx/X11
total 0
drwxr-xr-x. 2 root root 6 Aug 18 14:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..

$ ls -lah /usr/lib64/nx/bin
total 0
drwxr-xr-x. 2 root root 36 Aug 17 21:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..
lrwxrwxrwx. 1 root root 16 Aug 17 16:09 nxagent -> /usr/bin/nxagent
lrwxrwxrwx. 1 root root 7 Aug 17 21:28 x2goagent -> nxagent

$ ls -lah /usr/bin/nxagent
-rwxr-xr-x. 1 root root 4.5M Jul 18 17:47 /usr/bin/nxagent


# Where I found the related libraries

$ ls -lah /usr/lib64/*NX*

lrwxrwxrwx. 1 root root 18 Aug 18 14:15 /usr/lib64/libNX_X11.so.6 ->
libNX_X11.so.6.3.0
-rwxr-xr-x. 1 root root 1.3M Jul 18 17:47 /usr/lib64/libNX_X11.so.6.3.0

[Ulrich Sibiller]

You seem to be missing a libX11 link to libNX_X11. Please run ldd
/usr/bin/nxagent and post the result.

Uli

Post by Alec Bayo
Hi all,
Asking for help please troubleshooting an issue with x2go.
Inbound connections stopped working after I updated the kernel and nvidia
drivers on my centos workstation (the x2go server).
On the client, the error message says "Connection failed. Failed to start
X2Go Agent session with ID account-_____. X2Go Agent terminated
unexpectedly. Aborting session startup."
[If I attempt a connecion again the client shows a previous apparently
pre-existing connection, but I cannot resume it.]
On the server, the session log says "Error: Aborting session with 'Unable
to open display 'nx/nx,options=...". I have attempted uninstalling and
reinstalling x2goserver from epel.
This symptoms are similar to an issue Mihai Moldovan resolved for Qianqian
However, I do not have the offending line set in LD_LIBRARY_PATH which
solved the issue for Qianqian.
#Contents of LD_LIBRARY_PATH (this is from env after ssh-in).
LD_LIBRARY_PATH=/usr/local/cuda/lib64:/home/[...]/caffe/distribute/lib/::/usr/lib:/usr/lib/python2.7
I also attempted commenting out this setting from .bashrc, and that did
not fix it.
I've copied below answers to some questions Mihai asked Qianqian. The only
strange thing I noticed is that Mihai asked for the contents of
/usr/lib64/nx/X11 , and this is empty. The library Qianqian listed in there
is located at /usr/lib64 on my server.
Some more details below, answering questions that were made on this
earlier, similar troubleshooting request.
Best regards, thanks for any help,
Ale
Package nx-libs-3.5.99.16-1.el7.x86_64 already installed and latest version
x2goclient is already the newest version (4.1.1.1-2).
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/1
QT_GRAPHICSSYSTEM_CHECKED=1
QTLIB=/usr/lib64/qt-3.3/lib
QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins
# Contents of /usr/lib64/nx/X11
$ ls /usr/lib64/nx
bin X11
$ ls -lah /usr/lib64/nx/X11
total 0
drwxr-xr-x. 2 root root 6 Aug 18 14:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..
$ ls -lah /usr/lib64/nx/bin
total 0
drwxr-xr-x. 2 root root 36 Aug 17 21:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..
lrwxrwxrwx. 1 root root 16 Aug 17 16:09 nxagent -> /usr/bin/nxagent
lrwxrwxrwx. 1 root root 7 Aug 17 21:28 x2goagent -> nxagent
$ ls -lah /usr/bin/nxagent
-rwxr-xr-x. 1 root root 4.5M Jul 18 17:47 /usr/bin/nxagent
# Where I found the related libraries
$ ls -lah /usr/lib64/*NX*
lrwxrwxrwx. 1 root root 18 Aug 18 14:15 /usr/lib64/libNX_X11.so.6 ->
libNX_X11.so.6.3.0
-rwxr-xr-x. 1 root root 1.3M Jul 18 17:47 /usr/lib64/libNX_X11.so.6.3.0
_______________________________________________
x2go-user mailing list
https://lists.x2go.org/listinfo/x2go-user

[Alec Bayo]

Thanks for the reply Uli. Pasted output below

Also, I was scratching my head some more after sending the previous email.
Would I run the two lines below to add these links manually? Or should I do
something more/else?

ln -s /usr/lib64/libNX_X11.so.6 /usr/lib64/nx/X11/libX11.so.6
ln -s /usr/lib64/libNX_X11.so.6.3.0 /usr/lib64/nx/X11/libX11.so.6.3.0


Best, A.

$ ldd /usr/bin/nxagent
linux-vdso.so.1 => (0x00007ffe21fc2000)
libtirpc.so.1 => /lib64/libtirpc.so.1 (0x00007fa9d1cd4000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007fa9d1995000)
libXcomp.so.3 => /lib64/libXcomp.so.3 (0x00007fa9d166c000)
libXcompshad.so.3 => /lib64/libXcompshad.so.3 (0x00007fa9d1460000)
libXrender.so.1 => /lib64/libXrender.so.1 (0x00007fa9d1254000)
libXfixes.so.3 => /lib64/libXfixes.so.3 (0x00007fa9d104e000)
libXfont.so.1 => /lib64/libXfont.so.1 (0x00007fa9d0e1a000)
libXcomposite.so.1 => /lib64/libXcomposite.so.1 (0x00007fa9d0c16000)
libXinerama.so.1 => /lib64/libXinerama.so.1 (0x00007fa9d0a13000)
libXdmcp.so.6 => /lib64/libXdmcp.so.6 (0x00007fa9d080d000)
libNX_X11.so.6 => /lib64/libNX_X11.so.6 (0x00007fa9d04c9000)
libXext.so.6 => /lib64/libXext.so.6 (0x00007fa9d02b7000)
libz.so.1 => /lib64/libz.so.1 (0x00007fa9d00a1000)
libm.so.6 => /lib64/libm.so.6 (0x00007fa9cfd9e000)
libXpm.so.4 => /lib64/libXpm.so.4 (0x00007fa9cfb8c000)
libxml2.so.2 => /lib64/libxml2.so.2 (0x00007fa9cf822000)
libpixman-1.so.0 => /lib64/libpixman-1.so.0 (0x00007fa9cf578000)
libpng15.so.15 => /lib64/libpng15.so.15 (0x00007fa9cf34d000)
libjpeg.so.62 => /lib64/libjpeg.so.62 (0x00007fa9cf0f8000)
libc.so.6 => /lib64/libc.so.6 (0x00007fa9ced2a000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fa9ceadd000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fa9ce7f5000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fa9ce5c1000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fa9ce3bd000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fa9ce1a3000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa9cdf86000)
libxcb.so.1 => /lib64/libxcb.so.1 (0x00007fa9cdd5e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fa9cdb5a000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa9cd852000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa9cd63c000)
libXdamage.so.1 => /lib64/libXdamage.so.1 (0x00007fa9cd439000)
libXrandr.so.2 => /lib64/libXrandr.so.2 (0x00007fa9cd22d000)
libXtst.so.6 => /lib64/libXtst.so.6 (0x00007fa9cd027000)
libfreetype.so.6 => /lib64/libfreetype.so.6 (0x00007fa9ccd81000)
libfontenc.so.1 => /lib64/libfontenc.so.1 (0x00007fa9ccb79000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa9d25bd000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fa9cc953000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fa9cc744000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fa9cc540000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fa9cc327000)
libXau.so.6 => /lib64/libXau.so.6 (0x00007fa9cc122000)
libXi.so.6 => /lib64/libXi.so.6 (0x00007fa9cbf12000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fa9cbcea000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fa9cba88000)

Post by Ulrich Sibiller
You seem to be missing a libX11 link to libNX_X11. Please run ldd
/usr/bin/nxagent and post the result.
Uli

Post by Alec Bayo
Hi all,
Asking for help please troubleshooting an issue with x2go.
Inbound connections stopped working after I updated the kernel and
nvidia drivers on my centos workstation (the x2go server).
On the client, the error message says "Connection failed. Failed to start
X2Go Agent session with ID account-_____. X2Go Agent terminated
unexpectedly. Aborting session startup."
[If I attempt a connecion again the client shows a previous apparently
pre-existing connection, but I cannot resume it.]
On the server, the session log says "Error: Aborting session with 'Unable
to open display 'nx/nx,options=...". I have attempted uninstalling and
reinstalling x2goserver from epel.
This symptoms are similar to an issue Mihai Moldovan resolved
for Qianqian Fang in April: https://www.mail-archive.com/
However, I do not have the offending line set in LD_LIBRARY_PATH which
solved the issue for Qianqian.
#Contents of LD_LIBRARY_PATH (this is from env after ssh-in).
LD_LIBRARY_PATH=/usr/local/cuda/lib64:/home/[...]/caffe/
distribute/lib/::/usr/lib:/usr/lib/python2.7
I also attempted commenting out this setting from .bashrc, and that did
not fix it.
I've copied below answers to some questions Mihai asked Qianqian. The
only strange thing I noticed is that Mihai asked for the contents of
/usr/lib64/nx/X11 , and this is empty. The library Qianqian listed in there
is located at /usr/lib64 on my server.
Some more details below, answering questions that were made on this
earlier, similar troubleshooting request.
Best regards, thanks for any help,
Ale
Package nx-libs-3.5.99.16-1.el7.x86_64 already installed and latest version
x2goclient is already the newest version (4.1.1.1-2).
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/1
QT_GRAPHICSSYSTEM_CHECKED=1
QTLIB=/usr/lib64/qt-3.3/lib
QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins
# Contents of /usr/lib64/nx/X11
$ ls /usr/lib64/nx
bin X11
$ ls -lah /usr/lib64/nx/X11
total 0
drwxr-xr-x. 2 root root 6 Aug 18 14:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..
$ ls -lah /usr/lib64/nx/bin
total 0
drwxr-xr-x. 2 root root 36 Aug 17 21:28 .
drwxr-xr-x. 4 root root 26 Aug 18 14:28 ..
lrwxrwxrwx. 1 root root 16 Aug 17 16:09 nxagent -> /usr/bin/nxagent
lrwxrwxrwx. 1 root root 7 Aug 17 21:28 x2goagent -> nxagent
$ ls -lah /usr/bin/nxagent
-rwxr-xr-x. 1 root root 4.5M Jul 18 17:47 /usr/bin/nxagent
# Where I found the related libraries
$ ls -lah /usr/lib64/*NX*
lrwxrwxrwx. 1 root root 18 Aug 18 14:15 /usr/lib64/libNX_X11.so.6 ->
libNX_X11.so.6.3.0
-rwxr-xr-x. 1 root root 1.3M Jul 18 17:47 /usr/lib64/libNX_X11.so.6.3.0
_______________________________________________
x2go-user mailing list
https://lists.x2go.org/listinfo/x2go-user

[Ulrich Sibiller]

Post by Alec Bayo
Thanks for the reply Uli. Pasted output below
Also, I was scratching my head some more after sending the previous email.
Would I run the two lines below to add these links manually? Or should I do
something more/else?
ln -s /usr/lib64/libNX_X11.so.6 /usr/lib64/nx/X11/libX11.so.6
ln -s /usr/lib64/libNX_X11.so.6.3.0 /usr/lib64/nx/X11/libX11.so.6.3.0

No, this would just be wrong!

Post by Alec Bayo
$ ldd /usr/bin/nxagent
linux-vdso.so.1 => (0x00007ffe21fc2000)
libtirpc.so.1 => /lib64/libtirpc.so.1 (0x00007fa9d1cd4000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007fa9d1995000)

...

Post by Alec Bayo
libNX_X11.so.6 => /lib64/libNX_X11.so.6 (0x00007fa9d04c9000)

An here's the problem. You should never see both libs in the ldd
output. NX brings libNX_X11 but does not directly require it
(linker-wise). Instead it requires a libX11.so.6. There must be a link
in an NX related dir (I don't know the exact path on CentOS/RHEL and
cannot look that up atm). That link must be named libX11.so.6 and
point to libNX_X11.so.6.

It is a trick that is required to ensure the other X11 libs (that are
required by NX) who require a libX11.so.6 themselves use the NX
variant of the lib instead of the one from the system (which would
break NX).

Now try objdump -x /usr/bin/nxagent. Grep for RUNPATH in the output
and you will find the directory where to put the missing link. In that
directory run

ln -s /lib64/libNX_X11.so.6 libX11.so.6

This should fix it.

It would be interesting to know the reason for the missing link.

Uli

[Alec Bayo]

Thank you so much Uli, that fixed it!

I'd be happy to report further if there's anything that I can look up as to
why the link might have been missing.

I don't know if it will be of general/debugging interest, however, because
my workstation's yum is not the most healthy state: I've followed some bad
advice from dodgy internet "how tos" in the past, e.g. installing a kernel
beyond the level supported by CentOS, which I've backtracked from. Each
kernel change requires reinstalling Nvidia drivers.

Currently, yum update generates the following errors, which might be
related:

yum update

Loaded plugins: fastestmirror, langpacks, versionlock
Loading mirror speeds from cached hostfile
* base: reflector.westga.edu
* elrepo: reflector.westga.edu
* epel: mirror.team-cymru.com
* extras: centos.mirrors.tds.net
* nux-dextop: mirror.li.nux.ro
* updates: centos.servint.com
Resolving Dependencies
--> Running transaction check
---> Package xorg-x11-drv-nvidia.x86_64 1:352.99-1.el7 will be updated
---> Package xorg-x11-drv-nvidia.x86_64 1:396.44-1.el7 will be an update
--> Processing Dependency: nvidia-kmod >= 1:396.44 for package:
1:xorg-x11-drv-nvidia-396.44-1.el7.x86_64
---> Package xorg-x11-drv-nvidia-devel.x86_64 1:352.99-1.el7 will be updated
---> Package xorg-x11-drv-nvidia-devel.x86_64 1:396.44-1.el7 will be an
update
---> Package xorg-x11-drv-nvidia-gl.x86_64 1:352.99-1.el7 will be updated
---> Package xorg-x11-drv-nvidia-gl.x86_64 1:396.44-1.el7 will be an update
---> Package xorg-x11-drv-nvidia-libs.x86_64 1:352.99-1.el7 will be updated
---> Package xorg-x11-drv-nvidia-libs.x86_64 1:396.44-1.el7 will be an
update
--> Finished Dependency Resolution
Error: Package: 1:xorg-x11-drv-nvidia-396.44-1.el7.x86_64 (cuda)
Requires: nvidia-kmod >= 1:396.44
Installed: 1:nvidia-kmod-352.99-2.el7.x86_64 (@cuda)
nvidia-kmod = 1:352.99-2.el7
Available: kmod-nvidia-390.67-1.el7_5.elrepo.x86_64 (elrepo)
nvidia-kmod = 390.67-1.el7_5.elrepo
Available: kmod-nvidia-390.77-1.el7_5.elrepo.x86_64 (elrepo)
nvidia-kmod = 390.77-1.el7_5.elrepo
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest



Thanks again, best, A.

Post by Alec Bayo

Post by Alec Bayo
Thanks for the reply Uli. Pasted output below
Also, I was scratching my head some more after sending the previous

email.

Post by Alec Bayo
Would I run the two lines below to add these links manually? Or should I

do

Post by Alec Bayo
something more/else?
ln -s /usr/lib64/libNX_X11.so.6 /usr/lib64/nx/X11/libX11.so.6
ln -s /usr/lib64/libNX_X11.so.6.3.0 /usr/lib64/nx/X11/libX11.so.6.3.0

No, this would just be wrong!

Post by Alec Bayo
$ ldd /usr/bin/nxagent
linux-vdso.so.1 => (0x00007ffe21fc2000)
libtirpc.so.1 => /lib64/libtirpc.so.1 (0x00007fa9d1cd4000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007fa9d1995000)

...

Post by Alec Bayo
libNX_X11.so.6 => /lib64/libNX_X11.so.6 (0x00007fa9d04c9000)

An here's the problem. You should never see both libs in the ldd
output. NX brings libNX_X11 but does not directly require it
(linker-wise). Instead it requires a libX11.so.6. There must be a link
in an NX related dir (I don't know the exact path on CentOS/RHEL and
cannot look that up atm). That link must be named libX11.so.6 and
point to libNX_X11.so.6.
It is a trick that is required to ensure the other X11 libs (that are
required by NX) who require a libX11.so.6 themselves use the NX
variant of the lib instead of the one from the system (which would
break NX).
Now try objdump -x /usr/bin/nxagent. Grep for RUNPATH in the output
and you will find the directory where to put the missing link. In that
directory run
ln -s /lib64/libNX_X11.so.6 libX11.so.6
This should fix it.
It would be interesting to know the reason for the missing link.
Uli