Unable to connect. Error: Cannot load module ... libdense_linear_algebra.so


#1

We just recently upgraded to scidb 16.9 community edition. I am unable to connect with python using scidb-py, everything else (shim, iquery, scidbr) appears to be working normally. I have pasted the error below, along with a successful connection using scidbr. We did not notice anything not compiling correctly during the upgrade process, but I suppose it’s possible we missed something.

Any help would be appreciated,
~David.

dwiltsee@hpc083 ~ $ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.9 (Santiago)

dwiltsee@hpc083 ~ $ pip freeze | grep scidb
scidb-py==14.10.0

dwiltsee@hpc083 ~ $ ipython
Python 2.7.13 (default, Feb 8 2017, 06:30:30)
Type “copyright”, “credits” or “license” for more information.

IPython 4.2.0 – An enhanced Interactive Python.
? -> Introduction and overview of IPython’s features.
%quickref -> Quick reference.
help -> Python’s own help system.
object? -> Details about ‘object’, use ‘object??’ for extra details.

In [1]: import scidbpy as sci

In [2]: sdb = sci.connect(‘http://10.17.17.82:8080’)

ValueError Traceback (most recent call last)
in ()
----> 1 sdb = sci.connect(‘http://10.17.17.82:8080’)

/opt/rh/python27/root/usr/lib/python2.7/site-packages/scidbpy/interface.pyc in connect(url, username, password)
1845 password = os.environ.get(‘SCIDB_PASSWORD’, None)
1846
-> 1847 return SciDBShimInterface(url, user=username, password=password)

/opt/rh/python27/root/usr/lib/python2.7/site-packages/scidbpy/interface.pyc in init(self, hostname, user, password, pam, digest)
1635 except Exception as e:
1636 raise ValueError(“Could not connect to a SciDB instance at {0}. {1}”
-> 1637 .format(self.hostname, e))
1638
1639 def login(self, user, password):

ValueError: Could not connect to a SciDB instance at http://10.17.17.82:8080. SystemException in file: src/util/PluginManager.cpp function: findModule line: 101
Error id: scidb::SCIDB_SE_PLUGIN_MGR::SCIDB_LE_CANT_LOAD_MODULE
Error description: Plugin manager error. Cannot load module ‘/opt/scidb/16.9/lib/scidb/plugins/libdense_linear_algebra.so’, dlopen returned ‘/opt/scidb/16.9/lib/scidb/plugins/libdense_linear_algebra.so: undefined symbol: _ZN5scidb11MPIPhysical8setQueryERKSt10shared_ptrINS_5QueryEE’.

#################### End Python / Start R Example ##############################

dwiltsee@hpc083 ~ $ R

R version 3.3.2 (2016-10-31) – "Sincere Pumpkin Patch"
Copyright © 2016 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type ‘license()’ or ‘licence()’ for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type ‘contributors()’ for more information and
’citation()’ on how to cite R or R packages in publications.

Type ‘demo()’ for some demos, ‘help()’ for on-line help, or
’help.start()’ for an HTML browser interface to help.
Type ‘q()’ to quit R.

library(scidb)
Loading required package: bit64
Loading required package: bit
Attaching package bit
package:bit © 2008-2012 Jens Oehlschlaegel (GPL-2)
creators: bit bitwhich
coercion: as.logical as.integer as.bit as.bitwhich which
operator: ! & | xor != ==
querying: print length any all min max range sum summary
bit access: length<- [ [<- [[ [[<-
for more help type ?bit

Attaching package: ‘bit’

The following object is masked from ‘package:base’:

xor

Attaching package bit64
package:bit64 © 2011-2012 Jens Oehlschlaegel (GPL-2 with commercial restrictions)
creators: integer64 seq :
coercion: as.integer64 as.vector as.logical as.integer as.double as.character as.bin
logical operator: ! & | xor != == < <= >= >
arithmetic operator: + - * / %/% %% ^
math: sign abs sqrt log log2 log10
math: floor ceiling trunc round
querying: is.integer64 is.vector [is.atomic} [length] is.na format print
aggregation: any all min max range sum prod
cumulation: diff cummin cummax cumsum cumprod
access: length<- [ [<- [[ [[<-
combine: c rep cbind rbind as.data.frame
for more help type ?bit64

Attaching package: ‘bit64’

The following object is masked from ‘package:bit’:

still.identical

The following objects are masked from ‘package:base’:

:, %in%, is.double, match, order, rank

/ /() _ / _ )
\ / / / // / _ |
/
/_//// Copyright 2016, Paradigm4, Inc.

sdb = scidbconnect(‘10.17.17.82’)
iquery(sdb, “list(‘macros’)”, return=TRUE)
No name
1 0 func_max
2 1 func_min
3 2 load
4 3 op_avg
5 4 op_count
6 5 op_distinct_count
7 6 op_max
8 7 op_min
9 8 op_now
10 9 op_range
11 10 op_scidbversion
12 11 op_set_cell_attr_1D
13 12 op_set_cell_attr_2D
14 13 op_stdev
15 14 op_sum
16 15 sys_create_array_att
17 16 sys_create_array_aux
18 17 sys_create_array_dim


#2

Hey David - try running that command with iquery?

iquery -aq "load_library('dense_linear_algebra')"

That should work. It contains some operators like gemm and gesvd. Looks like the Python package tries to load it when it connnects. The failure looks like some kind of a build problem: a missing symbol. Were there build errors?

Also - note there is a newer lighter Python package currently under development. It’s on a special d"devel" branch right now:
https://github.com/paradigm4/scidb-py/tree/devel


#3

Thanks. Looks like that command fails as well.

Looking into it, it looks like we have some circular references going on with libdense_linear_algebra.so and liblinear_algebra.so. So, it looks like those libraries were never installed properly. My admin tells me he did not remember seeing any build errors.

scidb@hpc082 ~ $ iquery -aq "load_library(‘dense_linear_algebra’)"
SystemException in file: src/util/PluginManager.cpp function: findModule line: 101
Error id: scidb::SCIDB_SE_PLUGIN_MGR::SCIDB_LE_CANT_LOAD_MODULE
Error description: Plugin manager error. Cannot load module ‘/opt/scidb/16.9/lib/scidb/plugins/libdense_linear_algebra.so’, dlopen returned ‘/opt/scidb/16.9/lib/scidb/plugins/libdense_linear_algebra.so: undefined symbol: _ZN5scidb11MPIPhysical8setQueryERKSt10shared_ptrINS_5QueryEE’.

scidb@hpc082 ~ $ cd /opt/scidb/16.9/lib/scidb/plugins/

scidb@hpc082 …b/scidb/plugins $ ls -l
total 50M
-rwxrwxr-x 1 scidb scidb 12M Mar 17 16:29 libaccelerated_io_tools.so*
-rw-r–r-- 1 scidb scidb 198K Mar 10 15:21 libbestmatch.so
-rw-r–r-- 1 scidb scidb 112K Mar 10 15:14 libcomplex.so
-rw-r–r-- 1 scidb scidb 730K Mar 10 15:04 libdense_linear_algebra-scidb.so
lrwxrwxrwx 1 scidb scidb 58 Mar 10 16:12 libdense_linear_algebra.so -> /opt/scidb/16.9/etc/alternatives/dense_linear_algebra.16.9
-rwxr-xr-x 1 scidb scidb 210K Mar 17 16:03 libdev_tools.so*
-rwxrwxr-x 1 scidb scidb 9.2M Apr 14 14:46 libequi_join.so*
-rw-r–r-- 1 scidb scidb 196K Mar 10 15:24 libexample_udos.so
-rwxrwxr-x 1 scidb scidb 8.2M Apr 17 08:34 libfaster_redimension.so*
-rw-r–r-- 1 scidb scidb 285K Mar 10 15:23 libfits.so
-rwxrwxr-x 1 scidb scidb 7.3M Apr 14 13:23 libgrouped_aggregate.so*
-rwxrwxr-x 1 scidb scidb 3.4M Apr 17 08:25 liblimit.so*
-rw-r–r-- 1 scidb scidb 596K Mar 10 15:05 liblinear_algebra-scidb.so
lrwxrwxrwx 1 scidb scidb 52 Mar 10 16:12 liblinear_algebra.so -> /opt/scidb/16.9/etc/alternatives/linear_algebra.16.9
-rw-r–r-- 1 scidb scidb 201K Mar 10 15:20 libmatch.so
-rw-r–r-- 1 scidb scidb 945K Mar 10 15:20 libmisc.so
-rw-r–r-- 1 scidb scidb 32K Mar 10 15:14 libmore_math.so
-rw-r–r-- 1 scidb scidb 1.2M Mar 10 14:56 libmpi.so
-rw-r–r-- 1 scidb scidb 93K Mar 10 15:13 libpoint.so
-rw-r–r-- 1 scidb scidb 74K Mar 10 15:14 libra_decl.so
-rw-r–r-- 1 scidb scidb 149K Mar 10 15:14 librational.so
-rwxrwxr-x 1 scidb scidb 4.9M Apr 17 08:29 libsummarize.so*
-rw-r–r-- 1 scidb scidb 245K Mar 10 15:25 libtile_integration.so
-rwxr-xr-x 1 scidb scidb 672K Mar 10 15:00 mpi_slave_scidb*

scidb@hpc082 …b/scidb/plugins $ ls -l /opt/scidb/16.9/etc/alternatives/
total 8.0K
lrwxrwxrwx 1 scidb scidb 66 Mar 10 16:12 dense_linear_algebra.16.9 -> /opt/scidb/16.9/lib/scidb/plugins/libdense_linear_algebra-scidb.so
lrwxrwxrwx 1 scidb scidb 60 Mar 10 16:12 linear_algebra.16.9 -> /opt/scidb/16.9/lib/scidb/plugins/liblinear_algebra-scidb.so


#4

Oops, never mind about the circular reference. I see, now, that it is pointing to a file with the suffix -scidb.so in the etc/alternatives/. My eyes deceived me.

So, I don’t know, does 730K look like the right size for this file? Is there a way we could just try to recompile libdense_linear_algebra-scidb.so without reinstalling all of scidb?