CentOS build from sources


#1

I made a clean boost 1.46.1, installed the headers in /usr/local/include.
Clean cmake, from a clean scidb-13.3 source tree, make install finished, etc …
restarted scidb with old config.ini, all my stuff was still there. The arrays, the data all good.
Now, I want to exercise gemm() and gesvd()
But look:

AFL% load_library('dense_linear_algebra'); SystemException in file: src/util/PluginManager.cpp function: findModule line: 112 Error id: scidb::SCIDB_SE_PLUGIN_MGR::SCIDB_LE_CANT_LOAD_MODULE Error description: Plugin manager error. Cannot load module '/opt/scidb/13.3/lib/scidb/plugins/libdense_linear_algebra.so', dlopen returned '/opt/scidb/13.3/lib/scidb/plugins/libdense_linear_algebra.so: undefined symbol: _ZN5scidb11MPIPhysical8setQueryERKN5boost10shared_ptrINS_5QueryEEE. Failed query id: 1100881127890 AFL%
Where did I go wrong? There are no old boost headers anywhere (Well, I hid them…)
The error message suggests a mangled template. Does the lib.so say that something is wrong?

[scidb@mas-nn plugins]$ pwd /opt/scidb/13.3/lib/scidb/plugins [scidb@mas-nn plugins]$ ldd libdense_linear_algebra.so linux-vdso.so.1 => (0x00007fff7ddff000) libmpi.so => /data/gfekete/src/scidb-13.3.0.6147/bin/plugins/libmpi.so (0x00007f5123b83000) libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f512387e000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f5123578000) libm.so.6 => /lib64/libm.so.6 (0x00007f51232f4000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f51230dd000) libc.so.6 => /lib64/libc.so.6 (0x00007f5122d4a000) /lib64/ld-linux-x86-64.so.2 (0x0000003122a00000) [scidb@mas-nn plugins]$
Cheers, George


#2

George,

This actually looks different. The “undefined symbol” is now a scidb-specific function, more precisely, it can’t find the method
MPIPhysical::setQuery(const boost::shared_ptr& query)
Which is defined in MPIPhysical.cpp.

Was there something wrong with the compilation? It’s a strange one…


#3

Did you run deployment/deploy.sh prepare_toolkit followed by run.py setup? Those take care of some problems. Examine the “cmake” output carefully - any problems there?


#4

Alex, yes there are problems with cmake:
http://scidb.org/forum/viewtopic.php?f=11&t=1119

I did not run deployment prepare_toolchain (not toolkit, right) then, but now I did.
Our sysadmin installed openmpi and openmpi-dev (instead of debian/ubuntu libopenmpi-dev as
suggested by cmake).

Frankly, I don’t understand cmake is saying. It says it found MPICH2, BLAS, LAPACK and using
"locally installed ScaLAPACK" Then it goes on saying thet MPI not present and ScaLAPACK-based functionality will NOT be built. But it builds libdense_linear_algebra plugin anyway.

MPI_C_FOUND is (blank) , but /usr/lib64/openmpi/bin/mpicc is there, it works and the directory is on the PATH.

Here is what cmake . had to say:

[code]-- CMAKE_INSTALL_PREFIX not Initialized to default keeping: /opt/scidb/13.3
– Reading revision from file: 6147
– Build type (use -DCMAKE_BUILD_TYPE=[RelWithDebInfo]/Debug/Release/Profile/CC/Valgrind):
DISTRO_NAME_VER is This-
– GPB - /usr/local/lib/libprotobuf.so
– LOG4CXX - /usr/local/lib/liblog4cxx.so
– Found LibPQ: /usr/bin/pg_config
– Found LibPQ: /usr/lib64
– Could NOT find FOP (missing: FOP_EXECUTABLE)
– Could NOT find docbook.xsl (missing: DOCBOOK_XSL_FILE)
– SWIG 2 and Python found. Python module will built
– Configuring /data/gfekete/src/scidb-13.3.0.6147/src/system/Constants.cpp.in as /data/gfeket
e/src/scidb-13.3.0.6147/src/system/Constants.cpp
– Configuring /data/gfekete/src/scidb-13.3.0.6147/src/system/System.h.in as /data/gfekete/src
/scidb-13.3.0.6147/src/system/System.h
– Configuring /data/gfekete/src/scidb-13.3.0.6147/src/util/scidb-sample.conf.in as /data/gfek
ete/src/scidb-13.3.0.6147/bin/packaging_only/scidb-sample.conf
– ****************** BEGIN mpi/CMakeLists.txt ******************
– WARNING-- using locally installed MPICH2
– Local MPICH2 path: UNDEFINED_FOR_THIS_PLATFORM
– Using platform BLAS
– A library with BLAS API found.
– Using platform LAPACK
– A library with BLAS API found.
– A library with LAPACK API found.
– WARNING-- using locally installed ScaLAPACK for MPICH2
– Debug: To refresh/rebuild the local copy of scalapack, remove /data/gfekete/src/scidb-13.3.
0.6147/src/mpi/local_scalapack/container.
– ****************** BEGIN mpi/local_scalapack/CMakeLists.txt ******************
– DEBUG BLAS_FOUND=TRUE LAPACK_FOUND=TRUE MPICH_FOUND=
– Debug: SCALAPACK_LIBRARIES is /data/gfekete/src/scidb-13.3.0.6147/src/mpi/local_scalapack/c
ontainer/src/scalapackForMPICH2-build/install/lib/libscalapack.a
– Debug: SCALAPACK_INC_DIR is /data/gfekete/src/scidb-13.3.0.6147/src/mpi/local_scalapack/con
tainer/src/scalapackForMPICH2-build/install/lib/include
CMake Warning at src/mpi/CMakeLists.txt:252 (message):
MPI not present. ScaLAPACK-based functionality will NOT be built

CMake Warning at src/mpi/CMakeLists.txt:253 (message):
Debug: MPI_C_FOUND is , MPI_CXX_FOUND is , MPI_Fortran_FOUND is

CMake Warning at src/mpi/CMakeLists.txt:254 (message):
Ubuntu Hint: $ sudo apt-get install libopenmpi-dev

– Debug: FINAL PACKAGE SETTINGS src/mpi ================================>
– Debug: SCALAPACK_LIBRARIES is /data/gfekete/src/scidb-13.3.0.6147/src/mpi/local_scalapack/c
ontainer/src/scalapackForMPICH2-build/install/lib/libscalapack.a
– Debug: LAPACK_LIBRARIES is /usr/local/lib/liblapack.a;/usr/local/lib/libblas.a
– Debug: BLAS_LIBRARIES is /usr/local/lib/libblas.a
– Debug: MPI_LIBRARIES is
– Debug: MPI_INCLUDE is
– Debug: MPI_Fortran_COMPILER is /usr/lib64/openmpi/bin/mpif77
– Debug: CMAKE_C_FLAGS is -pedantic -W -Wextra -Wall -Wno-long-long -Wno-unused-parameter -f
PIC -D__STDC_FORMAT_MACROS
– Debug: CMAKE_CXX_FLAGS is -pedantic -W -Wextra -Wall -Wno-strict-aliasing -Wno-long-long -
Wno-unused-parameter -fPIC -D__STDC_FORMAT_MACROS -Wno-system-headers -isystem /opt/local/incl
ude/ -isystem /usr/local/include/ -D__STDC_LIMIT_MACROS -DPROJECT_ROOT="/data/gfekete/src/sci
db-13.3.0.6147/" -DOMPI_IGNORE_CXX_SEEK
– Debug: CMAKE_Fortran_FLAGS is -fno-f2c -I/opt/scidb/13.3/3rdparty/mpich2/include/mpi
– Debug: CMAKE_Fortran_COMPLIER is /usr/bin/gfortran
– Debug: CMAKE_CXX_COMPILER is /usr/bin/c++
– Debug: FINAL PACKAGE SETTINGS <================================
CMake Warning at src/mpi/CMakeLists.txt:318 (message):
Can’t build ScaLAPACK extensions

CMake Warning at src/mpi/CMakeLists.txt:319 (message):
without ScaLAPACK, MPI, LAPACK, and BLAS

CMake Warning at src/mpi/CMakeLists.txt:320 (message):
This is only a warning, SciDB will build but without

CMake Warning at src/mpi/CMakeLists.txt:321 (message):
the libdense_linear_algebra.so plugin.

CMake Warning at src/mpi/CMakeLists.txt:322 (message):
If that is not desired, please make sure the following are correctly
resolved:

CMake Warning at src/mpi/CMakeLists.txt:323 (message):

CMake Warning at src/mpi/CMakeLists.txt:324 (message):
Debug: MPI_Fortran_COMPLIER is /usr/lib64/openmpi/bin/mpif77

CMake Warning at src/mpi/CMakeLists.txt:325 (message):
Debug: CMAKE_Fortran_COMPLIER is /usr/bin/gfortran

CMake Warning at src/mpi/CMakeLists.txt:326 (message):
Debug: CMAKE_CXX_COMPILER is /usr/bin/c++

CMake Warning at src/mpi/CMakeLists.txt:327 (message):
Debug: SCALAPACK_FOUND is 1

CMake Warning at src/mpi/CMakeLists.txt:328 (message):
Debug: LAPACK_FOUND is TRUE

CMake Warning at src/mpi/CMakeLists.txt:329 (message):
Debug: BLAS_FOUND is TRUE

CMake Warning at src/mpi/CMakeLists.txt:330 (message):
Debug: MPI_CXX_FOUND is

CMake Warning at src/mpi/CMakeLists.txt:331 (message):
Debug: MPI_Fortran_FOUND is

CMake Warning at src/mpi/CMakeLists.txt:332 (message):

– *****BEGIN linear_algebra/CMakeLists.txt ***********************
– MPI_Fortran_COMPILER=/usr/lib64/openmpi/bin/mpif77
– Debug: BLAS_LIBRARIES is
– Debug: LAPACK_LIBRARIES is
– Debug: SCALAPACK_LIBRARIES is
– Debug: MPI_LIBRARIES is /usr/lib64/openmpi/lib/libmpi_cxx.so;/usr/lib64/openmpi/lib/libmpi.
so;/usr/lib64/libdl.so
– Debug: CMAKE_C_FLAGS is -pedantic -W -Wextra -Wall -Wno-long-long -Wno-unused-parameter -f
PIC -D__STDC_FORMAT_MACROS
– Debug: CMAKE_CXX_FLAGS is -pedantic -W -Wextra -Wall -Wno-strict-aliasing -Wno-long-long -
Wno-unused-parameter -fPIC -D__STDC_FORMAT_MACROS -Wno-system-headers -isystem /opt/local/incl
ude/ -isystem /usr/local/include/ -D__STDC_LIMIT_MACROS -DPROJECT_ROOT="/data/gfekete/src/sci
db-13.3.0.6147/" -DOMPI_IGNORE_CXX_SEEK
– Debug: CMAKE_Fortran_FLAGS is -fno-f2c -I/opt/scidb/13.3/3rdparty/mpich2/include/mpi
– Debug: MPI_Fortran_COMPILER is /usr/lib64/openmpi/bin/mpif77
– Configuring /data/gfekete/src/scidb-13.3.0.6147/doc/api/Doxyfile.in as /data/gfekete/src/sc
idb-13.3.0.6147/doc/api/Doxyfile
– Documentation will not be generated automatically. Use -DSCIDB_DOC_TYPE={NONE,FULL,API} to
set type of generated documentation. Use make doc to generate FULL documentation manually.
– Configuring done
– Generating done
– Build files have been written to: /data/gfekete/src/scidb-13.3.0.6147
[/code]
Any hints? George


#5

The first error is here:
– Local MPICH2 path: UNDEFINED_FOR_THIS_PLATFORM

For me it says
– Local MPICH2 path: /usr/lib64/mpich2

Also, you say:
– Debug: MPI_LIBRARIES is
– Debug: MPI_INCLUDE is

And for me its
– Debug: MPI_LIBRARIES is /usr/lib64/mpich2/lib/libmpichf90.a
– Debug: MPI_INCLUDE is -I /usr/include/mpich2-x86_64

That looks like a missing piece. Yes, I meant “prepare_toolchain” and that should put all of the right packages on your system. Puzzled that it did not. Did that complete OK?


#6

I did

I had to sudo because it uses root@… for its ssh. Was the right command?
When it returned it showed this:

[code][gfekete@mas-nn scidb-13.3.0.6147]$ sudo deployment/deploy.sh prepare_toolchain localhost
Source path: /data/gfekete/src/scidb-13.3.0.6147
Script common path: /data/gfekete/src/scidb-13.3.0.6147/deployment/common
Build path: /data/gfekete/src/scidb-13.3.0.6147
SciDB version: 13.3
Executing: prepare_toolchain localhost

Prepare toolchain @localhost
Register SciDB 3rdparty repository on localhost
spawn ssh -o StrictHostKeyChecking=no root@localhost rm -rf /tmp/root/deployment && mkdir -p /tmp/root
This U.S. Government resource is for authorized users only. By accessing
this system you are consenting to complete monitoring with no expectation
of privacy. Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.

spawn scp -r -q -o StrictHostKeyChecking=no /data/gfekete/src/scidb-13.3.0.6147/deployment/common root@localhost:/tmp/root/deployment
spawn ssh -o StrictHostKeyChecking=no root@localhost cd /tmp/root/deployment && ./register_3rdparty_scidb_repository.sh
This U.S. Government resource is for authorized users only. By accessing
this system you are consenting to complete monitoring with no expectation
of privacy. Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.

spawn ssh -o StrictHostKeyChecking=no root@localhost rm -rf /tmp/root/deployment
This U.S. Government resource is for authorized users only. By accessing
this system you are consenting to complete monitoring with no expectation
of privacy. Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.

spawn ssh -o StrictHostKeyChecking=no root@localhost rm -rf /tmp/root/deployment && mkdir -p /tmp/root
This U.S. Government resource is for authorized users only. By accessing
this system you are consenting to complete monitoring with no expectation
of privacy. Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.

spawn scp -r -q -o StrictHostKeyChecking=no /data/gfekete/src/scidb-13.3.0.6147/deployment/common root@localhost:/tmp/root/deployment
spawn ssh -o StrictHostKeyChecking=no root@localhost cd /tmp/root/deployment && ./prepare_toolchain.sh 13.3
This U.S. Government resource is for authorized users only. By accessing
this system you are consenting to complete monitoring with no expectation
of privacy. Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.

[gfekete@mas-nn scidb-13.3.0.6147]$ [/code]
Of course I can’t say whether or not it did anything, since it said nothing. If anywhere down the line deploy depends on /etc/issue to figure out the platform, it won’t work.
George


#7

Mmm… Yeah there could be a mismatch somewhere. You are not supposed to run it as root. You are supposed to run it as USER_X, but USER_X should be able to say “ssh root@localhost” without a password. Maybe it worked, maybe it didn’t. Your environment is different…

Let’s approach the problem from a different angle. The files that CMAKE says you are missing - where do they come from?

[apoliakov@localhost harness]$ rpm -qf /usr/lib64/mpich2/lib/libmpichf90.a
mpich2-devel-1.2.1-2.3.el6.x86_64
[apoliakov@localhost harness]$ rpm -qf /usr/include/mpich2-x86_64/
mpich2-devel-1.2.1-2.3.el6.x86_64

Do you have that package?