Issues with install of SciDB 14.8 on Ubuntu 12.04


#1

I just installed SciDB 14.8 on a new clean VM with Ubuntu 12.04 and when I call ‘scidb.py startall cluster’ from user scidb I get the following error:

scidb@SciDB:~$ scidb.py startall cluster/opt/scidb/14.8/bin/scidb.py: ERROR: Remote command exceptions:Abnormal return code: 2 stderr: sh: 1: Syntax error: Unterminated quoted string

The config file that I used is listed below. Any ideas as to why this is happening? I saw an older post with this occuring but didn’t find anything useful there. The version of postgres that I have installed is 8.4.

[cluster]
server-0=SciDB,3
install_root=/opt/scidb/14.8
metadata=/opt/scidb/14.8/share/scidb/meta.sql
pluginsdir=/opt/scidb/14.8/lib/scidb/plugins
logconf=/opt/scidb/14.8/share/scidb/log4cxx.properties
db_user=pguser
db_passwd=pguserpwd
base-port=1239
base-path=/home/scidb/scidb_data
redundancy=0
execution-threads=1
result-prefetch-threads=1
result-prefetch-queue-size=1
operator-threads=1


#2

Hi,

Could you please post the output for the following command?

Thanks,
Sunny


#3

Hi Sunny,

That directory is empty. I am assuming that since this fails early on start up that it is just not generating logs.

Tony


#4

Hi Tony,

Did you install using cluster_install ?
Have you, as scidb user, sourced the .bashrc file so as to pickup the SCIDB_ environment variables ?

Martin


#5

Martin,

I did install using the cluster_install installer and I also have sourced the .bashrc file.

Tony


#6

Hi Tony,
I’m going to pass you on to ilikhotkin who knows the scidb.py script.
I’ve emailed him to look.

Meanwhile I am going to make a fresh VM and try it your way.

Martin


#7

FYI I tried cluster_install on a fresh Ubuntu12.04 VM and there was no problem installing and no problem running scidb.py startall cluster.

Maybe the problem is with naming the host “SciDB” ?
I don’t know if scidb.py does some text processing that does something with the string since that string is in error messages.

Just a thought.


#8

First - let’s see which actual command is breaking. Please try to restart the cluster with -v switch:

Post the output.

Also, it would be interesting to know where your /bin/sh is pointing: is it dash. Sometimes, Ubuntu machines come with dash as the default shell. Post the result of this test also.


#9

The results of running with the -v option are:

/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'ps --no-headers -e -o pid,cmd | awk \'{print $1 " " $2}\' | grep "/home/scidb/scidb_data/000/*" | awk \'{print $1}\''
Traceback (most recent call last):
  File "/opt/scidb/14.8/bin/scidb.py", line 248, in parallelRemoteExec
    raise Exception("Abnormal return code: %s stderr: %s" % (exits[index],errors[index]))
Exception: Abnormal return code: 2 stderr: sh: 1: Syntax error: Unterminated quoted string

/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command exceptions:
['exec /bin/bash -c $\'ps --no-headers -e -o pid,cmd | awk \\\'{print $1 " " $2}\\\' | grep "/home/scidb/scidb_data/000/*" | awk \\\'{print $1}\\\'\'']
Abnormal return code: 2 stderr: sh: 1: Syntax error: Unterminated quoted string

/opt/scidb/14.8/bin/scidb.py: ERROR: Remote command exceptions:
Abnormal return code: 2 stderr: sh: 1: Syntax error: Unterminated quoted string

As for /bin/sh, yes it is pointing to dash. Does this not work with dash?


#10

All of our scripts rely on standard shell scripting techniques. Dash takes quite a few “shortcuts” with shell syntax. Can you please try re-pointing your /bin/sh to /bin/bash? Try starting the cluster and see if that makes a difference.


#11

I changed /bin/sh to point to bash instead and I did get further along. What I get when I run with -v again I now get this.

/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'(cd /home/scidb/scidb_data/000/0 && rm -f /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/0/SciDB-000-0-cluster -i SciDB -p 1239 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -j 1 -x 1 -t 1 -q 1 -s /home/scidb/scidb_data/000/0/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/0/scidb-stdout.log 2> /home/scidb/scidb_data/000/0/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/1 && rm -f /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/1/SciDB-000-1-cluster -i SciDB -p 1240 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -j 1 -x 1 -t 1 -q 1 -s /home/scidb/scidb_data/000/1/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/1/scidb-stdout.log 2> /home/scidb/scidb_data/000/1/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/2 && rm -f /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/2/SciDB-000-2-cluster -i SciDB -p 1241 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -j 1 -x 1 -t 1 -q 1 -s /home/scidb/scidb_data/000/2/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/2/scidb-stdout.log 2> /home/scidb/scidb_data/000/2/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/3 && rm -f /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/3/SciDB-000-3-cluster -i SciDB -p 1242 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -j 1 -x 1 -t 1 -q 1 -s /home/scidb/scidb_data/000/3/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/3/scidb-stdout.log 2> /home/scidb/scidb_data/000/3/scidb-stderr.log ) &'
/opt/scidb/14.8/bin/scidb.py: ERROR: Failed to start SciDB!

#12

Alright - now we are getting somewhere. Please comment out these lines (with a #) in your /opt/scidb/14.8/etc/config.ini:

execution-threads=1 result-prefetch-threads=1 result-prefetch-queue-size=1 operator-threads=1

Comment the lines out and try starting the cluster again. Post whether it works or does not.


#13

Okay, just ran that and it still fails. The following is the output I got:


/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'ps --no-headers -e -o pid,cmd | awk \'{print $1 " " $2}\' | grep "/home/scidb/scidb_data/000/*" | awk \'{print $1}\''
Found 0 scidb processes
start(server 0 (SciDB) local instance 0)
Starting SciDB server.
start(server 0 (SciDB) local instance 1)
Starting SciDB server.
start(server 0 (SciDB) local instance 2)
Starting SciDB server.
start(server 0 (SciDB) local instance 3)
Starting SciDB server.
/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'(cd /home/scidb/scidb_data/000/0 && rm -f /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/0/SciDB-000-0-cluster -i SciDB -p 1239 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/0/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/0/scidb-stdout.log 2> /home/scidb/scidb_data/000/0/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/1 && rm -f /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/1/SciDB-000-1-cluster -i SciDB -p 1240 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/1/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/1/scidb-stdout.log 2> /home/scidb/scidb_data/000/1/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/2 && rm -f /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/2/SciDB-000-2-cluster -i SciDB -p 1241 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/2/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/2/scidb-stdout.log 2> /home/scidb/scidb_data/000/2/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/3 && rm -f /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/3/SciDB-000-3-cluster -i SciDB -p 1242 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/3/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/3/scidb-stdout.log 2> /home/scidb/scidb_data/000/3/scidb-stderr.log ) &'
/opt/scidb/14.8/bin/scidb.py: ERROR: Failed to start SciDB!

#14

Alright - it seems the problem is somewhere else. Let’s check your password-less ssh. Please type the following on the command prompt and see if the login succeeds (you should not have to enter your password):

If login succeeds, then just exit. Please post if the password-less ssh succeeds or not.
You can also try putting the startup command into a shell file and running that shell. Take the output starting with “exec /bin/bash…” and place it into its own run_scidb.sh file (make sure to set its x permission bit). Then you can try starting scidb with that shell script:

#!/bin/bash
#start command goes here

#15

Password-less ssh works.


#16

Did it work to start scidb with the shell script?
Let’s see what your shell is now. Type this and post the output:

Also - did you source in ~/.bashrc file before attempting to start the cluster?


#17

Yep, it is running bash now however I am still getting the startup error.

After running ps -p $$ I get:

  PID TTY          TIME CMD
16107 pts/3    00:00:00 bash

The error I am getting is still this:


scidb@SciDB:~$ ./run_scidb.sh 
/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'ps --no-headers -e -o pid,cmd | awk \'{print $1 " " $2}\' | grep "/home/scidb/scidb_data/000/*" | awk \'{print $1}\''
Found 0 scidb processes
start(server 0 (SciDB) local instance 0)
Starting SciDB server.
start(server 0 (SciDB) local instance 1)
Starting SciDB server.
start(server 0 (SciDB) local instance 2)
Starting SciDB server.
start(server 0 (SciDB) local instance 3)
Starting SciDB server.
/opt/scidb/14.8/bin/scidb.py: DEBUG: Remote command=exec /bin/bash -c $'(cd /home/scidb/scidb_data/000/0 && rm -f /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/0/SciDB-000-0-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/0/SciDB-000-0-cluster -i SciDB -p 1239 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/0/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/0/scidb-stdout.log 2> /home/scidb/scidb_data/000/0/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/1 && rm -f /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/1/SciDB-000-1-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/1/SciDB-000-1-cluster -i SciDB -p 1240 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/1/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/1/scidb-stdout.log 2> /home/scidb/scidb_data/000/1/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/2 && rm -f /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/2/SciDB-000-2-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/2/SciDB-000-2-cluster -i SciDB -p 1241 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/2/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/2/scidb-stdout.log 2> /home/scidb/scidb_data/000/2/scidb-stderr.log ) & (cd /home/scidb/scidb_data/000/3 && rm -f /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && ln -fs /opt/scidb/14.8/bin/scidb /home/scidb/scidb_data/000/3/SciDB-000-3-cluster && LD_LIBRARY_PATH=/opt/scidb/14.8/lib:/opt/scidb/14.8/lib/scidb/plugins:/opt/scidb/14.8/lib:/opt/scidb/14.8/lib:  exec /home/scidb/scidb_data/000/3/SciDB-000-3-cluster -i SciDB -p 1242 --cache 256 -k -l /opt/scidb/14.8/share/scidb/log4cxx.properties --plugins /opt/scidb/14.8/lib/scidb/plugins --redundancy=0 --install_root=/opt/scidb/14.8 -s /home/scidb/scidb_data/000/3/storage.cfg -c \'host=SciDB port=5432 dbname=cluster user=pguser password=pguserpwd\' 1> /home/scidb/scidb_data/000/3/scidb-stdout.log 2> /home/scidb/scidb_data/000/3/scidb-stderr.log ) &'
/opt/scidb/14.8/bin/scidb.py: ERROR: Failed to start SciDB!


#18

Try sourcing ~/.bashrc again and then starting the cluster. Make sure you do it as scidb user.


#19

Recalled source .bashrc and then tried to start the server again and no change, same output saying Failed to start SciDB.


#20

Hmmm… Try running this command:

If the command succeeds, try re-running the startall command.
Post the output of the initall command if it fails.