SciDB community edition installer script


#1

Zero to SciDB in under two minutes

Folks, you have been asking for a easy way to install SciDB for a while now. Well, here you go – an easy install script for SciDB 18.1 CE :slight_smile:

What this script does

  1. Do a vanilla install of SciDB 18.1 community edition on a Unix machine
  2. Start SciDB with 4 instances

The following script has been tested on

  • Centos 6 / Centos 7
  • Ubuntu 14.04 (NOTE: does not work on Ubuntu 16)
  • RHEL 6 / RHEL 7
wget https://downloads.paradigm4.com/community/18.1/install-scidb-ce.sh
chmod +x install-scidb-ce.sh
sudo ./install-scidb-ce.sh

Then

# add /opt/scidb/... to path i.e. .bashrc or .bash_profile
# then run
iquery -aq "list('instances')"
{No} name,port,instance_id,online_since,instance_path
{0} '127.0.0.1',1239,0,'2018-02-02 20:41:27','/home/ubuntu/scidb_data/0/0'
{1} '127.0.0.1',1240,1,'2018-02-02 20:41:27','/home/ubuntu/scidb_data/0/1'
{2} '127.0.0.1',1241,2,'2018-02-02 20:41:27','/home/ubuntu/scidb_data/0/2'
{3} '127.0.0.1',1242,3,'2018-02-02 20:41:27','/home/ubuntu/scidb_data/0/3'

Please let us know if you find any issues while running this script.

Update


#2

I am trying to use your Zero to SciDB in 2 mins method to install – unfortunately – I don’t know how to go about the errror in the code below.

Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Installing scidb-18.1-ce.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 scidb-18.1-ce : Depends: scidb-18.1-all-coord but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
=================================================================
SciDB not successfully installed.  Please examine
messages above and look for errors.

If you are unable to resolve the issues, collect the output
and contact Paradigm4 at info@paradigm4.com or post to our forum
forum.paradigm4.com for help.
=================================================================


#3

@balekaki

Sorry to hear. It seems you are having this issue on Ubuntu – note that we ship only for Ubuntu 14.04.

I just tried the instructions after starting a brand new Ubuntu 14.04 instance off Amazon EC2 AMI-s and they worked OK.

Can you give us some more details of the machine where you are trying this e.g.

$ cat /etc/lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"

Also you do not have scidb installed on this machine previously (by any other mechanism) right?


#4

Hi @Kriti_Sen_Sharma,

Thanks for the response. I am running Ubuntu 16.04 LTS, core i7 Proline box. Could that be the reason why it can not run successfully?


#5

@balekaki

SciDB does not officially ship for Ubuntu 16. Also I tested the script on Ubuntu 16.04 and can confirm that I got the same error (have updated the original post to reflect this).

Is there any chance you can run your work on Ubuntu 14?

Also, depending on the nature of your work (e.g. if you do not intend to use SciDB as a cluster i.e. if you care only about simple prototyping work on a single node SciDB) – you can consider using SciDB on docker https://hub.docker.com/r/rvernica/scidb/, and that will run on any system that runs docker.


#6

@balekaki BTW if you want, you can try building on Ubuntu 16 from source. The link to the source tarball is here.


#7

@Kriti_Sen_Sharma – thanks. that i definitely gonna try on Ubuntu 16 and get back to you. What i had done initially was to re-install Ubuntu 14.04LTS – where it run successfully – however, i’m still failing to get sciDB to start running – maybe kind of confused with my configurations. Do you know of any resources that can pick my up from that point. i.e install successfully but failed to start sciDB.


#8

Any success @balekaki?

BTW I updated the original post – the most updated install documentation is now maintained at this SciDB page on installation.

Also see instructions for using SciDB on Docker (recommended only for quick prototyping and testing; not recommended for SciDB installs in production / cluster mode).


#9

Hi @Kriti_Sen_Sharma — i got it to run on Ubuntu 14.04LTS and on Ubuntu 16 via Docker successfully. I am now working smoothly with it. However – i am having trouble to load my csv data file in my scidb array. For example

AFL% load(intensityFlat ,'/tmp/intensity_data.csv', -2, 'CSV');

I am confused with the path --do i create tmp folder and save the csv file inside it or what? How do i go about this?


Trouble loading CSV
#10

@balekaki

Glad to hear that you are up and running with Docker. Spawned a new ticket for your other query.


#11

sciDB installed successful. But execute command

return fatal error

3 [0x7f253908b800] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239 (Connection refused)
3 [0x7f253908b800] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239 (Connection refused)
iquery SystemException in file: src/network/BaseConnection.cpp function: connect line: 78
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CONNECTION_ERROR
Error description: Network error. Error #system:111 when connecting to localhost:1239.

#12

Hi @d.stepanov,

That error typically means the cluster is not running. If the cluster start command

$ scidb.py start-all CLUSTERNAME

isn’t working, look in the data directories for init-std{err,out}.log files. That’s where problems on startup are logged.


#13

Hi @mjl and everybody

I’m trying to install scidb on a Centos 7 following page " Zero-to-SciDB in Two Minutes"
Installation seems all right, at least I cannot find any error in the install log file
cat 20181223-205720/install-20181223-205720.txt | grep -i error

However, I have the same problem than @d.stepanov
FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239

Sorry but I don’t know where are the data directories in order to search init-std{err,out}.log files.

I have three new files on /home/scidb called srv-0-1-20181223-205720-system.rpt, srv-0-2-20181223-205720-system.rpt and srv-0-3-20181223-205720-system.rpt containing almost the same text, including incomprehensible error messages [Errno 2] No such file or directory

srv-0-1-20181223-205720
=======================
Host: scidbinstance

sysctl -a
---------
[Errno 2] No such file or directory
ip a
----
[Errno 2] No such file or directory
netstat -i
----------
Kernel Interface table
Iface             MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0             1500   146161      0      0 0          4844      0      0      0 BMRU
lo              65536      880      0      0 0           880      0      0      0 LRU

netstat -r -n
-------------
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         93.189.88.1     0.0.0.0         UG        0 0          0 eth0
93.189.88.0     0.0.0.0         255.255.254.0   U         0 0          0 eth0
169.254.169.254 93.189.88.107   255.255.255.255 UGH       0 0          0 eth0

arp -an
-------
[Errno 2] No such file or directory

Best regards


#14

@vdebuen

Any updates since you posted? If you are simply prototyping on scidb, I would suggest trying scidb on docker

  1. https://paradigm4.github.io/scidblog/scidb-docker/scidb-on-docker/
  2. https://hub.docker.com/r/rvernica/scidb/

If you have a full service workload in mind, then try the following and send us the output.

# config file
cat /opt/scidb/18.1/etc/config.ini

#15

Hi Kriti

The docker version works for me. I’m evaluating now.

Thanks!