Scidb single machine and scidb cluster cost the same time when executing the command


#1

I predict that the scidb cluster will cost less time when execute the command than scidb single machine, but the fact is backfired. Dose the config.ini file cause the result or other reason?
for example, when I execute the command “load(arrayname, ‘datafile’, -2, ‘csv’)” ,scidb single machine and scidb cluster cost the same time all about 500s
when I execute the command "store(project ( apply( apply( ab,res, (val1-val2)/(val1+val2) ) , er, iif(res>0,1,0) ) , er) , ndviriver) ",scidb single machine and scidb cluster also cost the same time all about 3s

the config.ini of scidb single machine is as following

[single_server]
server-0=localhost,3
install_root=/opt/scidb/14.12
pluginsdir=/opt/scidb/14.12/lib/scidb/plugins
logconf=/opt/scidb/14.12/share/scidb/log4cxx.properties
db_user=scidb_pg_user
db_passwd=scidb_pg_user_pasw
base-port=1239
base-path=/home/scidb/scidb_data
redundancy=0

/### Threading: max_concurrent_queries=2, threads_per_query=1
/# max_concurrent_queries + 2:
execution-threads=4
/# max_concurrent_queries * threads_per_query:
result-prefetch-threads=2
/# threads_per_query:
result-prefetch-queue-size=1
operator-threads=1

/### Memory: 1000MB per instance, 750MB reserved
/# network: 300MB per instance assuming 5MB average chunks
/# in units of chunks per query:
sg-send-queue-size=15
sg-receive-queue-size=15
/# caches: 300MB per instance
smgr-cache-size=150
mem-array-threshold=150
/# sort: 150MB per instance (specified per thread)
merge-sort-buffer=75

**

the config.ini of scidb cluster is as following

**

[mycluster]
server-0=localhost,3
server-1=192.168.0.202,3
install_root=/opt/scidb/14.12
pluginsdir=/opt/scidb/14.12/lib/scidb/plugins
logconf=/opt/scidb/14.12/share/scidb/log4cxx.properties
db_user=scidb_pg_user
db_passwd=scidb_pg_user_pasw
base-port=1239
base-path=/home/scidb/scidb_data
redundancy=0

/### Threading: max_concurrent_queries=2, threads_per_query=1
/# max_concurrent_queries + 2:
execution-threads=4
/# max_concurrent_queries * threads_per_query:
result-prefetch-threads=2
/# threads_per_query:
result-prefetch-queue-size=1
operator-threads=1

/### Memory: 1000MB per instance, 750MB reserved
/# network: 300MB per instance assuming 5MB average chunks
/# in units of chunks per query:
sg-send-queue-size=15
sg-receive-queue-size=15
/# caches: 300MB per instance
smgr-cache-size=150
mem-array-threshold=150
/# sort: 150MB per instance (specified per thread)
merge-sort-buffer=75

Thank you for your reply!


#2

Hello,

That config file is not correctly populated. You should not use “localhost” in config files, especially in multi-node configurations. Don’t do this:
server-0=localhost,3
Instead, use the actual IP address of the machine.

Also, there is a prototype loader plugin we made that lives here:


You may find that to be much more scalable than the current load() operator.

load_tools can be built for and installed for 14.12; it is part of the package for 15.7 and there’s something different coming out with the upcoming 15.12