SCIDB_SE_NETWORK::SCIDB_LE_CANT_SEND_RECEIVE


#1

I am getting the following error when i run a redmension_store, it take very long time and throws the following error:

SystemException in file: src/network/BaseConnection.h function: receive line: 294 Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CANT_SEND_RECEIVE Error description: Network error. Cannot send or receive network messages.

Version 12.10.

Is Iquery getting timed out and is there any setting ?


#2

Hello,

That’s a crash. That error happens when SciDB crashes and quickly restarts. The meaning of the error is iquery-centric: “I can’t talk to SciDB”.
Can you please post your create array statement and your redimension_store query. I’ll try to take a look.
Thanks and sorry for the inconvenience.

  • Alex Poliakov

#3

Single VM with 4Gb of Ram and 4 VPROC. Please find the execution log of the script and config.ini

[code]#

Load the base data into a 1D Raw Load array.

iquery -aq "
remove ( pipe_load)
"
UserQueryException in file: src/query/parser/ALTranslator.cpp function: createArrayReferenceParam line: 863
Error id: scidb::SCIDB_SE_QPROC::SCIDB_LE_ARRAY_DOESNT_EXIST
Error description: Query processor error. Array ‘pipe_load’ does not exist.
remove ( pipe_load)
^^^^^^^^^^^

Failed query id: 1100864143704

iquery -aq "
CREATE EMPTY ARRAY pipe_load
<
id : int64,
pipelineId : int64,
segmentId : int64,
pointId : int64,
mesasurementDate : datetime,
nomCycle : string NULL,
locationType :string NULL,
scheduledVolume : double NULL,
actualVolume : double NULL,
utlization : double NULL,
designCapacity : double NULL,
operationalCapacity : double NULL,
actualCapacity : double NULL,
operationallyAvailable : double NULL,
flowdirection : string NULL,
interruptibleflow : string NULL

[ Line=0:*,1000,0 ]
"
Query was executed successfully

rm -rf /tmp/pipe_load_pipe
mkfifo /tmp/pipe_load_pipe

./pre_process.sh pipe.files | csv2scidb -c 1000 -p NNNNSSSNNNNNNNSS > /tmp/pipe_load_pipe &

iquery -naq "
load ( pipe_load, ‘/tmp/pipe_load_pipe’)
"
Query was executed successfully

iquery -o lcsv+ -aq "
aggregate ( pipe_load, count(id) )
"
i,id_count
0,71000343
Query execution time: 3810ms

iquery -aq "remove ( pipe_load_sorted )"
UserQueryException in file: src/query/parser/ALTranslator.cpp function: createArrayReferenceParam line: 863
Error id: scidb::SCIDB_SE_QPROC::SCIDB_LE_ARRAY_DOESNT_EXIST
Error description: Query processor error. Array ‘pipe_load_sorted’ does not exist.
remove ( pipe_load_sorted )
^^^^^^^^^^^^^^^^^^
Failed query id: 1101967409225

Sort the array

iquery -naq "
store(sort( pipe_load,mesasurementDate,pipelineId,segmentId,pointId,nomCycle,locationType),pipe_load_sorted)
"
SystemException in file: src/network/BaseConnection.h function: receive line: 294
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CANT_SEND_RECEIVE
Error description: Network error. Cannot send or receive network messages.

Move the 1D load file data into a 3D array for Symbols and Timeseries.

iquery -aq "remove ( pipe )"
93 [0x7f5713876780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
93 [0x7f5713876780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
iquery SystemException in file: src/network/BaseConnection.cpp function: connect line: 262
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CONNECTION_ERROR
Error description: Network error. Error #system:111 when connecting to localhost:1239.

iquery -aq "
CREATE EMPTY ARRAY pipe
<
id : int64,
segmentId : int64,
nomCycle : string NULL,
locationType :string NULL,
scheduledVolume : double NULL,
actualVolume : double NULL,
utlization : double NULL,
designCapacity : double NULL,
operationalCapacity : double NULL,
actualCapacity : double NULL,
operationallyAvailable : double NULL,
flowdirection : string NULL,
interruptibleflow : string NULL

[ pipelineId=0:,1000,0, mesasurementDate(datetime)=,366,1, pointId=0:*,1000,0 ]
"
2 [0x7f9859abf780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
2 [0x7f9859abf780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
iquery SystemException in file: src/network/BaseConnection.cpp function: connect line: 262
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CONNECTION_ERROR
Error description: Network error. Error #system:111 when connecting to localhost:1239.

iquery -naq "
redimension_store (
pipe_load,
pipe
)
"
2 [0x7ff2dcf48780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
2 [0x7ff2dcf48780] FATAL scidb.services.network null - Error #system:111 when connecting to localhost:1239
iquery SystemException in file: src/network/BaseConnection.cpp function: connect line: 262
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_CONNECTION_ERROR
Error description: Network error. Error #system:111 when connecting to localhost:1239.

[/code]

Here is the config.ini

[test] server-0=scidb-controller,1 db_user=user1 db_passwd=user1passwd install_root=/opt/scidb/12.10 metadata=/opt/scidb/12.10/share/scidb/meta.sql pluginsdir=/opt/scidb/12.10/lib/scidb/plugins logconf=/opt/scidb/12.10/share/scidb/log4cxx.properties base-path=/data base-port=1239 interface=eth0 network-buffer=1024 mem-array-threshold=1024 smgr-cache-size=1024 execution-threads=16 result-prefetch-queue-size=4 result-prefetch-threads=4 chunk-segment-size=100485760


#4

I re-ran the script and did see the swap space slowly getting filled and the Scidb process recycled. I have only 1Gb swap space. I increaded the swap space and testing again. I also decreased the chunk size to 100.

The memory is slowly growing, is this behaviour normal or is it a memory leak?

Thank you,
Vijay


#5

It worked after the memory is increaded to 6 Gb and Swap space is increaded to 6 Gb. The memory is at a stable state @ 6 Gb and did not see swap usage. But it took few hours to complete.

Is this performace due to the VM or do we get similar performace on a high end server will good storage and memory? I have 70Million rows and 15 attributes approximatly.

Thanks
Vijay


#6

So it’s the sort() query that is causing the problem.

This line:
server-0=scidb-controller,1
means that you are actually running 2 scidb processes. 2 instances. Sorry if the notation is confusing. The existence of the coordinator is implied, so if you want 1 instance only, you should say “scidb-controller,0”.

These lines:
network-buffer=1024
mem-array-threshold=1024
smgr-cache-size=1024

Mean that there is ~ give or take ~ 3GB of memory that the instance is allowed to consume. This usage is not very precise, but I think it explains what you are seeing. 2 instances, at 3GB each, eating 6GB total.

Does this make sense?