Binary load different in 13.12?


#1

Hi, SciDB Gurus,

In the past, I was able to distribute my binary files among all and sometime most but not all instances and to execute

iquery -p 1439 -anq "load(GF_Loadarr, '/dev/shm/scidbCommon.bin', -1, '(int64, int64, int64, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null)')"

The .bin file was at the same place on each node, but with different content as appropriate to the slice being loaded.
It worked like a charm, even when there was a warning message that the .bin file was missing on some nodes. For example, the coordinator, instance 0, would never get a .bin file, only the nodes 1, 2, …, n

I am finding that in 13.12, I get an error instead of a warning message and the load does not complete. Is there a magic value for the optional max_errors code in the load function that restores the old behavior? I tried 0, 1, 34, … same results every time.
The manual is not specific about what other options besides 0 exist and their ramifications.

Can you set me straight, please? It was so nice to have these files on all but a few nodes and ignore the warning.
Thanks, George


#2

Hi George,

Can you send over the error message code and text please?


#3

Alex, the query and the error messages are included:
To recap, scidbCommon.bin is the file on all nodes (except instance 0)
It exists on all nodes (except node 0)
In the old days, the warning message would say which instance is missing the file and continue and complete successfully.

iquery -p 1439 -anq "load(GF_Loadarr, '/dev/shm/scidbCommon.bin', -1, '(int64, int64, int64, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null)', 2)" UserQueryException in file: src/query/ops/input/LogicalInput.cpp function: inferSchema line: 152 Error id: scidb::SCIDB_SE_INFER_SCHEMA::SCIDB_LE_FILE_NOT_FOUND Error description: Error during schema inferring. File '/dev/shm/scidbCommon.bin' not found. load(GF_Loadarr, '/dev/shm/scidbCommon.bin', -1, '(int64, int64, int64, ^^^^^^^^^^^^^^^^^^^^^^^^^^ float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null)', 2) Failed query id: 1100907670319

Thanks for the help! George


#4

The previous message appeared on the "terminal"
This is the is the scidb.log excerpt. Note the warning of missing file from instance #0 and the subsequent error.

2014-01-26 20:01:38,203 [0x7f5177141700] [DEBUG]: Parsing query(1100908572496): load(GF_Loadarr, '/dev/shm/scidbCommon.bin', -1, '(int64, int64, int64, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null)', 99) 2014-01-26 20:01:38,204 [0x7f5177141700] [WARN ]: File '/dev/shm/scidbCommon.bin' not found on instance #0 2014-01-26 20:01:38,205 [0x7f5177141700] [DEBUG]: Query::done: queryID=1100908572496, _commitState=0, errorCode=37 2014-01-26 20:01:38,205 [0x7f5177141700] [ERROR]: prepareClientQuery failed to complete: UserQueryException in file: src/query/ops/input/LogicalInput.cpp function: inferSchema \ line: 152 Error id: scidb::SCIDB_SE_INFER_SCHEMA::SCIDB_LE_FILE_NOT_FOUND Error description: Error during schema inferring. File '/dev/shm/scidbCommon.bin' not found. load(GF_Loadarr, '/dev/shm/scidbCommon.bin', -1, '(int64, int64, int64, ^^^^^^^^^^^^^^^^^^^^^^^^^^ float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null, float null)', 99) Failed query id: 1100908572496 2014-01-26 20:01:38,205 [0x7f5177141700] [DEBUG]: Query (1100908572496) is being aborted 2014-01-26 20:01:38,205 [0x7f5177141700] [ERROR]: Query (1100908572496) error handlers (1) are being executed 2014-01-26 20:01:38,205 [0x7f5177141700] [DEBUG]: Broadcast ABORT message to all instances for query 1100908572496

Cheers, George