Canceling a load() from fifo query


#1

I’m unable to cancel a load() that reads data from a named pipe (fifo). The query is visible with list(‘queries’):

AFL% list(‘queries’);
inst,n,query_id,coordinator,query_string,creation_time,error_code,error,idle
0,0,1100968851615,0,‘load(TEST00004_1_1_var_load, ‘/panfs/pan1/genotype/data/TEST00004/v1.1/load/var_pipe’, -2, ‘TSV’)’,‘2016-02-04 22:09:46’,62,‘Query 1100968851615 was cancelled’,false

I tried killing the linux process, then iquery’s cancel():

AFL% cancel(1100968851615);
SystemException in file: src/network/MessageHandleJob.cpp function: handleError line: 1136
Error id: scidb::SCIDB_SE_NETWORK::SCIDB_LE_NO_QUORUM
Error description: Network error. Instance liveness has changed.

The only solution has been to restart Scidb. What is a more elegant solution to abort a load query?
I’m using Scidb 15.7.

Thanks.

===============================

Things get a little better if I increase the ‘execution-threads’ in config.ini. I’d been exhausting these by opening three fifos in a python script. After increasing, the execution-threads parameter, I can run cancel without error:

AFL% cancel(1100973089244);
Query was executed successfully

However, the query lingers, and I cannot remove the array targeted by the load:

AFL% list(‘queries’);
inst,n,query_id,coordinator,query_string,creation_time,error_code,error,idle
0,0,1100973089244,0,‘load(TEST00004_1_1_var_load, ‘/panfs/pan1/genotype/data/TEST00004/v1.1/load/var_pipe’, -2, ‘TSV’)’,‘2016-02-05 16:55:47’,62,‘Query 1100973089244 was cancelled’,false


#2

Hi there, sorry for the delay.

Yes I think this can happen - if the FIFO is never written to. A more elegant way might be to write a few characters to the fifo, and then issue the cancel. That has been working for some time - not sure which version you are using tho.

We also recommend using load_tools for TSV loads - you should get much better load rates. If you have 15.7, it comes as part of the package. See

Also, 15.12 will be out soon and we have made more improvements there.


#3

Thanks Alex. The behavior is triggered intentionally. I was trying to make sure my program exits cleanly, even if it is prematurely interrupted. I’ll take another look at load_tools.