Read-only chunk?


#1

i did backup of 14.8 array using parallel backup
iquery -an -q “save(current_speed,‘current_speed_backup3.csv’,-1,‘opaque’);”

i had upgraded scidb to 14.12 , I had recreated the target array
and now trying to do parallel restore
time iquery -an -q “load(current_speed,‘current_speed_backup3.csv’,-1,‘opaque’);”

now i am getting this strange error msg:

serException in file: src/query/ops/input/InputArray.cpp function: moveNext line: 371
Error id: scidb::SCIDB_SE_IMPORT_ERROR::SCIDB_LE_FILE_IMPORT_FAILED
Error description: Import error. Import from file ‘current_speed_backup3.csv’ (instance 0) to array ‘current_speed’ failed at line 1035673600, column 0, offset 78248, value=’’: Chunk is read-only and cannot be updated.
Command exited with non-zero status 1

what is that?? why chunk is read-only?


#2

more info from log file
2015-04-08 20:24:23,088 [0x7feded6f0700] [INFO ]: Executing query(1100942219121): load(current_speed,‘current_speed_backup3.csv’,-1,‘opaque’); from program
: 127.0.0.1:47837/opt/scidb/14.12/bin/iquery -an -q load(current_speed,‘current_speed_backup3.csv’,-1,‘opaque’); ;
2015-04-08 20:24:23,088 [0x7feded6f0700] [DEBUG]: Waiting notification in queryID from 31 instances
2015-04-08 20:24:23,091 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 1
2015-04-08 20:24:23,091 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 2
2015-04-08 20:24:23,091 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 5
2015-04-08 20:24:23,091 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 7
2015-04-08 20:24:23,092 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 9
2015-04-08 20:24:23,092 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 10
2015-04-08 20:24:23,092 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 13
2015-04-08 20:24:23,092 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 12
2015-04-08 20:24:23,092 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 11
2015-04-08 20:24:23,092 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 4
2015-04-08 20:24:23,092 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 14
2015-04-08 20:24:23,092 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 6
2015-04-08 20:24:23,093 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 8
2015-04-08 20:24:23,093 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 20
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 17
2015-04-08 20:24:23,094 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 3
2015-04-08 20:24:23,094 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 22
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 16
2015-04-08 20:24:23,094 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 19
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 15
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 18
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 26
2015-04-08 20:24:23,094 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 24
2015-04-08 20:24:23,094 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 21
2015-04-08 20:24:23,094 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 27
2015-04-08 20:24:23,096 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 23
2015-04-08 20:24:23,096 [0x7fede225a700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 31
2015-04-08 20:24:23,096 [0x7fede2159700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 28
2015-04-08 20:24:23,096 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 25
2015-04-08 20:24:23,096 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 29
2015-04-08 20:24:23,096 [0x7fede2058700] [DEBUG]: ServerMessageHandleJob::handleNotify: Notify on processing query 1100942219121 from instance 30
2015-04-08 20:24:23,096 [0x7feded6f0700] [DEBUG]: Send message from coordinator for waiting instances in queryID: 1100942219121
2015-04-08 20:24:23,097 [0x7feded6f0700] [DEBUG]: SG started with partitioning schema = 1, instanceID = 18446744073709551615
2015-04-08 20:24:23,106 [0x7feded6f0700] [DEBUG]: Array current_speed@1 was opened
2015-04-08 20:24:23,106 [0x7feded6f0700] [DEBUG]: Sending barrier to every one and waiting for 31 barrier messages

2015-04-08 20:24:23,470 [0x7feded6f0700] [DEBUG]: All barrier messages received - continuing
2015-04-08 20:24:31,293 [0x7fede2058700] [ERROR]: ServerMessageHandleJob::handleError: Error on processing query 1100942219121 on instance 20. Query coord
inator ID: 18446744073709551615. Message errorCode: 167. Message txt: UserException in file: src/smgr/io/InternalStorage.h function: updateChunk line: 340
Error id: scidb::SCIDB_SE_REDISTRIBUTE::SCIDB_LE_CANT_UPDATE_READ_ONLY_CHUNK
Error description: Error during redistribute. Chunk is read-only and cannot be updated.
2015-04-08 20:24:31,310 [0x7feded6f0700] [INFO ]: Loading of current_speed is completed: loaded 53 chunks and 0 cells with 0 errors
2015-04-08 20:24:31,310 [0x7feded6f0700] [DEBUG]: Query::done: queryID=1100942219121, _commitState=0, errorCode=167
2015-04-08 20:24:31,310 [0x7feded6f0700] [ERROR]: executeClientQuery failed to complete: UserException in file: src/query/ops/input/InputArray.cpp function
: moveNext line: 371
Error id: scidb::SCIDB_SE_IMPORT_ERROR::SCIDB_LE_FILE_IMPORT_FAILED
Error description: Import error. Import from file ‘current_speed_backup3.csv’ (instance 0) to array ‘current_speed’ failed at line 1035673600, column 0, of
fset 78248, value=’’: Chunk is read-only and cannot be updated.
2015-04-08 20:24:31,310 [0x7feded6f0700] [DEBUG]: Query (1100942219121) is being aborted
2015-04-08 20:24:31,310 [0x7feded6f0700] [ERROR]: Query (1100942219121) error handlers (2) are being executed
2015-04-08 20:24:31,310 [0x7feded6f0700] [DEBUG]: Update error handler is invoked for query (1100942219121)
2015-04-08 20:24:31,340 [0x7feded6f0700] [DEBUG]: UpdateErrorHandler::handleErrorOnCoordinator: the new version 1 of array current_speed (arrId=2) is being
rolled back for query (1100942219121)
2015-04-08 20:24:31,340 [0x7feded6f0700] [DEBUG]: Performing rollback
2015-04-08 20:24:31,340 [0x7feded6f0700] [DEBUG]: End of log at position 14280 rc=136
2015-04-08 20:24:31,340 [0x7feded6f0700] [DEBUG]: End of log at position 0 rc=0
2015-04-08 20:24:31,563 [0x7feded6f0700] [DEBUG]: Rolling back arrId = 1, version = 0


#3

Please provide the 14.8 config.ini, 14.12 config.ini, make sure that the .csv files are actually the opaque format files saved by the save() command on the 14.8 system (and not some csv format files left behind by an earlier load process).
The error suggests that the data has collisions, i.e. cells/chunks with the same coordinates.
If you look at scidb.log on instance 20, you should see something like
…[WARN] … Data collision … at pos XXX
XXX is the cell/chunk coordinate which scidb thinks is duplicated.
Executing
/opt/scidb/14.12/bin/iquery -an -q load(current_speed,‘current_speed_backup3.csv’,-1,‘opaque’,0,true)
will make load() detect data collisions and should force it to error out with the same information as in the log file on instance 20.


#4

ok, but do i recover from this?
i cannot load the data at all and backup file is corrupted essentially?

how did it got corrupted? i could query the data before export and array was in good shape.


#5

Before we can figure out how to recover, we need to diagnose the problem. If you dont mind, please provide us with the config.ini (14.8 & 14.12) and the scidb.log files preferably from all the instances. One way of getting most of that information is by running ‘scidb.py dbginfo[-lt] …’ And please make sure the .csv files that you are trying to load as the saved opaque backup files are in fact what they are on all the instances.