Spurious instance id in list chunk map (tombstones)


#1

In What is this chunk map thingy?, it says:

#  instn    :   uint32     // INSTANCE_ID
#                          //  The instance number within the SciDB cluster 
#                          // on which the chunk is located. Note that this
#                          // is also carried in the 'inst' dimension. 

My chunk map query showed some spurious outputs:

$ iquery -aq "filter(list('chunk map'), inst != instn)" | head
{inst,n} svrsn,instn,dguid,dhdrp,doffs,uaid,arrid,attid,coord,comp,flags,nelem,csize,usize,asize,addrs,clnof,clons,next,prev,data,accnt,nwrit,tstmp,raw,waitn,lpos,fposo,lposo,strge
{0,81990} 4294967295,4294967295,18446744073709551615,8932736,18446744073709551615,162,180,0,'{20150821, 0, 1450}',-1,8,18446744073709551615,18446744073709551615,18446744073709551615,18446744073709551615,0,4294967295,'[]',18446744073709551615,18446744073709551615,18446744073709551615,-1,-1,18446744073709551615,false,false,'','','',18446744073709551615
{0,81996} 4294967295,4294967295,18446744073709551615,8958720,18446744073709551615,162,180,0,'{20150821, 400000, 6000}',-1,8,18446744073709551615,18446744073709551615,18446744073709551615,18446744073709551615,0,4294967295,'[]',18446744073709551615,18446744073709551615,18446744073709551615,-1,-1,18446744073709551615,false,false,'','','',18446744073709551615

Was wondering what the spurious instn numbers are about?


#2

Here is the answer:

From the MAC Storage Explained document:

Creating a new version of an array involves adding new chunks and chunk map entries, only for the chunks that are different from the previous version. A special “tombstone” entry is added for chunks that used to exist in a previous version but are missing entirely from the new version.

Those instn numbers are for tombstone entries.

For more details, read the document at: SciDB's MAC(tm) Storage Explained

Also take a look at the screen-shots below (from a different machine):

scidb@scidb-VirtualBox$ iquery -aq "list('instances')"
{No} name,port,instance_id,online_since,instance_path
{0} '127.0.0.1',1239,0,'2016-03-08 18:51:08','/home/scidb/mydb-DB/000/0'
{1} '127.0.0.1',1240,1,'2016-03-08 18:51:08','/home/scidb/mydb-DB/000/1'
scidb@scidb-VirtualBox$ cat /opt/scidb/15.7/etc/config.ini | grep redundancy 
redundancy=1

Thus my box has two instances with a redundancy factor of 1

scidb@scidb-VirtualBox$ iquery -aq "filter(list('chunk map'), instn > 1)" 
{inst,n} svrsn,instn,dguid,dhdrp,doffs,uaid,arrid,attid,coord,comp,flags,nelem,csize,usize,asize,addrs,clnof,clons,next,prev,data,accnt,nwrit,tstmp,raw,waitn,lpos,fposo,lposo,strge

Thus, no chunk map entries with spurious instn values.

Now I run some queries to effectively delete values at a co-ordinate

scidb@scidb-VirtualBox$ iquery -aq "store(build(<val:double>[i=0:4,1,0], i), temp)"
{i} val
{0} 0
{1} 1
{2} 2
{3} 3
{4} 4
scidb@scidb-VirtualBox$ iquery -aq "store(between(temp, 0,3), temp)"
{i} val
{0} 0
{1} 1
{2} 2
{3} 3

Now my list of chunk map looks quite different:

scidb@scidb-VirtualBox$ iquery -aq "filter(list('chunk map'), instn > 1)" 
{inst,n} svrsn,instn,dguid,dhdrp,doffs,uaid,arrid,attid,coord,comp,flags,nelem,csize,usize,asize,addrs,clnof,clons,next,prev,data,accnt,nwrit,tstmp,raw,waitn,lpos,fposo,lposo,strge
{0,14750} 4294967295,4294967295,18446744073709551615,16282624,18446744073709551615,33373,33375,0,'{4}',-1,8,18446744073709551615,18446744073709551615,18446744073709551615,18446744073709551615,0,4294967295,'[]',18446744073709551615,18446744073709551615,18446744073709551615,-1,-1,18446744073709551615,false,false,'','','',18446744073709551615
{0,14760} 4294967295,4294967295,18446744073709551615,16283520,18446744073709551615,33373,33375,1,'{4}',-1,8,18446744073709551615,18446744073709551615,18446744073709551615,18446744073709551615,0,4294967295,'[]',18446744073709551615,18446744073709551615,18446744073709551615,-1,-1,18446744073709551615,false,false,'','','',18446744073709551615
...

Notice the unusual instn numbers popping up again.