Insert array at position


#1

I have a two dimensional array which is stored in scidb. It has two axis called frequency and time where time is an infinite dimension which represents the epoch time in milliseconds:

<density:uint8> [frequency=0:199,200,0,time=0:*,1,0]

Now I would like to load a numpy array into scidb and insert it into the existing array. But when I load the numpy array into scidb, the time dimension of the array does not start at the correct position:

Shape: (200, 5)

<f0:int64> [i0=0:199,1000,0,i1=0:4,1000,0]

i1 starts (obviously) at index 0 and ends at 4.

So my question is how to set the start of this dimension to the correct time. For example 50 milliseconds. I tried to reshape it to following schema:

data_sdb.reshape('<f0:uint8> [i0=0:199,200,0,i1=50:55,4,0]')

But then I get following error message:

TypeError: cannot perform reduce with flexible type

So what would be the correct way?


#2

You need to use the redimension operator of SciDB.

I have added a Jupyter Notebook demo to solve this problem at https://github.com/ksens/scidbpy-insert-at-position

My solution replicates the error you saw and uses SciDB-Py, numpy.

The question you bring up is more general and @plumber provides a great demo to provide a solution below:

--
--  File: Shift_and_Insert_New_Data_Into_Target_Array.aql
--
-- About: 
--
--------------------------------------------------------------------------------
--
--  Hygiene ... 
--
SET LANG AFL;
SET NO FETCH;
--
remove ( A );
remove ( B );
--
--------------------------------------------------------------------------------
--
--  Create the two arrays. The objective is to insert the data that's in the 'A'
--  array, into the B array. 
CREATE ARRAY A 
< f0:uint8 > 
[i0=0:199,1000,0,i1=0:4,1000,0];
--
CREATE ARRAY B
< density:uint8 > 
[frequency=0:199,200,0,time=0:*,1,0];
--
--------------------------------------------------------------------------------
--
-- Populate those arrays with some data. 
SET NO FETCH;
store ( 
  build ( A, i0 * 199 + i1 ),
  A
);
--
store ( 
  build ( < density:uint8 >
          [ frequency=0:199,200,0,time=0:49,1,0],
          uint8((frequency*199 + time)%127)
        ),
  B
);
--
-- What does the shape of those arrays "look like"? 
SET FETCH;
dimensions ( A );
--
--  {No} name,start,length,chunk_interval,chunk_overlap,low,high,type
--  {0} 'i0', 0,    200,   1000,          0,            0,  199, 'int64'
--  {1} 'i1', 0,    5,     1000,          0,            0,  4,   'int64'
--
dimensions ( B );
--
--  {No} name,      start,length,chunk_interval,chunk_overlap,low,high,type
--  {0} 'frequency',0,    200,   200,           0,            0,  199, 'int64'
--  {1} 'time',     0,    *,     1,             0,            0,  49,  'int64'
--
-- So. You want to put A into B. 
--
SET LANG AFL;
SET FETCH;
op_count ( A );
--
--  {i} count
--  {0} 1000
--
op_count ( B );
--
--  {i} count
--  {0} 10000
--
SET NO FETCH;
insert ( 
  redimension ( 
    apply ( 
      A,
      frequency, i0,
      time,      i1 + 50,
      density,   f0
    ),
    B
  ),
  B
);
--
SET FETCH;
op_count ( B );
--
-- {i} count
-- {0} 11000
--
dimensions ( B );
--
--  {No} name,      start,length,chunk_interval,chunk_overlap,low,high,type
--  {0} 'frequency',0,    200,   200,           0,            0,  199, 'int64'
--  {1} 'time',     0,    *,     1,             0,            0,  54,  'int64'
--
-- -- END --