Assign several values


#1

Hello!

The example below assign a value, like this:
mesh3d[0][0][0][0][0] = 1.5

I have success to do this because I have only one attribute, the attribute value:double.

create empty array mesh3d 
<value:double> 
[simulation_number=0:*,1,0, time_step=0:*,1,0, x_axis=0:*,1,0, y_axis=0:*,1,0, z_axis=0:*,1,0]
Query was executed successfully
store(merge(build_sparse(<value:double> [simulation_number=0:0,1,0, time_step=0:0,1,0, x_axis=0:0,1,0, y_axis=0:0,1,0, z_axis=0:0,1,0], 1.5, simulation_number=0 and time_step=0 and x_axis=0 and y_axis=0 and z_axis=0),mesh3d), mesh3d)
[[[[[{0,0,0,0,0}(1.5)]]]]]

I don’t have success to assign value when I have more than one attribute. Can anybody help me?
What is the sintaxe to put over ??? in the second code below?
such like this:
geometry3d[0][0][0][0][0] = (1,2,3,4,5,6,7);
where, velocity_x=1, velocity_y=2, …

create empty array geometry3d 
<velocity_x:double, velocity_y:double, velocity_z:double, pression:double, displacement_x:double, displacement_y:double, displacement_z:double>
[simulation_number=0:*,1,0, time_step=0:*,1,0, x_axis=0:*,1,0, y_axis=0:*,1,0, z_axis=0:*,1,0];
Query was executed successfully
store(merge(build_sparse(<velocity_x:double, velocity_y:double, velocity_z:double, pression:double, displacement_x:double, displacement_y:double, displacement_z:double> [simulation_number=0:0,1,0, time_step=0:0,1,0, x_axis=0:0,1,0, y_axis=0:0,1,0, z_axis=0:0,1,0], ???????, simulation_number=0 and time_step=0 and x_axis=0 and y_axis=0 and z_axis=0),geometry3d), geometry3d)

Thank you very much for the attention.


#2

Hi rcosta

You can use “join()” to organize your data. For example:

CMD="
join ( 
   build_sparse ( < x : double > [I=0:9,10,0], 100-I, I % 2 = 0),
   build_sparse ( < Y : double > [I=0:9,10,0], I, I % 2 = 0)
)
"
iquery -aq "$CMD"
[{0}(100,0),{2}(98,2),{4}(96,4),{6}(94,6),{8}(92,8)]

To increase the number of attributes, you have to nest the “join()” operators.

CMD="
join ( 
   build_sparse ( < x : double > [I=0:9,10,0], 100-I, I % 2 = 0),
   join (
       build_sparse ( < y : double > [I=0:9,10,0], I, I % 2 = 0),
       build_sparse ( < z : double > [I=0:9,10,0], 5 - I, I % 2 = 0)
    )
)
"
iquery -aq "$CMD"
[{0}(100,0,5),{2}(98,2,3),{4}(96,4,1),{6}(94,6,-1),{8}(92,8,-3)]

So if you think that’s a complete PITA, you’re not alone. I suspect what you really want to do is to generate data over the entire array space. ie. For each simulation, for each time step, for each X and Y, generate a value. So - here’s some thoughts.

  1. First, don’t use chunk length = 1. The whole point of our little Array DBMS is to permit folk to cluster by region. What you want is, for every chunk, to have a sizeable portion of data in it. We usually shoot for chunk sizes of between 2 and 16 meg (YMMV). What you have here, with every chunk_length = 1 is a size of 1 element - say, 8 bytes. Here’s what I suggest. You probably have a min/max range for all of the X,Y,Z and T dimensions, with some number of simulations. And you have (say) 8 bytes per attribute. So - that’s about (say) 1,000,000 elements per chunk. So try to find values for X.chunk_length, Y.chunk_length, Z.chunk_length and T.chunk_length which when multiplied get you to about 1,000,000.

  2. My second intuition is that you don’t necessarily want more than one T per chunk. I might be wrong about this; you might be computing windows of some length through T to “smooth” your < X, Y, Z > data over time. It depends on your workload. But if what you’re doing is to compare T=t with T=t+1 (say), or your slicing 3D boxes out for Simulation=S, and T=t, then I would set your T value to 1, and set X.chunk_length, Y.chunk_length and Z.chunk_length to 100 each. That will give you 8M “chunks”, each of which contains 100x100x100 “box” in the X,Y,Z space, each for a fixed S and T. Note that I am assuming here that your 5D space is dense. If it’s not, you’ll need to adjust your chunk lengths accordingly. Chunk lengths are in the logical space.

  3. If you’re trying to simulate this data, I wouldn’t use a bunch of build() or build_sparse() ops. The join() is ugly. I would take one of the following two options.

3.1 Use cross_join. The idea is that you can build a vector of the length you want, and then use cross() to build a large, nD array. Something like this:

CMD="
cross_join (   
   build_sparse (<x : double> [I=0:9,10,0], random()/2147483648.0, I % 2 = 0),
   build_sparse (<y : double> [I=0:9,10,0], random()/2147483648.0, I % 2 = 0)
)
"
iquery -aq "$CMD"

Now, there are a lot of problems with this from a data generation point of view. You won’t end up with something that looks even semi-random. You’ll end up with lots of stripes and planes through the data. So - there’s another approach.

3.2 Generate the data in 1D, then use redimension_store() to convert it to the target shape. The following query generates exactly 10,000 points in the 5D space of 100x100x100x100x10. You can fiddle with the types in the query to get the ranges and types you want.

CMD="
SELECT floor(random()/2147483648.0 * 100.0) AS x_axis,
       floor(random()/2147483648.0 * 100.0) AS y_axis,
       floor(random()/2147483648.0 * 100.0) AS z_axis,
       floor(random()/2147483648.0 * 100.0) AS time_step,
       floor(random()/2147483648.0 * 10.0) AS simulation_number,
       random()/2147483648.0 * 1000.0 AS value
 INTO Temp_Array
  FROM build ( < v : int64 > [I=0:9999,1000,0], I)
" 
iquery -nq "$CMD"
#
iquery -aq "show ( Temp_Array )"

From there, use the contents of the Temp_Array to populate the target schema …

#
CMD="create array mesh3d
<value:double> 
[ simulation_number=0:*,1,0,
  time_step=0:*,1,0,   
  x_axis=0:*,99,0, 
  y_axis=0:*,99,0,
  z_axis=0:*,99,0]"
     
iquery -q "$CMD"

CMD="
SELECT * 
  INTO mesh3d
 FROM Temp_Array
"
#
iquery -q "$CMD"

Note that when you run this script above, you’re going to create a very, very sparse array. 100x100x100x100x10 = 1,000,000,000 cells, of which you’re only going to populate 10,000. You’re asking SciDB to store more information about what isn’t there (all of the “empty” space needs to be logged) than you are about what actually is there. To overcome this, consider adjusting the chunk_length values for your dimensions.

Hope this helps!

KR

Pb