I’m currently working on a university project, which requires me to write a program in Java, which loads data into SciDB. The original data is already stored in a multidimensional array.
The solution I have in mind, is to transform the multidimensional array in a one-dimensional (flat) array. I store the flat array in a file and then load it into SciDB and reshape it. Theoretically this works, but I’m afraid it might turn out to be too slow, as I have to load about 5GB of data spread over multiple files.
Now my question is, is it possible to load data from multidimensional arrays into SciDB without having to flatten and reshape it?