Suppose I initialize a scidb array fed by a file on a remote physical instance. For example here’s a snippet of my python script:
sdb.query(“store(aio_input(‘paths=/tmp/mri_3.out’, ‘instances=4294967299’, ‘num_attributes=1’), mri_4_3)”)
If I apply some operations to the array (e.g., aggregate(mri_4_3,max(a0)), would the processing (e.g., CPU usage, memory usage, etc.) happen on the remote instance (i.e., 4294967299)? It would be very risky to load remote files all into the local memory… which may cause out-of-memory error if the data set is huge.
I guess a more general question is: how to control which instance works (both computation and I/O) on which data chunk?