Thanks for the details! They certainly help.
So. The way the load(…) (or more strictly the input(…) ) operator work is that they expect a “file name” as a parameter. But Linux / Unix possess a number of file-system features that will allow you to achieve your goal of avoiding interacting with the disk multiple times (makin’ the magnet bounce).
( a ) Instead of files, use “named pipes”, which are essentially FIFO queues for data. “man mkfifo” for details. Everything in the SciDB manual that talks about reading from “files” or mentions “file-names” applies equally to these named pipes.
( b ) From your LAMMP application, write the result data from each MPI process into one of these named pipes. You will almost certainly need to explicitly write coordinates information into the stream. That is, serialize the state of the model run, including the coordinates information. (This last is necessary because we’re going to have to re-organize the data once it’s inside SciDB … it’s probable that data from one MPI process might need to be written to multiple SciDB instances, especially if you’re using a SciDB feature like chunk overlap.)
( c ) Then, so far as SciDB is concerned, for reading purposes the “named pipes” you’ve created behave just like files. Only instead of reading data from a disk file, you’re actually reading from an in-memory, inter-process FIFO queue managed by the Linux kernel.
( d ) Once you’ve got the data into SciDB, you’re probable going to need to re-organize it to slot it into an array to track the changes to the model state over time. But that part of the work must be done regardless of whether you’re using a file or a fifo.
Is this clear? And hopefully helpful?