Transfer from Oracle to SciDB & recommended connectors


#1

Hello

I have a Java Swing application for my own personal use to run some calculations that uses an OracleXE instance. Well I saw a talk by Michael Stonebreaker about how bad it is to do linear algebra (matrix multiplication in my case) in a standard RDBMS, and then I hit the size limit of 11 GB on OracleXE so I am now migrating to SciDB.

What is the best way to migrate my data? Should I export it as text files and then run import commands?

I’m guessing there is a driver that will let me connect from Java to SciDB to issue commands, but when I look through the documentation I can’t find reference to it.

Thanks in advance,
Dave


#2

So there are references on the wiki to connecting from C/C++
trac.scidb.org/wiki/Docs/Archite … tInterface

But I haven’t found any docs yet on how to use it, or any other languages.


#3

To migrate data simply export as CSV files from your source database and load into SciDB using csv loading.
Our documentation describes csv loading in detail. Here’s a snapshot of the User Guide updated for the upcoming release:
p4-releases-12-10.s3.amazonaws. … -12.10.pdf

We don’t currently have a Java connector, however, we do have a python connector.


#4

Thanks, that documentation is very helpful.

I ended up outputing single column data in the scidb format with some simple chunks that were easy / consistent with the original. And now as the documentation mentions I can convert to other arrays with more appropriate chunk sizes. I wrote it up in detail here:
viewtopic.php?f=11&t=598