Where to find documentation for plugins/extensions?


#1

Dear Contributors,

I’m looking into extending SciDB with some custom operators/aggregates for my application. Unfortunately the Section of the User Guide “16. SciDB Plugins: Extending SciDB Functionality” referenced in this forum is for some reason missing from both 12.03 and 12.10 guide versions. There’s a useful piece on user types and functions on the Wiki, but it doesn’t cover UDO and UDA. Examples that come with the source have something, but I couldn’t figure it out just by reading the code.

Can you please point me to some docs?

Thanks,
Kosty


#2

Hi kmaleyev !

Sorry. No. What you see is all there is, documentation wise. We’ve not made documenting the UDT/UDF/UDA and UDO extensibility a priority for several reasons.

  1. The features aren’t “finished” yet. If you look at our UDT/UDF extensibility, there are several features found in other extensible DBMS frameworks (like PostgreSQL, for example) that we don’t (yet) support. For example, we don’t have a mechanism to pass state between UDF invocations, or to allow the engine to manage memory for the UDFs (so we can clean it up in the case of a query abort).

  2. We have plans that go beyond what you get with other platform’s UDAs. If you look at the code, you’ll see that we currently combine four UDFs into a UDA; init(), accum(), merge() and final(). This works fine for most things, but we want to be able to support features like “sliding windows” more efficiently, which means we’ll have to extend the UDA framework.

  3. We really don’t have much of a handle on what people want to do with UDOs. We use them heavily, and we think they’re a very powerful feature. But it’s hard to standardize and document an API for them when we don’t really know how they’re going to be used “in the wild”, as it were.

What we’ve tried to do is to provide lots of examples–have a look at the UDTs/UDFs in the examples directory, and the findStarrs operator that was written as part of the SS-DB benchmark. That’s about all the advice we can give you at this point …

But! What did you have in mind? How would you like to extend SciDB?