Shim & scidbpy using multiple SciDB ports


#1

Scenario: Multiple SciDB ports on the same hardware (to allow for testing without having to frequently shut down the production cluster)

Question: Is it possible to launch shim to allow for more than one SciDB port and if so, is it then possible to select a SciDB port to use for connection with scidbpy?

Thanks.


#2

According to localhost:8080


#3

Looks like I didn’t explain the problem properly. Assuming I have two SciDB coordinator instances on the same physical node using different SciDB ports, is there a way to choose which SciDB coordinator instance I want to connect to with scidbpy? Since scidbpy uses shim, once shim is running (say started by root) configured with some http port(s) and SciDB port, it doesn’t look like a user can start another shim in user space to overwrite the already running shim with options that will allow the selection of a new SciDB port that scidbpy running in that user space can then use. On the other hand, if the shim started by root can admit multiple SciDB ports and the connect interface of scidbpy allows a user to choose which of the alternate ports to use, that is what I was asking about.

Thanks,


#4

The short answer is no. Shim can’t be configured right now to use more than one SciDB port.

You can start multiple copies of shim, and use command-line options to control the port. See shim -h or man shim or the shim webpage for more about this. Note that you’ll want to start the 2nd shim service also running on a different http port to differentiate it from the default shim service.

Here is an example:

shim -p 8090 -s 2239

That would start a shim server running on http port 8090 (that one could use from python or R or wherever), and talking to a hypothetical SciDB server running on port 2239. Then in python use port 8090 to talk to the 2239 SciDB and the default port 8080 to talk to SciDB on its default port of 1239.

This shows why I picked the name shim! It’s a network service that sits in-between SciDB and other clients, and translates things into HTTP.

Looking back at this, I think it’s more complicated than it needs to be. We should probably incorporate a way to select the SCiDB network port as part of shims HTTP API. This should not be too hard.

Is this something that you would be interested in helping with? Maybe at least testing some new ideas out?

Best regards,

Bryan


#5

The short answer is no. Shim can’t be configured right now to use more than one SciDB port.

You can start multiple copies of shim, and use command-line options to control the port. See shim -h or man shim or the shim webpage for more about this. Note that you’ll want to start the 2nd shim service also running on a different http port to differentiate it from the default shim service.

Here is an example:

shim -p 8090 -s 2239

That would start a shim server running on http port 8090 (that one could use from python or R or wherever), and talking to a hypothetical SciDB server running on port 2239. Then in python use port 8090 to talk to the 2239 SciDB and the default port 8080 to talk to SciDB on its default port of 1239.

This shows why I picked the name shim! It’s a network service that sits in-between SciDB and other clients, and translates things into HTTP.

Looking back at this, I think it’s more complicated than it needs to be. We should probably incorporate a way to select the SCiDB network port as part of shims HTTP API. This should not be too hard.

Is this something that you would be interested in helping with? Maybe at least testing some new ideas out?

Best regards,

Bryan


#6

Thank you very much for your comprehensive answer.
Yes to your question about modifying shim - will take a look after some pending deadlines. If I do make any worthwhile changes, I’ll let you know.

Hamid


#7

Has there been any more progress on this front?