How to using JDBC by SciDB 18.1?


#1

I want using JDBC for SciDB 18.1.
But I didn’t find jar file.
Does SciDB support JDBC?
If SciDB support JDBC, what version of SciDB support JDBC?

Thanks.


#2

We apologize for being late in making the JDBC package available for release 18.1. Package scidb-18.1-jdbc will be available on the repository shortly for all supported platforms. Again, very sorry for the oversight!


#3

I’m told that the 18.1.8 packages for both CE and EE are now available on https://downloads.paradigm4.com . Once you install, your package repositories will be set up and you’ll be able to use yum or apt-get to install scidb-18.1-jdbc. The availability of the JDBC connector is the only change since 18.1.7. Again, sorry for the inconvenience!


#4

Hmm, I see… I’ll try to install it. Thank you.


#5

Hello, I installed to scidb-18.1-jdbc on CentOS7.4.
And imported the scidb4j.jar file of the [/opt/scidb/18.1/jdbc] path and I code it using Java.

    try {
		Connection conn = new Connection("192.168.10.156", 1239);
		System.out.println("SciDB Connect: "+conn.isClosed());
		Statement st = conn.createStatement();
		System.out.println("Create scidb statement");
		conn.getSciDBConnection().setAfl(true);
		ResultSet res = st.executeQuery("scan(foo)");
		
		ResultSetMetaData meta = res.getMetaData();
		System.out.println("Source array name: " + meta.getTableName(0));
		System.out.println(meta.getColumnCount() + " columns:");
		
		while (res.next()){
			ret.add(res.getDouble("x")+"");	 //x:inta64
		}
		conn.close();	
	} catch (SQLException e){
		e.printStackTrace();
	}

I’m using org.scidb.jdbc.Connection and an error occurred in connection .

2018-04-27 15:14:41.162 ERROR 6656 --- [io-8080-exec-10] o.s.b.w.servlet.support.ErrorPageFilter  : Forwarding to error page from request [/home] due to exception [org/scidb/jdbc/Connection]

java.lang.NoClassDefFoundError: org/scidb/jdbc/Connection
        at kr.re.inspace.support.SciDB.execute(SciDB.java:126) ~[classes/:na]
        at kr.re.inspace.controller.HomeController.home(HomeController.java:66) ~[classes/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_161]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_161]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_161]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_161]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[servlet-api.jar:na]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851) ~[spring-webmvc-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[servlet-api.jar:na]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-websocket.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:117) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:61) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:92) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:110) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.30]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.30]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [catalina.jar:8.5.30]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [catalina.jar:8.5.30]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) [catalina.jar:8.5.30]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [catalina.jar:8.5.30]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [catalina.jar:8.5.30]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650) [catalina.jar:8.5.30]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [catalina.jar:8.5.30]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [catalina.jar:8.5.30]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-coyote.jar:8.5.30]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-coyote.jar:8.5.30]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) [tomcat-coyote.jar:8.5.30]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-coyote.jar:8.5.30]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:8.5.30]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.5.30]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Caused by: java.lang.ClassNotFoundException: org.scidb.jdbc.Connection
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1308) ~[catalina.jar:8.5.30]
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1136) ~[catalina.jar:8.5.30]
        ... 64 common frames omitted

But I tried working same code on CentOS6.5 and it’s well works!
Jdbc not work in CentOS 7? Or I’m using it wrong?

If you’re developing jdbc, shim etc… , When can I use it on CentOS 7?

I have a lot of questions.:sweat_smile:
I’ll wait for a good answer.
Thank you :slight_smile:


#6

This looks like a -classpath issue. Make sure that the scidb4j.jar file appears in your class path.

Our JDBC tests are passing on CentOS 7.4 with OpenJDK 1.8.0_161-b14 .


#7

@derbar

We like “lots of questions” :slight_smile:

While @mjl had some suggestions on fixing your JDBC issues, I noticed that you also mentioned shim in your last response.

At Paradigm4, we use shim across Ubuntu and Redhat/CentOS (6 and 7) for building almost all our customer applications.

Would it be OK for you to use Shim instead of JDBC? If so, there are two options for installing Shim:


#8

I used jdk1.8.0_172.
I wrote SciDBTest.java and excuted the code in Windows.

Two servers tried to connect.
Server1: CentOS7.4, SciDB 18.1
Server2: CentOS 6.5, SciDB 18.1

Server2 successfully executed the code, but Server1 did not even connect to SciDB.
Server1 printed “connect start”, but not printed “connect end” .

I attach my code as well.

SciDBTest.java
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;

import org.scidb.jdbc.Connection;
import org.scidb.jdbc.IResultSetWrapper;

public class SciDBTest {
  private static String iqueryHost = "192.168.10.152";
  private static int iqueryPort = 1239;
  private static Connection conn;

  public static void main(String[] args) {
    // TODO Auto-generated method stub
    System.out.println("scidb start");

    try {
      System.out.println("connect start");
      conn = new Connection(iqueryHost, iqueryPort );
      System.out.println("connect end");
      Statement st = conn.createStatement();

      conn.getSciDBConnection().setAfl(false);
      ResultSet res = st.executeQuery("select * from foo");
      ResultSetMetaData meta = res.getMetaData();

      System.out.println("Source array name: " + meta.getTableName(0));
      System.out.println(meta.getColumnCount() + " columns:");

      IResultSetWrapper resWrapper = (IResultSetWrapper)res.unwrap(IResultSetWrapper.class);
      for (int i = 1; i <= meta.getColumnCount(); i++) {
        System.out.println(meta.getColumnName(i) + " - " + meta.getColumnTypeName(i) + " - is attribute:" + resWrapper.isColumnAttribute(i));
      }
      System.out.println("=====");
      System.out.println("i");
      System.out.println("-----");
      while (res.next()) {
        System.out.println(res.getLong("i"));
      }

    } catch (SQLException e){
      System.out.println(e);
      e.printStackTrace();
    }

    System.out.println("scidb end");
  }
}

#9

Thanks. I tried install Shim.

Then, Can I used SciDB-py in 18.1? The document says that it can be used up to 16.9…


#10

@derbar

It seems you are consulting the old scidb-py docs.

As mentioned on that page, the updated scidb-py is here. We do use this one with 18.1 all the time. So do let us know if you face any issues.


#11

Yes, Installation of SciDB 18.1 is possible, but an error occurs…
To use Scidb-py, I need run Shim.
So I tried to install Shim.

In CentOS 7, SciDB18.1 requires libprotobuf.so.8.
But Shim requires libprotobuf.so.7.

./shim: error while loading shared libraries: libprotobuf.so.7: cannot open shared object file: No such file or directory

What should I do? :joy:


#12

@derbar Sorry I seemed to have dropped the ball on this.

This seems to be a bug with our packages. I use Shim on SciDB 18.1 on RHEL7 – so it should work on Centos 7 too. The only difference I notice is that I use scidb EE.

Can you give me a more detailed repro instruction? Specifically,

  • assuming that you are using scidb CE
  • did you use Docker for setting up SciDB?
  • where did you get shim from
  • can you run scidb --version and shim -v and give me the output?

#13

I solved this problem myself :slight_smile:
I created a symbolic link to libprotobuf.so.8 and then Shim work!

p.s.

  • [$ scidb --version ]
    SciDB Version: 18.1.4
    Build Type: RelWithDebInfo
    Commit: b12ac1b
    Copyright © 2008-2017 SciDB, Inc.
  • [$ shim -version]
    SciDB Version: 18.1.2
    Shim Commit: b8025ba

Thanks to reply :smiley:


#14

Glad to know you have stuff working. Noticed that you are at 18.1.4 – do take a note of my comment about minor releases for 18.1.

BTW can you still give us repro instructions so that we can replicate the problem. Specifically,

  • Did you use Docker of the scidb install script to install scidb?
  • From where did you install Shim after that?

#15

I didn’t use Docker, installed directly on the server .