Gesvd "ChunkInterval is too small/large"


#1

The error could be reproduced by

  1. Upload a 2d matrix from R to scidb. x = matrix(rnorm(400), 20, 20); as.scidb(x)'
  2. perform gesvd in scidb client iquery
AFL% list('arrays');
No,name,id,schema,availability
0,'A',1,'A<val:double> [i=0:1,32,0,j=0:1,32,0]',true
1,'B',3,'B<value:double> [i=0:1,32,0,j=0:1,32,0]',true
2,'C',5,'C<value:double> [i=0:1,32,0,j=0:1,32,0]',true
4,'product',7,'not empty product<multiply:double> [i=0:1,32,0,j=0:1,32,0]',true
5,'x',27,'x<val:double> [i=0:19,20,0,j=0:19,20,0]',true
6,'y',29,'y<val:double> [i=0:99,100,0,j=0:99,100,0]',true
AFL% gesvd(product, 'right');
i,j,v
0,0,-0.866025
0,1,-0.5
1,0,-0.5
1,1,0.866025
AFL% gesvd(x, 'right');
UserException in file: src/linear_algebra/scalapackUtil/ScaLAPACKLogical.cpp function: checkScaLAPACKInputs line: 118
Error id: DLA::SCIDB_SE_INFER_SCHEMA::DLA_ERROR41
Error description: Error during schema inferring. ChunkInterval is too small.
Failed query id: 1100890966839
AFL% gesvd(y, 'right');
UserException in file: src/linear_algebra/scalapackUtil/ScaLAPACKLogical.cpp function: checkScaLAPACKInputs line: 126
Error id: DLA::SCIDB_SE_INFER_SCHEMA::DLA_ERROR42
Error description: Error during schema inferring. ChunkInterval is too large.
Failed query id: 1100890966846

gesvd works for array ‘product’ but not the others. Any idea? Many thanks.


#2

Yes, the Scalapack-based operators GEMM and GESVD are presently really restrictive on chunk sizes. That will be fixed this month with SciDB 13.9

Also your example exposed a subtle bug/deficiency in the R package that does not allow users to explicitly set chunk sizes larger than array dimensions.

Consider this example instead:

x = matrix(rnorm(1600),40,40)
as.scidb(x, name=“x”, rowChunkSize=32, colChunkSize=32, gc=FALSE)
iquery(“gesvd(x, ‘left’)”

I’ll fix up the R package so that it also will work for your tiny example.

–Bryan


#3

That’s great. I am very excited to see fast improvements in scidb. It would be great if there are some documentation about implementing custom functions (e.g. regression in my case) using Scalapack (only specific to scidb). Many users may be happy to contribute useful packages to scidb.