Fastest way to alter chunk size


#1

Hello,

I currently use

store(redimension(test, <value:double>[time=0:*,1,0,lat=0:72,10,0,lon=0:143,10,0]), test2);

to change (alter) chunk size of array test and save new array to test2.

I have two questions:

  1. is this the fastest way to alter chunk size?
  2. how do I tune store not to print out the resulting test2 which takes much longer than rechunking itself?

Thanks!


#2
  1. Yes.

  2. It depends what client tool you’re using. In iquery you can turn off printing the result of a query either by suppling a ‘-n’ option to the tool’s command line (most useful if you’re using iquery on a “query at a time” basis) or else by using the “SET NO FETCH;” command that’s part of the tool’s REPL functionality.

    (py27)[plumber@wolvi scidbtrunk]$ iquery --help
    Available options:
    -w [ --precision ] arg Precision for printing floating point numbers.
    Default is 6
    -c [ --host ] arg Host of one of the cluster instances. Default is
    ’localhost’
    -p [ --port ] arg Port for connection. Default is 1239
    -q [ --query ] arg Query to be executed
    -f [ --query-file ] arg File with query to be executed
    -r [ --result ] arg Filename with result array data
    -o [ --format ] arg Output format. Type ‘iquery -q help’ for details.
    Default is ‘dcsv’.
    -v [ --verbose ] Print debug info. Disabled by default
    -t [ --timer ] Print query execution time (in milliseconds)
    -n [ --no-fetch ] Skip data fetching. Disabled by default’
    -a [ --afl ] Switch to AFL query language mode. AQL by default
    -u [ --pluginsdir ] arg Path to the plugins directory
    -h [ --help ] Show this help text
    -V [ --version ] Show version info
    –ignore-errors Ignore execution errors in batch mode
    -A [ --auth-file ] arg User authentication file
    –admin Create an administrative connection to use reserved
    system resources

    (py27)[plumber@wolvi scidbtrunk]$ iquery
    AQL% help;
    set - List current options
    set lang afl - Set AFL as querying language
    set lang aql - Set AQL as querying language
    set fetch - Start retrieving query results
    set no fetch - Stop retrieving query results
    set timer - Start reporting query setup time
    set no timer - Stop reporting query setup time
    set verbose - Start reporting details from engine
    set no verbose - Stop reporting details from engine
    set format <base_format> [ : ] - Switch output format
    quit or exit - End iquery session-

    Available base_format values are:
    csv - Comma separated values
    csv+ - Comma separated values including coordinates
    dcsv - “Display CSV”, a more human-readable CSV variant
    dense - A variant of ‘text’ suitable for dense data
    sparse - A variant of ‘text’ suitable for sparse data
    opaque - SciDB raw storage format
    store - A variant of ‘text’ that includes overlap regions
    text - SciDB native text format
    tsv - Tab separated values (LinearTSV dialect)
    tsv+ - Tab separated values including coordinates (LinearTSV)

    Option flags for modifying output format. Not all flags affect all
    formats. These may be combined:
    d - Use double quotes in CSV output
    E - Use an empty string for null in TSV or CSV output
    l - Print label line for TSV or CSV output
    N - Use \N for null in CSV output
    n - Use the unquoted token ‘null’ for null in TSV or CSV output
    s - Use single quotes in CSV output (normally the default)
    ? - Use ?0 for null in TSV or CSV output
    AQL% SET LANG AFL;
    AFL% SET NO FETCH;
    AFL% store ( build ( < val : int64 > [I=0:9,10,0], I ), A );
    Query was executed successfully
    AFL% SET FETCH;
    AFL% store ( build ( < val : int64 > [I=0:9,10,0], I ), B );
    {I} val
    {0} 0
    {1} 1
    {2} 2
    {3} 3
    {4} 4
    {5} 5
    {6} 6
    {7} 7
    {8} 8
    {9} 9
    AFL%
    (py27)[plumber@wolvi scidbtrunk]$ iquery -naq "store ( build ( < val : int64 > [I=0:9,10,0], I ), A );"
    Query was executed successfully
    (py27)[plumber@wolvi scidbtrunk]$ iquery -aq “store ( build ( < val : int64 > [I=0:9,10,0], I ), B );”
    {I} val
    {0} 0
    {1} 1
    {2} 2
    {3} 3
    {4} 4
    {5} 5
    {6} 6
    {7} 7
    {8} 8
    {9} 9
    (py27)[plumber@wolvi scidbtrunk]$