DACP

Records queries: filter, projection, and limit

Clients can attach a records scan to an Item stream. The server executes filter → projection → limit. Only matching records and requested columns cross the network. Logical Item metadata and count(path) still describe the complete Item; they do not reflect a scan. Bytes views reject scan options.

Python

from dacp import Source, col

with Source.open("dacp://localhost:8815/data1") as ds:
    with ds.open(
        "/reports/orders",
        predicate=(col("id") > 1) & col("amount").is_not_null(),
        columns=["amount"],
        limit=100,
    ) as stream:
        records = stream.as_records()
        try:
            for batch in records:
                print(batch.to_pydict())
        finally:
            records.close()

Use parentheses around comparisons combined with & and |; negate a predicate with ~. Python and, or, and not are not expression builders. Use col("id").isin([1, 2]), .is_null(), or .is_not_null() for membership and null checks.

Scala

import dacp.client.Source
import dacp.client.Expressions.col
import scala.util.Using

Using.resource(Source.open("dacp://localhost:8815/data1")) { ds =>
  Using.resource(ds.open(
    "/reports/orders",
    predicate = Some((col("id") > 1) && col("amount").isNotNull),
    columns = Some(Seq("amount")),
    limit = Some(100L)
  )) { item =>
    Using.resource(item.asRecords()) { records =>
      while (records.next()) println(records.root.contentToTSVString())
    }
  }
}

Scala equality and inequality builders are === and =!=; boolean builders are &&, ||, and unary !. Membership is .isin(Seq(1, 2)); null checks are .isNull and .isNotNull. Both SDKs use the same protobuf expression tree; raw SQL is never accepted.

SQL in dacp-shell

Start the shell against a running DACP server (with DACP_CA, DACP_CERT, and DACP_KEY configured):

dacp-shell dacp://localhost:8815
# shell:
# datasets
# use data1

Enter a complete single-line query at dacp-shell>:

SELECT amount FROM "/reports/orders" WHERE id > 1 LIMIT 10
SELECT amount FROM 2 WHERE id > 1 LIMIT 10
SELECT id, amount FROM "/reports/orders" WHERE NOT (id IN (3, 4)) AND amount IS NOT NULL
SELECT * FROM "/events" WHERE name = 'beta'

SQLGlot parses the statement locally into the existing typed scan expression; the server receives no SQL text. The source is exactly one double-quoted absolute public Item path in the current Dataset, or an unquoted positive Item number from the latest ls result. For example, FROM 2 selects row 2 of that listing. ls PATH replaces the numbered listing; numbers are resolved locally to logical paths and never sent as source identifiers to the server. Zero and out-of-range numbers fail before opening a stream. Single quotes delimit string literals, with doubled quotes for escaping ('O''Brien'). Double quotes also allow column names containing spaces or SQL keywords. Column names retain DACP's exact, case-sensitive matching; SQL keywords are case insensitive. A trailing semicolon is optional.

Supported syntax:

Only records views support SQL. Numeric fractional literals require the Item's records schema in metadata. Native structured plugins should advertise records first when supplying that schema. Queries without fractional literals can still be constructed when the metadata does not provide a records schema; the server validates them against the streaming schema.

Results use the shell's bordered records table. Empty results keep their selected column names. --rows remains a display cap: the shell sends min(SQL LIMIT, --rows) to the server, or --rows when SQL LIMIT is omitted. For larger previews, start the shell with a larger --rows value. help select describes the command.

The initial implementation rejects joins, aliases, qualified column references, CTEs, subqueries, multiple statements, sorting, grouping, aggregation, computed columns, arbitrary functions, and offset. Unsupported clauses are never silently discarded. Missing/invalid paths, unknown columns, invalid literal types, and sources without records produce errors while leaving the shell connected.

Semantics and extension implementations

See XStream records scans for expression semantics, source pushdown and plugin contracts. DACP handles request decoding, authorization and transport; the data layer executes the scan.