Development guide
Integrate DACP into applications, embed a data service, or work on remote-access and collaboration protocols. Start with a complete example and use the corresponding API reference while developing.
flowchart TD
Goal["What are you building?"] --> Client["An application accessing a service"]
Goal --> Host["An application publishing data"]
Goal --> Collaboration["Member collaboration integration"]
Client --> SDK["Python or Scala client"]
Host --> Embedded["DacpServer + application catalog"]
Collaboration --> Protocol["Collaboration SDK + Union protocol"]
1. Application integration
| Task | Example and reference |
|---|---|
| Discover data and consume remote streams | Python examples, Scala SDK |
| Submit Jobs and consume attempt/result events | Job submission, execution policies |
| Handle authentication and capabilities | Security and role checks, protocol |
| Integrate member collaboration operations | Python collaboration SDK, Scala collaboration SDK |
Ordinary and collaboration clients share transport and certificate implementations. Servers enforce permissions independently of the chosen SDK.
2. Embed and extend a service
| Task | Example and reference |
|---|---|
| Start and close a server from Scala | Embedding API, complete embedded-server example |
| Publish an application catalog | Embedding API, server configuration |
| Load existing backend and runtime plugins | Plugin deployment, plugin example |
| Configure service-side execution history | History API and administration |
Implement Dataset backends, format readers, operators, stream contracts and execution-storage plugins using the XStream Development guide. Use DACP configuration to load them into a service.
3. Protocol reference
| Topic | Reference |
|---|---|
| Discovery, reads, tickets and caller capabilities | DACP protocol |
| Wire messages and service configuration | Service reference, dacp.proto |
| Membership, subjob dispatch and inter-node streams | Union collaboration, Union transport API |
| Public Union namespace and incomplete catalogs | Union public service |
| Credentials, authorization and cancellation ownership | Security |
4. Validate integrations
Run the checks matching the changed component from the repository root:
(cd dacp-server && sbt test)
.venv/bin/python -m pytest -q dacp-client/tests
(cd dacp-client-scala && sbt test)
(cd dacp-union-server && sbt test)
For cross-node container integration, run bash scripts/test-collaboration.sh with Docker and the certificates and runtimes specified by that script. Cover discovery, role denial, partial directories, cancellation, stream closure and failure propagation. Keep protocol definitions, callers, examples and English/Chinese references synchronized.