Details
-
New Feature
-
Status: Resolved
-
Minor
-
Resolution: Won't Fix
-
7.12.x
-
None
-
None
Description
/* Notes on protocol:
*
- A JSON request with these top-level fields:
- "format" - the format of the reply. Supported formats = "binary", "xml", "json"
- "handle" - the handle of for a file session that was previously open (for continuation)
- "commCompression" - compression format of the communication protocol. Supports "LZ4", "LZW", "FLZ" (TBD: "ZLIB")
- "replyLimit" - Number of K to limit each reply size to. (default 1024)
- "node" - contains all 'activity' properties below:
* - For a secured dafilesrv (streaming protocol), requests will only be accepted if the meta blob ("metaInfo") has a matching signature.
- The request must specify "filePart" (1 based) to denote the partition # being read from or written to.
* - "filePartCopy" (1 based) defaults to 1
* - "kind" - supported kinds = "diskread", "diskwrite", "indexread", "indexcount" (TBD: "diskcount", "indexwrite", "disklookup")
- NB: disk vs index will be auto detected if "kind" is absent.
* - "action" - supported actions = "count" (used if "kind" is auto-detected to specify count should be performed instead of read)
* - "keyFilter" - filter the results by this expression (See: HPCC-18474 for more details).
* - "chooseN" - maximum # of results to return
* - "compressed" - specifies whether input file is compressed. NB: not relevant to "index" types. Default = false. Auto-detected.
* - "input" - specifies layout on disk of the file being read.
* - "output" - where relavant, specifies the output format to be returned
* - "fileName" is only used for unsecured non signed connections (normally forbidden), and specifies the fully qualified path to a physical file.
*
*/
void cmdStreamCommon(IPropertyTree *requestTree, MemoryBuffer &rest, MemoryBuffer &reply, CRemoteClientHandler &client, CClientStats &stats)
{
/* Example JSON request:
* - {
- "format" : "binary",
- "handle" : "1234",
- "replyLimit" : "64",
- "commCompression" : "LZ4",
- "node" : {
- "metaInfo" : "",
- "filePart" : 2,
- "filePartCopy" : 1,
- "kind" : "diskread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "chooseN" : 5,
- "compressed" : "false"
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- }
- }
- OR
- {
- "format" : "binary",
- "handle" : "1234",
- "replyLimit" : "64",
- "commCompression" : "LZ4",
- "node" : {
- "kind" : "diskread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "chooseN" : 5,
- "compressed" : "false"
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- }
- }
- OR
- {
- "format" : "xml",
- "handle" : "1234",
- "replyLimit" : "64",
- "node" : {
- "kind" : "diskread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "chooseN" : 5,
- "compressed" : "false"
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- }
- }
- OR
- {
- "format" : "xml",
- "handle" : "1234",
- "node" : {
- "kind" : "indexread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- }
- OR
- {
- "format" : "xml",
- "node" : {
- "kind" : "xmlread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- "ActivityOptions" : { // usually not required, options here may override file meta info. * "rowTag" : "/Dataset/OtherRow" * }
- }
- OR
- {
- "format" : "xml",
- "node" : {
- "kind" : "csvread",
- "fileName": "examplefilename",
- "keyFilter" : "f1='1 '",
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- "output" : { * "f2" : "string", * "f1" : "real" * }
- "ActivityOptions" : { // usually not required, options here may override file meta info. * "csvQuote" : "\"", * "csvSeparate" : "," * "csvTerminate" : "\\n,\\r\\n", * }
- }
- OR
- {
- "format" : "xml",
- "node" : {
- "action" : "count", // if present performs count with/without filter and returns count
- "fileName": "examplefilename", // can be either index or flat file
- "keyFilter" : "f1='1 '",
- "input" :
{
* "f1" : "string5",
* "f2" : "string5"
* }
,
- }
- }
- OR
- {
- "format" : "binary",
- "handle" : "1234",
- "replyLimit" : "64",
- "commCompression" : "LZ4",
- "node" : {
- "kind" : "diskwrite",
- "fileName": "examplefilename",
- "compressed" : "false" (or "LZ4", "FLZ", "LZW")
- "input" : { * "f1" : "string5", * "f2" : "string5" * }
- }
- }
- OR
- {
- "format" : "binary",
- "handle" : "1234",
- "replyLimit" : "64",
- "node" : {
- "kind" : "indexwrite",
- "fileName": "examplefilename",
- "input" : { * "f1" : "string5", * "f2" : "string5" * }
- }
- }
*
*/
Attachments
Issue Links
- relates to
-
JAPI-415 WsClient calls should be aware of maxreqentity behavior
-
- Resolved
-
-
HPCC-21445 Add zlib compression support to ease compatibility with external clients (e.g. java clients)
-
- New
-