this issue based on conversation in https://track.hpccsystems.com/browse/JAPI-120?filter=17700
"This is actually controlled by the WsDFU, the ESP service generates a soapfault when a superfile containing multiple subfiles is interrogated:
2018-07-09 17:44:33 GMT: CWsDfuEx::onDFUGetFileMetaData: This feature is not designed to work with a superfile which contains multiple subfiles."
Jake Smith can confirm, but I think it is very common for the subfiles to all have the same format - I think some engines may not even work if it is different.
Permalink [ Edit|https://track.hpccsystems.com/secure/EditComment!default.jspa?id=90816&commentId=80187] [ Delete|https://track.hpccsystems.com/secure/DeleteComment!default.jspa?id=90816&commentId=80187]
Jake Smith added a comment - 6 hours ago
DFS forces subfiles to be of the same format, and should be commoning up attributes such that interrogating the super for meta data is sufficient to represent all of the subfiles.
I'm not sure what onDFUGetFileMetaData does, but it should be possible to return the meta info for a super.
Permalink [ Edit|https://track.hpccsystems.com/secure/EditComment!default.jspa?id=90816&commentId=80190] [ Delete|https://track.hpccsystems.com/secure/DeleteComment!default.jspa?id=90816&commentId=80190]
Kevin Wang added a comment - 4 hours ago
If DFS forces subfiles to be of the same format, may I return the format of one subfile as the format of the superfile?
Permalink [ Edit|https://track.hpccsystems.com/secure/EditComment!default.jspa?id=90816&commentId=80191] [ Delete|https://track.hpccsystems.com/secure/DeleteComment!default.jspa?id=90816&commentId=80191]
Jake Smith added a comment - 4 hours ago
The format should be in the super, common attributes are propagated into the super's attributes.
- relates to
-
HPCC-20039 Spark accepting superfiles with multiple sub-files
-
- Resolved
-