Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-19373

Spark use of keys

    Details

    • Type: Improvement
    • Status: Scheduled
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Spark-HPCC
    • Compatibility:
      Minor

      Description

      There are 3 use cases: keyed lookups; filtered index reads, and keyed joins.

      The keyed lookup will be the equivalent of an index read inside a project transform with a keyed field and optional non-keyed filters.  For example, SELF.foo := indx_attr(KEYED(bar=LEFT.my_var) AND extra_field=LEFT.my_extra);

      The filtered index read will be the equivalent of a PULL(indx_attr(KEYED(...) AND ...)).

      The keyed join will be the equivalent of JOIN(ds, ks, KEYED(...) ..., ...)

      The details are covered in the individual JIRA tickets.

        Attachments

          Issue Links

          is blocked by

          Sub-task - The sub-task of the issue HPCC-18475 Extend remote filtering to include indexes

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HPCC-19480 Add keyed lookup and fetch support.

          • Major - Major loss of function.
          • Scheduled
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HPCC-19375 Spark use of keys: filtered index read

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HPCC-19374 Spark use of Keys, keyed lookups

          • Major - Major loss of function.
          • Scheduled

          Improvement - An improvement or enhancement to an existing feature or task. HPCC-19376 Spark use of keys: keyed join

          • Major - Major loss of function.
          • Scheduled

            Activity

              People

              • Assignee:
                rpastrana Rodrigo Pastrana
                Reporter:
                johnholt John Holt
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated: