Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-26994

Document remote storage plane configuration

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 9.2.0
    • Documentation
    • None

    Description

      HPCC-26958 introduces the ability to access logical files from other remote environments in a cloud configuration.

      This is acheived by configuring the storage section of hpcc helm chart with a "remote" section.
      The storage.remote section is a list of named remote environments that define the remote service url and a section that maps remote plane names to local plane names.
      The local planes referenced are special planes with category 'remote'. They are read-only and only exposed to the engines which can read from them.

      Example:

      storage:
        planes:
      ...
        - name: hpcc2-stddata
          pvc: hpcc2-stddata-pv
          prefix: "/var/lib/HPCCSystems/hpcc2/hpcc-stddata"
          category: remote
        - name: hpcc2-fastdata
          pvc: hpcc2-fastdata-pv
          prefix: "/var/lib/HPCCSystems/hpcc2/hpcc-fastdata"
          category: remote
        remote:
        - name: hpcc2
          service: http://20.102.22.31:8010
          planes:
          - remote: data
            local: hpcc2-stddata
          - remote: fastdata
            local: hpcc2-fastdata
      

      This defines a remote target called "hpcc2" whose DFS service url is http://20.102.22.31:8010 and whose local plane is "hpcc2data".

      The local plane must be defined such that it shares the same storage as the remote environment. This is expected to be done via a PVC that has been pre-configured to use the same storage.

      A logical file will be access in ECL in the following form, e.g.:

      ds := DATASET('~remote::hpcc2::somescope1::somelfn1', rec);
      

      Attachments

        Issue Links

          Activity

            People

              g-pan Greg Panagiotatos
              jakesmith Jake Smith
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: