Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-20344

Document HPCC + SPARK integration

    XMLWordPrintable

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 7.0.0
    • Documentation

    Description

      7.0.0 provides a compile-time option to integrate SPARK environment into the HPCC cluster.

      The HPCC install package will include the SPARK install binary and the spark-hpcc.jar library along with spark environment configuration files.

      • Spark will be installed here: /opt/HPCCSystems/externals/spark
      • Spark-hpcc is installed here: /opt/HPCCSystems/jars/spark-hpccXYZ.jar

      Upon startup of the platform, the spark cluster will be configured to have a master node on the eclwatch node, and a slave node on all Thor nodes, and the spark-hpcc library will be added to the spark env path.

      jamesdefabia g-pan Michael-Gardner mcmuja01

       

      Attachments

        Activity

          People

            g-pan Greg Panagiotatos
            rpastrana Rodrigo Pastrana
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: