Uploaded image for project: 'JAPI'
  1. JAPI
  2. JAPI-271

Spark binary should not be downloaded at build-time

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 7.6.x
    • 7.12.x
    • Spark
    • None

    Description

      Noticed that spark is downloaded at build-time:
      [ 86%] Performing download step (download, verify and extract) for 'fetch-spark-source'
      – Downloading...
      dst='/home/hadoop/GIT/HPCC-build/downloads/spark-2.4.3-bin-hadoop2.7.tgz'
      timeout='300 seconds'
      – Using src='https://archive.apache.org/dist/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz'
      [download 0% complete]
      [download 1% complete]
      [download 2% complete]
      [download 3% complete]
      [download 4% complete]
      [download 5% complete]

      the spark binary should be submoduled in an attempt to avoid downloading many times.

      Attachments

        Issue Links

          Activity

            People

              Michael-Gardner Michael Gardner
              rpastrana Rodrigo Pastrana
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: