Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-19788

Ability to get data from Amazon S3 to Thor

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Scheduled
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: DFU Server
    • Labels:
      None
    • Compatibility:
      Minor

      Description

      I'm trying to bring S3 data into Thor. We have a usecase where in data not being in Thor and in S3 is the constraint.

      There is a pilot initiative to try out Spark since most of the data is in S3 and the ability to build a solution using Spark is driven coz of the data being in S3.

      I would like to evaluate / have the feature to bring S3 data into Thor and compare timings of the builds for downstream data modeling work; As part of FIDO, we build aggregated data for Dimension model / Analytical reporting. For now, our biggest constraint is data being in S3.

      Thanks.

        Attachments

          Activity

            People

            • Assignee:
              mcmuja01 James McMullan
              Reporter:
              rajunagarajan Raju Nagarajan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Due:
                Created:
                Updated: