Uploaded image for project: 'Machine Learning Library'
  1. Machine Learning Library
  2. ML-415

Begin development of a software library (consisting of ECL and Python code) to provide HPCC Systems distributed neural network training

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 7.0.0
    • Fix Version/s: None
    • Component/s: ecl-ml

      Description

      The project I intend to work on during the Summer 2018 internship at HPCC is to research and develop distributed deep learning algorithms on HPCC Systems. The training process for modern deep neural networks require big data and large computational power. Though HPCC Systems excels at both of these, HPCC is limited to a single node when dealing with neural networks and its training. This project will greatly enhance HPCC System’s neural network capabilities.

      This project would aim to begin development of a software library (consisting of ECL and Python code) that would provide HPCC Systems distributed neural network training, using a popular configuration that is well suited for a cluster computer. Called “data parallelism”, this paradigm provides asynchronous training with minimal network overhead and can be used with different neural network training algorithms. This framework would also serve as a building block for future development for different distributed configurations and distributed neural network algorithms.

       The deliverables for the scope of this internship: (2 weeks each)

      1. Functions for interpolating the data between ECL records to TensorFlow runtime
      2. Functions for interpolating the neural network model between TensorFlow and ECL
      3. Functions for interpolating NN model parameters between TensorFlow and ECL
      4. Optimizer: Distributed Batch Gradient Descent
      5. Statistical performance analysis of the implementation
      6. Test Cases and Documentation

      Wish list:

      1. Distributed convolutional computation for CNNs
      2. Optimizer: DOWNPOUR
      3. Optimizer: Synchronous SGD

        Attachments

          Activity

            People

            • Assignee:
              rkennedy Robert Kennedy
              Reporter:
              lorraineachapman Lorraine Chapman
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: